performance - Impossibly high user and system times shown by time command in Linux -


i running following command in directory ~7000 files:

time find . | xargs -p8 -n1 grep -f 'xxx' 

the results are:

real    0m1.638s user    1m1.090s sys     0m5.080s 

i understand (user+cpu) can < or > cputime, following bound should hold:

(user + sys) < real * ncpu 

where ncpu number of logical cores on system. @ instant there should @ ncpu processes running, , being assigned either user or sys time. yet have 12 logical cores (6 real cores x 2 hyperthreads), 1.638 * 12 = ~20 seconds, whereas somehow process managed consume more minute of cpu time.

subjectively 1.6s real time right (and tried on larger directories).

messing -p value varies reported real , sys times, plateaus around value somewhere around 8-12.

note none of files contain xxx, results same if use string gets hits.

most common mechanism of "user" , "system" time accounting based in linux on periodic timer. method inaccurate, short processes:

http://www.cs.rochester.edu/courses/252/spring2014/notes/xx_timing

kernel uses regular timer interrupts trigger context switches, deliver sigalarm signals, track time of day (literally, counting timer ticks), schedule bookkeeping tasks, etc.

kernel keeps every process count of how many times timer interrupt handler found (the process) in (a) user mode, or (b) kernel (system) mode. these counts work statistical profiling of gprof give sense, averaged on long period of time, of fraction of time process running, , fraction of spent in kernel (system). ... because granularity of statistical sampling equivalent scheduling quantum (~5-20ms), interval timing terribly inaccurate times below 100ms. beyond that, it's within 10%. tends charge processes of overhead of processing timer interrupts. authors report on linux system overestimates consumed time 4-5%.

also http://www.cs.toronto.edu/~demke/469f.06/lectures/lecture5.pdf slides 5, 6, 7 "accuracy of interval counting".

if check basic implementation, know "cpu" , "system" times updated in kernel/timer.c file of linux kernel, in "update_process_times" http://lxr.free-electrons.com/source/kernel/timer.c#l1349

1345 /* 1346  * called timer interrupt handler charge 1 tick current 1347  * process.  user_tick 1 if tick user time, 0 system. 1348  */ 1349 void update_process_times(int user_tick) 1350 { 1351         struct task_struct *p = current; 1352         int cpu = smp_processor_id(); 1353  1354         /* note: timer irq context must accounted well. */ 1355         account_process_tick(p, user_tick);  // <<<<<<<<<<< here 1356         run_local_timers(); 1357         rcu_check_callbacks(cpu, user_tick);     ... 1362         scheduler_tick(); 1363         run_posix_cpu_timers(p); 1364 } 

the "update_process_times" called tick_nohz_handler or tick_sched_timer -> tick_sched_handle (kernel/time/tick-sched.c#l147) , tick_handle_periodic -> tick_periodic (kernel/time/tick-common.c#l90). think cases update_process_times may called more timer interrupt.


Comments

Popular posts from this blog

Android layout hidden on keyboard show -

google app engine - 403 Forbidden POST - Flask WTForms -

c - Why would PK11_GenerateRandom() return an error -8023? -