Skip to content
Peace Lee edited this page Jul 17, 2018 · 81 revisions

Welcome to guider wiki!

  • Contents
1. What is guider?
2. How to use
3. Options
4. Mode
   - Thread mode
	Interval Info
	Graph Info
	Syscall Info
	Module Info
	Custom Event Info
	Preemption Info
	Signal Info
        Lock Info
	Custom Event Info
	Dependency Info
   - Function Mode
	CPU Info
	Page Info
	Heap Info
	Block Info
        Lock Info
	Custom Event Info
	Function Graph Info
   - Top Mode
	File Info
	Stack Info
	Memory Info
	Summary Info
	Report Info
	Image Info
   - File Mode
	Process Info
	File Info
   - System Mode
	System Info
  • What is guider?
<pre><code>
                _      _
   __ _  _   _ (_)  __| |   
  / _` || | | || | / _` | / _ \| '__|   
 | (_| || |_| || || (_| ||  __/| |   
  \__, | \__,_||_| \__,_| \___||_|   
   |___/   

</pre></code>

Do you struggle to improve system performance or to find root cause that makes system abnormal?   
Guider is made to measure amount of system resource usage and to trace system behavior.   
You can analyze your performance issues effectively with this tool.   

Guider pursues characteristics as bellow.
>1. Easy to use: just run without any setting and package installation
>2. Measure correctly: count, time in from us, size in from byte
>3. Provides all features: enough functions for experiment and analysis
>4. Report in detail: show as much information as possible

Documents and links are available.
- [Examples](https://github.com/iipeace/guider/files/1149974/guider_20170715.pdf)
- [Tutorial](http://events.linuxfoundation.org/sites/events/files/slides/guider_20170715.pdf)
- [GENIVI](https://at.projects.genivi.org/wiki/display/PROJ/Projects+Home)

The features of Guider are as follows.

guider_mindmap

How to use

Input a command as bellow to start tracing for all threads.
    # guider.py record -a
And input 'Ctrl + c' on keyboard to finish tracing.

Input a command as bellow to start monitoring for all processes.
    $ guider.py top -a
And Input 'Ctrl + c' on keyboard to finish monitoring.

Input a command as bellow to see more instructions.
    $ guider.py -h

Visit a bellow link to see output of guider.
- https://github.com/iipeace/guider/wiki

Requirement

- linux kernel (>= 2.6)
- python (>= 2.7)

Build & Installation

If you can run 'pip' on your system then just input a command as bellow
    $ sudo pip install guider
Then you can use 'guider' command

Otherwise download source from https://github.com/iipeace/guider
Then you can just run "guider.py" but it is little bit heavy 
If you want to run guider lightly then input a command as bellow
    # make && make install
Then you can use 'guider' command

Kernel Configuration

Enable kernel options as below to take advantage of all the features
And if CONFIG_STRICT_MEMORY_RWX is enabled then disable it

CONFIG_RING_BUFFER
CONFIG_FTRACE
CONFIG_TRACING
CONFIG_TRACING_SUPPORT
CONFIG_EVENT_TRACING
CONFIG_NOP_TRACER
CONFIG_TRACEPOINTS
CONFIG_DYNAMIC_FTRACE
CONFIG_HAVE_DYNAMIC_FTRACE
CONFIG_FTRACE_SYSCALLS
CONFIG_HAVE_SYSCALL_TRACEPOINTS
CONFIG_TRACE_IRQFLAGS
CONFIG_TRACE_IRQFLAGS_SUPPORT

CONFIG_STACKTRACE
CONFIG_STACKTRACE_SUPPORT
CONFIG_USER_STACKTRACE_SUPPORT
CONFIG_FUNCTION_TRACER
CONFIG_FUNCTION_GRAPH_TRACER
CONFIG_UPROBES
CONFIG_UPROBE_EVENT
CONFIG_KPROBES
CONFIG_KPROBE_EVENTS

CONFIG_TASKSTATS
CONFIG_TASK_DELAY_ACCT
CONFIG_TASK_XACCT
CONFIG_TASK_IO_ACCOUNTING
CONFIG_PERF_EVENTS 
CONFIG_HW_PERF_EVENT

Modes

[analysis]
    top        [realtime]
    record     [thread]
    record -y  [system]
    record -f  [function]
    record -F  [file]
    mem        [page]
[control]
    kill|setsched|cpulimit [proc]
[communication]
    list|start|stop|send|kill [proc]
[convenience]
    draw       [image]
    event      [event]
    threadtop  [thread]
    filetop    [file]
    stacktop   [stack]

Options

  • Use comma(,) as delimiter for multiple option values
[record]
    -e  [enable_optionsPerMode - belowCharacters]
          [function] {m(em)|b(lock)|h(eap)|l(ock)|p(ipe)|g(raph)}
          [thread]   {m(em)|b(lock)|i(rq)|l(ock)|n(et)|p(ipe)|
                      P(ower)|r(eset)|g(raph)}
          [top]      {t(hread)|b(lock)|wf(c)|s(tack)|m(em)|w(ss)|
                      P(erf)|G(pu)|i(rq)|ps(S)|u(ss)|I(mage)|a(ffinity)|
                      g(raph)|r(eport)|R(file)|r(ss)|v(ss)|l(leak)}
    -d  [disable_optionsPerMode - belowCharacters]
          [thread]   {c(pu)|a(ll)}
          [function] {c(pu)|a(ll)|u(ser)}
          [top]      {c(pu)|p(rint)|P(erf)|W(chan)|n(net)|e(ncoding)}
    -s  [save_traceData - path]
    -S  [sort - c(pu)/m(em)/b(lock)/w(fc)/p(id)/n(ew)/r(untime)/f(ile)]
    -u  [run_inBackground]
    -W  [wait_forSignal]
    -b  [set_bufferSize - kb]
    -D  [trace_threadDependency]
    -t  [trace_syscall - syscalls]
    -T  [set_fontPath]
    -j  [set_reportPath - path]
    -U  [set_userEvent - name:func|addr:file]
    -K  [set_kernelEvent - name:func|addr{:%reg/argtype:rettype}]
    -C  [set_commandScriptPath - file]
    -w  [set_customRecordCommand - BEFORE|AFTER|STOP:file{:value}]
    -x  [set_addressForLocalServer - {ip:port}]
    -X  [set_requestToRemoteServer - {req@ip:port}]
    -N  [set_addressForReport - req@ip:port]
    -n  [set_addressForPrint - ip:port]
    -M  [set_objdumpPath - file]
    -k  [set_killList - comms|tids]
[analysis]
    -o  [save_outputData - path]
    -O  [set_coreFilter - cores]
    -P  [group_perProcessBasis]
    -p  [show_preemptInfo - tids]
    -l  [set_addr2linePath - files]
    -r  [set_targetRootPath - dir]
    -I  [set_inputValue - file|addr]
    -q  [configure_taskList]
    -Z  [convert_textToImage]
    -L  [set_graphLayout - CPU|MEM|IO{:proportion}]
    -m  [set_terminalSize - {rows:cols}]
[common]
    -a  [show_allInfo]
    -Q  [print_allRowsInaStream]
    -i  [set_interval - sec]
    -R  [set_repeatCount - {interval,}count]
    -g  [set_filter - comms|tids{:files}]
    -A  [set_arch - arm|aarch64|x86|x64]
    -c  [set_customEvent - event:filter]
    -E  [set_errorLogPath - file]
    -H  [set_functionDepth]
    -Y  [set_schedPriority - policy:prio{:pid:CONT}]
    -v  [verbose]

Examples

[thread mode examples]
    - record cpu usage of threads
        # ./guider.py record -s .
    - record specific resource usage of threads in background
        # ./guider.py record -s . -e m b i -u
    - record specific resource usage excluding cpu of threads in background
        # ./guider.py record -s . -e m b i -d c -u
    - record specific systemcalls of specific threads
        # ./guider.py record -s . -t sys_read, write -g 1234
    - record lock events of threads
        # ./guider.py record -s . -e l
    - record specific user function events
        # ./guider.py record -s . -U evt1:func1:/tmp/a.out, evt2:0x1234:/tmp/b.out -M $(which objdump)
    - record specific kernel function events
        # ./guider.py record -s . -K evt1:func1, evt2:0x1234
    - record specific kernel function events with register values
        # ./guider.py record -s . -K strace32:func1:%bp/u32.%sp/s64, strace:0x1234:$stack:NONE
    - record specific kernel function events with return value
        # ./guider.py record -s . -K openfile:getname::**string, access:0x1234:NONE:*string
    - excute special commands before recording
        # ./guider.py record -s . -w BEFORE:/tmp/started:1, BEFORE:ls
    - analyze data by expressing all possible information
        # ./guider.py guider.dat -o . -a -i
    - analyze data on specific interval
        # ./guider.py guider.dat -o . -R 3
    - analyze data including preemption info of specific threads
        # ./guider.py guider.dat -o . -p 1234, 4567
    - analyze specific threads that are involved in the specific processes
        # ./guider.py guider.dat -o . -P -g 1234, 4567
    - draw graph and chart in image file
        # ./guider.py draw guider.dat

[function mode examples]
    - record cpu usage of functions in all threads
        # ./guider.py record -f -s .
    - record cpu usage of specific functions having tid bigger than 1024 in all threads
        # ./guider.py record -f -s . -g 1024\<
    - record specific events of functions of all threads in kernel level
        # ./guider.py record -f -s . -d u -c sched/sched_switch
    - record resource usage of functions of specific threads
        # ./guider.py record -f -s . -e m b h -g 1234
    - excute special commands before recording
        # ./guider.py record -s . -w BEFORE:/tmp/started:1, BEFORE:ls
    - analyze function data for all
        # ./guider.py guider.dat -o . -r /home/target/root -l $(which arm-addr2line) -a
    - analyze function data for only lower than 3 levels
        # ./guider.py guider.dat -o . -r /home/target/root -l $(which arm-addr2line) -H 3
    - record segmentation fault event of all threads
        # ./guider.py record -f -s . -K segflt:bad_area -ep
    - record blocking event except for cpu usage of all threads
        # ./guider.py record -f -s . -dc -K block:schedule

[top mode examples]
    - show resource usage of processes in real-time
        # ./guider.py top
    - show resource usage of processes with fixed terminal size in real-time
        # ./guider.py top -m
    - show files opened via processes in real-time
        # ./guider.py top -e f
    - show specific files opened via specific processes in real-time
        # ./guider.py top -e f -g init, lightdm : home, var
    - show performance stats of specific processes in real-time
        # ./guider.py top -e P -g init, lightdm
    - show resource usage of processes by sorting memory in real-time
        # ./guider.py top -S m
    - show resource usage of processes by sorting file in real-time
        # ./guider.py top -S f
    - show resource usage of processes only 5 times in real-time
        # ./guider.py top -R 5
    - show resource usage of processes only 5 times per 3 sec interval in real-time
        # ./guider.py top -R 3, 5
    - show resource usage including block of threads per 2 sec interval in real-time
        # ./guider.py top -e t b -i 2 -a
    - show resource usage of specific processes/threads involved in specific process group in real-time
        # ./guider.py top -g 1234,4567 -P
    - record resource usage of processes and write to specific file in real-time
        # ./guider.py top -o . -e p
    - record and print resource usage of processes
        # ./guider.py top -o . -Q
    - record resource usage of processes and write to specific file in background
        # ./guider.py top -o . -u
    - record resource usage of processes, system status and write to specific file in background
        # ./guider.py top -o . -e r -j . -u
    - record resource usage of processes, system status and write to specific file if some events occur
        # ./guider.py top -o . -e r R
    - record resource usage of processes, system status and write to specific image
        # ./guider.py top -o . -e r I
    - record resource usage of processes and write to specific file when specific conditions met
        # ./guider.py top -o . -e R
    - excute special commands every interval
        # ./guider.py top -w AFTER:/tmp/touched:1, AFTER:ls
    - trace memory working set for specific processes
        # ./guider.py top -e w -g chrome
    - draw graph and chart in image file
        # ./guider.py draw guider.out
        # ./guider.py top -I guider.out -e g
    - draw graph and chart for specific process group in image file
        # ./guider.py draw guider.out -g chrome
        # ./guider.py top -I guider.out -e g -g chrome
    - draw cpu and memory graphs of specific processes in image file propotionally
        # ./guider.py draw guider.out -g chrome -L cpu:5, mem:5
    - draw VSS graph and chart for specific processes in image file
        # ./guider.py draw guider.out -g chrome -e v
    - report system status to specific server
        # ./guider.py top -n 192.168.0.5:5555
    - report system status to specific server if only some events occur
        # ./guider.py top -er -N [email protected]:5555
    - report system status to specific clients that asked it
        # ./guider.py top -x 5555
    - receive report data from server
        # ./guider.py top -x 5555 -X
    - set configuration file path
        # ./guider.py top -I guider.json

[file mode examples]
    - record memory usage of files mapped to processes
        # ./guider.py record -F -o .
    - record memory usage of files mapped to processes each intervals
        # ./guider.py record -F -i

[etc examples]
    - check property of specific pages
        # ./guider.py mem -g 1234 -I 0x7abc1234-0x7abc6789
    - convert a text fle to a image file
        # ./guider.py guider.out -Z
    - wait for signal
        # ./guider.py record|top -W
    - show guider processes running
        # ./guider.py list
    - send noty signal to guider processes running
        # ./guider.py send
        # ./guider.py kill
    - send stop signal to guider processes running
        # ./guider.py stop
    - send specific signals to specific processes running
        # ./guider.py send -9 1234, 4567
        # ./guider.py kill -9 1234, 4567
    - change priority of task
        # ./guider.py setsched c:-19, r:90:1217, i:0:1209
    - change priority of tasks in a group
        # ./guider.py setsched c:-19, r:90:1217 -P
    - update priority of tasks continuously
        # ./guider.py top -Y r:90:task:ALL
    - limit cpu usage of specific processes
        # ./guider.py cpulimit 1234:40, 5678:10
    - limit cpu usage of specific threads
        # ./guider.py cpulimit 1234:40, 5678:10 -e t
  • Mode

    1. Thread mode

      [Thread Info] [ Elapsed: 2.050 ] [ Start: 2849868.198 ] [ Running: 112 ] [ CtxSwc: 3357 ] [ LogSize: 4054 KB ] [ Unit: Sec/MB/NR ]
      ==========================================================================================================================================================
      __________Thread Info___________|_____________CPU Info______________|______SCHED Info______|________BLOCK Info________|_____________MEM Info_____________|
                                      |                                   |                      |                          |                                  |
                  Name(  Tid/  Pid)|LF|Usage(    %)|Delay(  Max)|Pri| IRQ |  Yld| Lose|Steal| Mig| Read( MB/  Cnt)|WCnt( MB)| Sum(Usr/Buf/Ker)|Rcl|Wst|DRcl(Nr)|
      ==========================================================================================================================================================
      # CPU: 12
      
                CORE/0(-----/-----)|--| 0.00(  0.1)| 0.00( 0.00)|  0| 0.00|    7|    -|    -|   -| 0.00(  0/    1)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                CORE/1(-----/-----)|--| 0.00(  0.1)| 0.10( 0.00)|  0| 0.00|  147|    -|    -|   -| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                CORE/2(-----/-----)|--| 0.00(  0.1)| 0.16( 0.00)|  0| 0.00|  211|    -|    -|   -| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                CORE/3(-----/-----)|--| 0.00(  0.1)| 0.11( 0.00)|  0| 0.00|  181|    -|    -|   -| 0.00(  0/    0)|  32(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                CORE/4(-----/-----)|--| 0.00(  0.1)| 0.11( 0.00)|  0| 0.00|  232|    -|    -|   -| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                CORE/5(-----/-----)|--| 0.30( 14.8)| 0.18( 0.00)|  0| 0.00|  179|    -|    -|   -| 1.26(  6/  495)|  19(  0)|  61( 57/  0/  3)|  0|  0|0.00( 0)|
                CORE/6(-----/-----)|--| 0.00(  0.0)| 0.35( 0.00)|  0| 0.00|   57|    -|    -|   -| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                CORE/7(-----/-----)|--| 0.00(  0.0)| 0.60( 0.00)|  0| 0.00|  100|    -|    -|   -| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                CORE/8(-----/-----)|--| 0.00(  0.0)| 0.44( 0.00)|  0| 0.00|   59|    -|    -|   -| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                CORE/9(-----/-----)|--| 0.00(  0.0)| 1.94( 0.00)|  0| 0.00|   37|    -|    -|   -| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
               CORE/10(-----/-----)|--| 0.07(  3.4)| 0.00( 0.00)|  0| 0.00|    2|    -|    -|   -| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
               CORE/11(-----/-----)|--| 0.00(  0.0)| 2.05( 0.00)|  0| 0.00|   39|    -|    -|   -| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
      
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
      # Hot: 4
      
              synergyc( 3604/ 3602)|  | 0.17(  8.5)| 0.00( 0.00)|  0| 0.00|    3|   14|    3|   0| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
       arm-starfish-li(16087/16087)|  | 0.13(  6.3)| 0.00( 0.00)|  0| 0.00|    0|   20|  157|   4| 1.26(  6/  496)|   0(  0)|  61( 57/  0/  3)|  0|  0|0.00( 0)|
                guider(16088/16088)|  | 0.07(  3.4)| 0.00( 0.00)|R90| 0.00|    2|    0|    2|   0| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
      
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
      
      

      This is a main output in thread mode.
      User can analyze resource usage of threads accurately.

      [CPU Info] section includes bellow information.

      Usage: cpu time(ms) that a thread has been running
      Delay: total time(ms) that a thread has been being preempted
      Pri: highest priority that a thread has been running at
      IRQ: cpu time(ms) that a thread has been running in interupt context

      [SCHED Info] section includes bellow information.

      Yld: yield number
      Lose: preempted number
      Steal: preemption number
      Mig: migration number

      [BLOCK Info] section includes bellow information.

      Read: total time(ms) that a thread has been being blocked for disk read
      MB: total size(MB) that a thread has been reading from disk
      Cnt: total number that a thread has been reading from disk
      WCnt: total number that a thread has been writing to disk
      MB: total size(MB) that a thread has been writing to disk

      [MEM Info] section includes bellow information.

      Sum: total size(MB) that a thread has been allocating to memory(PAGE)
      Usr: user-space size(MB) that a thread has been allocating to memory(PAGE)
      Buf: page cache size(MB) that a thread has been allocating to memory(PAGE)
      Ker: kernel-space size(MB) that a thread has been allocating to memory(PAGE)
      Rcl: total size(MB) that a thread has been being reclaimed by other threads
      Wst: wasting size(MB) that a thread has been allocating to memory(PAGE) in kernel space
      Drcl(Nr): total time(ms) and number that a thread has been reclaiming directly

    +  Interval Info [-i option]    
        ``````````````````````````````````````````
        [Thread Interval Info] [ Unit: 1 Sec ]
        ==========================================================================================================================================================
                    Name(  Tid/  Pid):   1   2  2.05
        ==========================================================================================================================================================
        # Total(%/MB/Cnt)
    
                  CORE/0(    0/-----):   0   0   0
                  CORE/1(    0/-----):   0   0   0
                  CORE/2(    0/-----):   0   0   0
                  CORE/3(    0/-----):   0   0   0
                  CORE/4(    0/-----):   0   0   0
                  CORE/5(    0/-----):  20   8   5
                  CORE/6(    0/-----):   0   0   0
                  CORE/7(    0/-----):   0   0   0
                  CORE/8(    0/-----):   0   0   0
                  CORE/9(    0/-----):   0   0   0
                 CORE/10(    0/-----):   6   0   0
                 CORE/11(    0/-----):   0   0   0
    
                     MEM(    0/-----):  15  44   1
    
                  BLK_RD(    0/-----):   3   3   0
                  BLK_WR(    0/-----):   0   0   0
    
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        # CPU(%)
    
                synergyc( 3604/ 3602):  17   0   0
         arm-starfish-li(16087/16087):   4   8   5
                  guider(16088/16088):   6   0   0
                KnottyUI(15896/15896):   0   0   0
    
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        # Delay(%)
    
         arm-starfish-li(16087/16087):   0   0   0
    
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        # MEM(MB)
    
         arm-starfish-li(16087/16087):  15  44   1
                  guider(16088/16088):   0   0   0
    
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        # BLK_RD(MB)
    
         arm-starfish-li(16087/16087):   3   3   0
         Plex Media Serv( 2965/ 2956):   0   0   0
    
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        # BLK_WR(MB)
    
         Plex Media Serv( 2965/ 2956):   0   0   0
    
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
    
        This is interval output about resource usage in thread mode.    
        User can analyze resource usage of threads in each specific seconds.    
    
    +  Graph Info [-i -e g option]    
        ![guider_interval](https://cloud.githubusercontent.com/assets/15862689/23349234/a6894a6a-fcf4-11e6-911f-7d2df9e1d571.png)    
        This is graphical output about resource usage in thread mode.    
        User can analyze resource usage of threads in graphical image.    
    
    +  IRQ Info  [-a -e i option]   
        ``````````````````````````````````````````
        [Thread IRQ Info]
        ==========================================================================================================================================================
              IRQ       (                             Name                             ):    Count       Usage     ProcMax    ProcMin    InterMax   InterMin
        ==========================================================================================================================================================
        # IRQ(9) / Total( 0.004) / Cnt(1226)
    
                  irq/88(                             ahci                             ):          527  0.001570   0.000013   0.000002   0.035694   0.000010
                  irq/90(                             eth0                             ):            6  0.000010   0.000002   0.000001   0.879542   0.000073
                  irq/92(                           nouveau                            ):           35  0.000535   0.000024   0.000014   0.016661   0.001491
               softirq/1(                            TIMER                             ):           85  0.000118   0.000005   0.000001   0.231687   0.000002
               softirq/3(                            NET_RX                            ):            6  0.000084   0.000033   0.000001   0.879542   0.000073
               softirq/4(                            BLOCK                             ):          527  0.001180   0.000019   0.000001   0.035696   0.000010
               softirq/7(                            SCHED                             ):           24  0.000050   0.000014   0.000001   0.231684   0.007992
               softirq/8(                           HRTIMER                            ):            1  0.000004   0.000004   0.000004   0.000000   0.000000
               softirq/9(                             RCU                              ):           15  0.000006   0.000001   0.000000   0.299594   0.003970
    
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
    
        This is output about interrupt details in thread mode.    
    
    +  Syscall Info [-t option]    
        ``````````````````````````````````````````
        [Thread Syscall Info]
        ==========================================================================================================================================================
                    Name( Tid)    Syscall     SysId        Usage       Count           Min             Max             Avg
        ==========================================================================================================================================================
         Chrome_ChildIOT(4264)
                            sys_recvmsg          47        0.000           2      0.000002        0.000010        0.000006
                              sys_futex         202        0.000           1      0.000010        0.000010        0.000010
                             sys_gettid         186        0.000           1      0.000001        0.000001        0.000001
                         sys_epoll_wait         232        0.000           2      0.000000        0.000001        0.000000
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
         Chrome_IOThread(3735)
                            sys_sendmsg          46        0.000           1      0.000015        0.000015        0.000015
                               sys_read           0        0.000           2      0.000001        0.000011        0.000006
                          sys_epoll_ctl         233        0.000           2      0.000001        0.000009        0.000005
                         sys_epoll_wait         232        0.000           2      0.000001        0.000001        0.000001
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
         Plex DLNA Serve(23531)
                              sys_futex         202        1.100          47      0.000001        0.099949        0.023407
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
        ``````````````````````````````````````````
        [Thread Syscall History]
        ==========================================================================================================================================================
                    Name( Tid)        Time        Diff     Type            Syscall     SysId  Core      Return    Parameter
        ==========================================================================================================================================================
                  guider(13089)   0.005889    0.000000     exit          sys_write         1    11           1    -
                  guider(13089)   0.005892    0.000002     both          sys_close         3    11           0    (3, 7f4f84292000, 7f4f84293000, 4f43e0, 252e7a0, 7f4f84267740)
                  guider(13089)   0.005897    0.000007     both         sys_munmap        11    11           0    (7f4f84292000, 1000, 0, 4f43e0, 252e7a0, 7f4f84267740)
                  guider(13089)   0.005924    0.000013     both           sys_open         2    11          -2    (1f83f20, 241, 1b6, 91a870, 208d106, 240)
                  guider(13089)   0.005981    0.000011     both           sys_open         2    11           3    (1f83f20, 241, 1b6, 91a870, 208d106, 240)
                  guider(13089)   0.005994    0.000001     both          sys_fstat         5    11           0    (3, 7ffdf528fa60, 7ffdf528fa60, 1, 0, 240)
                  guider(13089)   0.005998    0.000000     both          sys_fstat         5    11           0    (3, 7ffdf528fa70, 7ffdf528fa70, d, 252e7a0, 7f4f84215414)
                  guider(13089)   0.005999    0.000004     both           sys_mmap         9    11    139979496431616 (0, 1000, 3, 22, ffffffff, 0)
                  guider(13089)   0.006009    0.000183     both          sys_write         1    11           1    (3, 7f4f84292000, 1, 4f43e0, 252e7a0, 7f4f84267740)
                  guider(13089)   0.006194    0.000001     both          sys_close         3    11           0    (3, 7f4f84292000, 7f4f84293000, 4f43e0, 252e7a0, 7f4f84267740)
                  guider(13089)   0.006198    0.000007     both         sys_munmap        11    11           0    (7f4f84292000, 1000, 0, 4f43e0, 252e7a0, 7f4f84267740)
                  guider(13089)   0.006222    0.000012     both           sys_open         2    11           3    (1f83f20, 241, 1b6, 91a870, 208d106, 240)
                  guider(13089)   0.006235    0.000001     both          sys_fstat         5    11           0    (3, 7ffdf528fa60, 7ffdf528fa60, 1, 0, 240)
                  guider(13089)   0.006239    0.000000     both          sys_fstat         5    11           0    (3, 7ffdf528fa70, 7ffdf528fa70, d, 252e7a0, 7f4f84215414)
        ``````````````````````````````````````````
    
        This is output about system call details in thread mode.    
    
    +  Module Info [-a option]    
        ``````````````````````````````````````````
        [Thread Module Info]
        ==========================================================================================================================================================
        _Type_|_Time_|_____Module_____|__Thread Name___( Tid )|Elapsed|
        ==========================================================================================================================================================
         FREE | 1.892|    psmouse     |           rmmod(13336)|
         LOAD | 3.525|    psmouse     |          insmod(13339)|  0.003|
         FREE | 5.545|    psmouse     |           rmmod(13342)|
    
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
    
        This is output about kernel module details in thread mode.    
    
    +  Creation Info [-a option]    
        ``````````````````````````````````````````
        [Thread Creation Info] [Alive: +] [Die: -] [CreatedTime: //] [ChildCount: ||] [CpuUsage: <>] [WaitTimeForChilds: {}] [WaitTimeOfParent: []]
        ==========================================================================================================================================================
        + Plex DLNA Serve(3249) <0.001> |2|
             - Plex(13345) /4.641/ <0.002> |1|
                  - Plex(13346) /6.643/ <0.004>
             + Plex(13348) /6.707/ <0.000>
        + Plex DLNA Serve(3243) <0.000> |3|
             - Plex(13343) /6.138/ <0.000>
             - Plex(13344) /6.138/ <0.000>
             - Plex(13347) /6.694/ <0.001>
        + systemd-udevd(449) <0.002> |2|
             + systemd-udevd(13337) /1.892/ <0.001>
             + systemd-udevd(13338) /1.893/ <0.001>
        + bash(27395) <0.010> |4|
             - rmmod(13336) /0.392/ <0.002>
             - insmod(13339) /2.018/ <0.010>
             - rmmod(13342) /4.045/ <0.002>
             + guider(13349) /6.441/ <0.104>
        + kthreadd(2) <0.000> |2|
             - kthreadd(13340) /3.525/ <0.000>
             + kthreadd(13341) /3.534/ <0.000>
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
    
        This is output about thread creation details in thread mode.    
        User can analyze thread tree and wait time that a parent thread waits its child threads.    
    
    +  Custom Event Info [-c option]    
        ``````````````````````````````````````````
        [Thread CUSTOM Event Info]
        ==========================================================================================================================================================
                     Event                           Comm( Tid )      Count  MaxPeriod  MinPeriod
        ==========================================================================================================================================================
                 hrtimer_cancel                     TOTAL(  -  )         52   0.111849   0.000109
                                                 synergyc( 3604)         33   0.003995   0.000373
                                                   guider(22424)          6   0.002951   0.000147
                                                   guider(14882)          2   0.111849   0.111849
                                          Chrome_IOThread( 3735)          2   0.000266   0.000266
                                                     Xorg( 1636)          2   0.000109   0.000109
                                                 synergyc( 3602)          2   0.000053   0.000053
                                                 KnottyUI(22243)          1   0.000000   0.000000
                                          Plex Media Serv(14880)          1   0.000000   0.000000
                                          chromium-browse( 4273)          1   0.000000   0.000000
                                          BrowserBlocking( 3729)          1   0.000000   0.000000
                                                    gmain( 2233)          1   0.000000   0.000000
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
        ``````````````````````````````````````````
        [Thread CUSTOM Event History]
        ==========================================================================================================================================================
                     EVENT                  TIME                COMM(  TID) ARG
        ==========================================================================================================================================================
                 hrtimer_cancel            0.033069         synergyc( 3602) hrtimer=ffff880fc45d1a70
                 hrtimer_cancel            0.033122         synergyc( 3602) hrtimer=ffff880fc45d1a70
                 hrtimer_cancel            0.065970           guider(14882) hrtimer=ffff88101fd6dce0
                 hrtimer_cancel            0.177819           guider(14882) hrtimer=ffff88101fd6dce0
                 hrtimer_cancel            0.521354           guider(22424) hrtimer=ffff88101fc4dce0
                 hrtimer_cancel            0.524193           guider(22424) hrtimer=ffff880e6b9a5de0
                 hrtimer_cancel            0.525349           guider(22424) hrtimer=ffff88101fc4dce0
                 hrtimer_cancel            0.526243           guider(22424) hrtimer=ffff8800c126bde0
                 hrtimer_cancel            0.526390           guider(22424) hrtimer=ffff8801009f3de0
                 hrtimer_cancel            0.529209            gmain( 2233) hrtimer=ffff880f8fe8fa70
                 hrtimer_cancel            0.529341           guider(22424) hrtimer=ffff88101fc4dce0
                 hrtimer_cancel            0.725987  Chrome_IOThread( 3735) hrtimer=ffff880f67b3ddf8
                 hrtimer_cancel            0.726215  BrowserBlocking( 3729) hrtimer=ffff8800c126bde0
                 hrtimer_cancel            0.726253  Chrome_IOThread( 3735) hrtimer=ffff880f67b3ddf8
                 hrtimer_cancel            0.726379  chromium-browse( 4273) hrtimer=ffff880f66e75de0
                 hrtimer_cancel            1.072608  Plex Media Serv(14880) hrtimer=ffff88101fccdce0
                 hrtimer_cancel            1.388184         synergyc( 3604) hrtimer=ffff88101fcadce0
                 hrtimer_cancel            1.392177         synergyc( 3604) hrtimer=ffff88101fcadce0
                 hrtimer_cancel            1.396172         synergyc( 3604) hrtimer=ffff88101fcadce0
                 hrtimer_cancel            1.400166         synergyc( 3604) hrtimer=ffff88101fcadce0
                 hrtimer_cancel            1.402252         KnottyUI(22243) hrtimer=ffff880f9176dde0
                 hrtimer_cancel            1.404160         synergyc( 3604) hrtimer=ffff88101fcadce0
                 hrtimer_cancel            1.408155         synergyc( 3604) hrtimer=ffff88101fcadce0
                 hrtimer_cancel            1.412149         synergyc( 3604) hrtimer=ffff88101fcadce0
                 hrtimer_cancel            1.415771         synergyc( 3604) hrtimer=ffff8800c127fde0
                 hrtimer_cancel            1.416144         synergyc( 3604) hrtimer=ffff88101fcadce0
                 hrtimer_cancel            1.420138         synergyc( 3604) hrtimer=ffff88101fcadce0
                 hrtimer_cancel            1.424133         synergyc( 3604) hrtimer=ffff88101fcadce0
                 hrtimer_cancel            1.428128         synergyc( 3604) hrtimer=ffff88101fcadce0
                 hrtimer_cancel            1.432122         synergyc( 3604) hrtimer=ffff88101fcadce0
                 hrtimer_cancel            1.436117         synergyc( 3604) hrtimer=ffff88101fcadce0
                 hrtimer_cancel            1.438559             Xorg( 1636) hrtimer=ffff880fc4abd938
        ``````````````````````````````````````````
    
        This is output about custom event in thread mode.    
        User can analyze ftrace tracing points.    
    
    +  Preemption Info [-p option]    
        ``````````````````````````````````````````
        [Thread Info] [ Elapsed: 2.742 ] [ Start: 75.657 ] [ Running: 139 ] [ CtxSwc: 20255 ] [ LogSize: 5500 KB ] [ Unit: Sec/MB/NR ]
        ==========================================================================================================================================================
        __________Thread Info___________|_____________CPU Info______________|______SCHED Info______|________BLOCK Info________|_____________MEM Info_____________|
                                        |                                   |                      |                          |                                  |
                    Name(  Tid/  Pid)|LF|Usage(    %)|Delay(  Max)|Pri| IRQ |  Yld| Lose|Steal| Mig| Read( MB/  Cnt)|WCnt( MB)| Sum(Usr/Buf/Ker)|Rcl|Wst|DRcl(Nr)|
        ==========================================================================================================================================================
        # CPU: 4
    
                  CORE/0(-----/-----)|--| 0.19(  6.8)| 0.00( 0.00)|  0| 0.00|    0|    -|    -|   -| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                  CORE/1(-----/-----)|--| 0.21(  7.5)| 0.00( 0.00)|  0| 0.00|    0|    -|    -|   -| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                  CORE/2(-----/-----)|--| 0.40( 14.6)| 0.00( 0.00)|  0| 0.00|    0|    -|    -|   -| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                  CORE/3(-----/-----)|--| 0.17(  6.4)| 0.65( 0.00)|  0| 0.00|    1|    -|    -|   -| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
    
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        # Hot: 9
    
         surface-manager( 1615/ 1615)|  | 0.29( 10.7)| 0.02( 0.00)|  0| 0.00|  161|  314|   77|   4| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                  guider( 3474/ 3474)|  | 0.06(  2.3)| 0.01( 0.00)|  0| 0.00|    2|   40|    1|   1| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
             tWindowCore( 1442/ 1407)|  | 0.06(  2.3)| 0.01( 0.00)|  0| 0.00|  333|  200|  314|   3| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
           mali-renderer( 1681/ 1615)|  | 0.05(  1.9)| 0.00( 0.00)|  0| 0.00|  559|  101|  540|  10| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                  guider( 4875/ 4875)|  | 0.05(  1.7)| 0.00( 0.00)|  0| 0.00|    2|   10|    2|   0| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                tvideoCC( 2422/ 1613)|  | 0.04(  1.4)| 0.00( 0.00)|  0| 0.00|  285|    7|  268|   2| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
               pq_thread( 2558/ 1613)|  | 0.04(  1.3)| 0.00( 0.00)|  0| 0.00|  103|   30|   97|   0| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                n0000028( 2450/ 1613)|  | 0.03(  1.0)| 0.00( 0.00)|  0| 0.00|  529|    3|  473|   5| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
    
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        # PRT: Tid(1615) / Comm(surface-manager) / Total( 0.017) / Threads(45)
    
           mali-renderer( 1681/    0)|  | 0.01( 40.5)
         com.webos.app.i( 1785/    0)|  | 0.00( 10.9)
                n0000034( 2512/    0)|  | 0.00(  7.0)
                 tHDMI5V( 2458/    0)|  | 0.00(  6.7)
                tvideoCC( 2422/    0)|  | 0.00(  4.4)
                n0000028( 2450/    0)|  | 0.00(  4.1)
                     sdx( 4858/    0)|  | 0.00(  2.8)
                 tAUDATV( 2470/    0)|  | 0.00(  2.7)
                tCMState( 2490/    0)|  | 0.00(  2.5)
                    tAVD( 2454/    0)|  | 0.00(  2.0)
                tAUDHDMI( 2469/    0)|  | 0.00(  1.6)
            kworker/u9:1( 1524/    0)|  | 0.00(  1.4)
             rcu_preempt(    7/    0)|  | 0.00(  1.1)
                  sleepd( 2251/    0)|  | 0.00(  0.9)
                n0000005( 2226/    0)|  | 0.00(  0.9)
                  tHDMI1( 2459/    0)|  | 0.00(  0.8)
         hdmi20_rx_video( 2451/    0)|  | 0.00(  0.8)
              te_monitor(  940/    0)|  | 0.00(  0.8)
          mali-event-hnd( 1676/    0)|  | 0.00(  0.8)
         physical-device( 3010/    0)|  | 0.00(  0.7)
            HSM Read Map( 2548/    0)|  | 0.00(  0.6)
                  tHDMI3( 2461/    0)|  | 0.00(  0.5)
                  tHDMI4( 2462/    0)|  | 0.00(  0.4)
                  tHDMI2( 2460/    0)|  | 0.00(  0.4)
                  ACCT00( 2524/    0)|  | 0.00(  0.4)
           tSettingsDone( 2563/    0)|  | 0.00(  0.4)
               ARTIMER_T( 2473/    0)|  | 0.00(  0.4)
        ``````````````````````````````````````````
    
        This is output about preemption details in thread mode.    
        User can analyze which threads had preempted a specific thread.    
    
    +  Signal Info [-a option]    
        ``````````````````````````````````````````
        [Thread Signal Info]
        ==========================================================================================================================================================
         TYPE        TIME           SENDER(  TID)   SIGNAL           RECEIVER(  TID)
        ==========================================================================================================================================================
         RECV    0.394702                          SIGCHLD               bash(27395)
         SEND    1.892381            rmmod(13336)  SIGCHLD               bash(27395)
         RECV    2.027533                          SIGCHLD               bash(27395)
         SEND    3.525273           insmod(13339)  SIGCHLD               bash(27395)
         RECV    4.047565                          SIGCHLD               bash(27395)
         SEND    5.545286            rmmod(13342)  SIGCHLD               bash(27395)
         RECV    6.547838                          SIGCHLD               bash(27395)
         SEND    7.946579        swapper/4( 0[4])  SIGALRM             screen( 7400)
         RECV    7.946588                          SIGALRM             screen( 7400)
         SEND    8.042445           guider(13349)   SIGINT             guider(13335)
         RECV    8.042476                           SIGINT             guider(13335)
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
    
        This is output about signal traffic in thread mode.    
    
    +  User Event Info [-U option]    
        ``````````````````````````````````````````
        [Thread USER Event Info]
        ==========================================================================================================================================================
                     Event                           Comm( Tid )      Usage      Count    ProcMax    ProcMin   InterMax   InterMin
        ==========================================================================================================================================================
                    diskFunc                        TOTAL(  -  )   0.004830          1   0.004830   0.004830   0.000000   0.000000
                                                    a.out(15158)   0.004830          1   0.004830   0.004830   0.000000   0.000000
    
                    cpuFunc                         TOTAL(  -  )   0.000000          1   0.000000   0.000000   0.000000   0.000000
                                                    a.out(15158)   0.000000          1   0.000000   0.000000   0.000000   0.000000
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
        ``````````````````````````````````````````
        [Thread USER Event History]
        ==========================================================================================================================================================
                     EVENT                TYPE     TIME                COMM(  TID)      CALLER         ELAPSED
        ==========================================================================================================================================================
                    diskFunc              ENTER   1.981194            a.out(15158)                           -
                    diskFunc               EXIT   1.986024            a.out(15158)        0x400ab3    0.004830
                    cpuFunc               ENTER   1.986029            a.out(15158)                           -
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
    
        This is output about user event in thread mode.    
        User can analyze user-level function call events.    
    
    
    +  Kernel Event Info [-K option]    
        ``````````````````````````````````````````
        [Thread KERNEL Event Info]
        ==========================================================================================================================================================
                     Event                           Comm( Tid )      Usage      Count    ProcMax    ProcMin   InterMax   InterMin
        ==========================================================================================================================================================
                      open                          TOTAL(  -  )   0.000000          0   0.000000   0.000000   0.000000   0.000000
                                                   guider(15403)   0.000000          0   0.000000   0.000000   0.000000   0.000000
                                                   guider(22424)   0.000000          0   0.000000   0.000000   0.000000   0.000000
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
        ``````````````````````````````````````````
        [Thread KERNEL Event History]
        ==========================================================================================================================================================
                     EVENT                TYPE     TIME                COMM(  TID)         CALLER            ELAPSED ARG
        ==========================================================================================================================================================
                      open                 EXIT   0.000000           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/sched/sched_switch/filter"
                      open                 EXIT   0.000064           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/sched/sched_switch/enable"
                      open                 EXIT   0.000186           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/sched/sched_migrate_task/enable"
                      open                 EXIT   0.000414           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/sched/sched_process_exit/enable"
                      open                 EXIT   0.000644           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/sched/sched_process_wait/enable"
                      open                 EXIT   0.000877           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/raw_syscalls/sys_enter/filter"
                      open                 EXIT   0.000921           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/raw_syscalls/sys_enter/enable"
                      open                 EXIT   0.000959           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/power/suspend_resume/enable"
                      open                 EXIT   0.001026           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/module/module_load/enable"
                      open                 EXIT   0.001260           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/module/module_free/enable"
                      open                 EXIT   0.001488           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/module/module_put/enable"
                      open                 EXIT   0.002246           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/power/cpu_idle/enable"
                      open                 EXIT   0.002985           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/power/cpu_frequency/enable"
                      open                 EXIT   0.003325           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/vmscan/mm_vmscan_direct_reclaim_begin/enable"
                      open                 EXIT   0.003552           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/vmscan/mm_vmscan_direct_reclaim_end/enable"
                      open                 EXIT   0.003778           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/task/enable"
                      open                 EXIT   0.004226           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/signal/enable"
        ``````````````````````````````````````````
    
        This is output about kernel event in thread mode.    
        User can analyze kernel-level function call events.    
    
    +  Dependency Info [-D option]    
        ``````````````````````````````````````````
        [Thread Dependency Info]
        ==========================================================================================================================================================
          Total/Inter                 From( Tid) ->               To( Tid)    Event
        ==========================================================================================================================================================
        # Dep: 182
    
          0.002/0.000               guider(3474) ->    mali-renderer(1681)    kick
          0.002/0.000        mali-renderer(1681) ->           tHDMI4(2462)    kick
          0.003/0.000               guider(3474) ->              sdx(4858)    kick
          0.003/0.000             n0000028(2450) ->           ACCT00(2524)    kick
          0.004/0.001                                  mali-renderer(1681)    wakeup
          0.012/0.008               guider(3474) ->             qosm(3484)    kick
          0.012/0.001                                    PmLogDaemon(1417)    recv
          0.019/0.007                                  mali-renderer(1681)    wakeup
          0.137/0.118                                  mali-renderer(1681)    wakeup
          0.139/0.002       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
          0.152/0.014      surface-manager(1615) ->    mali-renderer(1681)    kick
          0.154/0.002       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
          0.169/0.015      surface-manager(1615) ->    mali-renderer(1681)    kick
          0.171/0.002       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
          0.185/0.015      surface-manager(1615) ->    mali-renderer(1681)    kick
          0.187/0.002                                  mali-renderer(1681)    wakeup
          0.204/0.017       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
          0.221/0.017       mali-event-hnd(1676) ->            MICOM(2380)    kick
          0.221/0.000       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
          0.269/0.048      surface-manager(1615) ->    mali-renderer(1681)    kick
          0.371/0.102       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
          0.654/0.283                                  mali-renderer(1681)    wakeup
          0.669/0.015      surface-manager(1615) ->    mali-renderer(1681)    kick
          0.804/0.135       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
          1.002/0.198      surface-manager(1615) ->    mali-renderer(1681)    kick
          1.004/0.002       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
          1.018/0.015      surface-manager(1615) ->    mali-renderer(1681)    kick
          1.021/0.002       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
          1.035/0.015      surface-manager(1615) ->    mali-renderer(1681)    kick
          1.037/0.002       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
          1.052/0.015      surface-manager(1615) ->    mali-renderer(1681)    kick
          1.054/0.002       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
          1.068/0.014      surface-manager(1615) ->    mali-renderer(1681)    kick
          1.071/0.002       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
          1.085/0.015      surface-manager(1615) ->    mali-renderer(1681)    kick
          1.087/0.002       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
          1.102/0.015      surface-manager(1615) ->    mali-renderer(1681)    kick
          1.104/0.002       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
          1.118/0.015      surface-manager(1615) ->    mali-renderer(1681)    kick
          1.121/0.002       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
        ``````````````````````````````````````````
    
        This is output about dependency of each threads in thread mode.    
    
    1. Function mode [-f option]

      [Function Thread Info] [ Elapsed: 3.569 ] [ Start: 2858278.707 ] [ Threads: 23 ] [ LogSize: 1541 KB ]
      ==========================================================================================================================================================
      _______________________Thread________________________|__CPU__|_________________________PAGE_________________________|__HEAP__|______BLOCK_______|_CUSTOM_|
                                                           |       |                                                      |        |                  |        |
      ______Name______|__Tid__|__Pid__|_Target_|_DIE_|_NEW_|__PER__|__ALLOC__(__USER__/__BUF___/__KERN__)|__FREE__|_UFREE_|__EXP___|__READ__|__WRITE__|___NR___|
      ==========================================================================================================================================================
                 a.out|  15939|  15939|   *    |     |  v  |  98.3%|   11416K(   3140K/      0K/   8276K)|   104K|     84K|      0K|      0K|       0K|       0|
       Plex DLNA Serve|  23531|   3010|   *    |     |     |   0.2%|       0K(      0K/      0K/      0K)|     0K|      0K|      0K|      0K|       0K|       0|
       Plex DLNA Serve|  15944|  -----|   *    |  v  |  v  |   0.2%|       0K(      0K/      0K/      0K)|  2024K|      0K|      0K|      0K|       0K|       0|
       Plex DLNA Serve|   9871|   3010|   *    |     |     |   0.2%|       0K(      0K/      0K/      0K)|     0K|      0K|      0K|      0K|       0K|       0|
                guider|  15936|  15936|   *    |     |     |   0.2%|       0K(      0K/      0K/      0K)|   188K|    120K|      0K|      0K|       0K|       0|
                  bash|   4011|   4011|   *    |     |     |   0.2%|      92K(     84K/      0K/      8K)|     0K|      0K|      0K|      0K|       0K|       0|
           ksoftirqd/1|     37|     37|   *    |     |     |   0.2%|       0K(      0K/      0K/      0K)|     0K|      0K|      0K|      0K|       0K|       0|
                compiz|   2571|   2571|   *    |     |     |   0.2%|      12K(     12K/      0K/      0K)|     0K|      0K|      0K|      0K|       0K|       0|
       Plex DLNA Serve|  15943|  -----|   *    |  v  |  v  |   0.0%|       0K(      0K/      0K/      0K)|     0K|      0K|      0K|      0K|       0K|       0|
       Plex DLNA Serve|   8385|   3010|   *    |     |     |   0.0%|       0K(      0K/      0K/      0K)|     4K|      0K|      0K|      0K|       0K|       0|
       Plex DLNA Serve|  15941|  -----|   *    |  v  |  v  |   0.0%|       0K(      0K/      0K/      0K)|     0K|      0K|      0K|      0K|       0K|       0|
         kworker/u24:2|    679|    679|   *    |     |     |   0.0%|       4K(      0K/      0K/      4K)|     0K|      0K|      0K|      0K|      64K|       0|
                  Xorg|   1636|   1636|   *    |     |     |   0.0%|       8K(      0K/      0K/      8K)|    40K|      8K|      0K|      0K|       0K|       0|
       chromium-browse|   4273|   4273|   *    |     |     |   0.0%|       4K(      4K/      0K/      0K)|     0K|      0K|      0K|      0K|       0K|       0|
       chromium-browse|   4507|   4507|   *    |     |     |   0.0%|       4K(      4K/      0K/      0K)|     0K|      0K|      0K|      0K|       0K|       0|
               rcuos/3|     11|     11|   *    |     |     |   0.0%|       0K(      0K/      0K/      0K)|     8K|      0K|      0K|      0K|       0K|       0|
                screen|   3565|   3565|   *    |     |     |   0.0%|       4K(      0K/      0K/      4K)|    40K|      4K|      0K|      0K|       0K|       0|
       Plex DLNA Serve|   3243|   3010|   *    |     |     |   0.0%|      40K(      0K/      0K/     40K)|     0K|      0K|      0K|      0K|       0K|       0|
           kworker/2:0|  13550|  13550|   *    |     |     |   0.0%|       0K(      0K/      0K/      0K)|   516K|     12K|      0K|      0K|       0K|       0|
       Plex DLNA Serve|   3246|   3010|   *    |     |     |   0.0%|       8K(      0K/      0K/      8K)|     0K|      0K|      0K|      0K|       0K|       0|
               rcuos/8|     16|     16|   *    |     |     |   0.0%|       0K(      0K/      0K/      0K)|    16K|      0K|      0K|      0K|       0K|       0|
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
      

      This is output about thread list profiled in function mode.
      User can select specific threads with -g option and analyze resource usage by function.
      But symbol of threads that already died will not be shown.

    +  CPU Info    
        ``````````````````````````````````````````
        [Function CPU Info] [Cnt: 394] [Interval: 8ms] (USER)
        ==========================================================================================================================================================
        __Usage__|___________________Function____________________|_____________________________________________Binary_____________________________________________
        ==========================================================================================================================================================
           99.0% |                    cpuTest                    | /media/disk/work/test/a.out
           +  100.0% | <- startTest [/media/disk/work/test/a.out] <- main [/media/disk/work/test/a.out]
                         <- __libc_start_main [/lib/x86_64-linux-gnu/libc-2.19.so]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
            0.5% |                    memset                     | /lib/x86_64-linux-gnu/libc-2.19.so
           +  100.0% | <- startTest [/media/disk/work/test/a.out] <- main [/media/disk/work/test/a.out]
                         <- __libc_start_main [/lib/x86_64-linux-gnu/libc-2.19.so]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
            0.3% |                  _int_malloc                  | /lib/x86_64-linux-gnu/libc-2.19.so
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
            0.3% |               00007f756e3e7ee4                | ??
           +  100.0% | <- 000000000044676f [/media/disk/work/test/a.out]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
    
        [Function CPU Info] [Cnt: 394] [Interval: 8ms] (KERNEL)
        ==========================================================================================================================================================
        __Usage__|____________________________________________________________________Function____________________________________________________________________
        ==========================================================================================================================================================
          100.0% |                                                          hrtimer_interrupt
           +   99.5% | <- local_apic_timer_interrupt <- smp_apic_timer_interrupt <- apic_timer_interrupt
           +    0.3% | <- local_apic_timer_interrupt <- smp_apic_timer_interrupt <- apic_timer_interrupt <- do_page_fault <- page_fault
           +    0.3% | <- local_apic_timer_interrupt <- smp_apic_timer_interrupt <- apic_timer_interrupt <- __do_fault <- handle_mm_fault <- __do_page_fault
                         <- do_page_fault <- page_fault
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
    
        This is output about cpu usage of specific threads by function in function mode.    
    
    +  Mem Info [-f -e m option]    
        ``````````````````````````````````````````
        [Function Page Info] [Total: 11416KB] [Alloc: 11444KB(817)] [Free: 188KB(47)] (USER)
        ==========================================================================================================================================================
         Usage ( Usr  / Buf  / Ker  )|___________________Function____________________|________________LifeTime________________|______________Binary_______________
        ==========================================================================================================================================================
         10256K(  2048/     0/  8208)|                    memset                     | AVR: 1.563 / MIN: 1.560 / MAX: 1.568   | /lib/x86_64-linux-gnu/libc-2.19.so
          +  10256K(  2048/     0/  8208)| <- startTest [/media/disk/work/test/a.out] <- main [/media/disk/work/test/a.out]
                                             <- __libc_start_main [/lib/x86_64-linux-gnu/libc-2.19.so]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
           960K(   956/     0/     4)|                  _int_malloc                  | AVR: 1.559 / MIN: 1.554 / MAX: 1.560   | /lib/x86_64-linux-gnu/libc-2.19.so
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
            56K(    16/     0/    40)|               00007f756e3e81e7                | AVR: 1.569 / MIN: 1.568 / MAX: 1.569   | ??
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
            44K(    36/     0/     8)|                   sysmalloc                   | AVR: 1.560 / MIN: 1.558 / MAX: 1.568   | /lib/x86_64-linux-gnu/libc-2.19.so
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
            12K(    12/     0/     0)|           elf_machine_rela_relative           | AVR: 1.568 / MIN: 1.568 / MAX: 1.568   | /lib/x86_64-linux-gnu/ld-2.19.so
          +     12K(    12/     0/     0)| <- dl_main [/lib/x86_64-linux-gnu/ld-2.19.so] <- _dl_sysdep_start [/lib/x86_64-linux-gnu/ld-2.19.so]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
             8K(     8/     0/     0)|                    realloc                    | AVR: 1.568 / MIN: 1.568 / MAX: 1.568   | /lib/x86_64-linux-gnu/ld-2.19.so
          +      4K(     4/     0/     0)| <- _dl_map_object [/lib/x86_64-linux-gnu/ld-2.19.so]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
             8K(     4/     0/     4)|                    dl_main                    | AVR: 1.568 / MIN: 1.568 / MAX: 1.568   | /lib/x86_64-linux-gnu/ld-2.19.so
          +      8K(     4/     0/     4)| <- _dl_sysdep_start [/lib/x86_64-linux-gnu/ld-2.19.so]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
    
        [Function Page Info] [Total: 11416KB] [Alloc: 11444KB(817)] [Free: 188KB(47)] (KERNEL)
        ==========================================================================================================================================================
         Usage ( Usr  / Buf  / Ker  )|___________________Function____________________|__________________________________LifeTime__________________________________
        ==========================================================================================================================================================
          8192K(     0/     0/  8192)|          do_huge_pmd_anonymous_page           |                    AVR: 1.563 / MIN: 1.562 / MAX: 1.564
          +   8192K(     0/     0/  8192)| <- handle_mm_fault <- __do_page_fault <- do_page_fault <- page_fault
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
          3084K(  3084/     0/     0)|                handle_mm_fault                |                    AVR: 1.563 / MIN: 1.554 / MAX: 1.569
          +   3076K(  3076/     0/     0)| <- __do_page_fault <- do_page_fault <- page_fault
          +      4K(     4/     0/     0)| <- __get_user_pages <- get_user_pages <- copy_strings.isra.17 <- copy_strings_kernel <- do_execve_common.isra.23
                                             <- SyS_execve <- stub_execve
          +      4K(     4/     0/     0)| <- __do_page_fault <- do_page_fault <- page_fault <- load_elf_binary <- search_binary_handler
                                             <- do_execve_common.isra.23 <- SyS_execve <- stub_execve
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
    
        This is output about memory usage of specific threads by function in function mode.    
    
    +  Heap Info [-f -e h option]    
        ``````````````````````````````````````````
        [Function Heap Info] [Total: 18913KB] [Alloc: 19074KB(16)] [Free: 161KB(1)] (USER)
        ==========================================================================================================================================================
        __Usage__|___________________Function____________________|_____________________Binary______________________|____________________Source____________________
        ==========================================================================================================================================================
          10244K |                    mmap64                     | /lib/x86_64-linux-gnu/libc-2.19.so              | /build/eglibc-oGUzwX/eglibc-2.19/misc/../sysdeps/unix/syscall-template.S:81
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
           4752K |                     __brk                     | /lib/x86_64-linux-gnu/ld-2.19.so                | /build/eglibc-oGUzwX/eglibc-2.19/elf/../sysdeps/unix/sysv/linux/x86_64/brk.c:31
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
           3917K |                    realloc                    | /lib/x86_64-linux-gnu/ld-2.19.so                | /build/eglibc-oGUzwX/eglibc-2.19/misc/../sysdeps/unix/syscall-template.S:81
          +    3897K | <- _dl_map_object [/lib/x86_64-linux-gnu/ld-2.19.so]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
    
        [Function Heap History] [Cnt: 15]
        ==========================================================================================================================================================
        _____________VAddr______________|____Size____|__Size(KB)__|__Size(MB)__|______COMM_______(__TID__)|__CORE__|______TIME_______|
        ==========================================================================================================================================================
                139733299458048         |       8192 |          8 |          0 |            a.out(  16284)|      5 |  2858989.603987 |
        ------------------------------------------------------------------------------------------------------------------------------
                                 [User] | realloc[/lib/x86_64-linux-gnu/ld-2.19.so]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                139733293277184         |    3949248 |       3856 |          3 |            a.out(  16284)|      5 |  2858989.604134 |
        ------------------------------------------------------------------------------------------------------------------------------
                                 [User] | realloc[/lib/x86_64-linux-gnu/ld-2.19.so] <- _dl_map_object[/lib/x86_64-linux-gnu/ld-2.19.so]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                139733297184768         |      24576 |         24 |          0 |            a.out(  16284)|      5 |  2858989.604164 |
        ------------------------------------------------------------------------------------------------------------------------------
                                 [User] | realloc[/lib/x86_64-linux-gnu/ld-2.19.so] <- _dl_map_object[/lib/x86_64-linux-gnu/ld-2.19.so]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                139733297209344         |      17088 |         16 |          0 |            a.out(  16284)|      5 |  2858989.604191 |
        ------------------------------------------------------------------------------------------------------------------------------
                                 [User] | realloc[/lib/x86_64-linux-gnu/ld-2.19.so] <- _dl_map_object[/lib/x86_64-linux-gnu/ld-2.19.so]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                139733299286016         |       4096 |          4 |          0 |            a.out(  16284)|      5 |  2858989.604241 |
        ------------------------------------------------------------------------------------------------------------------------------
                                 [User] | realloc[/lib/x86_64-linux-gnu/ld-2.19.so]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
    
        This is output about heap usage of specific threads by function in function mode.    
    
    +  Block Info [-f -e b option]    
        ``````````````````````````````````````````
        [Function BLK_RD Info] [Size: 10396KB] [Cnt: 90] (USER)
        ==========================================================================================================================================================
        __Usage__|___________________Function____________________|_____________________Binary______________________|____________________Source____________________
        ==========================================================================================================================================================
          10384K |                    __read                     | /lib/x86_64-linux-gnu/libc-2.19.so              | /build/eglibc-oGUzwX/eglibc-2.19/io/../sysdeps/unix/syscall-template.S:81
          +   10384K | <- startTest [/media/disk/work/test/a.out] <- main [/media/disk/work/test/a.out]
                         <- __libc_start_main [/lib/x86_64-linux-gnu/libc-2.19.so]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
             12K |               00007f756e3e81e7                | ??                                              |
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
    
        [Function BLK_RD Info] [Size: 10396KB] [Cnt: 90] (KERNEL)
        ==========================================================================================================================================================
        __Usage__|____________________________________________________________________Function____________________________________________________________________
        ==========================================================================================================================================================
          10396K |                                                                   submit_bio
          +   10016K | <- mpage_readpages <- ext4_readpages <- __do_page_cache_readahead <- ondemand_readahead <- page_cache_async_readahead
                         <- generic_file_aio_read <- do_sync_read <- vfs_read <- SyS_read <- tracesys
          +     128K | <- mpage_readpages <- ext4_readpages <- __do_page_cache_readahead <- ondemand_readahead <- page_cache_sync_readahead
                         <- generic_file_aio_read <- do_sync_read <- vfs_read <- SyS_read <- tracesys
          +      96K | <- do_mpage_readpage <- mpage_readpages <- ext4_readpages <- __do_page_cache_readahead <- ondemand_readahead
                         <- page_cache_async_readahead <- generic_file_aio_read <- do_sync_read <- vfs_read <- SyS_read <- tracesys
          +      80K | <- do_mpage_readpage <- mpage_readpages <- ext4_readpages <- __do_page_cache_readahead <- ondemand_readahead <- page_cache_sync_readahead
                         <- generic_file_aio_read <- do_sync_read <- vfs_read <- SyS_read <- tracesys
          +      48K | <- do_mpage_readpage <- mpage_readpages <- ext4_readpages <- __do_page_cache_readahead <- ondemand_readahead <- page_cache_sync_readahead
                         <- generic_file_aio_read <- do_sync_read <- vfs_read <- SyS_read <- tracesys
          +      12K | <- _submit_bh <- bh_submit_read <- ext4_get_branch <- ext4_ind_map_blocks <- ext4_map_blocks <- _ext4_get_block <- ext4_get_block
                         <- do_mpage_readpage <- mpage_readpages <- ext4_readpages <- __do_page_cache_readahead <- ondemand_readahead
                         <- page_cache_async_readahead <- generic_file_aio_read <- do_sync_read <- vfs_read <- SyS_read <- tracesys
          +      12K | <- mpage_readpages <- ext4_readpages <- __do_page_cache_readahead <- ondemand_readahead <- page_cache_sync_readahead
                         <- generic_file_aio_read <- do_sync_read <- vfs_read <- kernel_read <- prepare_binprm <- do_execve_common.isra.23 <- SyS_execve
                         <- stub_execve
          +       4K | <- _submit_bh <- bh_submit_read <- ext4_get_branch <- ext4_ind_map_blocks <- ext4_map_blocks <- _ext4_get_block <- ext4_get_block
                         <- do_mpage_readpage <- mpage_readpages <- ext4_readpages <- __do_page_cache_readahead <- ondemand_readahead
                         <- page_cache_sync_readahead <- generic_file_aio_read <- do_sync_read <- vfs_read <- SyS_read <- tracesys
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
    
        This is output about disk(block) usage of specific threads by function in function mode.    
    
    +  Kernel Event Info [-f -K option]    
        ```````````````````````````````````````
        [Function segflt_exit, segflt_enter Info] [Cnt: 2] [Total: 2] (USER)
        ==========================================================================================================================================================
        __Usage__|___________________Function____________________|_____________________Binary______________________|____________________Source____________________
        ==========================================================================================================================================================
              2  |                   faultTest                   | /media/disk/work/test/a.out                     | ??:?
               +      2 | <- startTest [/media/disk/work/test/a.out] <- main [/media/disk/work/test/a.out]
                            <- __libc_start_main [/lib/x86_64-linux-gnu/libc-2.19.so]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
    
        [Function segflt_exit, segflt_enter Info] [Cnt: 2] [Total: 2] (KERNEL)
        ==========================================================================================================================================================
        __Usage__|____________________________________________________________________Function____________________________________________________________________
        ==========================================================================================================================================================
              2  |                                                            do_page_fault
               +      2 | <- page_fault
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
        ``````````````````````````````````````````
        [Function segflt_exit, segflt_enter History] [Cnt: 2] [Total: 2]
        ==========================================================================================================================================================
        _____________Event______________|______COMM_______(__TID__)|__CORE__|______TIME_______|
        ==========================================================================================================================================================
                  segflt_enter          |            a.out(  15657)|    010 |  2870585.085374 |
        ---------------------------------------------------------------------------------------
                                 [Args] | (bad_area+0x0/0x4a <ffffffff81722b0e>)
                                 [User] | faultTest[/media/disk/work/test/a.out] <- startTest[/media/disk/work/test/a.out] <- main[/media/disk/work/test/a.out]
                                          <- __libc_start_main[/lib/x86_64-linux-gnu/libc-2.19.so]
                               [Kernel] | do_page_fault <- page_fault
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                  segflt_exit           |            a.out(  15657)|    010 |  2870585.085420 |
        ---------------------------------------------------------------------------------------
                                 [Args] | (__do_page_fault+0x462/0x560 <ffffffff817362a2> <- bad_area <ffffffff81722b0e>)
                                 [User] | faultTest[/media/disk/work/test/a.out] <- startTest[/media/disk/work/test/a.out] <- main[/media/disk/work/test/a.out]
                                          <- __libc_start_main[/lib/x86_64-linux-gnu/libc-2.19.so]
                               [Kernel] | do_page_fault <- page_fault
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
    
        This is output about stacks of specific threads when a specific kernel function is called in function mode.    
    
    +  Function Graph Info [-f -e g option]    
        ``````````````````````````````````````````
        # tracer: function_graph
        #
        #     TIME        CPU  DURATION                  FUNCTION CALLS
        #      |          |     |   |                     |   |   |   |
        2868141.800659 |    2)               |  SyS_read() {
        2868141.800660 |    2)   0.090 us    |    fget_light();
        2868141.800660 |    2)               |    vfs_read() {
        2868141.800661 |    2)               |      rw_verify_area() {
        2868141.800661 |    2)               |        security_file_permission() {
        2868141.800661 |    2)               |          apparmor_file_permission() {
        2868141.800661 |    2)               |            common_file_perm() {
        2868141.800661 |    2)   0.077 us    |              aa_file_perm();
        2868141.800662 |    2)   0.373 us    |            }
        2868141.800662 |    2)   0.605 us    |          }
        2868141.800662 |    2)   0.072 us    |          __fsnotify_parent();
        2868141.800662 |    2)   0.069 us    |          fsnotify();
        2868141.800662 |    2)   1.446 us    |        }
        2868141.800662 |    2)   1.727 us    |      }
        2868141.800663 |    2)               |      do_sync_read() {
        2868141.800663 |    2)               |        pipe_read() {
        2868141.800663 |    2)               |          mutex_lock() {
        2868141.800663 |    2)   0.028 us    |            _cond_resched();
        2868141.800663 |    2)   0.366 us    |          }
        2868141.800664 |    2)   0.021 us    |          generic_pipe_buf_confirm();
        2868141.800664 |    2)   0.020 us    |          generic_pipe_buf_map();
        2868141.800664 |    2)   0.364 us    |          pipe_iov_copy_to_user();
        2868141.800665 |    2)   0.027 us    |          generic_pipe_buf_unmap();
        2868141.800665 |    2)   0.066 us    |          anon_pipe_buf_release();
        2868141.800665 |    2)   0.020 us    |          generic_pipe_buf_confirm();
        2868141.800665 |    2)   0.029 us    |          generic_pipe_buf_map();
        2868141.800666 |    2)   0.304 us    |          pipe_iov_copy_to_user();
        2868141.800666 |    2)   0.027 us    |          generic_pipe_buf_unmap();
        2868141.800666 |    2)               |          anon_pipe_buf_release() {
        2868141.800666 |    2)               |            put_page() {
        2868141.800666 |    2)               |              __put_single_page() {
        2868141.800667 |    2)               |                free_hot_cold_page() {
        2868141.800667 |    2)   0.042 us    |                  free_pages_prepare();
        2868141.800667 |    2)   0.086 us    |                  get_pageblock_flags_mask();
        2868141.800667 |    2)               |                  free_pcppages_bulk() {
        2868141.800667 |    2)   0.042 us    |                    _raw_spin_lock();
        2868141.800668 |    2)   0.042 us    |                    get_pageblock_flags_mask();
        2868141.800668 |    2)   0.019 us    |                    __mod_zone_page_state();
        2868141.800668 |    2)   0.043 us    |                    get_pageblock_flags_mask();
        2868141.800668 |    2)   0.020 us    |                    __mod_zone_page_state();
        2868141.800669 |    2)   0.042 us    |                    get_pageblock_flags_mask();
        2868141.800669 |    2)   0.020 us    |                    __mod_zone_page_state();
        2868141.800669 |    2)   0.028 us    |                    get_pageblock_flags_mask();
        2868141.800670 |    2)   0.029 us    |                    __mod_zone_page_state();
        2868141.800670 |    2)   0.029 us    |                    get_pageblock_flags_mask();
        2868141.800670 |    2)   0.028 us    |                    __mod_zone_page_state();
        2868141.800670 |    2)   0.042 us    |                    get_pageblock_flags_mask();
        2868141.800671 |    2)   0.028 us    |                    __mod_zone_page_state();
        2868141.800671 |    2)   0.027 us    |                    get_pageblock_flags_mask();
        2868141.800671 |    2)   0.020 us    |                    __mod_zone_page_state();
        2868141.800671 |    2)   0.043 us    |                    get_pageblock_flags_mask();
        2868141.800671 |    2)   0.029 us    |                    __mod_zone_page_state();
        ``````````````````````````````````````````
    
        This is output about function graph of specific threads in function mode.    
        User can analyze function calls in kernel mode.    
    
    1. Top mode [top option]

      [Top Info] [Time: 2863246.420] [Interval: 1.0] [Ctxt: 1711] [Fork: 0] [IRQ: 898] [Core: 12] [Task: 337/814] [RAM: 64374] [Swap: 65477] [Unit: %/MB/NR]
      ==========================================================================================================================================================
        ID   | CPU (Usr/Ker/Blk/IRQ)| Mem (Free/Anon/File/Slab)| Swap (Used/ InOut )| Reclaim  | BlkRW | NrFlt | NrBlk | NrSIRQ  | NrMlk | NrDrt  |   NetIO    |
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
      Total  |  0 %( 0 / 0 / 0 / 0 )|60942( -2 / 0  / 0  / 0  )| 481  ( 0  /  0/0  )|   0/0    |  0/0  |   0   |   0   |   125   |  13   |   9    |   3K/2K    |
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
      Core/0 |  0 %( 0 / 0 / 0 / 0 )|                                                                                                     | 1171 Mhz [1171-3418]
      Core/1 |  0 %( 0 / 0 / 0 / 0 )|                                                                                                     | 1171 Mhz [1171-3418]
      Core/2 |  0 %( 0 / 0 / 0 / 0 )|                                                                                                     | 1171 Mhz [1171-3418]
      Core/3 |  0 %( 0 / 0 / 0 / 0 )|                                                                                                     | 1171 Mhz [1171-3418]
      Core/4 |  0 %( 0 / 0 / 0 / 0 )|                                                                                                     | 1171 Mhz [1171-3418]
      Core/5 | 17 %(17 / 0 / 0 / 0 )|#################                                                                                    | 1171 Mhz [1171-3418]
      Core/6 |  0 %( 0 / 0 / 0 / 0 )|                                                                                                     | 3418 Mhz [1171-3418]
      Core/7 |  0 %( 0 / 0 / 0 / 0 )|                                                                                                     | 1171 Mhz [1171-3418]
      Core/8 |  0 %( 0 / 0 / 0 / 0 )|                                                                                                     | 1171 Mhz [1171-3418]
      Core/9 |  0 %( 0 / 0 / 0 / 0 )|                                                                                                     | 1171 Mhz [1171-3418]
      Core/10|  0 %( 0 / 0 / 0 / 0 )|                                                                                                     | 1171 Mhz [1171-3418]
      Core/11|  0 %( 0 / 0 / 0 / 0 )|                                                                                                     | 1171 Mhz [1171-3418]
      ==========================================================================================================================================================
          Process      ( ID  / Pid / Nr / Pri)| CPU(Usr/Ker/Dly)|  Mem(RSS/Txt/Shr/Swp)| Blk( RD / WR /NrFlt)| Yld | Prmt | FD | LifeTime|    SignalHandler    |
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
              synergyc ( 3602/    1/   3/C  0)|  17( 17/  0/  -)|  285(152/  0/  1/  0)|   0(   -/   -/    0)|    -|     -|  64|795:19:42|      180000200      |
         firewire_ohci (  216/    2/   1/C-20)|   0(  0/  0/  -)|    0(  0/  0/  0/  -)|   0(   -/   -/    0)|    -|     -|  64|795:20:45|                     |
       chromium-browse ( 3715/ 3711/   1/C  0)|   0(  0/  0/  -)|  654(  3/ 40/  0/  7)|   0(   -/   -/    0)|    -|     -| 256|795:19:29|      1800104e8      |
                vsftpd ( 1306/    1/   1/C  0)|   0(  0/  0/  -)|   22(  0/  0/  0/  0)|   0(   -/   -/    0)|    -|     -|  64|795:20:41|        12001        |
              firewire (  215/    2/   1/C-20)|   0(  0/  0/  -)|    0(  0/  0/  0/  -)|   0(   -/   -/    0)|    -|     -|  64|795:20:45|                     |
             scsi_eh_5 (  224/    2/   1/C  0)|   0(  0/  0/  -)|    0(  0/  0/  0/  -)|   0(   -/   -/    0)|    -|     -|  64|795:20:45|                     |
        NetworkManager ( 1147/    1/   4/C  0)|   0(  0/  0/  -)|  335(  4/  1/  3/  0)|   0(   -/   -/    0)|    -|     -|  64|795:20:42|      180010000      |
       unity-panel-ser (11628/ 1803/   3/C  0)|   0(  0/  0/  -)|  476( 15/  0/ 10/  0)|   0(   -/   -/    0)|    -|     -|  64|  1:46:31|      180004002      |
       Plex Media Serv ( 2956/ 2931/  16/C  0)|   0(  0/  0/  -)|  436( 40/  9/  9/ 29)|   0(   -/   -/    0)|    -|     -| 128|795:20:34|      1800044ee      |
                 getty ( 1785/    1/   1/C  0)|   0(  0/  0/  -)|   17(  0/  0/  0/  0)|   0(   -/   -/    0)|    -|     -|  64|795:20:41|                     |
       unity-scope-loa ( 8422/ 1803/   4/C  0)|   0(  0/  0/  -)|  652( 23/  0/ 11/  0)|   0(   -/   -/    0)|    -|     -| 128| 22:14:19|      180000000      |
             scsi_eh_0 (  219/    2/   1/C  0)|   0(  0/  0/  -)|    0(  0/  0/  0/  -)|   0(   -/   -/    0)|    -|     -|  64|795:20:45|                     |
       chromium-browse ( 4348/ 3715/  10/C  0)|   0(  0/  0/  -)| 1105( 31/ 40/ 13/ 23)|   0(   -/   -/    0)|    -|     -| 512|795:17:56|      1c0014eed      |
                  bash ( 4027/ 3565/   1/C  0)|   0(  0/  0/  -)|   26(  3/  0/  1/  1)|   0(   -/   -/    0)|    -|     -| 256|795:19: 1|      4b817efb       |
       indicator-datet ( 2097/ 1803/   6/C  0)|   0(  0/  0/  -)| 1133(  4/  0/  3/  1)|   0(   -/   -/    0)|    -|     -|  64|795:20:41|      180000000      |
           gvfsd-trash ( 2658/ 1803/   4/C  0)|   0(  0/  0/  -)|  421(  2/  0/  2/  0)|   0(   -/   -/    0)|    -|     -|  64|795:20:39|      180000000      |
       indicator-bluet ( 2091/ 1803/   3/C  0)|   0(  0/  0/  -)|  256(  1/  0/  1/  0)|   0(   -/   -/    0)|    -|     -|  64|795:20:41|      180000000      |
                 osspd ( 1610/    1/  12/C  0)|   0(  0/  0/  -)|  419(  0/  0/  0/  4)|   0(   -/   -/    0)|    -|     -|  64|795:20:41|      180004003      |
       indicator-power ( 2093/ 1803/   3/C  0)|   0(  0/  0/  -)|  270(  2/  0/  2/  0)|   0(   -/   -/    0)|    -|     -|  64|795:20:41|      180000000      |
       upstart-event-b ( 1910/ 1803/   1/C  0)|   0(  0/  0/  -)|   20(  1/  0/  0/  0)|   0(   -/   -/    0)|    -|     -|  64|795:20:41|      180014002      |
       chromium-browse ( 3711/ 3692/   1/C  0)|   0(  0/  0/  -)|  654(  7/ 40/  4/  7)|   0(   -/   -/    0)|    -|     -| 256|795:19:30|      1800104e8      |
       chromium-browse ( 4472/ 3715/  10/C  0)|   0(  0/  0/  -)| 1092( 43/ 40/ 23/ 10)|   0(   -/   -/    0)|    -|     -| 512|795:17:50|      1c0014eed      |
          rtkit-daemon ( 2189/    1/   3/C  1)|   0(  0/  0/  -)|  164(  1/  0/  0/  0)|   0(   -/   -/    0)|    -|     -|  64|795:20:41|      180000000      |
       chromium-browse ( 4306/ 3715/  10/C  0)|   0(  0/  0/  -)| 1310(152/ 40/ 77/ 18)|   0(   -/   -/    0)|    -|     -| 512|795:17:58|      1c0014eed      |
      
      

      This is output about resource usage of processes profiled in top mode.
      User can analyze resource usage of process / thread(-e t option) in real-time.

    +  File Info [top -e f option]    
        ``````````````````````````````````````````
        [Top File Info] [Time: 2863483.030] [Proc: 337] [FD: 5979] [File: 1543] [Unit: %/MB/NR]
        ==========================================================================================================================================================
              PROC       ( ID  / Pid / Nr / Pri)| FD |                                                   PATH                                                    |
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
         chromium-browse ( 3790/ 3692/   3/C  0)| 285|    FILE: 253   SOCKET: 21   DEVICE: 5   PIPE: 4   EVENT: 2   PROC: 0                                        |
                                                | 383|    /run/shm/.org.chromium.Chromium.nxq6Ay (deleted)                                                         |
                                                | 336|    /run/shm/.org.chromium.Chromium.YW8qa2 (deleted)                                                         |
                                                | 335|    /run/shm/.org.chromium.Chromium.9rkJ0S (deleted)                                                         |
                                                | 332|    /run/shm/.org.chromium.Chromium.OTJc3T (deleted)                                                         |
                                                | 326|    socket:[33361129]                                                                                        |
                                                | 316|    /run/shm/.org.chromium.Chromium.L1Qv1V (deleted)                                                         |
                                                | 305|    /run/shm/.org.chromium.Chromium.rdnIXH (deleted)                                                         |
                                                | 301|    /run/shm/.org.chromium.Chromium.0iBtn5 (deleted)                                                         |
                                                | 298|    /run/shm/.org.chromium.Chromium.x6rmSQ (deleted)                                                         |
                                                | 297|    /run/shm/.org.chromium.Chromium.Lu3ppd (deleted)                                                         |
                                                | 295|    /run/shm/.org.chromium.Chromium.P715y8 (deleted)                                                         |
                                                | 290|    /run/shm/.org.chromium.Chromium.oM0SpM (deleted)                                                         |
                                                | 288|    socket:[33345211]                                                                                        |
                                                | 287|    /run/shm/.org.chromium.Chromium.1NqMr4 (deleted)                                                         |
                                                | 285|    /run/shm/.org.chromium.Chromium.O3VbrY (deleted)                                                         |
                                                | 284|    /run/shm/.org.chromium.Chromium.SDOlTq (deleted)                                                         |
                                                | 283|    /run/shm/.org.chromium.Chromium.zmSpuP (deleted)                                                         |
                                                | 281|    /run/shm/.org.chromium.Chromium.qNsy7E (deleted)                                                         |
                                                | 280|    /run/shm/.org.chromium.Chromium.dkoRM4 (deleted)                                                         |
                                                | 279|    /run/shm/.org.chromium.Chromium.SvUOXX (deleted)                                                         |
                                                | 278|    /run/shm/.org.chromium.Chromium.V3OReG (deleted)                                                         |
                                                | 277|    /run/shm/.org.chromium.Chromium.ZXafZ4 (deleted)                                                         |
                                                | 276|    /run/shm/.org.chromium.Chromium.yFXvT0 (deleted)                                                         |
                                                | 270|    /run/shm/.org.chromium.Chromium.f0pFHz (deleted)                                                         |
                                                | 269|    /run/shm/.org.chromium.Chromium.N98Qhi (deleted)                                                         |
                                                | 268|    /run/shm/.org.chromium.Chromium.Ig99pZ (deleted)                                                         |
                                                | 267|    /run/shm/.org.chromium.Chromium.4fhSXu (deleted)                                                         |
                                                | 266|    socket:[943325]                                                                                          |
        ``````````````````````````````````````````
    
        This is output about file usage of system profiled in top mode.    
        User can analyze file usage of system.    
    
    +  Stack Info [top -e s option]    
        ``````````````````````````````````````````
        [Top Info] [Time: 2870953.430] [Interval: 1.0] [Ctxt: 3121] [Fork: 5] [IRQ: 1633] [Core: 12] [Task: 336/815] [RAM: 64374] [Swap: 65477] [Unit: %/MB/NR]
        ==========================================================================================================================================================
          ID   | CPU (Usr/Ker/Blk/IRQ)| Mem (Free/Anon/File/Slab)| Swap (Used/ InOut )| Reclaim  | BlkRW | NrFlt | NrBlk | NrSIRQ  | NrMlk | NrDrt  |   NetIO    |
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        Total  |  1 %( 1 / 0 / 0 / 0 )|60891( -2 / 0  / 0  / 0  )| 481  ( 0  /  0/0  )|   0/0    |  0/0  |   0   |   0   |   184   |  13   |   24   |   8K/1K    |
        ==========================================================================================================================================================
            Process      ( ID  / Pid / Nr / Pri)| CPU(Usr/Ker/Dly)|  Mem(RSS/Txt/Shr/Swp)| Blk( RD / WR /NrFlt)| Yld | Prmt | FD | LifeTime|    SignalHandler    |
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                 rcuob/1 (   22/    2/   1/C  0)|   0(  0/  0/  -)|    0(  0/  0/  0/  -)|   0(   -/   -/    0)|    0|     0|  64|797:29:13|                     |
                                           100% | rcu_nocb_kthread+0x2a8/0x2e0 <- kthread+0xc9/0xe0 <- ret_from_fork+0x58/0x90
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
             kworker/3:1 (13295/    2/   1/C  0)|   0(  0/  0/  0)|    0(  0/  0/  0/  -)|   0(   -/   -/    0)|    2|     0|  64| 65: 6:55|                     |
                                           100% | worker_thread+0x1d1/0x410 <- kthread+0xc9/0xe0 <- ret_from_fork+0x58/0x90
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
             kworker/0:1 (22466/    2/   1/C  0)|   0(  0/  0/  -)|    0(  0/  0/  0/  -)|   0(   -/   -/    0)|    0|     0|  64| 17: 6:53|                     |
                                           100% | worker_thread+0x1d1/0x410 <- kthread+0xc9/0xe0 <- ret_from_fork+0x58/0x90
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
            kworker/11:0 (17707/    2/   1/C  0)|   0(  0/  0/  -)|    0(  0/  0/  0/  -)|   0(   -/   -/    0)|    0|     0|  64| 39:20:14|                     |
                                           100% | worker_thread+0x1d1/0x410 <- kthread+0xc9/0xe0 <- ret_from_fork+0x58/0x90
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
           kworker/11:0H (   89/    2/   1/C-20)|   0(  0/  0/  -)|    0(  0/  0/  0/  -)|   0(   -/   -/    0)|    0|     0|  64|797:29:13|                     |
                                           100% | worker_thread+0x1d1/0x410 <- kthread+0xc9/0xe0 <- ret_from_fork+0x58/0x90
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
            ksoftirqd/10 (   82/    2/   1/C  0)|   0(  0/  0/  -)|    0(  0/  0/  0/  -)|   0(   -/   -/    0)|    0|     0|  64|797:29:13|                     |
                                           100% | smpboot_thread_fn+0xd3/0x1a0 <- kthread+0xc9/0xe0 <- ret_from_fork+0x58/0x90
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
             watchdog/10 (   80/    2/   1/F 99)|   0(  0/  0/  -)|    0(  0/  0/  0/  -)|   0(   -/   -/    0)|    0|     0|  64|797:29:13|                     |
                                           100% | smpboot_thread_fn+0xd3/0x1a0 <- kthread+0xc9/0xe0 <- ret_from_fork+0x58/0x90
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
            migration/10 (   81/    2/   1/F 99)|   0(  0/  0/  -)|    0(  0/  0/  0/  -)|   0(   -/   -/    0)|    0|     0|  64|797:29:13|                     |
                                           100% | smpboot_thread_fn+0xd3/0x1a0 <- kthread+0xc9/0xe0 <- ret_from_fork+0x58/0x90
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
            migration/11 (   86/    2/   1/F 99)|   0(  0/  0/  -)|    0(  0/  0/  0/  -)|   0(   -/   -/    0)|    0|     0|  64|797:29:13|                     |
                                           100% | smpboot_thread_fn+0xd3/0x1a0 <- kthread+0xc9/0xe0 <- ret_from_fork+0x58/0x90
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
    
        This is output about stacks of threads in top mode.    
        User can analyze stacks of threads.    
    
    +  Memory Info [top -e m option]    
        ``````````````````````````````````````````
        [Top Info] [Time: 2871726.780] [Interval: 1.0] [Ctxt: 3184] [Fork: 0] [IRQ: 1136] [Core: 12] [Task: 349/830] [RAM: 64374] [Swap: 65477] [Unit: %/MB/NR]
           ==========================================================================================================================================================
          ID   | CPU (Usr/Ker/Blk/IRQ)| Mem (Free/Anon/File/Slab)| Swap (Used/ InOut )| Reclaim  | BlkRW | NrFlt | NrBlk | NrSIRQ  | NrMlk | NrDrt  |   NetIO    |
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        Total  |  8 %( 8 / 0 / 0 / 0 )|46126(-11 / 0  / 10 / 0  )| 481  ( 0  /  0/0  )|   0/0    |  0/0  |   0   |   0   |   350   |  13   |  3548  |   0K/0K    |
           ==========================================================================================================================================================
            Process      ( ID  / Pid / Nr / Pri)| CPU(Usr/Ker/Dly)|  Mem(RSS/Txt/Shr/Swp)| Blk( RD / WR /NrFlt)| Yld | Prmt | FD | LifeTime|    SignalHandler    |
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                    gzip (21266/21265/   1/C  0)|  98( 98/  0/  -)|   11(  0/  0/  0/  0)|   0(   -/   -/    0)|    -|     -|  64|  0: 1:36|      181805001      |
                                     (1)[STACK] | SIZE:   0M / RSS:   0M / PSS:   0M / SWAP:   0M / HUGE:   0M / LOCK:   0K / SDRT:   0K / PDRT:  20K / NONE:   0K
                                      (6)[FILE] | SIZE:  10M / RSS:   0M / PSS:   0M / SWAP:   0M / HUGE:   0M / LOCK:   0K / SDRT:   0K / PDRT:  68K / NONE:8184K
                                       (2)[ETC] | SIZE:   0M / RSS:   0M / PSS:   0M / SWAP:   0M / HUGE:   0M / LOCK:   0K / SDRT:   0K / PDRT:   0K / NONE:   0K
                                      (8)[ANON] | SIZE:   0M / RSS:   0M / PSS:   0M / SWAP:   0M / HUGE:   0M / LOCK:   0K / SDRT:   0K / PDRT: 376K / NONE:   0K
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                synergyc ( 3602/    1/   3/C  0)|  16( 16/  0/  -)|  285(152/  0/  1/  0)|   0(   -/   -/    0)|    -|     -|  64|797:41: 2|      180000200      |
                                     (3)[STACK] | SIZE:  16M / RSS:   0M / PSS:   0M / SWAP:   0M / HUGE:   0M / LOCK:   0K / SDRT:   0K / PDRT:  28K / NONE:   0K
                                     (20)[FILE] | SIZE:  47M / RSS:   1M / PSS:   0M / SWAP:   0M / HUGE:   0M / LOCK:   0K / SDRT:   0K / PDRT: 116K / NONE:  35M
                                       (2)[ETC] | SIZE:   0M / RSS:   0M / PSS:   0M / SWAP:   0M / HUGE:   0M / LOCK:   0K / SDRT:   0K / PDRT:   0K / NONE:   0K
                                     (13)[ANON] | SIZE: 221M / RSS: 151M / PSS: 151M / SWAP:   0M / HUGE: 150M / LOCK:   0K / SDRT:   0K / PDRT: 151M / NONE:  51M
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                     tar (21265/21262/   1/C  0)|   2(  0/  2/  -)|   25(  1/  0/  1/  0)|   0(   -/   -/    0)|    -|     -|  64|  0: 1:36|      180000000      |
                                     (1)[STACK] | SIZE:   0M / RSS:   0M / PSS:   0M / SWAP:   0M / HUGE:   0M / LOCK:   0K / SDRT:   0K / PDRT:  24K / NONE:   0K
                                     (11)[FILE] | SIZE:  24M / RSS:   1M / PSS:   0M / SWAP:   0M / HUGE:   0M / LOCK:   0K / SDRT:   0K / PDRT: 104K / NONE:  15M
                                       (2)[ETC] | SIZE:   0M / RSS:   0M / PSS:   0M / SWAP:   0M / HUGE:   0M / LOCK:   0K / SDRT:   0K / PDRT:   0K / NONE:   0K
                                      (9)[ANON] | SIZE:   0M / RSS:   0M / PSS:   0M / SWAP:   0M / HUGE:   0M / LOCK:   0K / SDRT:   0K / PDRT: 288K / NONE:   0K
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
    
        This is output about memory details of processes in top mode.    
        User can analyze memory details of processes.    
    
    +  Summary Info [top -o option]    
        ``````````````````````````````````````````
        [Top Summary Info]
        ==========================================================================================================================================================
         IDX  |          Interval           | CPU(%) | MEM(MB)  | BlkRW(MB) | BlkWait(%) | SWAP(MB) | Rclm(MB) | NrFlt | NrCtxt | NrIRQ  |  NrTask  |   NetIO    |
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
            1 |        START -  2871178.270 |      4 |    60463 |    5/0    |          0 |      481 |   0/0    |    17 |   3547 |   1658 |  340/819 |  205K/15K  |
            2 |  2871178.270 -  2871179.270 |      8 |    60451 |    0/0    |          0 |      481 |   0/0    |     0 |   6815 |   1210 |  340/818 |   0K/0K    |
            3 |  2871179.270 -  2871180.280 |      7 |    60439 |    0/0    |          0 |      481 |   0/0    |     0 |   6329 |   1204 |  340/818 |   0K/0K    |
            4 |  2871180.280 -  2871181.280 |     10 |    60429 |    0/0    |          0 |      481 |   0/0    |     0 |   6261 |   1262 |  340/818 |   0K/0K    |
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
    
        [Top CPU Info] [Unit: %]
        ==========================================================================================================================================================
              COMM       ( ID  / Pid / Nr / Pri)| Avg |   1    2    3    4
        ==========================================================================================================================================================
             [CPU]       (  -  /  -  / -  /   -)|   7 |   4    8    7   10
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                  Cooker (19737/19733/   2/C  0)|  82 |  27   103  99   101
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                synergyc ( 3602/    1/   3/C  0)|   8 |  18    0    0   17
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                KnottyUI (19733/19732/   1/C  0)|   3 |   0    6    4    4
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                  screen ( 3565/    1/   1/C  0)|   0 |   0    1    0    2
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
             kworker/2:0 (13550/    2/   1/C  0)|   0 |   0    1    0    0
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                  guider (19823/27395/   1/C  0)|   0 |   0    1    0    1
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
         Plex DLNA Serve ( 3010/ 2956/  48/C  0)|   0 |   0    1    0    1
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
    
        [Top VSS Info] [Unit: MB]
        ==========================================================================================================================================================
              COMM       ( ID  / Pid / Nr / Pri)| Max  |   1    2    3    4
        ==========================================================================================================================================================
             [FREE]      (  -  /  -  / -  /   -)| 60463 | 60463604516043960429
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                synergyc ( 3602/    1/   3/C  0)|  285 |  285   0    0   285
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                  screen ( 3565/    1/   1/C  0)|   33 |   0   33    0   33
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                  Cooker (19737/19733/   2/C  0)|  460 |  425  437  448  460
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
             kworker/2:0 (13550/    2/   1/C  0)|    0 |   0    0    0    0
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                KnottyUI (19733/19732/   1/C  0)|  125 |   0   125  125  125
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                  guider (19823/27395/   1/C  0)|   41 |   0   41    0   41
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
         Plex DLNA Serve ( 3010/ 2956/  48/C  0)| 1286 |   0  1286   0  1286
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
    
        [Top RSS Info] [Unit: MB]
        ==========================================================================================================================================================
              COMM       ( ID  / Pid / Nr / Pri)| Max  |   1    2    3    4
        ==========================================================================================================================================================
             [FREE]      (  -  /  -  / -  /   -)| 60463 | 60463604516043960429
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                synergyc ( 3602/    1/   3/C  0)|  152 |  152   0    0   152
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                  screen ( 3565/    1/   1/C  0)|    6 |   0    6    0    6
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                  Cooker (19737/19733/   2/C  0)|  320 |  286  297  309  320
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
             kworker/2:0 (13550/    2/   1/C  0)|    0 |   0    0    0    0
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                KnottyUI (19733/19732/   1/C  0)|   48 |   0   48   48   48
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                  guider (19823/27395/   1/C  0)|   15 |   0   14    0   15
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
         Plex DLNA Serve ( 3010/ 2956/  48/C  0)|  580 |   0   580   0   580
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
    
        [Top Block Info] [Unit: %]
        ==========================================================================================================================================================
              COMM       ( ID  / Pid / Nr / Pri)| Sum |   1    2    3    4
        ==========================================================================================================================================================
                  Cooker (19737/19733/   2/C  0)|   4 |   4    0    0    0
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
    
        [Top Memory Details] [Unit: MB]
        ==========================================================================================================================================================
              COMM       ( ID  / Pid ) |   Type   |  Cnt  |  VIRT  |  RSS   |  PSS   |  SWAP  |  HUGE  |  LOCK(KB)  |   PDRT(KB)   |   SDRT(KB)   |   NONE(KB)   |
        ==========================================================================================================================================================
         Plex DLNA Serve ( 3010/ 2956) |  [TOTAL] |   273 |   1285 |    580 |    578 |     98 |    574 |          0 |       590044 |            0 |       107580 |
                                       |    STACK |    50 |    392 |     42 |     42 |      0 |     42 |          0 |        43376 |            0 |            0 |
                                       |      SHM |     2 |      0 |      0 |      0 |      0 |      0 |          0 |            0 |            0 |            0 |
                                       |     FILE |    63 |    169 |      4 |      2 |      4 |      0 |          0 |          160 |            0 |       107376 |
                                       |      ETC |     2 |      0 |      0 |      0 |      0 |      0 |          0 |            0 |            0 |            0 |
                                       |     ANON |   156 |    724 |    534 |    534 |     94 |    532 |          0 |       546508 |            0 |          204 |
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                  Cooker (19737/19733) |  [TOTAL] |    75 |    470 |    331 |    320 |      0 |      2 |          0 |       318872 |        17840 |       124720 |
                                       |    STACK |     2 |     17 |     15 |     15 |      0 |      0 |          0 |        15448 |           12 |            0 |
                                       |      SHM |     6 |      0 |      0 |      0 |      0 |      0 |          0 |            4 |           12 |            0 |
                                       |     FILE |    32 |     77 |      4 |      1 |      0 |      0 |          0 |          240 |          772 |        59312 |
                                       |      ETC |     2 |      0 |      0 |      0 |      0 |      0 |          0 |            0 |            0 |            0 |
                                       |     ANON |    33 |    376 |    312 |    304 |      0 |      2 |          0 |       303180 |        17044 |        65408 |
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
         chromium-browse ( 4263/ 3715) |  [TOTAL] |  1327 |   1666 |    192 |    164 |      8 |     38 |          0 |       161364 |         3460 |       989704 |
                                       |    STACK |    13 |     96 |      0 |      0 |      0 |      0 |          0 |          496 |            0 |            0 |
                                       |      SHM |    27 |     59 |      8 |      6 |      0 |      0 |          0 |         6296 |          152 |            0 |
                                       |     FILE |   204 |    617 |     33 |      7 |      5 |      0 |          0 |          148 |         3064 |       411188 |
                                       |      ETC |     2 |      0 |      0 |      0 |      0 |      0 |          0 |            0 |            0 |            0 |
                                       |     ANON |  1081 |    894 |    151 |    151 |      3 |     38 |          0 |       154424 |          244 |       578516 |
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
         chromium-browse ( 4273/ 3715) |  [TOTAL] |   855 |   1371 |    173 |    146 |      6 |     50 |          0 |       145552 |         3456 |       715588 |
                                       |    STACK |    10 |     72 |      0 |      0 |      0 |      0 |          0 |          480 |            0 |            0 |
                                       |      SHM |    25 |     65 |      4 |      3 |      0 |      0 |          0 |         3528 |          148 |            0 |
                                       |     FILE |   204 |    617 |     31 |      5 |      5 |      0 |          0 |          148 |         3064 |       411188 |
                                       |      ETC |     2 |      0 |      0 |      0 |      0 |      0 |          0 |            0 |            0 |            0 |
                                       |     ANON |   614 |    617 |    138 |    138 |      1 |     50 |          0 |       141396 |          244 |       304400 |
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                synergyc ( 3602/    1) |  [TOTAL] |    38 |    284 |    152 |    151 |      0 |    150 |          0 |       155292 |            0 |        89688 |
                                       |    STACK |     3 |     16 |      0 |      0 |      0 |      0 |          0 |           28 |            0 |            0 |
                                       |      SHM |     0 |      0 |      0 |      0 |      0 |      0 |          0 |            0 |            0 |            0 |
                                       |     FILE |    20 |     47 |      1 |      0 |      0 |      0 |          0 |          116 |            0 |        36808 |
                                       |      ETC |     2 |      0 |      0 |      0 |      0 |      0 |          0 |            0 |            0 |            0 |
                                       |     ANON |    13 |    221 |    151 |    151 |      0 |    150 |          0 |       155148 |            0 |        52880 |
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
         chromium-browse ( 4306/ 3715) |  [TOTAL] |  1327 |   1308 |    151 |    124 |     18 |     30 |          0 |       123172 |         3460 |       741656 |
                                       |    STACK |    10 |     72 |      0 |      0 |      0 |      0 |          0 |          372 |            0 |            0 |
                                       |      SHM |    30 |     99 |     49 |     48 |      0 |      0 |          0 |        49712 |          152 |            0 |
                                       |     FILE |   204 |    617 |     31 |      5 |      6 |      0 |          0 |          160 |         3064 |       411188 |
                                       |      ETC |     2 |      0 |      0 |      0 |      0 |      0 |          0 |            0 |            0 |            0 |
                                       |     ANON |  1081 |    520 |     71 |     71 |     12 |     30 |          0 |        72928 |          244 |       330468 |
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
    
        This is output about summarized resource usage of system profiled in top mode.    
        User can analyze summarized resource usage of system.    
    
    +  Report Info [top -e r -j option]    
        ``````````````````````````````````````````
        {
          "task": {
            "nrThread": 836,
            "nrBlocked": 3,
            "nrCtx": 7466,
            "nrProc": 355
          },
          "mem": {
            "anonDiff": 7,
            "slabDiff": 7,
            "free": 40103,
            "anon": 2843,
            "dirty": 443,
            "file": 19713,
            "freeDiff": -15,
            "total": 64374,
            "slab": 734,
            "fileDiff": 0
          },
          "system": {
            "nrSoftIrq": 603,
            "uptime": 2872587.2,
            "interval": 1.0100000002421439,
            "pid": 22424,
            "nrIrq": 1322
          },
          "event": {},
          "swap": {
            "usage": 481,
            "drReclaim": 0,
            "total": 65477,
            "usageDiff": 0,
            "bgReclaim": 0
          },
          "cpu": {
            "kernel": 2,
            "nrCore": 12,
            "total": 3,
            "user": 1,
            "irq": 0
          },
          "block": {
            "read": 0,
            "ioWait": 3,
            "write": 34,
            "nrFault": 0
          }
        }
        ``````````````````````````````````````````
    
        This is output about report file as json format in top mode.    
        User can analyze resource usage and event of system.    
    
    +  Image Info [top -I guider.out -e g option]    
        ![guider_top_graph](https://cloud.githubusercontent.com/assets/15862689/23285445/a03e0bf0-fa74-11e6-9f5a-872a3f10fe48.png)    
        ![guider_top_chart](https://cloud.githubusercontent.com/assets/15862689/24597375/67f31f22-1880-11e7-8290-64554ed2859c.png)    
        This is a graph about resource usage of system profiled in top mode.    
        User can analyze resource usage of process / thread(with -e t) by graph.    
    
    1. File mode [-F option]

      [File Process Info] [ Process : 347 ] [ RAM: 216496(KB) ][ Keys: Foward/Back/Save/Quit ] [ Capture: Ctrl+\ ]
      ==========================================================================================================================================================
      ____Process_____(_Pid_)|_RAM(KB)_|_____Thread_____(_Tid_) |
      ==========================================================================================================================================================
       chromium-browse( 3692)|  141756 |   LevelDBEnv   (4573 ) |ppapi_crash_upl (4337 ) |SimpleCacheWork (4243 ) |SimpleCacheWork (4242 ) |
                                       |SimpleCacheWork (3928 ) |extension_crash (3843 ) |renderer_crash_ (3794 ) |SimpleCacheWork (3792 ) |
                                       |SimpleCacheWork (3791 ) |gpu-process_cra (3789 ) |BrowserBlocking (3788 ) |BrowserBlocking (3787 ) |
                                       |Chrome_HistoryT (3786 ) |BrowserWatchdog (3740 ) |  AudioThread   (3739 ) |  threaded-ml   (3738 ) |
                                       |CompositorTileW (3737 ) |   IndexedDB    (3736 ) |Chrome_IOThread (3735 ) |Chrome_CacheThr (3734 ) |
                                       |Chrome_ProcessL (3733 ) |Chrome_FileUser (3732 ) |Chrome_FileThre (3731 ) |Chrome_DBThread (3730 ) |
                                       |BrowserBlocking (3729 ) | inotify_reader (3726 ) |CrShutdownDetec (3725 ) |  D-Bus thread  (3724 ) |
                                       |NetworkChangeNo (3723 ) |     gmain      (3719 ) |     gdbus      (3718 ) |  dconf worker  (3717 ) |
                                       |chromium-browse (3716 ) |sandbox_ipc_thr (3709 ) |chromium-browse (3692 ) |BatteryStatusNo (21714) |
                                       |WorkerPool/1966 (19666) |
       chromium-browse( 3892)|  131528 |chromium-browse (3892 ) |
       chromium-browse( 3790)|  131528 |Chrome_ChildIOT (3901 ) |    Watchdog    (3900 ) |chromium-browse (3790 ) |
       chromium-browse( 4472)|  130292 |ScriptStreamerT (4481 ) |HTMLParserThrea (4480 ) |CompositorTileW (4479 ) |CompositorTileW (4478 ) |
                                       |CompositorTileW (4477 ) |CompositorTileW (4476 ) |CompositorTileW (4475 ) |   Compositor   (4474 ) |
                                       |Chrome_ChildIOT (4473 ) |chromium-browse (4472 ) |
       chromium-browse( 4263)|  130284 |DedicatedWorker (8208 ) |ScriptStreamerT (4328 ) |HTMLParserThrea (4284 ) |CompositorTileW (4270 ) |
                                       |CompositorTileW (4269 ) |CompositorTileW (4268 ) |CompositorTileW (4267 ) |CompositorTileW (4266 ) |
                                       |   Compositor   (4265 ) |Chrome_ChildIOT (4264 ) |chromium-browse (4263 ) | FFmpegDemuxer  (19496) |
                                       |     Media      (12691) |
       chromium-browse( 4273)|  130260 |ScriptStreamerT (4282 ) |HTMLParserThrea (4281 ) |CompositorTileW (4280 ) |CompositorTileW (4279 ) |
                                       |CompositorTileW (4278 ) |CompositorTileW (4277 ) |CompositorTileW (4276 ) |   Compositor   (4275 ) |
                                       |Chrome_ChildIOT (4274 ) |chromium-browse (4273 ) |
       chromium-browse( 4393)|  130224 |ScriptStreamerT (4411 ) |HTMLParserThrea (4408 ) |CompositorTileW (4400 ) |CompositorTileW (4399 ) |
                                       |CompositorTileW (4398 ) |CompositorTileW (4397 ) |CompositorTileW (4396 ) |   Compositor   (4395 ) |
                                       |Chrome_ChildIOT (4394 ) |chromium-browse (4393 ) |
       chromium-browse( 4306)|  130180 |ScriptStreamerT (4336 ) |HTMLParserThrea (4322 ) |CompositorTileW (4317 ) |CompositorTileW (4315 ) |
                                       |CompositorTileW (4314 ) |CompositorTileW (4313 ) |CompositorTileW (4312 ) |   Compositor   (4310 ) |
                                       |Chrome_ChildIOT (4307 ) |chromium-browse (4306 ) |
       chromium-browse( 4348)|  129976 |ScriptStreamerT (4379 ) |HTMLParserThrea (4370 ) |CompositorTileW (4355 ) |CompositorTileW (4354 ) |
                                       |CompositorTileW (4353 ) |CompositorTileW (4352 ) |CompositorTileW (4351 ) |   Compositor   (4350 ) |
                                       |Chrome_ChildIOT (4349 ) |chromium-browse (4348 ) |
       chromium-browse( 4507)|  129868 |ScriptStreamerT (4542 ) |HTMLParserThrea (4515 ) |CompositorTileW (4514 ) |CompositorTileW (4513 ) |
                                       |CompositorTileW (4512 ) |CompositorTileW (4511 ) |CompositorTileW (4510 ) |   Compositor   (4509 ) |
                                       |Chrome_ChildIOT (4508 ) |chromium-browse (4507 ) |
       chromium-browse( 4434)|  129844 |ScriptStreamerT (4443 ) |HTMLParserThrea (4442 ) |CompositorTileW (4441 ) |CompositorTileW (4440 ) |
                                       |CompositorTileW (4439 ) |CompositorTileW (4438 ) |CompositorTileW (4437 ) |   Compositor   (4436 ) |
                                       |Chrome_ChildIOT (4435 ) |chromium-browse (4434 ) |
       chromium-browse(22287)|  129764 |HTMLParserThrea (22295) |CompositorTileW (22294) |CompositorTileW (22293) |CompositorTileW (22292) |
                                       |CompositorTileW (22291) |CompositorTileW (22290) |   Compositor   (22289) |Chrome_ChildIOT (22288) |
                                       |chromium-browse (22287) |
      
      [File Usage Info] [ File: 1345 ] [ RAM: 216496(KB) ] [ Keys: Foward/Back/Save/Quit ]
      ==========================================================================================================================================================
      __RAM(KB)___|_File(KB)_|__%___|_____________________________________________________Library & Process_____________________________________________________
      ==========================================================================================================================================================
            19392 |    43180 |   44 | /usr/lib/chromium-browser/chromium-browser [Proc: 17] [Link: 1]
                                    |  chromium-browse ( 3715) |  chromium-browse ( 4273) |  chromium-browse ( 4263) |  chromium-browse ( 3711) |
                                    |  chromium-browse ( 4306) |  chromium-browse ( 3692) |  chromium-browse ( 4289) |  chromium-browse ( 3790) |
                                    |  chromium-browse ( 4393) |  chromium-browse ( 4434) |  chromium-browse ( 4507) |  chromium-browse (22287) |
                                    |  chromium-browse (22277) |  chromium-browse ( 4348) |  chromium-browse ( 3892) |  chromium-browse ( 4472) |
                                    |  chromium-browse (22307) |
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
            17360 |    23652 |   73 | /usr/lib/chromium-browser/libs/libwebcore_shared.so [Proc: 17] [Link: 1]
                                    |  chromium-browse ( 3715) |  chromium-browse ( 4273) |  chromium-browse ( 4263) |  chromium-browse ( 3711) |
                                    |  chromium-browse ( 4306) |  chromium-browse ( 3692) |  chromium-browse ( 4289) |  chromium-browse ( 3790) |
                                    |  chromium-browse ( 4393) |  chromium-browse ( 4434) |  chromium-browse ( 4507) |  chromium-browse (22287) |
                                    |  chromium-browse (22277) |  chromium-browse ( 4348) |  chromium-browse ( 3892) |  chromium-browse ( 4472) |
                                    |  chromium-browse (22307) |
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
            13132 |    25284 |   51 | /usr/lib/chromium-browser/libs/libcontent.so [Proc: 17] [Link: 1]
                                    |  chromium-browse ( 3715) |  chromium-browse ( 4273) |  chromium-browse ( 4263) |  chromium-browse ( 3711) |
                                    |  chromium-browse ( 4306) |  chromium-browse ( 3692) |  chromium-browse ( 4289) |  chromium-browse ( 3790) |
                                    |  chromium-browse ( 4393) |  chromium-browse ( 4434) |  chromium-browse ( 4507) |  chromium-browse (22287) |
                                    |  chromium-browse (22277) |  chromium-browse ( 4348) |  chromium-browse ( 3892) |  chromium-browse ( 4472) |
                                    |  chromium-browse (22307) |
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
             9536 |     9536 |  100 | /usr/lib/chromium-browser/libs/libv8.so [Proc: 17] [Link: 1]
                                    |  chromium-browse ( 3715) |  chromium-browse ( 4273) |  chromium-browse ( 4263) |  chromium-browse ( 3711) |
                                    |  chromium-browse ( 4306) |  chromium-browse ( 3692) |  chromium-browse ( 4289) |  chromium-browse ( 3790) |
                                    |  chromium-browse ( 4393) |  chromium-browse ( 4434) |  chromium-browse ( 4507) |  chromium-browse (22287) |
                                    |  chromium-browse (22277) |  chromium-browse ( 4348) |  chromium-browse ( 3892) |  chromium-browse ( 4472) |
                                    |  chromium-browse (22307) |
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
             4172 |     4172 |  100 | /usr/lib/x86_64-linux-gnu/dri/nouveau_dri.so [Proc: 4] [Link: 1]
                                    |  chromium-browse ( 3892) |  chromium-browse ( 3790) |             Xorg ( 1636) |           compiz ( 2571) |
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
             3880 |     8548 |   45 | /usr/lib/chromium-browser/libs/libmodules.so [Proc: 17] [Link: 1]
                                    |  chromium-browse ( 3715) |  chromium-browse ( 4273) |  chromium-browse ( 4263) |  chromium-browse ( 3711) |
                                    |  chromium-browse ( 4306) |  chromium-browse ( 3692) |  chromium-browse ( 4289) |  chromium-browse ( 3790) |
                                    |  chromium-browse ( 4393) |  chromium-browse ( 4434) |  chromium-browse ( 4507) |  chromium-browse (22287) |
                                    |  chromium-browse (22277) |  chromium-browse ( 4348) |  chromium-browse ( 3892) |  chromium-browse ( 4472) |
                                    |  chromium-browse (22307) |
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
             3780 |     5868 |   64 | /usr/lib/chromium-browser/libs/libnet.so [Proc: 17] [Link: 1]
                                    |  chromium-browse ( 3715) |  chromium-browse ( 4273) |  chromium-browse ( 4263) |  chromium-browse ( 3711) |
                                    |  chromium-browse ( 4306) |  chromium-browse ( 3692) |  chromium-browse ( 4289) |  chromium-browse ( 3790) |
                                    |  chromium-browse ( 4393) |  chromium-browse ( 4434) |  chromium-browse ( 4507) |  chromium-browse (22287) |
                                    |  chromium-browse (22277) |  chromium-browse ( 4348) |  chromium-browse ( 3892) |  chromium-browse ( 4472) |
                                    |  chromium-browse (22307) |
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
      

      This is output about file usage of processes profiled in file mode.
      User can analyze file usage (on-memory) of process.

    2. System mode [-y option]

      [System General Info]
      ==========================================================================================================================================================
              TYPE         Information
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
      Launch               # ./guider.py record -F . -o . -
      Arch                 x64
      Time                 2017-08-15 04:31:23
      OS                   #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016
      Kernel               Linux 3.13.0-92-generic
      RunningTime          797 hour  35 min
      Load                 192% (1 min)    /   71% (5 min) /   28% (15 min)
      Threads              3/829 (running/total)
      LastPid              20150
      Cmdline              BOOT_IMAGE=/boot/vmlinuz-3.13.0-92-generic.efi.signed root=UUID=54329ccb-dcd9-4835-81c5-e63189905219 ro quiet splash vt.handoff=7
      ==========================================================================================================================================================
      
      [System CPU Info]
      ==========================================================================================================================================================
              TYPE         Information
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
      Physical             1
      CoresPerCPU          6
      Logical              12
      Vendor               GenuineIntel
      Model                Intel(R) Core(TM) i7-3970X CPU @ 3.50GHz
      Cache(L2)            15360 KB
      Perf                 7004.23
      Address              46 bits physical, 48 bits virtual
      ==========================================================================================================================================================
      
      [System Memory Info] [ Unit: MB ]
      ==========================================================================================================================================================
      [  DESC]     Memory       Swap     Buffer      Cache     Shared     Mapped     Active   Inactive PageTables       Slab SReclaimable SUnreclaim    Mlocked
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
      [ TOTAL]      64374      65477
      [  FREE]      51910      64996
      [USAGE1]      12463        481        144       8392        120        258       1950       9132         60        376        257        119          0
      [USAGE2]      12549        481        144       8392        120        258       2031       9132         59        381        257        124          0
      [  DIFF]         86          0          0         -1         -1          0         80          0         -1          5          0          5          0
      ==========================================================================================================================================================
      
      [System Disk Info] [ Unit: ms/KB ]
      ==========================================================================================================================================================
            Dev         Maj   Min  RdSize RdTime WrSize WrTime FileSystem MountPoint <Option>
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
      /dev/sdb2          8    18     4     296    356   45940     ext3    /media/disk <rw,relatime,data=ordered>
      /dev/sda1          8     1     0      0      0      0       vfat    /boot/efi <rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortnam
                                                                                                                                       =mixed,errors=remount-ro>
      /dev/sda2          8     2     0      0      0      0       ext4    / <rw,relatime,errors=remount-ro,data=ordered>
      ==========================================================================================================================================================
      

      This is output about system information in system mode.
      But this information will be shown in all outputs.

Clone this wiki locally