Installing Perf Performance Analysis Tool on CentOS 8

0
742

In this guide, we are going to learn on installing Perf Performance Analysis Tool on CentOS 8. Perf aka perf_events aka perf tools is a powerful performance counters monitoring tool for Linux systems that gathers data about hardware events such as instructions executed, branches mispredicted or cache-misses suffered.

It also provides very low overhead profiling of applications to trace dynamic control flow and identify hotspots, per task, per CPU and per-workload counters, sampling and source code event annotation, dynamic creation of tracepoints using the kprobes and uprobes frameworks, for kernel and userspace dynamic tracing respectively.

Installing Perf Performance Analysis Tool on CentOS 8

Perf provide a userspace controlling utility called perf. You need to install this Perf userland utility in order to be able to use Perf on command line.

Perf is available on the default CentOS 8 repos through the perf package

dnf info perf
Available Packages
Name         : perf
Version      : 4.18.0
Release      : 147.5.1.el8_1
Architecture : x86_64
Size         : 3.4 M
Source       : kernel-4.18.0-147.5.1.el8_1.src.rpm
Repository   : BaseOS
Summary      : Performance monitoring for the Linux kernel
URL          : http://www.kernel.org/
License      : GPLv2
Description  : This package contains the perf tool, which enables performance monitoring
             : of the Linux kernel.

Run system update.

dnf update

Install Perf on CentOS 8

dnf install perf

The basic command line Syntax of perf utility is;

perf [--version] [--help] [OPTIONS] COMMAND [ARGS]

The most common perf sub-commands include;

  • stat: runs a command and gathers performance counter statistics from it.
  • top: generates and displays a performance counter profile in real time.
  • record: runs a command and gathers a performance counter profile from it. The gathered data is saved into perf.data without displaying anything.
  • report: It reads perf.data to display the performance counter profile information recorded via perf record sub-command.
  • list: displays the symbolic event types which can be selected in the various perf commands with the -e option.
  • trace: Just like strace, perf trace sub-command shows various system calls being executed by a particular thread or process and the signals it receives.
  • annotate: Perf annotate command displays a report file and an annotated version of the executed code.

If you just type perf command on the terminal, you should be able to see quite a number of subcommands;

perf
perf

 usage: perf [--version] [--help] [OPTIONS] COMMAND [ARGS]

 The most commonly used perf commands are:
   annotate        Read perf.data (created by perf record) and display annotated code
   archive         Create archive with object files with build-ids found in perf.data file
   bench           General framework for benchmark suites
   buildid-cache   Manage build-id cache.
   buildid-list    List the buildids in a perf.data file
   c2c             Shared Data C2C/HITM Analyzer.
   config          Get and set variables in a configuration file.
   data            Data file related processing
   diff            Read perf.data files and display the differential profile
   evlist          List the event names in a perf.data file
   ftrace          simple wrapper for kernel's ftrace functionality
   inject          Filter to augment the events stream with additional information
   kallsyms        Searches running kernel for symbols
   kmem            Tool to trace/measure kernel memory properties
   kvm             Tool to trace/measure kvm guest os
   list            List all symbolic event types
   lock            Analyze lock events
   mem             Profile memory accesses
   record          Run a command and record its profile into perf.data
   report          Read perf.data (created by perf record) and display the profile
   sched           Tool to trace/measure scheduler properties (latencies)
   script          Read perf.data (created by perf record) and display trace output
   stat            Run a command and gather performance counter statistics
   test            Runs sanity tests.
   timechart       Tool to visualize total system behavior during a workload
   top             System profiling tool.
   version         display the version of perf binary
   probe           Define new dynamic tracepoints
   trace           strace inspired tool

To obtain help about a specific sub-command, just execute;

perf help SUB-COMMAND

Example Usage of Perf on CentOS 8

Next, let us go over a few example usage of perf command on CentOS 8 and similar derivatives.

Gather Command Performance Counter statistics

The stat sub-command of perf utility enables you to run a command and gather its performance counter statistics

perf stat ls
con  file  file1

 Performance counter stats for 'ls':

              1.72 msec task-clock                #    0.827 CPUs utilized          
                 0      context-switches          #    0.000 K/sec                  
                 0      cpu-migrations            #    0.000 K/sec                  
               108      page-faults               #    0.063 M/sec                  
   <not supported>      cycles                                                      
   <not supported>      instructions                                                
   <not supported>      branches                                                    
   <not supported>      branch-misses                                               

       0.002075544 seconds time elapsed

       0.002106000 seconds user
       0.000000000 seconds sys

Read more on man perf-stat.

Display Live Performance Counter Profile

perf-top is a system profiling tool used to generate and display a performance counter profile in real time. This can be achieved using the perf top command.

perf top
Samples: 29  of event 'cpu-clock', 4000 Hz, Event count (approx.): 7250000 lost: 0/0 drop: 0/0                                                         
Overhead  Shared Object  Symbol                                                                                                                         
  34.48%  [kernel]      [k] e1000_watchdog
  13.79%  [kernel]      [k] _raw_spin_unlock_irqrestore
   6.90%  [kernel]      [k] format_decode
   6.90%  [kernel]      [k] vsnprintf
   3.45%  [kernel]      [k] __x86_indirect_thunk_rax
   3.45%  [kernel]      [k] __x86_indirect_thunk_rdx
   3.45%  [kernel]      [k] kallsyms_expand_symbol.constprop.1
   3.45%  [kernel]      [k] mem_cgroup_throttle_swaprate
   3.45%  [kernel]      [k] memcpy
   3.45%  [kernel]      [k] number
   3.45%  [kernel]      [k] update_iter
   3.45%  libc-2.28.so  [.] _IO_getdelim
   3.45%  libc-2.28.so  [.] __memmove_avx_unaligned
   3.45%  perf          [.] dso__next_symbol
   3.45%  perf          [.] rb_next
...

Consult man perf-top to see more usage.

List all symbolic event types

To list all symbolic event types using perf command;

perf list
  alignment-faults                                   [Software event]
  bpf-output                                         [Software event]
  context-switches OR cs                             [Software event]
  cpu-clock                                          [Software event]
...
  msr/pperf/                                         [Kernel PMU event]
  msr/smi/                                           [Kernel PMU event]
  msr/tsc/                                           [Kernel PMU event]
...
  rNNN                                               [Raw hardware event descriptor]
  cpu/t1=v1[,t2=v2,t3 ...]/modifier                  [Raw hardware event descriptor]
   (see 'man perf-list' on how to encode it)

  mem:<addr>[/len][:access]                          [Hardware breakpoint]
...

Record Command Profile

To run a command and record its profile into perf.data;

perf record ls -ld /etc
drwxr-xr-x. 78 root root 8192 Mar 19 20:32 /etc
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.011 MB perf.data ]

The basic syntax and a comprehensive description of using perf record command is stated on man perf-record.

perf record -e cpu-clock -a -g -- sleep 3
[ perf record: Woken up 2 times to write data ]
[ perf record: Captured and wrote 0.628 MB perf.data (4514 samples) ]

Read perf.data and display the profile

To read the performance record, use the report sub-command. The command syntax is perf report [-i <file> | --input=file]

perf report -i perf.data
Samples: 4K of event 'cpu-clock', Event count (approx.): 1128500000                                                                                    
  Children      Self  Command          Shared Object      Symbol                                                                                       
+   99.87%    99.87%  swapper          [kernel.kallsyms]  [k] mwait_idle
+   99.87%     0.00%  swapper          [kernel.kallsyms]  [k] secondary_startup_64
+   99.87%     0.00%  swapper          [kernel.kallsyms]  [k] start_kernel
+   99.87%     0.00%  swapper          [kernel.kallsyms]  [k] cpu_startup_entry
+   99.87%     0.00%  swapper          [kernel.kallsyms]  [k] do_idle
     0.13%     0.00%  kworker/0:3-eve  [kernel.kallsyms]  [k] ret_from_fork
     0.13%     0.00%  kworker/0:3-eve  [kernel.kallsyms]  [k] kthread
     0.13%     0.00%  kworker/0:3-eve  [kernel.kallsyms]  [k] worker_thread
     0.13%     0.00%  kworker/0:3-eve  [kernel.kallsyms]  [k] process_one_work
     0.11%     0.09%  kworker/0:3-eve  [kernel.kallsyms]  [k] e1000_watchdog
...

Well, Perf tool is quite a comprehensive command and we can’t exhaust all if its usage in a simple introductory guide.

To read more about Perf Performance Analysis Tool, check;

Perf Wiki

Linux Perf Examples

Perf Events and tool security

Want to install Perf on Ubuntu 18.04?

Installing Perf Performance Analysis Tool on Ubuntu 18.04

Other Tutorials

How to Perform System Security Auditing with Lynis on Ubuntu 18.04

Install and Setup Adiscon LogAnalyzer on CentOS 8

Restrict SFTP User Access to Specific Directories in Linux

LEAVE A REPLY

Please enter your comment!
Please enter your name here