Ceci est une ancienne révision du document !


Version : 2022.01

Updated: 2022/10/03 04:54

Topic 1 - Capacity Planning

Contents

  • Topic 1 - Capacity Planning
    • Contents
    • LAB #1 - Measuring System Resource Usage
      • 1.1 - free
      • 1.2 - uptime ou w
      • 1.3 - iostat
      • 1.4 - hdparm
      • 1.5 - vmstat
      • 1.6 - mpstat
      • 1.7 - sar
    • LAB #2 - Viewing Open Files and Running Processes
      • Presentation
      • Process Types
      • Process Commands
        • The ps Command
        • The pstree Command
        • The top Command

LAB#1 - Interpreting Information in /proc

The information found in files in the /proc filesystem can be interpreted using the following commands:

  • free,
  • uptime et w,
  • iostat,
  • hdparm
  • vmstat,
  • mpstat,
  • sar.

1.1 - The free Command

The free command shows total, used, free, shared, buffered, cached and swapped memory:

[root@centos8 ~]# free -m
              total        used        free      shared  buff/cache   available
Mem:           3735        1135        1818          14         782        2363
Swap:          3279           0        3279

In the above example, you can see:

  • 3735 MB of total physical memory,
  • 1135 MB of used physical memory and 1818 MB of free physical memory,
  • 3279 MB of swap space with 0MB being used.

The command line switches of this command are:

[root@centos8 ~]# free --help

Usage:
 free [options]

Options:
 -b, --bytes         show output in bytes
     --kilo          show output in kilobytes
     --mega          show output in megabytes
     --giga          show output in gigabytes
     --tera          show output in terabytes
     --peta          show output in petabytes
 -k, --kibi          show output in kibibytes
 -m, --mebi          show output in mebibytes
 -g, --gibi          show output in gibibytes
     --tebi          show output in tebibytes
     --pebi          show output in pebibytes
 -h, --human         show human-readable output
     --si            use powers of 1000 not 1024
 -l, --lohi          show detailed low and high memory statistics
 -t, --total         show total for RAM + swap
 -s N, --seconds N   repeat printing every N seconds
 -c N, --count N     repeat printing N times, then exit
 -w, --wide          wide output

     --help     display this help and exit
 -V, --version  output version information and exit

For more details see free(1).

1.2 - The uptime and w Commands

Each of these commands show the load average over the past 1, 5 and 15 minutes:

[root@centos8 ~]# uptime
 04:39:03 up 1 day,  2:34,  1 user,  load average: 0.00, 0.00, 0.00
 
[root@centos8 ~]# w
 04:39:04 up 1 day,  2:34,  1 user,  load average: 0.00, 0.00, 0.00
USER     TTY      FROM             LOGIN@   IDLE   JCPU   PCPU WHAT
trainee  pts/0    10.0.2.1         03:58    0.00s  0.11s  0.02s sshd: trainee [priv]

The load average indicates the number of processes being executed and waiting to be executed for the period concerned.

If the load average of a single-core system was 3.48 4.00 3.85 this would indicate a bottleneck since, on average:

  • 2.48 processes would have been waiting to be executed over the last minute,
  • 3.00 processes would have been waiting to be executed over the last 5 minutes,
  • 2.85 processes would have been waiting to be executed over the last 15 minutes,

The command line switches of these commands are:

[root@centos8 ~]# uptime --help

Usage:
 uptime [options]

Options:
 -p, --pretty   show uptime in pretty format
 -h, --help     display this help and exit
 -s, --since    system up since
 -V, --version  output version information and exit

For more details see uptime(1).

[root@centos8 ~]# w --help

Usage:
 w [options]

Options:
 -h, --no-header     do not print header
 -u, --no-current    ignore current process username
 -s, --short         short format
 -f, --from          show remote hostname field
 -o, --old-style     old style output
 -i, --ip-addr       display IP address instead of hostname (if possible)

     --help     display this help and exit
 -V, --version  output version information and exit

For more details see w(1).

1.3 - The iostat Command

The iostat command show disk, terminal and streamer statistics:

[root@centos8 ~]# iostat
bash: iostat: command not found...
Install package 'sysstat' to provide command 'iostat'? [N/y] y


 * Waiting in queue... 
The following packages have to be installed:
 lm_sensors-libs-3.4.0-22.20180522git70f7e08.el8.x86_64 Lm_sensors core libraries
 sysstat-11.7.3-5.el8.x86_64    Collection of performance monitoring tools for Linux
Proceed with changes? [N/y] y


 * Waiting in queue... 
 * Waiting for authentication... 
 * Waiting in queue... 
 * Downloading packages... 
 * Requesting data... 
 * Testing changes... 
 * Installing packages... 
Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc)        29/06/21        _x86_64_(8 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.03    0.00    0.03    0.01    0.00   99.93

Device             tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
sda               0.28         7.67         1.49     735338     142510
sdb               0.00         0.03         0.00       2995          0
scd0              0.00         0.00         0.00          2          0
dm-0              0.26         7.25         1.55     694786     148837
dm-1              0.00         0.02         0.00       2220          0
[root@centos8 ~]# iostat -d -x
Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc)        30/06/21        _x86_64_        (8 CPU)

Device            r/s     w/s     rkB/s     wkB/s   rrqm/s   wrqm/s  %rrqm  %wrqm r_await w_await aqu-sz rareq-sz wareq-sz  svctm  %util
sda              0.20    0.16     11.67      1.81     0.00     0.03   0.48  17.45    6.79   69.99   0.01    58.28    11.33   9.13   0.33
sdb              0.00    0.00      0.02      0.00     0.00     0.00   0.00   0.00    0.13    0.00   0.00     8.81     0.00   0.51   0.00
scd0             0.00    0.00      0.00      0.00     0.00     0.00   0.00   0.00    0.20    0.00   0.00     0.20     0.00   0.90   0.00
dm-0             0.12    0.19      3.98      1.90     0.00     0.00   0.00   0.00    4.18   79.07   0.02    32.88    10.07  10.34   0.32
dm-1             0.00    0.00      0.01      0.00     0.00     0.00   0.00   0.00    0.76    0.00   0.00    21.35     0.00   0.68   0.00

The command line switches of this command are:

[root@centos8 ~]# iostat --help
Usage: iostat [ options ] [ <interval> [ <count> ] ]
Options are:
[ -c ] [ -d ] [ -h ] [ -k | -m ] [ -N ] [ -s ] [ -t ] [ -V ] [ -x ] [ -y ] [ -z ]
[ -j { ID | LABEL | PATH | UUID | ... } ] [ --human ] [ -o JSON ]
[ [ -H ] -g <group_name> ] [ -p [ <device> [,...] | ALL ] ]
[ <device> [...] | ALL ]

1.4 - The hdparm Command

The hdparm command measures disk reads:

[root@centos8 ~]# hdparm -t /dev/sda

/dev/sda:
 Timing buffered disk reads: 1410 MB in  3.00 seconds = 469.98 MB/sec

1.5 - La Commande vmstat

The vmstat commands shows memory, pagination and processor statistics:

[root@centos8 ~]# vmstat 1 10
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0      0 1765216   2256 866336    0    0     3     0    9   12  0  0 100  0  0
 0  0      0 1765136   2256 866336    0    0     0     0   57   80  0  0 100  0  0
 0  0      0 1765136   2256 866376    0    0     0     0   54   77  0  0 100  0  0
 0  0      0 1765136   2256 866376    0    0     0     0   66  100  0  0 100  0  0
 0  0      0 1765136   2256 866376    0    0     0     0  103  125  0  0 100  0  0
 0  0      0 1765108   2256 866376    0    0     0     0   64   86  0  0 100  0  0
 0  0      0 1765108   2256 866376    0    0     0     0   62   88  0  0 100  0  0
 0  0      0 1765108   2256 866376    0    0     0     0   68   97  0  0 100  0  0
 0  0      0 1765108   2256 866376    0    0     0     0   60   88  0  0 100  0  0
 0  0      0 1765108   2256 866376    0    0     0     0  177  251  0  0 100  0  0

The command line switches of this command are:

[root@centos8 ~]# vmstat --help

Usage:
 vmstat [options] [delay [count]]

Options:
 -a, --active           active/inactive memory
 -f, --forks            number of forks since boot
 -m, --slabs            slabinfo
 -n, --one-header       do not redisplay header
 -s, --stats            event counter statistics
 -d, --disk             disk statistics
 -D, --disk-sum         summarize disk statistics
 -p, --partition <dev>  partition specific statistics
 -S, --unit <char>      define display unit
 -w, --wide             wide output
 -t, --timestamp        show timestamp

 -h, --help     display this help and exit
 -V, --version  output version information and exit

For more details see vmstat(8).

Important : By default vmstat shows statistics from boot until current time.

1.6 - The mpstat Command

La commande mpstat affiche des statistiques détaillées sur le CPU :

[root@centos8 ~]# mpstat
Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc)        29/06/21        _x86_64_(8 CPU)

04:53:22     CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle
04:53:22     all    0.03    0.00    0.01    0.02    0.01    0.01    0.00    0.00    0.00   99.92

If there are several CPU's in the system, statistics can be viewed by core and as an average:

[root@centos8 ~]# mpstat -P ALL
Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc)        29/06/21        _x86_64_(8 CPU)

04:54:28     CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle
04:54:28     all    0.03    0.00    0.01    0.02    0.01    0.01    0.00    0.00    0.00   99.92
04:54:28       0    0.03    0.00    0.01    0.00    0.01    0.00    0.00    0.00    0.00   99.94
04:54:28       1    0.02    0.00    0.02    0.03    0.02    0.00    0.00    0.00    0.00   99.91
04:54:28       2    0.02    0.00    0.01    0.01    0.01    0.04    0.00    0.00    0.00   99.90
04:54:28       3    0.01    0.00    0.01    0.02    0.00    0.00    0.00    0.00    0.00   99.95
04:54:28       4    0.05    0.00    0.02    0.03    0.01    0.00    0.00    0.00    0.00   99.88
04:54:28       5    0.03    0.01    0.01    0.02    0.01    0.00    0.00    0.00    0.00   99.92
04:54:28       6    0.02    0.00    0.02    0.01    0.01    0.00    0.00    0.00    0.00   99.95
04:54:28       7    0.02    0.00    0.01    0.01    0.01    0.00    0.00    0.00    0.00   99.94

Finally, mpstat is capable of showing statistics over time. In the following example you can see 5 data sets, one taken every 2 seconds:

[root@centos8 ~]# mpstat -P ALL 2 5
Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc)        29/06/21        _x86_64_        (8 CPU)

04:55:11     CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle
04:55:13     all    0.00    0.00    0.00    0.00    0.00    0.00    0.06    0.00    0.00   99.94
04:55:13       0    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:13       1    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:13       2    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:13       3    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:13       4    0.00    0.00    0.00    0.00    0.00    0.00    0.50    0.00    0.00   99.50
04:55:13       5    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:13       6    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:13       7    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00

04:55:13     CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle
04:55:15     all    0.00    0.00    0.00    0.00    0.06    0.00    0.00    0.00    0.00   99.94
04:55:15       0    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:15       1    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:15       2    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:15       3    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:15       4    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:15       5    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:15       6    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:15       7    0.00    0.00    0.00    0.00    0.50    0.00    0.00    0.00    0.00   99.50

04:55:15     CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle
04:55:17     all    0.00    0.00    0.00    0.00    0.06    0.00    0.00    0.00    0.00   99.94
04:55:17       0    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:17       1    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:17       2    0.00    0.00    0.00    0.00    0.50    0.00    0.00    0.00    0.00   99.50
04:55:17       3    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:17       4    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:17       5    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:17       6    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:17       7    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00

04:55:17     CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle
04:55:19     all    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:19       0    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:19       1    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:19       2    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:19       3    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:19       4    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:19       5    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:19       6    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:19       7    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00

04:55:19     CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle
04:55:21     all    0.06    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00   99.94
04:55:21       0    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:21       1    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:21       2    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:21       3    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:21       4    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:21       5    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:21       6    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
04:55:21       7    0.50    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00   99.50

Average:     CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle
Average:     all    0.01    0.00    0.00    0.00    0.02    0.00    0.01    0.00    0.00   99.95
Average:       0    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
Average:       1    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
Average:       2    0.00    0.00    0.00    0.00    0.10    0.00    0.00    0.00    0.00   99.90
Average:       3    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
Average:       4    0.00    0.00    0.00    0.00    0.00    0.00    0.10    0.00    0.00   99.90
Average:       5    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
Average:       6    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
Average:       7    0.10    0.00    0.00    0.00    0.10    0.00    0.00    0.00    0.00   99.80

The command line switches of this command are:

[root@centos8 ~]# mpstat --help
Usage: mpstat [ options ] [ <interval> [ <count> ] ]
Options are:
[ -A ] [ -n ] [ -u ] [ -V ] [ -I { SUM | CPU | SCPU | ALL } ]
[ -N { <node_list> | ALL } ] [ -o JSON ] [ -P { <cpu_list> | ALL } ]

1.7 - The sar Command

The sar command can survey all system resources dependant upon the switch that is used. Several important switches are:

Switch Description
-u CPU usage in %
-q Number of processes in the process queue
-r Memory usage
-w Swap usage
-p Pagination usage
-b Buffer usage
-d Disk usage

The /usr/lib64/sa/sadc command is used to collect data:

[root@centos8 ~]# ls /usr/lib64/sa
sa1  sa2  sadc

The /usr/lib64/sa/sa1 script calls the /usr/lib/sa/sadc command and can use two switches:

Switch Description
-t Interval
-n Count

The /usr/lib64/sa/sa2 script also creates a log at /var/log/sa/sar<dd>, where <dd> is the day of the month.

[root@centos8 ~]# ls /var/log/sa/
sa29  s


ar29

Using CentOS / RHEL 8, the interval between collects is configured using systemd timers de systemd and not cron as was previously the case:

[root@centos8 ~]# cat /usr/lib/systemd/system/sysstat-collect.timer
# /usr/lib/systemd/system/sysstat-collect.timer
# (C) 2014 Tomasz Torcz <tomek@pipebreaker.pl>
#
# sysstat-11.7.3 systemd unit file:
#        Activates activity collector every 10 minutes

[Unit]
Description=Run system activity accounting tool every 10 minutes

[Timer]
OnCalendar=*:00/10

[Install]
WantedBy=sysstat.service

The OnCalendar value indicates a collect every 10 minutes.

To change this value, you need to create an override file in /etc/systemd/system/ by using the systemctl edit command. You should never edit directly files in /usr/lib/systemd/system :

[root@centos8 ~]# systemctl edit sysstat-collect.timer
[root@centos8 ~]# cat /etc/systemd/system/sysstat-collect.timer.d/override.conf 
[Unit]
Description=Run system activity accounting tool every 2 minutes

[Timer]
OnCalendar=
OnCalendar=*:00/2
AccuracySec=0

Important : Note the line OnCalendar= which is required to override the default value.

Now check if the configuration has been applied:

[root@centos8 ~]# systemctl status sysstat-collect.timer
● sysstat-collect.timer - Run system activity accounting tool every 2 minutes
   Loaded: loaded (/usr/lib/systemd/system/sysstat-collect.timer; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/sysstat-collect.timer.d
           └─override.conf
   Active: active (waiting) since Tue 2021-06-29 06:16:04 EDT; 3h 2min ago
  Trigger: Tue 2021-06-29 09:20:00 EDT; 1min 19s left

Jun 29 06:16:04 centos8.ittraining.loc systemd[1]: Started Run system activity accounting tool every 10 minutes.
[root@centos8 ~]# journalctl -g sysstat-collect.service
-- Logs begin at Mon 2021-06-28 02:04:10 EDT, end at Tue 2021-06-29 09:18:00 EDT. --
Jun 29 06:20:33 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded.
Jun 29 06:26:29 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded.
Jun 29 06:30:33 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded.
Jun 29 06:40:33 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded.
Jun 29 06:50:33 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded.
Jun 29 07:00:33 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded.
Jun 29 07:10:33 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded.
Jun 29 07:20:33 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded.
Jun 29 07:30:33 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded.
Jun 29 07:40:33 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded.
Jun 29 07:50:33 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded.
Jun 29 07:53:56 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded.
Jun 29 07:54:00 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded.
Jun 29 07:56:00 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded.
Jun 29 07:58:00 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded.
Jun 29 08:00:00 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded.
...

Execute the sar command:

[root@centos8 ~]# sar
Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc)        29/06/21        _x86_64_        (8 CPU)

06:16:04     LINUX RESTART      (8 CPU)

06:20:33        CPU     %user     %nice   %system   %iowait    %steal     %idle
06:26:29        all      0.03      0.00      0.03      0.00      0.00     99.94
06:30:33        all      0.02      0.00      0.03      0.00      0.00     99.95
06:40:33        all      0.02      0.00      0.03      0.00      0.00     99.94
06:50:33        all      0.02      0.00      0.02      0.00      0.00     99.95
07:00:33        all      0.02      0.00      0.02      0.00      0.00     99.95
07:10:33        all      0.02      0.00      0.02      0.00      0.00     99.95
07:20:33        all      0.02      0.00      0.03      0.00      0.00     99.95
07:30:33        all      0.02      0.01      0.03      0.00      0.00     99.94
07:40:33        all      0.03      0.00      0.04      0.00      0.00     99.93
07:50:33        all      0.03      0.00      0.03      0.00      0.00     99.94
07:53:56        all      0.08      0.00      0.06      0.00      0.00     99.86
07:54:00        all      0.09      0.00      0.06      0.00      0.00     99.85
Average:        all      0.03      0.00      0.03      0.00      0.00     99.94

07:55:44     LINUX RESTART      (8 CPU)

07:56:00        CPU     %user     %nice   %system   %iowait    %steal     %idle
07:58:00        all      0.03      0.00      0.03      0.00      0.00     99.94
08:00:00        all      0.02      0.00      0.04      0.00      0.00     99.94
08:02:00        all      0.02      0.00      0.03      0.00      0.00     99.94
08:04:00        all      0.02      0.00      0.03      0.00      0.00     99.95
08:06:00        all      0.02      0.00      0.03      0.00      0.00     99.95
08:08:00        all      0.02      0.00      0.04      0.00      0.00     99.94
08:10:00        all      0.02      0.00      0.03      0.00      0.00     99.95
08:12:00        all      0.03      0.00      0.03      0.00      0.00     99.95
08:14:00        all      0.02      0.00      0.03      0.01      0.00     99.94
08:16:00        all      0.02      0.00      0.03      0.00      0.00     99.95
08:18:00        all      0.02      0.00      0.03      0.00      0.00     99.95
08:20:00        all      0.02      0.00      0.04      0.00      0.00     99.94
08:22:00        all      0.02      0.00      0.03      0.00      0.00     99.95
08:24:00        all      0.02      0.00      0.02      0.00      0.00     99.95
08:26:00        all      0.02      0.00      0.03      0.00      0.00     99.95
08:28:00        all      0.02      0.00      0.04      0.00      0.00     99.94
08:30:00        all      0.02      0.05      0.05      0.00      0.00     99.87
08:32:00        all      0.02      0.00      0.04      0.00      0.00     99.94
08:34:00        all      0.02      0.00      0.04      0.00      0.00     99.94
08:36:00        all      0.03      0.00      0.04      0.00      0.00     99.94
08:38:00        all      0.02      0.00      0.04      0.00      0.00     99.94
08:40:00        all      0.02      0.00      0.04      0.00      0.00     99.94
08:42:00        all      0.02      0.00      0.03      0.00      0.00     99.94
08:44:00        all      0.03      0.00      0.03      0.00      0.00     99.94
08:46:00        all      0.02      0.00      0.03      0.00      0.00     99.94
08:48:00        all      0.03      0.00      0.03      0.00      0.00     99.95
08:50:00        all      0.02      0.00      0.04      0.00      0.00     99.94
08:52:00        all      0.02      0.00      0.06      0.00      0.00     99.92
08:54:00        all      0.02      0.00      0.03      0.00      0.00     99.95
08:56:00        all      0.02      0.00      0.04      0.00      0.00     99.94
08:58:00        all      0.02      0.00      0.02      0.00      0.00     99.96
09:00:00        all      0.07      0.00      0.05      0.00      0.00     99.88
09:02:00        all      0.02      0.00      0.04      0.00      0.00     99.94
09:04:00        all      0.02      0.00      0.03      0.00      0.00     99.95
09:06:00        all      0.02      0.00      0.04      0.00      0.00     99.94
09:08:00        all      0.02      0.00      0.04      0.00      0.00     99.94
09:10:00        all      0.02      0.00      0.03      0.00      0.00     99.95
09:12:00        all      0.02      0.00      0.03      0.00      0.00     99.94

09:12:00        CPU     %user     %nice   %system   %iowait    %steal     %idle
09:14:00        all      0.02      0.00      0.03      0.00      0.00     99.95
09:16:00        all      0.02      0.00      0.06      0.00      0.00     99.92
09:18:00        all      0.03      0.00      0.03      0.00      0.00     99.95
09:20:00        all      0.02      0.00      0.03      0.00      0.00     99.94
Average:        all      0.02      0.00      0.03      0.00      0.00     99.94

CPU Stats

Use the -u switch:

[root@centos8 ~]# sar -u 5 3
Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc)        29/06/21        _x86_64_        (8 CPU)

09:22:52        CPU     %user     %nice   %system   %iowait    %steal     %idle
09:22:57        all      0.03      0.00      0.03      0.00      0.00     99.95
09:23:02        all      0.03      0.00      0.03      0.00      0.00     99.95
09:23:07        all      0.00      0.00      0.03      0.00      0.00     99.97
Average:        all      0.02      0.00      0.03      0.00      0.00     99.96

More information can be shown by using the ALL argument:

[root@centos8 ~]# sar -u ALL 5 3
Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc)        29/06/21        _x86_64_(8 CPU)

01:49:14        CPU      %usr     %nice      %sys   %iowait    %steal      %irq     %soft    %guest    %gnice     %idle
01:49:19        all      0.03      0.00      0.00      0.00      0.00      0.03      0.00      0.00      0.00     99.95
01:49:24        all      0.03      0.00      0.03      0.00      0.00      0.10      0.05      0.00      0.00     99.80
01:49:29        all      0.00      0.00      0.00      0.25      0.00      0.10      0.05      0.00      0.00     99.60
Average:        all      0.02      0.00      0.01      0.08      0.00      0.08      0.03      0.00      0.00     99.78

To see the statistics from a specific core, use the -P switch:

[root@centos8 ~]# sar -u -P 1 5 3
Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc)        29/06/21        _x86_64_        (8 CPU)

01:51:52        CPU     %user     %nice   %system   %iowait    %steal     %idle
01:51:57          1      0.00      0.00      0.00      0.00      0.00    100.00
01:52:02          1      0.20      0.00      0.00      0.00      0.00     99.80
01:52:07          1      0.00      0.00      0.00      0.00      0.00    100.00
Average:          1      0.07      0.00      0.00      0.00      0.00     99.93
[root@centos8 ~]# sar -u -P 5 5 3
Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc)        29/06/21        _x86_64_        (8 CPU)

01:52:16        CPU     %user     %nice   %system   %iowait    %steal     %idle
01:52:21          5      0.00      0.00      0.00      0.00      0.00    100.00
01:52:26          5      0.00      0.00      0.00      0.00      0.00    100.00
01:52:31          5      0.00      0.00      0.00      0.00      0.00    100.00
Average:          5      0.00      0.00      0.00      0.00      0.00    100.00

Memory and Swap Statistics

Use the -r switch to see memory statistics:

[root@centos8 ~]# sar -r 5 3
Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc)        30/06/21        _x86_64_(8 CPU)

07:33:32    kbmemfree   kbavail kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit  kbactive   kbinact   kbdirty
07:33:37      1647240   2297232   2177792     56.94      3356    827396   5096432     70.94    359072   1486368         0
07:33:42      1647232   2297224   2177800     56.94      3356    827396   5095788     70.93    359072   1486300         0
07:33:47      1647232   2297224   2177800     56.94      3356    827396   5095788     70.93    359072   1486376         0
Average:      1647235   2297227   2177797     56.94      3356    827396   5096003     70.94    359072   1486348         0

Use the -S switch to see swap statistics:

[root@centos8 ~]# sar -S 5 3
Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc)        30/06/21        _x86_64_(8 CPU)

07:31:58    kbswpfree kbswpused  %swpused  kbswpcad   %swpcad
07:32:03      3358716         0      0.00         0      0.00
07:32:08      3358716         0      0.00         0      0.00
07:32:13      3358716         0      0.00         0      0.00
Average:      3358716         0      0.00         0      0.00

I/O Statistics

Use the -b switch:

[root@centos8 ~]# sar -b 5 3
Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc)        29/06/21        _x86_64_        (8 CPU)

09:24:49          tps      rtps      wtps   bread/s   bwrtn/s
09:24:54         0.00      0.00      0.00      0.00      0.00
09:24:59         1.20      0.00      1.20      0.00     20.20
09:25:04         0.00      0.00      0.00      0.00      0.00
Average:         0.40      0.00      0.40      0.00      6.73

Disk I/O Statistics

Use the -d switch:

[root@centos8 ~]# sar -d 5 3
Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc)        29/06/21        _x86_64_        (8 CPU)

09:25:45          DEV       tps     rkB/s     wkB/s   areq-sz    aqu-sz     await     svctm     %util
09:25:50       dev8-0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
09:25:50      dev8-16      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
09:25:50      dev11-0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
09:25:50     dev253-0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
09:25:50     dev253-1      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00

09:25:50          DEV       tps     rkB/s     wkB/s   areq-sz    aqu-sz     await     svctm     %util
09:25:55       dev8-0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
09:25:55      dev8-16      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
09:25:55      dev11-0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
09:25:55     dev253-0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
09:25:55     dev253-1      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00

09:25:55          DEV       tps     rkB/s     wkB/s   areq-sz    aqu-sz     await     svctm     %util
09:26:00       dev8-0      0.60      0.00      0.30      0.50      0.01     13.00     13.00      0.78
09:26:00      dev8-16      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
09:26:00      dev11-0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
09:26:00     dev253-0      0.60      0.00      0.50      0.83      0.01     12.67     13.00      0.78
09:26:00     dev253-1      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00

Average:          DEV       tps     rkB/s     wkB/s   areq-sz    aqu-sz     await     svctm     %util
Average:       dev8-0      0.20      0.00      0.10      0.50      0.00     13.00     13.00      0.26
Average:      dev8-16      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
Average:      dev11-0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
Average:     dev253-0      0.20      0.00      0.17      0.83      0.00     12.67     13.00      0.26
Average:     dev253-1      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00

The DEV column identifies the disks by their major/minor numbers. To see the names of the disks add the, -p switch:

[root@centos8 ~]# sar -p -d 5 3
Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc)        30/06/21        _x86_64_        (8 CPU)

07:48:32          DEV       tps     rkB/s     wkB/s   areq-sz    aqu-sz     await     svctm     %util
07:48:37          sda      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
07:48:37          sdb      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
07:48:37          sr0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
07:48:37    cl_centos8-root      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
07:48:37    cl_centos8-swap      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00

07:48:37          DEV       tps     rkB/s     wkB/s   areq-sz    aqu-sz     await     svctm     %util
07:48:42          sda      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
07:48:42          sdb      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
07:48:42          sr0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
07:48:42    cl_centos8-root      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
07:48:42    cl_centos8-swap      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00

07:48:42          DEV       tps     rkB/s     wkB/s   areq-sz    aqu-sz     await     svctm     %util
07:48:47          sda      0.40      0.00      0.40      1.00      0.02     56.00     56.50      2.26
07:48:47          sdb      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
07:48:47          sr0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
07:48:47    cl_centos8-root      0.40      0.00      0.80      2.00      0.02     56.00     56.50      2.26
07:48:47    cl_centos8-swap      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00

Average:          DEV       tps     rkB/s     wkB/s   areq-sz    aqu-sz     await     svctm     %util
Average:          sda      0.13      0.00      0.13      1.00      0.01     56.00     56.50      0.75
Average:          sdb      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
Average:          sr0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
Average:    cl_centos8-root      0.13      0.00      0.27      2.00      0.01     56.00     56.50      0.75
Average:    cl_centos8-swap      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00

The command line switches of this command are:

[root@centos8 ~]# sar --help
Usage: sar [ options ] [ <interval> [ <count> ] ]
Main options and reports (report name between square brackets):
        -B      Paging statistics [A_PAGE]
        -b      I/O and transfer rate statistics [A_IO]
        -d      Block devices statistics [A_DISK]
        -F [ MOUNT ]
                Filesystems statistics [A_FS]
        -H      Hugepages utilization statistics [A_HUGE]
        -I { <int_list> | SUM | ALL }
                Interrupts statistics [A_IRQ]
        -m { <keyword> [,...] | ALL }
                Power management statistics [A_PWR_...]
                Keywords are:
                CPU     CPU instantaneous clock frequency
                FAN     Fans speed
                FREQ    CPU average clock frequency
                IN      Voltage inputs
                TEMP    Devices temperature
                USB     USB devices plugged into the system
        -n { <keyword> [,...] | ALL }
                Network statistics [A_NET_...]
                Keywords are:
                DEV     Network interfaces
                EDEV    Network interfaces (errors)
                NFS     NFS client
                NFSD    NFS server
                SOCK    Sockets (v4)
                IP      IP traffic      (v4)
                EIP     IP traffic      (v4) (errors)
                ICMP    ICMP traffic    (v4)
                EICMP   ICMP traffic    (v4) (errors)
                TCP     TCP traffic     (v4)
                ETCP    TCP traffic     (v4) (errors)
                UDP     UDP traffic     (v4)
                SOCK6   Sockets (v6)
                IP6     IP traffic      (v6)
                EIP6    IP traffic      (v6) (errors)
                ICMP6   ICMP traffic    (v6)
                EICMP6  ICMP traffic    (v6) (errors)
                UDP6    UDP traffic     (v6)
                FC      Fibre channel HBAs
                SOFT    Software-based network processing
        -q      Queue length and load average statistics [A_QUEUE]
        -r [ ALL ]
                Memory utilization statistics [A_MEMORY]
        -S      Swap space utilization statistics [A_MEMORY]
        -u [ ALL ]
                CPU utilization statistics [A_CPU]
        -v      Kernel tables statistics [A_KTABLES]
        -W      Swapping statistics [A_SWAP]
        -w      Task creation and system switching statistics [A_PCSW]
        -y      TTY devices statistics [A_SERIAL]
Menu