System Monitoring : sar

sar: the System Activity Reporter

sar is part of the sysstat package. Check to see if you have it:

rpm -q sysstat

and if you don’t, install it. In Red Hat/Fedora:

yum install sysstat

In SUSE:

yast # for a text-mode installation
or
yast2 # for a GUI installation

will get you started.

 

Initialize sar by running (as root):

/usr/lib/sa/sa1 1 1
and
/usr/lib/sa/sa2 -A

If you don’t initialize it this way, cron will run these commands for you anyway (eventually). The sysstat package puts these commands in /etc/cron.d/systat/. You won’t be able to run sar without arguments and get intelligible output until you’ve done this first.

 

The basic output of sar (run as a naked command) is a familiar-looking screen, except that what you’re seeing is the cumulative averages taken every ten minutes.

These values come from running sar in cron. Fedora Core 4 puts these entries in /etc/cron.d/sysstat:

# run system activity accounting tool every 10 minutes
*/10 * * * * root /usr/lib/sa/sa1 1 1
# generate a daily summary of process accounting at 23:53
53 23 * * * root /usr/lib/sa/sa2 -A

The sa1 script collects binary data into the system activity daily summary file.

The sa2 script collects a daily report in a file by the name of /var/log/sa/sa.#, where # is the day of the month. Thus you have a month’s records at all times (after the first month).

 

sar can also provide real-time statistics on the fly. Use the command:

sar

for an instant snapshot of the CPU statistics for your system, or:

sar 1 10

to collect data at one second intervals 10 times. This is a great method for finding bottlenecks; notice the iowait column, which pinpoints how long a process is having to wait for execution. Of course, the next question (“Why?”) is harder to answer, but it’s a start.

 

sar options
-A All information
-b I/O statistics (similar to the iostat command)
-B Swap statistics (formerly in Blocks, now in kilobytes)
-c Processes created per second
-d I/O stats for each block device (similar to the iostat command)
-f <filename> Display info from a specific file (e.g. one day’s log file)
-n FULL Full network statistics
-o <filename> Save output to a file
-q Processor queue statistics
-r report memory and swap statistics
-R Report memory statistics
-u CPU statistics (similar to the mpstat command)
-v Kernel-related filesystem statistics
-W sWapping statistics

On a computer that has been running sysstat for some time, run the command:

sar -q 1 5

The columns that will be returned are:

Time
runq-sz
plist-sz
ldavg-1
ldavg-5
ldavg-15
Time of reading run queue size (actually count) process list size (actually count) load average – 1 minute load average – 5 minute load average – 15 minute

The run queue size is the number of processes waiting for execution on the processor. A run queue size that persists above 2 processes means the CPU is overwhelmed by system requests.

The process list size is the number of processes loaded in memory.

The load averages give the loads over the past one, five and fifteen minutes. You’re looking for sudden increases, and for gradual increases that never decrease.

Now run:

sar -W 1 5

The columns that will be returned are:

Time
pswpin/s
pswpout/s
Time of reading memory pages swapped in per second memory pages swapped out per second

When it comes to these numbers, the lower the better. Under heavy load, the only solution is more RAM.