Sysstat: Essential Tools for Analyzing Performance Issues
Introduction
Sysstat is a package containing the sar and iostat binaries. The latter is used to monitor only disk I/O, while sar is used to monitor almost everything.
Installation
Debian
On Debian, you need to install sysstat:
| |
Red Hat
On Red Hat, you need to install sysstat:
| |
Solaris
On Solaris, you’ll need to use Sun Freeware packages to find this tool:
| |
iostat
iostat allows you to measure disk I/O. If you want to test the performance of disks mounted on your machines, I recommend using screen to see real-time results.
Linux
On Linux, here’s how to use it on the sda disk, for example:
| |
- -x: extended statistics mode
- 1: This means that every second, iostat will analyze the performance of all disks
- 5: For 5 seconds
The first lines of the iostat command provide an average of I/O since the machine was booted. Here are some explanations of the columns:
- r/s: read operations per second
- w/s: write operations per second
- await: wait time (r/s + w/s)
To know if there are sequential reads/writes on the disk:
- rrqm/s: number of merged read requests per second
- wrqm/s: number of merged write requests per second
The more sequential the reads, the faster (on non-SSD disks), the more scattered the sectors, the longer the wait.
For testing, here is some useful information1:
- Advise to drop cache for whole file
| |
- Ensure drop cache for whole file
| |
- Drop cache for part of file
| |
- Stream data just using readahead cache
| |
Solaris
On Solaris, the commands are a bit different:
| |
To stress the disk, we will use the dd command, which is a low-level command:
| |
Here is a small shell script that does everything for you (bench_disk.sh):
| |
sar
sar is a tool that will allow us to monitor many things. When installing sysstat, sar will set itself up in crontab to regularly execute probes placed in /var/log/sa.
You can change the default crontab at any time:
| |
Uncomment all lines or adjust according to your needs. To read these files afterward, use the sar command like this:
| |
- XX: day of the month
One important thing before using sar, create an alias in your bashrc or the preferences file for your favorite shell so that the hours are displayed correctly:
| |
Disks
To monitor disks, use the -d option:
| |
As you can see, the last lines correspond to the averages since the machine was booted.
If the disk devices don’t make much sense to you, you can use the -p option:
| |
CPU
To analyze the CPU:
| |
Memory
To monitor memory:
| |
You can also use vmstat to monitor memory:
| |
- free, buff, and cache: the amount of memory in KiB that is idle
- si and so: correspond to swap usage
- swpd: the size in KiB of swap used
To monitor the rate of change:
| |
Swap
To analyze swap:
| |
Network
For network analysis:
| |
IO Operations
To monitor all I/O operations:
| |
Processes
It is possible to get a lot of information about a specific process using the pidstat command:
| |
It is also possible to monitor I/O (-d) or even the top 5 processes in page fault:
| |
FAQ
sar: can’t open /var/adm/sa/saXX: No such file or directory
You want to use the “sar” command on Solaris to perform monitoring or performance analysis on your server, but when you execute the command, you get an error similar to the following:
sar: can't open /var/adm/sa/saXX: No such file or directoryThe answer is in the manpage for “sadc”. You need to execute the following command, and you should be able to execute the command without issue after running this:
| |
Resources
Last updated 24 Aug 2013, 12:57 CEST.