UNIX System Administration Handbook - Evi Nemeth [451]
Analyzing memory usage
On a workstation, your best memory analysis tools are your ears. The amount of paging activity is generally proportional to the amount of crunching you hear from the disk. On most disks, the seeking of the heads is what you actually hear, but the correspondence is good enough to give you a general idea of what’s going on.
There are basically two numbers that quantify memory activity: the total amount of active virtual memory and the paging rate. The first number tells you the total demand for memory, and the second suggests the proportion of that memory that is actively used. The goal is to reduce activity or increase memory until paging remains at an acceptable level. Occasional paging is inevitable, so don’t worry about trying to eliminate it completely.
The amount of swap space currently in use can be determined with the command swap -l under Solaris, swapinfo under HP-UX, swapon -s on Red Hat, and pstat -s on FreeBSD. sar -r can also be used on Solaris systems (as always, with arguments to specify the monitoring interval), but for some reason it gives results that are not entirely consistent with swap -l.
% swap -l
swapfile dev swapl blocks free
/dev/dsk/c0t0d0s1 32,1 16 164400 162960
% sar -r 5
17:58:52 freemem freeswap
17:58:57 361 179616
% pstat -s
Device 1K-blocks Used Avail Capacity Type
/dev/wd0s1b 70784 0 70656 0% Interleaved
/dev/da0b 1048920 0 1048792 0% Interleaved
Total 1119448 0 1119448 0%
pstat shows the swap sizes in kilobytes; swap -l and sar -r use 512-byte disk blocks. The sizes quoted by these programs do not include the contents of core memory, so you must compute the total amount of virtual memory yourself:
VM = size of real memory + amount of swap space used
On most systems, paging statistics are obtained with vmstat:
% vmstat 5 5
procs memory page disk faults
r b w swap free re mf pi po fr de sr s0 s6 s4 -- in sy cs
0 0 0 338216 10384 0 3 1 0 0 0 0 0 0 0 0 132 101 58
0 0 0 341784 11064 0 26 1 1 1 0 0 0 0 1 0 150 215 100
0 0 0 351752 12968 1 69 0 9 9 0 0 0 0 2 0 173 358 156
0 0 0 360240 14520 0 30 6 0 0 0 0 0 0 1 0 138 176 71
1 0 0 366648 15712 0 73 0 8 4 0 0 0 0 36 0 390 474 237
CPU information has been removed from this example. Under the procs heading is shown the number of processes that are immediately runnable, blocked on I/O, and runnable but swapped. If the value in the w column is ever nonzero, it is likely that the system’s memory is pitifully inadequate relative to the current load.
The swap column gives the number of available kilobytes of virtual memory. The free column tells the number of kilobytes on the system’s free list; values lower than 3% of the system’s total memory generally indicate problems.
The next seven columns give information about paging activity. All columns represent average values per second. Their meanings are:
• re – the number of pages reclaimed (rescued from the free list)
• mf – the number of minor faults (minor meaning “small # of pages”)
• pi – the number of kilobytes paged in
• po – the number of kilobytes paged out
• fr – the number of kilobytes placed on the free list
• de – the number of kilobytes of “predicted short-term memory shortfall”
• sr – the number of pages scanned by the clock algorithm
The de column is the best indicator of serious memory problems. If it often jumps above 100, the machine is starved for memory. Unfortunately, some versions of vmstat don’t show this number.
vmstat -S shows statistics for swapping rather than paging.
The apparent inconsistencies among the memory-related columns are for the most part illusory. Some columns count pages, others count kilobytes. The clock algorithm is not responsible for all frees because some pages may be voluntarily released by processes. All values are rounded averages. Furthermore, some are averages of scalar quantities and others are average deltas. For example, you can’t compute the next value of free from the current free and paging information