Managing NFS and NIS, 2nd Edition - Mike Eisler [241]
Ensure that the RPC mixture of the benchmark matches that of your NFS clients. Running a benchmark that does a large percentage of write operations tells you little about how NFS servers perform if your clients mostly read files. Conversely, if you have a large percentage of write operations, the wrong benchmark RPC mixture overstates expected server performance. Use the nfsstat tool to determine accurate RPC mixtures for your servers. You may want to run several benchmarks, testing performance with client loads simulating normal and heavy conditions. The SPEC website, http://www.spec.org, contains information about the SFS97 RPC-generating benchmark, which is widely used by NFS vendors to compare their servers to one another.
Watch out for cache effects. Clients cache parts of files that have been recently read and not modified. Repeatedly reading the same file may only generate a fraction of the desired number of read RPC requests.
When gauging a particular limit, such as the maximum number of short RPCs or the maximum NFS disk transfer rate, try to isolate the quantity under test as much as possible. Stress testing is often useful for determining a server's behavior under severe loads, but it helps to stress only one component at a time.
The last point rings of Heisenberg's Uncertainty Principle. In short, Heisenberg stated that the process of observing something changes it. A goal of NFS performance measurement should be to change the actual performance being measured as little as possible. Using networked measurement tools that add to the traffic level on a congested network, or running suites of utilities that drain the server's CPU, color the results of any benchmarks.
When benchmarking a network router or gateway, ensure that you are measuring the desired capacity and not another constraint. To determine maximum IP packet forwarding rates, for example, you should put a packet generator on one side of the router and a packet counting device such as a LAN analyzer on the other. Timing rpc transfers of large files through the router gives a fair indication of maximum disk transfer rates or maximum network data transfer rates, but tells you little about the router's network interface because the packets forwarded are not "typical" in size.
The goal of the next section is to indicate the common areas in which performance bottlenecks are created. The remainder of this chapter covers techniques for relaxing these constraints on the server as much as possible. The majority of the following discussion concerns NFS, although NIS-specific topics will be introduced where applicable.
Identifying NFS performance bottlenecks
The stateless design of NFS makes crash recovery simple, but it also makes it impossible for a client to distinguish between a server that is slow and one that has crashed. In either case, the client does not receive an RPC reply before the RPC timeout period expires. Clients can't tell why a server appears slow, either: packets could be dropped by the network and never reach the server, or the server could simply be overloaded. Using NFS performance figures alone, it is hard to distinguish a slow server from an unreliable network. Users complain that "the system is slow," but there are several areas that contribute to system sluggishness.
An overloaded server responds to all packets that it enqueues for its nfsd daemons, perhaps dropping some incoming packets due to the high load. Those requests that are received generate a response, albeit a response that arrives sometime after the client has retransmitted the request. If the network itself is to blame, then packets may not make it from the client or server onto the wire, or they may vanish in transit between the two hosts.
Problem areas
The potential bottlenecks in the client-server relationship are:
Client network interface
The client may not be able to transmit or receive packets due to hardware or configuration problems at its network