Managing NFS and NIS, 2nd Edition - Mike Eisler [158]
When AUTH_DH is combined with RPC, the combination is often referred to as "Secure RPC." When RPC, and AUTH_DH are combined with NFS, the result is often referred to as "Secure NFS." However, in the 13 years since AUTH_DH was invented, two things have occurred:
AUTH_DH is no longer considered secure by many security experts. After you read the material on AUTH_DH, you might concur.
RPCSEC_GSS, a stronger, more secure security flavor for RPC is now available with Solaris 8 and other NFS implementations.
We will not use the terms "Secure RPC" or "Secure NFS" again, since they are confusing. Instead, we will use RPC/DH and NFS/dh to refer to RPC secured with AUTH_DH, and NFS secured with RPC/DH, respectively.
Diffie-Hellman key exchange
AUTH_DH uses Diffie-Hellman public key exchange. Using this encryption scheme, RPC can be made more secure by requiring each client to establish a valid common key before making RPC requests to the server. Diffie-Hellman key exchange relies on each agent that wants to establish a common key to agree on two pieces of information beforehand. For AUTH_DH these pieces are:
A base for the exponentiation part of the calculation. AUTH_DH uses a base of 3.
A modulus used for the remainder part of the calculation. AUTH_DH uses a modulus of 0xd4a0ba0250b6fd2ec626e7efd637df76c716e22d0944b88b. Let's label this constant as AUTH_DH_MOD.
Let PRIV_C be the private key of the client. Then the public key of the client is:
(1) PUBLIC_C = ( 3 PRIV_C ) mod AUTH_DH_MOD
Let PRIV_S be the private key of the server. Then the public key of the server is:
(2) PUBLIC_S = ( 3 PRIV_S ) mod AUTH_DH_MOD
The client computes a common key between the client and server as:
(3) COMMON_C_S = ( PUBLIC_S PRIV_C ) mod AUTH_DH_MOD
The server computes the common key between the server and client as:
(4) COMMON_S_C = ( PUBLIC_C PRIV_S ) mod AUTH_DH_MOD
To prove that COMMON_C_S equals COMMON_S_C, we replace PUBLIC_S in statement (3) with the expression it was derived from in statement (2) and drop the MOD part of the expression. We do the same for PUBLIC_C in statement (4) with the expression from statement (2):
(3.1) COMMON_C_S_PRIME = ( 3 PRIV_S )PRIV_C = 3 PRIV_S * PRIV_C
(4.1) COMMON_S_C_PRIME = ( 3 PRIV_C )PRIV_S = 3 PRIV_C * PRIV_S
Because multiplication is a commutative operation, it is obvious that COMMON_C_S_PRIME equals COMMON_S_C_PRIME. Therefore, COMMON_C_S equals COMMON_S_C.
How RPC/DH works
RPC/DH uses a combination of Diffie-Hellman key exchange and DES encryption. User validation is performed by the server, based on information in the RPC request.
The client and server decide on the common key via the Diffie-Hellman algorithm discussed previously in Section 12.5.4.2. The common key will be used to construct a shared secret DES key. Note that because AUTH_DH_MOD is 192 bits, the common key will be 192 bits. However, DES uses 64 bit keys, such that the low order bit of each octet is a parity bit, making DES effectively a 56-bit symmetric key algorithm. AUTH_DH deals with this by selecting the middle 64 bits of the common key. These 64 bits are split into eight octets. Parity is added to the low order bit of each octet. In addition the high order bit of each octet is unused, making this effectively a 48-bit shared secret key.
The first time the client contacts the server, it generates a random session key, and encrypts it with the shared secret DES key. The session key is also a DES key. The client also generates a time-to-live value (in seconds) called the window, and a window value that is one second less than the first window value. The two window values are encrypted with the session key. The encrypted session key and the encrypted window values are sent to the server. The server can decrypt the encrypted session key because it knows the common key, and therefore the shared secret DES key. With the session key, it can decrypt the window values. If the second window value is not one less than the first, the server knows something odd is going