Classic Shell Scripting - Arnold Robbins [108]
This script must be usable by any user at any Unix site, so we cannot embed information about particular hosts in it. Instead, we assume that the user has provided two customization files: directories to list places to look for the package distribution files, and userhosts to list usernames, remote hostnames, remote build directories, and special environment variables. We place these, and other related files, in a hidden directory, $HOME/.build, to reduce clutter. However, since the list of source directories is likely to be similar for all users at a given site, we include a reasonable default list so that the directories file may not be needed.
A build should sometimes be done on only a subset of the normal build hosts, or with archive files in unusual locations, so the script should make it possible to set those values on the command line.
The script that we develop here can be invoked like this:
$ build-all coreutils-5.2.1 gawk-3.1.4
Build two packages everywhere
$ build-all --on loaner.example.com gnupg-1.2.4
Build one package on a specific host
$ build-all --source $HOME/work butter-0.3.7
Build package from nonstandard location
These commands do a lot of work. Here is an outline of the steps that they carry out for each specified software package and each of the default, or selected, build hosts:
Find the package distribution in the local filesystem.
Copy the distribution to the remote build host.
Initiate login connections on the remote host.
Change to the remote build directory and unbundle the distribution file.
Change to the package build directory and configure, build, and test the package.
Record all of the output on the initiating host in separate log files for each package and build environment.
The builds on the remote hosts proceed in parallel, so the total wall-clock time required is that for the slowest machine, rather than the sum of the individual times. Thanks to build-all, builds in up to 100 environments are routine for us, and provide a challenging workout for package developers.
The build-all script is long, so we present it in parts, with surrounding commentary, and then for reader convenience we show the complete program later in this chapter, in Example 8-2.
We begin with the usual introductory comment header:
#! /bin/sh -
# Build one or more packages in parallel on one or more build hosts.
#
# Usage:
# build-all [ --? ]
# [ --all "..." ]
# [ --cd "..." ]
# [ --check "..." ]
# [ --configure "..." ]
# [ --environment "..." ]
# [ --help ]
# [ --logdirectory dir ]
# [ --on "[user@]host[:dir][,envfile] ..." ]
# [ --source "dir ..." ]
# [ --userhosts "file(s)" ]
# [ --version ]
# package(s)
#
# Optional initialization files:
# $HOME/.build/directories list of source directories
# $HOME/.build/userhosts list of [user@]host[:dir][,envfile]
We initialize the input field separator, IFS, to newline-space-tab:
IFS='
'
Next, we set the search path to a limited list and make it global with export, so that all subprocesses on the initiating host use it:
PATH=/usr/local/bin:/bin:/usr/bin
export PATH
We set the permission mask (see Section B.6.1.3 in Appendix B) to allow full access for user and group, and read access for other. The group is given full access because, on some of our systems, more than one system manager handles software installations, and the managers all belong to a common trusted group. The same mask is needed later on the remote systems, so we follow our programming convention by giving it an uppercase name:
UMASK=002
umask $UMASK
It proves convenient to delegate part of the work to separate functions, so that we can limit code blocks to a comfortable size. Nine such functions are defined at this point in the program. However, for tutorial purposes, we delay their presentation until we have discussed the main