Classic Shell Scripting - Arnold Robbins [115]
Although we usually avoid writing chatty programs, the echo command before each communication with a remote system is intentional: it gives the user essential feedback. The remote copy can be time-consuming and can fail or hang: without that feedback, the user has no simple way to tell why the script is taking unexpectedly long, or what host is responsible for the failure. The parbaselocal variable serves to distinguish between a temporary copy of the archive file and a preexisting one:
echo $SSH $SSHFLAGS $userhost "test -f $PARFILE"
if $SSH $SSHFLAGS $userhost "test -f $PARFILE"
then
parbaselocal=$PARFILE
else
parbaselocal=$parbase
echo $SCP $PARFILE $userhost:$builddir
$SCP $PARFILE $userhost:$builddir
fi
Ideally, we should use a pipeline to do the unbundling, since that halves the amount of input/output, and also the disk space requirements. Alas, only jar and tar can read their archives that way: unzip requires an actual file. In principle, jar can read an InfoZip file, allowing us to replace unzip with jar and use the pipeline. Unfortunately, at the time of this writing, jar is still immature, and we have found at least one implementation that chokes on a .zip file.
Observe that the remote copies are done sequentially, rather than in parallel. The latter would be possible, but at the expense of extra complexity in the main body, which would have to first find and distribute packages, wait for their distribution to complete, and then build them. However, the build time is usually much longer than the remote copy time, so sequential copies do not contribute much to the total runtime.
Our log files are named with the package, remote host, and a timestamp with one-second resolution. If multiple builds are done on a single remote host, then we risk a filename collision. Using the process ID variable, $$, in the log filenames does not provide a solution because it is constant within the single invocation of build-all. We could use $$ to initialize a counter that is incremented with each build and used in the log filenames, but that would simply clutter the filenames with a meaningless number. The solution is to guarantee that at least one second elapses between the creation of two successive log files: sleep is just what we need. GNU date offers a %N (nanoseconds) format item that should suffice to generate unique filenames, allowing us to do away with sleep, but POSIX and older date implementations lack that format item. In the interests of maximal portability, we rest for a second:
sleep 1
now="`date $DATEFLAGS`"
logfile="$package.$host.$now.log"
We have now come to the last part of this exposition: the lengthy commands to carry out the build on the remote host. $SSH is prefixed with the nice command to lower its priority, reducing competition with interactive jobs on the system. Even though most of the work happens on the remote system, build logs are sometimes large, giving $SSH more to do.
Notice that the second argument to $SSH is a long string delimited with double quotes. Inside that string, variables prefixed with a dollar sign are expanded in the context of the script, and need not be known on the remote host.
The command syntax that we need in the $SSH argument string depends on the user's login shell on the remote host. We carefully restrict the syntax to work in all common Unix shells so that build-all works for any user, including users with different login shells on different hosts. We cannot demand the same login shell everywhere, because on many systems, users cannot choose their login shells. The alternative would be to pipe the command stream into the Bourne shell on each host, but that would start yet another process for each build, and get us into an even deeper quoting mess: dealing with three shells at a time is already hard enough.
nice $SSH $SSHFLAGS $userhost "
echo '= = = = = = = = = = = = = = = = = = = = = = =
= = = = = = = = = = =