UNIX System Administration Handbook - Evi Nemeth [466]
Most power hits are of short duration, but plan for two hours of battery life so that you have time to shut down machines properly in the event of a longer outage. Some UPSs have a serial port or Ethernet interface that you can use to initiate a graceful shutdown of noncritical machines after 5 minutes (configurable) of power outage.
Take advantage of power outages to do any 5-minute upgrades that you already have planned, such as adding a disk to a server. You’re down anyway, so people expect to be inconvenienced. In some shops, an extra 5 minutes during a power outage is easier to accept than a scheduled downtime with a week’s notice. If you have old machines that you think are not in use anymore, leave them turned off until someone complains. It might not be until weeks later—or never—that the “missing” machine is noticed.
Cooling systems often have a notification system that can call you if the temperature gets too high. Tune the value of “too high” so that you have time to get in after the cooling system pages you before machines start to fry; we use 76 degrees instead of 90. Keep a mechanical or battery operated thermometer in the machine room—losing power means that you lose all those nifty electronic indicators that normally tell you the temperature.
See page 749 for more information about environment issues.
A large U.S. government lab recently built a fancy new machine room and filled it with a 256-node Alpha cluster for running large scientific models. Everything was plugged into a UPS, and all the facilities were state of the art. Unfortunately, a minor power outage brought the center down for four hours. Why? The PC that controlled the HVAC (air conditioner) was not on the UPS. It failed and messed up the air conditioning system. Test carefully.
Miscellaneous tidbits
ISPs are merging and being acquired at a fantastic rate. These mergers have demolished many companies’ carefully laid plans for maintaining redundant connections to the Internet. A post-merger ISP will often consolidate circuits that belonged to the independent companies. Customers that formerly had independent paths to the Internet may then have both connections running through a single conduit and once again be at the mercy of a single backhoe fiber cut.
When CNN or Slashdot announces that your web site is down, the same effect that makes highway traffic slow down to look at an accident at the side of the road causes your Internet traffic to increase enormously, often to the point of breaking whatever it was that you just fixed. If your web site cannot handle an increase in traffic of 25% or more, consider having your load balancing software route excess connections to a server that presents a single page that says “Sorry, we are too busy to handle your request right now.”
Use tripwire to keep abreast of what your sysadmins are doing, especially if different groups are responsible for different aspects of the same machine. Oracle database patches and OS patches can conflict with each other without either group realizing that they should ask what the other group has been up to. tripwire snooping is also useful if you are a sysadmin service organization and you find yourself having to clean up after a customer’s somewhat clueless in-house sysadmin. It can clearly identify what has changed and when, making it easier to respond if the local sysadmin tries to blame you for his mistakes.
See page 668 for more information about tripwire .
27.2 LEGAL ISSUES
The U.S. federal government and several states have laws regarding computer crime. At the federal level, there are two from the early 1990s and two more recent ones:
• The Federal Communications Privacy Act
• The Computer