Design of Everyday Things - Norman, Don [72]
The crew of flight 007 probably misprogrammed the INS, but the INS couldn’t be reprogrammed in flight: if an error were detected the aircraft would have to go back to the original airport, land (jettisoning fuel to get to a safe landing weight), and then reset the INS and take off again—an expensive proposition. Three Korean Air flights had returned to their airport in the six months preceding the flight 007 incident, and the airline had told its pilots that the next pilot who returned would be punished. Was this a factor in the accident? It’s hard to know, but the design of the INS sounds badly deficient. The social pressures on the crew not to find (or admit to) an error in the INS were clearly strong. But punishment for following a safety procedure is never wise. The proper approach would be to redesign either the INS’s or the procedures for using them.14
The real culprit, almost always, is the design. Design that makes it easy to make wrong settings, or to misread an instrument, or to misclassify an event. Design of the social structure that makes false reporting of danger punishable. Turn a nuclear power plant off by mistake and you cost the company hundreds of thousands of dollars; you’ll probably lose your job. Fail to turn it off when there is a real incident, and you might lose your life. If you refuse to fly a crowded airliner because the weather looks bad, the company loses lots of money and the passengers get very angry. Take off under those situations and most of the time it works out fine, which encourages risk taking. But every so often there is a disaster.
Tenerife, the Canary Islands, in 1977. A KLM Boeing 747 that was taking off crashed into a Pan American 747 that was taxiing on the runway, killing 583 people. The KLM plane should not have tried to take off then, but the weather was starting to get bad, and the crew had already been delayed for too long (even being on the Canary Islands was a diversion from the scheduled flight—they had to land there because bad weather had prevented them from landing at their scheduled destination); they had not received clearance to take off. And the Pan American flight should not have been on the runway, but there was considerable misunderstanding between the pilots and the air traffic controllers. Furthermore, the fog was coming in so neither plane could see the other.
There were time pressures and economic pressures acting together. The Pan American pilots questioned their orders to taxi on the run way, but they continued anyway. The co-pilot of the KLM flight voiced minor objections to the pilot, suggesting that they were not yet cleared for takeoff. All in all, a tragedy occurred due to a complex mixture of social pressures and logical explaining away of discrepant observations.
The Air Florida flight from National Airport, Washington, D.C., crashed at takeoff into the 14th Street bridge over the Potomac River, killing seventy-eight people, including four who were on the bridge. The plane should not have taken off because there was ice on the wings, but it had already been delayed over an hour and a half; this and other factors “may have predisposed the crew to hurry.” The accident occurred despite the first officer’s (the co-pilot’s) concern: “Although the First officer expressed concern that something ‘was not right’ to the captain four times during the takeoff, the captain took no action to reject the takeoff. ”Again we see social pressures coupled with time and economic forces.15
Designing for Error
Error is often thought of as something to be avoided or something done by unskilled or unmotivated people. But everyone makes errors. Designers make the mistake of not taking error into account. Inadvertently, they can make it easy to err and difficult or impossible to discover error or to recover from it. Consider the London stock market story that opened this chapter. The