Proofiness - Charles Seife [46]
From a mathematical point of view, this dependence on question phrasing is a systematic error that prevents polls from reflecting reality. To some pollsters, though, reality is merely a burden. They can make more money if they give their clients what they want—and if their clients want a Potemkin number, they seem to be willing to provide it.
In April 2005, a deeply divisive drama was drawing to a close in the United States. Terri Schiavo was a woman who had been in a persistent vegetative state for more than a decade. Her husband, Michael, wanted her feeding tube removed; her parents wanted Terri to be kept alive. After a long battle in the courts and a great deal of posturing by politicians, Michael prevailed and the tube was removed. It was a grisly business; unable to feed herself, Terri was slowly starving. Though people felt strongly on both sides of the issue, the polls in March seemed to show that Americans tended to favor the removal of the tube. An ABC poll, for example, had 63 percent in favor of the feeding tube’s being removed.
The Christian Defense Coalition, horrified by the results, evidently sought to bend public opinion to reflect their reality. So they hired Zogby International, one of the premier polling firms in the nation, to conduct a new poll. Zogby agreed, and when the results were released in early April, an article reproduced on Zogby’s website trumpeted the results: “A new Zogby poll with fairer questions shows the nation clearly supporting Terri and her parents and wanting to protect the lives of other disabled patients.” What were these fairer questions exactly? A small selection:
• Do you agree or disagree that the representative branch of governments should intervene when the judicial branch appears to deny basic rights to the disabled?
• Do you agree or disagree that the representative branch of governments should intervene when the judicial branch appears to deny basic rights to minorities?
• Do you agree or disagree that it is proper for the federal government to intervene when basic civil rights are being denied?
• If a disabled person is not terminally ill, not in a coma, and not being kept alive on life support, and they have no written directive, should or should they not be denied food and water?
• Michael Schiavo has had a girlfriend for 10 years and has two children with her. Considering this, do you agree or disagree that Michael Schiavo should turn guardianship of Terri over to her parents?
This wasn’t an attempt to discover public opinion. It was an attempt to shape it. The questions about minorities and the disabled were intended to frame the issue as one where Schiavo’s civil liberties were being violated. Then the use of “denied”—a very evocative word—in the key question makes it hard for anyone but a monster to say that Schiavo (however abstractly she’s presented) should be disconnected from the feeding tube. And then, in case you had any doubts about where your loyalties lie, the poll paints Michael Schiavo as a philanderer. Naturally, the Zogby poll had quite different results from the other polls that used less inflammatory language—results that the Christian Defense Coalition clearly wanted. Another client pleased, reality be damned.
Polling organizations make money by producing