Academic Legal Writing - Eugene Volokh [87]
* The least popular First Amendment right is freedom of the press, with 42 percent saying the news media have too much freedom.
* More than 40 percent of those polled said newspapers should not be allowed to freely criticize the U.S. military's strategy and performance.
* About half said the American press has been too aggressive in asking government officials for information about the war on terrorism ....
Freedom of the press exists not only for the news media, but for the very public that it strives to serve. In times when our democratic form of government is under attack, we should fight even harder to preserve our freedoms.
A powerful assertion: People are afraid of terrorism, and they're taking this fear out on the press.
The facts, though, don't fully bear out this story. Here is the fraction of people who think “the press in America has too much freedom to do what it wants” (the 2001 survey was conducted before Sept. 11):
The margin of error is ±3%, so some of the fluctuations may be random, but there's no reason to think that the Sept. 2001 attacks caused more people to think the press has too much freedom; in fact, from 2000 to 2002, the fraction of people holding this view declined by a statistically significant margin. Likewise, here's the result of another question, which seems to reveal no statistically significant change in attitudes about media freedom from the pre-Sept. 2001 survey to the post-Sept. 2001 survey:
Some people believe that the media has too much freedom to publish whatever it wants. Others believe there is too much government censorship. Which of these beliefs lies closest to your own?
On balance, the rest of the survey likewise doesn't seem to show any material change in public attitudes after Sept. 11, 2001—most differences between the 2001 and 2002 numbers are statistically insignificant, and where the difference is significant, the 2002 numbers tend to be similar to numbers from some of the pre–2001 years. It's hard to explain these results as flowing from a fear of terrorism.
But in any event, those who want to make a case for the claim that “Many Americans, spooked by the Sept. 11, 2001, terrorist attacks ... seem inclined to clamp down on ... freedom of the press,” need to confront all the data in the survey, and not just cite the few numbers that seem to support the claim.
5. Respondents giving incorrect answers to pollsters
Finally, all this assumes that the respondents answered the questions accurately. This assumption may be wrong for various reasons:
a. If the question asks about past events, some respondents might not have remembered the events well enough.
b. Some respondents might have concealed their past behavior (for instance, drug or gun use) because it's illegal or embarrassing.
c. Some respondents might have concealed their present views, for instance, views that they think might be seen by the surveytaker as racist or otherwise unpopular.
d. Some respondents might have been unwilling to admit their views even to themselves.
e. Some respondents might have misunderstood even carefully designed questions.
f. Some respondents might not have had firmly held views on the subject, but might have said whatever came to their minds just to avoid looking ignorant or apathetic, either to the surveytaker or to themselves.
Unfortunately, it's not easy to deal with these problems, which is why relying on surveys is risky (though sometimes unavoidable). If you think that a particular survey might yield unsound results because of these problems, look at a good textbook on survey evidence for more information on how serious these problems are likely to be.
6. An exercise
Here's an exercise, based on a graphic on the front page of the July 16, 2002 USA Today; the question refers to a Ninth Circuit case that concluded that the use of the words “under God” in the Pledge of Allegiance violates the Establishment Clause.
There are at least four errors here (not all of them related just to the statistics); what are they? See