unSpun_ Finding Facts in a World of Disinformation - Brooks Jackson [45]
What they found, after 400 separate observations, was that there was no real difference. In fact, crows were just slightly more likely to drop a walnut on the pavement when no car was approaching. The birds also were slightly more likely to fly away and leave a nut on the pavement in the absence of a car, contrary to what would be expected if the birds really expected cars to crack the nuts for them. Furthermore, the scientists noted that they frequently saw crows dropping walnuts on rooftops, on sidewalks, and in vacant parking lots, where there was no possibility of a car coming along. Not once during the study did a car crack even a single walnut dropped by a crow.
The authors concluded, reasonably enough: “Our observations suggest that crows merely are using the hard road surface to facilitate opening walnuts, and their interactions with cars are incidental.” The title of their article: “Crows Do Not Use Automobiles as Nutcrackers: Putting an Anecdote to the Test.” The anecdote flunked.
LESSON: Don’t Confuse Anecdotes with Data
ONE OF OUR FAVORITE SAYINGS—VARIOUSLY ATTRIBUTED TO DIFFERENT economists—is “The plural of ‘anecdote’ is not ‘data.’” That means simply this: one or two interesting stories don’t prove anything. They could be far from typical. In this case, it’s fun to think that crows might be clever enough to learn such a neat trick as using human drivers to prepare their meals for them. It’s also easy to see how spotting a few crows getting lucky can encourage even serious scientists to think the behavior might be deliberate. But we have to consider the term “anecdotal evidence” as something close to an oxymoron, a contradiction in terms.
Now, it’s true that the crow debate continues. Millions of people saw a PBS documentary by David Attenborough that showed Japanese crows putting walnuts in a crosswalk and then returning to eat after passing cars had cracked them. That scene was inspired by an article in the Japanese Journal of Ornithology by a psychologist at Tohoku University. But the Japanese article wasn’t based on a scientific study; it merely reported more anecdotes: “Because the [crows’] behavior was so sporadic, most observation was made when the author came across the behavior coincidentally on his commute to the campus.” That was two years before Cristol and his colleagues finally published their truly systematic study. So for us, the notion that crows deliberately use cars as nutcrackers has been debunked, until and unless better evidence comes along. Even Theodore Pietsch, who coauthored the 1978 article that said crows do use cars as nutcrackers, has changed his view. “When Grobecker and I wrote that paper so long ago, we did it on a whim, took about an hour to write it, and we were shocked that it was accepted for publication almost immediately, with no criticism at all from outside referees,” he told us. “I would definitely put much more credibility in a study supported by data rather than random observation.” So do we, and so should all of us.
Seeing versus Believing
Avoiding spin and getting a solid grip on hard facts requires not only an open mind and a willingness to consider all the evidence, it requires us to have some basic skills in telling good evidence from bad, and to recognize that mere assertion is not fact and that not all facts are good evidence. As counterintuitive as it may seem, the most basic lesson is that our own personal experience isn’t necessarily very good evidence. It’s natural to trust what we can see with our own eyes, what we can touch with our own hands and hear with our own ears. But our own experience can mislead us.
LESSON: Remember the Blind Men and the Elephant
IT IS A NATURAL HUMAN TENDENCY TO GIVE GREAT WEIGHT TO OUR immediate experience, as the ancient fable of the blind men and the elephant should