Academic Legal Writing - Eugene Volokh [94]
Curiously, the table that the opinion cites actually lists the county-by-county differences in the death penalty verdicts as a fraction of all homicides. Those differences are indeed huge, ranging, among listed counties with more than 100 homicides, from 3.04/1000 to 128.44/1000, which does suggest that different communities have different views on when the death penalty is proper. (The homicides in the 3.04/1000 county might tend to be less heinous than the ones in the 128.44/1000 county, but that effect isn't likely to be large enough to explain the 40–fold difference in the death penalty imposition rate.)
The opinion's conclusion about the diversity in death penalty views is thus correct. But rather than relying on a comparison that strongly supported its thesis (the county-by-county disparity in death sentences per homicide), the opinion relied on a comparison that didn't support it (the county-by-county disparity in absolute numbers of death sentences).
2. Make sure that cost/benefit comparisons sensibly quantify costs and benefits
When you present a comparison as part of a cost-benefit analysis, make sure that the two sides of the comparison bear some proportion to the activity's true costs and benefits. Consider the statement that “a gun in the home is 43 times more likely to kill its owner or a friend than an intruder,” used to support the assertion that “guns in the home, rather than providing protection and safety, actually increase the risk of injury and death to their owners.” This claim is a technically fairly accurate summary of a study (albeit one that was limited to five years in one county) that “noted 43 suicides, criminal homicides, or accidental gunshot deaths involving a gun kept in the home for every case of homicide for self-protection.”54
But while the statement is framed as a cost-benefit comparison, it mischaracterizes the benefit. The primary benefit of guns for “protection and safety” isn't killing intruders, but scaring them away; most defensive gun uses don't even involve the gun being shot, and only a tiny fraction involve an intruder being killed.55
The comparison between “likely to kill its owner or a friend” and “likely to kill ... an intruder” is thus unhelpful. It is so far removed from measuring the true benefits of gun ownership as well as the true costs—deaths aren't the only possible cost of gun misuse, and suicides (in the study, 37 of the 43 deaths per self-protective homicide) are, to many people, not equivalent to accidental deaths or criminal homicides—that it adds nearly nothing to the discussion. That it originally seems so telling only makes it more misleading.
3. Say how many cases the comparison is based on, and how small changes in selection may change the result
Imagine someone pointing out that “In the first ninety years of the 20th century, all major American wars began under Democratic presidents.” This implicitly compares Democrats and Republicans, and implies that Democratic presidents are more warlike than Republicans.
This assertion (which I have heard in various versions) is potentially misleading in two ways. First, the claim is based on four data points: World War I, World War II, the Korean War, and the Vietnam War. It's hard to infer much from four data points, especially since there are so many other factors besides the President's party affiliation that can influence whether a country goes to war. If casual readers aren't explicitly told that the claim is based on just four items, they may miss this important limitation of the comparison.
Second, if we broadened the sample by a few years in both directions, we'd get the Spanish–American War (1898) and the Gulf War (1991), both started under Republican Presidents. This doesn't mean that Republicans