Academic Legal Writing - Eugene Volokh [88]
H. Be Explicit About Your Assumptions
Often, your evidence won't precisely match the claim you're making. You might be making a claim about what's happening now, but the available studies might only report what was happening five years ago. You might be making a claim about what's happening throughout the country, but the available studies might focus only on certain regions.
You might be making a claim about the crime rate, but the available studies might measure only the arrest rate. Riskiest of all, you might be arguing that some policy will lead to a certain result, but the studies might only say that when a similar policy was implemented, it was followed by that result. Such studies may merely reflect coincidence, rather than causation.
These gaps in the data need not be fatal to your argument. Most policy analysis requires inferring from correlation to causation, or extrapolating from one place, time, group, or variable to another.
But you must be explicit about the inferences and extrapolations that you make, and the assumptions on which they rest (for instance, that things haven't changed much from 1998 to 2004). Clearly acknowledge them, at least in the footnotes and, if they're important or controversial enough, in the text. And if it's not obvious that the inference or extrapolation is sound, you need to explain to the reader why it's sound.
There are three reasons why you must do this. First, you need to do this to be honest with your readers. If you say “Studies show that there are X contract killings in the country per year,” and it turns out that the studies showed only that there were X contract killings in the country in 1980, then you're being inaccurate or even dishonest.
Second, you need to do it to maintain your credibility with your readers. Many readers will be savvy enough to notice any unspoken assumptions that you make. They won't be deceived by your silence—but they'll be annoyed, and they'll assume that you're sloppy, dishonest, or oblivious to the logical leaps that you're making.
Third, this explicitness will help you see and therefore correct the potential flaws in your article. When you make clear that you're only inferring or extrapolating something, you might think to yourself: “I wonder why this inference is sound.” Perhaps it's not sound, and you need to find a more apt study, or to change or qualify your claim. Or perhaps it is sound, but you realize that you need to explain further why it's sound. In either case, you'll be able to make your article more well-reasoned and more persuasive.
1. Inferring from correlation to causation
“There are more guns in the U.S. than in England; there is also more murder in the U.S. than in England. Therefore, the prevalence of guns causes an increase in murder.” This is an argument from correlation (the murder rate seems higher where gun ownership is higher) to causation (the higher murder rate is caused by higher gun ownership).
Here's another argument from correlation to causation: “There are more guns in rural areas in the United States than in urban areas; there is also less murder in rural areas than in urban areas. Therefore, the prevalence of guns causes a decrease in murder.” The premises of both these arguments are true, but the conclusions can't be. This illustrates the danger of inferring causation from correlation.
For another illustration, consider ice cream production and rape in the United States. Within any particular year, the two are highly correlated: In 2000, for instance, the correlation was 0.84, which is very high (1 would be perfect correlation) and statistically significant; look how closely the two coincide on the graph below.50 Does ice cream production cause rape? Does rape cause ice cream production?
No, it seems more likely that some third factor (often called the “confounding factor”) causes both. Here, the third factor is time of year: The rape rate is higher during the summer, probably partly because people are out in public more in the summer. Ice cream sales and therefore