Case Studies and Theory Development in the Social Sciences - Alexander L. George [20]
I have overlooked a major source of discipline (i.e., degrees of freedom if I persist in using this statistical concept for the analogous problem in nonstatistical settings). In a case study done by an alert social scientist who has thorough local acquaintance, the theory he uses to explain the focal difference also generates predictions or expectations on dozens of other aspects of the culture, and he does not retain the theory unless most of these are also confirmed. In some sense, he has tested the theory with degrees of freedom coming from the multiple implications of any one theory.61
Thus, as long as competing theories make different predictions on the causal processes thought to have taken place in a case—and sufficient evidence is accessible for process-tracing and congruence testing—case study researchers may have the means to reject many of the possible alternative explanations of a case.62
We would go even further than Campbell on this issue. While Campbell states that “most” predictions or expectations a theory makes regarding a case must be confirmed in order for the theory to be retained, we would distinguish retaining a theory that has general utility in many cases from retaining a historical explanation of a particular case. A satisfactory historical explanation of a particular case needs to address and explain each of the significant steps in the sequence that led to the outcome of that case. If even one step in the hypothesized causal process in a particular case is not as predicted, then the historical explanation of the case needs to be modified, perhaps in a trivial way that is consistent with the original theory, or perhaps in a crucial way that calls into question the theory’s general utility and its applicability to other cases. It is this insistence on providing a continuous and theoretically based historical explanation of a case, in which each significant step toward the outcome is explained by reference to a theory, that makes process-tracing a powerful method of inference (a point that we take up in detail in Chapter 10).
The misguided focus on case studies’ supposed “degrees of freedom problem” has diverted attention from a more fundamental problem of indeterminacy that affects all research methods, even experimental methods. This is the problem that evidence, whether from a case or a database, can be equally consistent with a large or even infinite number of alternative theories. The pragmatic (but necessarily incomplete) approach we and others suggest to this problem is that researchers limit themselves to testing alternative theories, which individuals have proposed, rather than worrying over the infinite number of potential theories that lack any proponent. Even so, a particular database or case might not be able to discriminate between which of two or more competing explanations fits best. This is more a matter of how the evidence in a particular case matches up with competing hypotheses than a mechanical issue of the number of cases and the number of variables. This is why case study researchers seek crucial cases in order to be able to definitively test which of several theories fits best and, when such cases are not available, why they look for instances where a theory fails to fit a most-likely case or fits a least-likely one. When more than one competing explanation fits a case equally well, it may still be possible to narrow the number of plausible explanations, and it is also important to indicate as clearly as possible the extent to which the remaining hypotheses appear to be complementary, competing, or incommensurate in explaining the case.63
LACK OF REPRESENTATIVENESS
Case researchers do not aspire to select cases that are directly “representative” of diverse populations and they usually do not and should not make claims that their findings are applicable