Case Studies and Theory Development in the Social Sciences - Alexander L. George [66]
GENERALIZING ACROSS TYPES: TOUGH TESTS AND MOST-LIKELY, LEAST-LIKELY, AND CRUCIAL CASES
It is difficult to judge the probative value of a particular test relative to the weight of prior evidence behind an existing theory. Harry Eckstein argues that “crucial cases” provide the most definitive type of evidence on a theory. He defines a crucial case as one “that must closely fit a theory if one is to have confidence in the theory’s validity, or conversely, must not fit equally well with any rule contrary to that proposed.” He adds that “in a crucial case it must be extremely difficult, or clearly petulant, to dismiss any finding contrary to the theory as simply ‘deviant’ (due to chance, or the operation of unconsidered factors).”242
Eckstein notes the difficulties in identifying such crucial cases when theories and their predictive consequences are not precisely stated, but notes that the foremost problem is that truly crucial cases rarely occur in nature or the social world. Therefore, he suggests the alternative of tough tests which entail studying most-likely and least-likely cases. In a most-likely case, the independent variables posited by a theory are at values that strongly posit an outcome or posit an extreme outcome. In a least-likely case, the independent variables in a theory are at values that only weakly predict an outcome or predict a low-magnitude outcome. Most-likely cases, he notes, are tailored to cast strong doubt on theories if the theories do not fit, while least-likely cases can strengthen support for theories that fit even cases where they should be weak.
Many case study researchers have identified the cases they choose for study as most-likely or least-likely cases, but it is necessary to be explicit and systematic in determining this status. One must consider not only whether a case is most or least likely for a given theory, but whether it is also most or least likely for alternative theories. One useful means of doing so, as noted in Chapter 11 on typological theory, is to include a typological table that shows the values of variables in the case or cases studied for competing hypotheses. Such a table helps the researcher and reader identify which variables in a case may favor alternative theories, and helps the researcher to address systematically whether alternative theories make the same or different predictions on processes and outcomes in a given case.
In general, the strongest possible supporting evidence for a theory is a case that is least likely for that theory but most likely for all alternative theories, and one where the alternative theories collectively predict an outcome very different from that of the least-likely theory. If the least-likely theory turns out to be accurate, it deserves full credit for a prediction that cannot also be ascribed to other theories (though it could still be spurious and subject to an as-yet undiscovered theory). This might be called a toughest test case.243 Theories that survive such a difficult test may prove to be generally applicable to many types of cases, as they have already proven their robustness in the presence of countervailing mechanisms.
The best possible evidence for weakening a theory is when a case is most likely for that theory and for alternative theories, and all these theories make the same prediction. If the prediction proves wrong, the failure of the theory cannot be attributed to the countervailing influence of variables from other theories (again, left-out variables can still weaken the strength of this inference). This might be called an easiest test case. If a theory and all the alternatives fail in such a case, it should be considered a deviant case and