Case Studies and Theory Development in the Social Sciences - Alexander L. George [81]
Mill’s methods can work well in identifying underlying causal relations only under three demanding assumptions. First, the causal relation being investigated must be a deterministic regularity involving only one condition that is either necessary or sufficient for a specified outcome. Second, all causally relevant variables must be identified prior to the analysis (whereas Mill’s methods are applicable only for explaining single-cause hypotheses). Third, cases that represent the full range of all logically and socially possible causal paths must be available for study.
These well-known requirements strongly constrain and limit the usefulness of Mill’s methods. The methods of agreement and difference both utilize the logic of what Mill called the “method of elimination.” Mill explained that his use of the logic of elimination was analogous to its use in the theory of equations “to denote the process by which one or another of the elements of the question is excluded, and the solution is made to depend on the relation between the remaining elements only.”303
In the method of agreement, the investigator employs the logic of elimination to exclude as a candidate cause (independent variable) for the common outcome (dependent variable) in two or more cases those conditions that are not present in both cases. A cause or condition that survives this method of elimination can be regarded as possibly associated (“connected,” in Mill’s terminology) with the case outcome. An inherent weakness of this method of causal inference is that another case may be discovered later in which the same outcome is not associated with the variable that survived the elimination procedure in the comparison of the two earlier cases. Thus, that variable cannot be regarded as either a necessary or sufficient condition for that type of outcome.304 Thus, the possibil-ity remains that the common condition identified for the similar outcome in two cases may turn out to be a “false positive.”
In the method of difference—in which two cases having different outcomes are compared—the investigator employs the logic of elimination to exclude as a candidate cause (independent variable) for the variance in the outcome (dependent variable) any condition that is present in both cases. On the face of it, the logic is quite simple: a condition present in both cases cannot account for the difference in case outcomes. However, conditions that were not present in both cases can only be regarded as possibly causally associated with the variance in case outcomes, for these conditions may not be present in other cases with the same outcome. In that event, the attribution of causal significance to the conditions that seemed to be associated with the variance in outcome in the first two cases would constitute a “false positive.”
In sum, in exercises that use the method of agreement and difference, the investigator cannot be sure that all of the possibly relevant independent variables have been identified or that the study has included a sufficient variety of cases of the phenomenon. Hence, inferences in both methods of agreement and difference may be spurious and invalid. On the other hand, if a much larger number of independent variables are included, we may well encounter the problem of underdetermination (also known as “too many variables, too few cases”). This dilemma cannot easily or adequately be resolved so long as the investigator relies solely on the logic of elimination and does not find sufficiently comparable cases that provide the functional equivalent of experimental control. However,