Case Studies and Theory Development in the Social Sciences - Alexander L. George [113]
Assessing Alternative Hypothesized Processes
We note in particular that process-tracing needs to consider the possibility of alternative processes that lead to the outcome in question. It is important to examine the process-tracing evidence not only on the hypothesis of interest, but on alternative hypotheses that other scholars, policy experts, and historians have proposed. Too often, researchers focus great attention on the process-tracing evidence on the hypothesis that interests them most, while giving the process-tracing evidence that bears on alternative explanations little attention or using it only to explain variance that is not adequately explained by the hypothesis of interest. This can create a strong confirmation bias, and it can overstate the causal weight that should be accorded to the hypothesis of interest.
Lawrence Mohr has given a useful account of the need to avoid confirmation bias, following Michael Scriven’s modus operandi method and his metaphor of a detective:
… when X causes Y it may operate so as to leave a “signature,” or traces of itself that are diagnostic. In other words, one can tell when it was X that caused Y because certain other things that happened and are observed unequivocally point to X. At the same time, one knows the signature of other possible causes of Y and one may observe that those traces did not occur. By using this technique, one can make a strong inference that X either did or did not cause Y in a certain case. For the present purpose, moreover, one notes in passing the affinity of this approach for the study of a single case. The kind of example of the modus operandi approach that is frequently given reminds one of the work of a detective or a diagnostician.437
Yet as Mohr himself points out, the theory in question may not leave an observable signature. It is also possible that the predictions about causal process attributed to, or claimed by, the theory may be questionable or ambiguous. Moreover, proving the negative and demonstrating that a particular process did not occur can be notoriously difficult. Both detectives and researchers face these difficulties. But the main difficulty may be that the theory is not sufficiently specified to allow one to identify confidently a causal process it predicts or is capable of predicting.
As Mohr’s detective metaphor suggests, when well-specified theories are available, process testing can proceed forward, from potential causes to effects; backward, from effects to their possible causes; or both. The use of process-tracing to verify the predictions of a theory should also ordinarily involve attempts to test and eliminate alternative causal processes (derived from other theories) that might lead to the same outcome. For example, the detective usually pursues several suspects and clues, constructing possible chronologies and causal paths backward from the crime scene and forward from the last known whereabouts of the suspects. With theories, as with suspects, the evidence might not be sufficient to eliminate all but one. In addition, alternative theories and the causal processes they specify may be complementary rather than mutually exclusive. Since more than one theory may be consistent with the process-tracing evidence, several may have contributed to the observed effect or even overdetermined it.
On the other hand, when theories make genuinely competing process predictions, the process-tracing evidence may be incomplete in ways that do not permit firm conclusions on which theory fits better. The detective’s colleague, the district attorney, would remind us that a potential causal path cannot explain a case if it does not establish an uninterrupted causal path from the alleged cause to