Case Studies and Theory Development in the Social Sciences - Alexander L. George [17]
In addition, in the early stages of a research program, selection on the dependent variable can serve the heuristic purpose of identifying the potential causal paths and variables leading to the dependent variable of interest. Later, the resulting causal model can be tested against cases in which there is variation on the dependent variable.51 Ideally, researchers
A related issue is whether researchers’ foreknowledge of the values of variables in cases—and perhaps their cognitive biases in favor of particular hypotheses—necessarily bias the selection of case studies.52 Selection with some preliminary knowledge of cases, however, allows much stronger research designs; cases can be selected with a view toward whether they are most-likely, least-likely, or crucial for a theory, making the process-tracing test of a theory more severe. Also, within-case analysis often leads to the finding that the researcher’s (or the literature’s) preliminary knowledge of the values of the independent and dependent variables was incomplete or simply wrong, and case study researchers sometimes conclude that none of the proposed theories adequately explains a case. In addition, researchers selecting cases can benefit from knowledge of the findings of existing studies, and be guided by estimations of whether the theories of interest are strong and previously tested or new and relatively weak.53 There are also methodological safeguards against investigator-induced bias in case studies, such as careful congruence testing and process-tracing.
Interestingly, statistical views of selection bias understate both the most severe and the most common kinds of selection biases in qualitative research. The most damaging consequences arise from selecting only cases whose independent and dependent variables vary as the favored hypothesis suggests, ignoring cases that appear to contradict the theory, and overgeneralizing from these cases to wider populations. This type of selection bias can occur even when there is variation in both independent and dependent variables and this variation covers the full range of values that these variables can assume.
Rather than understating the relationship between independent and dependent variables, as in the statistical view of selection bias, this selection bias can understate or overstate the relationship.54 While this form of selection bias seems too obvious to require a warning to social scientists, case researchers may fail to realize that by implicitly or explicitly limiting their sample of cases (say, to history that is contemporary, Western, specific to one country, or easily researchable), they may bias their sample with regard to a wider set of cases about which they are trying to make inferences—unless they carefully define and limit the scope of their findings to a well-specified population that shares the same key characteristics as the cases studied.
This form of selection bias is far more common in political argumentation than in social science case studies. Several other case selection biases, however, are quite common in case study research and deserve increased attention. These include selection of cases based on their “intrinsic” historical importance or on the accessibility of evidence.
IDENTIFYING SCOPE CONDITIONS AND “NECESSITY”
A limitation of case studies is that they can make only tentative conclusions on how much gradations of a particular variable affect the outcome in a particular case or how much they generally contribute to the outcomes in a class or type of cases. Case studies are much stronger at identifying the scope conditions of theories and assessing arguments about causal necessity or sufficiency in particular cases than they are at estimating the generalized causal effects or causal