Case Studies and Theory Development in the Social Sciences - Alexander L. George [91]
The first of these suggestions is similar to that advanced by Arend Lijphart for increasing the numbers of cases, as DSI acknowledges.356 The second method “involves a partial replication of the theory or hypothesis that uses a new dependent variable but keeps the same explanatory variables” (emphasis added). The problem here is that changing the dependent variable alters the research objective of the study—and, indeed, the theory itself—since the choice of a new dependent variable changes the nature of the phenomenon that is to be explained or predicted.
This criticism applies with even greater force to DSI’s third proposal: for “a new (or greatly revised) hypothesis implied by our original theory that uses a new dependent variable and applies the hypothesis to new instances.” 357 DSI explicitly acknowledges that this suggestion involves “new variables and new units. The measures used to test what are essentially new hypotheses that are derived from the original ones may be quite different from those used thus far.”358 They acknowledge that this “may involve the introduction of explanatory variables not applicable to the original unit.”359
The second and third ways of increasing the number of observations put the technical requirements of their method ahead of the objective of testing the initial theory. They endorse changing the starting theory when necessary to obtain a large number of observable implications of some other, perhaps related, theory. A similar critical observation is made by Charles Ragin in commenting on DSI’s effort to gain analytic leverage through empirical disaggregation. Most such attempts “undermine the question that inspired the investigation in the first place.”360
We should make clear that the reservations expressed here do not question the general desirability of attempting to identify observable implications of a given theory, both within and among cases, in order to facilitate the task of assessing it. Our disagreement with DSI is that we believe the search for observable implications should be confined to those clearly relevant to the original theory. DSI anticipates this criticism by saying that the changes in an original theory suggested by their second and third ways of increasing observations should be consistent with the original theory. This is a basic prerequisite of their approach which rests on the questionable assumption that the different observations are not independent of each other and are not independent of the original outcome variable of the starting theory.
In our view, switching the effort to assess a given theory by altering it raises serious questions. First, how can one decide whether one has correctly assigned observable implications to a theory? This is the question of the validity of the imputed observations. Second, are all of the observations imputed to a theory equally important for assessing that theory? DSI makes only passing reference to both of these questions in its repeated emphasis on increasing the number of observations. For example: “Maximizing leverage is so important and so general that we strongly recommend that researchers routinely list all possible observable implications.” 361 Similarly, readers are enjoined to “collect data on as many of its [the theory’s] observable implications as possible.”362
Following such advice may lead to indiscriminate listing of all questionable implications of a theory. Little guidance is given for distinguishing between genuine, questionable, and highly speculative implications of a theory, as a more Bayesian approach to theory testing would require. The emphasis on all observable implications, moreover, fails to indicate the importance of identifying strong, valid implications of a theory even if they are not readily observable at present. DSI’s method