School Choice or Best Systems_ What Improves Education_ - Margaret C. Wang [7]
The difference in outcomes can be straightforwardly assessed with a far smaller set of assumptions than required in nonexperimental research (such as that the groups have indeed been subjected to the specified conditions, which can be investigated). Experimental studies do not require the usually disputable and often ideological questions about specifying, controlling for, and reliably measuring all other causes. In contrast, social scientists have come to conflicting conclusions from the same data, depending on social scientists’ initial causal views, which are often assumed rather than probed.
Because a student’s academic achievement prior to moving to a different school often accounts for the bulk of variations in later test scores, studies with measures of achievement gains from one occasion to another (or more than two occasions) are given special weight in this book. Measuring value-added gains or “over-time” growth in achievement during intervening periods increases the sensitivity of the study, reduces possible biases attributable to preexisting differences among students, and thus makes the findings more creditable. Such prior information helps to take into account the powerful influence of families and measures the separate and distinct contribution of the school to a student’s achievement.
Other things being equal, “cross-sectional” studies of scores obtained on only one occasion as well as studies that follow grouped rather than individual students’ progress are much less credible, and only a few are discussed in this book in cases in which no individual value-added, learning-gain research is available. Once again, with other things being equal, large, randomly chosen samples of large, well-defined populations (preferably a state or nation) allow correspondingly more general causal inferences than small samples within a single community or city.
Social and educational research involves many plausible variables and difficult measurement and sampling problems. Any study is likely to have several flaws. Therefore, scholars in such professions as medicine and psychology weight findings more heavily that replicate, that is, repeat the same findings, preferably many times in a variety of circumstances. This book describes the methods and findings of particularly rigorous studies, but it also draws on previous summaries of many studies on a number of topics. These are called “reviews of research” since they critically evaluate multiple studies and point out findings that are consistent across them.
Unlike news accounts, they avoid putting exclusive weight on a single study when other studies are also available. They also provide a better indication of whether an effect is broadly found in many circumstances by several investigators rather than by only a single study that may be flawed in known and unknown ways.
On some topics, studies that do not meet the above standards may shed light on causality. An example is studies of economically advanced and developing countries that rapidly and substantially introduced vouchers, and for which investigators have documented massive changes in test scores; private school enrollments; and integration of immigrant, special needs, and minority students. In one case discussed, a randomized experimental trial was employed to test the effects of school choice on achievement. Of course, there remains the question of whether such findings can be generalized to the United States and to other countries. In addition to studies of massive national changes, systematic observational studies of chosen schools that have had outstanding success may be reasonable to review to see what sets those schools apart from other schools.
This book draws on the largest body of rigorous evidence the author could amass on school choice and competitive effects. Admittedly, no single study or piece of evidence is definitive. The situation may be likened to the search for “dose-response” connections between