School Choice or Best Systems_ What Improves Education_ - Margaret C. Wang [6]
In addition to achievement and consumer opinion, other measures of school success are reported here when available. These include high school and college graduation rates and students’ voluntary charitable activities in school and later adult life. Because of the American problem of low public school cost-effectiveness, the costs of choice schools and traditional public schools are also considered.
Credibility and Selection of Evidence
This book sets aside philosophical controversies about school choice and confines itself largely to empirical research on its effects. Some research on this topic is set aside since it does not measure up to modern social science research standards. Opinion research counts for little, for example, unless results are obtained from surveys of large, well-defined populations or large, random samples of them. Expert observers’ observations may be subjective and merely confirm presuppositions. The rampant anecdotalism common in many public discussions of school choice cannot be trusted and is ignored here, even though it is often highly influential in both policy and practice.
Perhaps the most difficult problem in evaluating school choice research is estimating causal effects. It is often said, and just as often ignored, in policy discussions that correlation does not mean causation. The research selected for discussion in this book is largely confined to several types regarded as scientific in the applied fields such as medicine, epidemiology, agriculture, engineering, psychology, and increasingly in education and the social sciences. With illustrations, these may be simply and nontechnically defined as follows:
• “Randomized field trials,” the gold standard of causality, compare the academic achievement and other measurable outcomes of admitted applicants to an oversubscribed voucher program or charter school with those of unadmitted applicants who attended the traditional public school. Since whether any applicant is admitted or not is determined by lottery, any “statistically significant” difference between the two groups of students is most likely attributable to the effectiveness of the “treatment,” in this case the type of school chosen, rather than chance.
• “Quasi-experiments” in which students have not been randomly assigned to schools, but statistical adjustments, usually based on achievement pretests, are made in an effort to remove preexisting differences among students before they enrolled in choice and other schools. In the hope that these adjustments eliminate possible “selection biases,” investigators compare the outcome results of students in choice and other schools.
• “Correlational analyses” (usually regression analyses), more often employed by economists than other scholars, compare non-randomly-assigned students in two or more groups of schools by statistically controlling for preexisting differences among students including achievement, race, socioeconomic status, and other characteristics.
These methods of research deserve further discussion here. The social sciences—anthropology, economics, political science, and sociology—are perhaps a half-century behind the applied natural and human sciences in drawing the causal inferences necessary to base policy and practice decisions on scientific conclusions. Random assignment of units to experimental and control or contrast conditions (or treatments) is now generally required in agronomy, medicine, public health, and the rigorous parts of psychology and educational research. Experiments require that the units of analysis be randomly assigned to alternative conditions purely by chance, for example, a coin flip (or, usually, randomly generated numbers). Thus, there is no reason to think