How Methodological Features Affect Effect Sizes in Education

Autor/inn/en	Cheung, Alan; Slavin, Robert
Institution	Society for Research on Educational Effectiveness (SREE)
Titel	How Methodological Features Affect Effect Sizes in Education
Quelle	(2016), (6 Seiten) PDF als Volltext kostenfreie Datei Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Monographie
Schlagwörter	Effect Size; Evidence Based Practice; Educational Change; Educational Policy; Research Design; Educational Experiments; Program Evaluation; Publications; Sample Size; Literature Reviews; Best Practices; Educational Research; Elementary Secondary Education; Mathematics Education; Science Education; Reading Instruction; Reading Programs; Early Childhood Education; Technology Uses in Education; Meta Analysis; Computer Software; Statistical Analysis; Q Methodology; Quasiexperimental Design; Randomized Controlled Trials + Suchen Sie Ihr Suchwort? Bildungsreform; Politics of education; Bildungspolitik; Forschungsdesign; Schulversuch; Programme evaluation; Programmevaluation; Bildungsforschung; Pädagogische Forschung; Mathematische Bildung; Naturwissenschaftliche Bildung; Leseunterricht; Early childhood; Education; Frühkindliche Bildung; Frühpädagogik; Technology enhanced learning; Technology aided learning; Technologieunterstütztes Lernen; Meta-analysis; Metaanalyse; Statistische Analyse
Abstract	As evidence-based reform becomes increasingly important in educational policy, it is becoming essential to understand how research design might contribute to reported effect sizes in experiments evaluating educational programs. The purpose of this study was to examine how methodological features such as types of publication, sample sizes, and research designs affect effect sizes in experiments. In order to investigate the relationships between study methodological features and effect sizes, the authors analyzed 645 studies that met the standards of inclusion for any of 12 reviews written for the Best Evidence Encyclopedia and (in most cases) published in review journals. The reviews cover programs in elementary and secondary math, elementary and secondary science, and elementary and secondary reading, as well as a review of elementary reading programs for struggling readers and a review of early childhood education. Studies included in reviews focusing on technology applications in reading and math were also included. Comprehensive Meta-Analysis software Version 2 (Borenstein, Hedges, Higgins, & Rothstein, 2005) was used to carry out all statistical analyses such as Q statistics and overall effect sizes. The findings suggest that effect sizes are roughly twice as large for published articles, small-scale trials, and experimenter-made measures, than for unpublished reports, large-scale studies, and independent measures, respectively. In addition, effect sizes are significantly higher in quasi-experiments than in randomized experiments. Based on the findings of the analyses, it is clear that researchers as well as policy makers need to take into account research design, sample size, measures, and type of publication before comparing effect sizes from program evaluations. Some specific recommendations are provided. (As Provided).
Anmerkungen	Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail: inquiries@sree.org; Web site: http://www.sree.org
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01