An Approach for Addressing the Multiple Testing Problem in Social Policy Impact Evaluations

Autor/in	Schochet, Peter Z.
Titel	An Approach for Addressing the Multiple Testing Problem in Social Policy Impact Evaluations
Quelle	In: Evaluation Review, 33 (2009) 6, S.539-567 (29 Seiten) PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0193-841X
DOI	10.1177/0193841X09350590
Schlagwörter	Policy; Evaluation; Testing Problems; Hypothesis Testing; Error of Measurement; Research Design; Statistical Analysis; Comparative Analysis; Factor Analysis + Suchen Sie Ihr Suchwort? Politik; Evaluierung; Hypothesenprüfung; Hypothesentest; Messfehler; Forschungsdesign; Statistische Analyse; Faktorenanalyse
Abstract	In social policy evaluations, the multiple testing problem occurs due to the many hypothesis tests that are typically conducted across multiple outcomes and subgroups, which can lead to spurious impact findings. This article discusses a framework for addressing this problem that balances Types I and II errors. The framework involves specifying confirmatory and exploratory analyses in study protocols, delineating confirmatory outcome domains, conducting t tests on composite domain outcomes, and applying multiplicity corrections to composites across domains to obtain summative impact evidence. The article presents statistical background and discusses multiplicity issues for subgroup analyses, designs with multiple treatments, and reporting. (Contains 4 tables and 5 notes.) (As Provided).
Anmerkungen	SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10