Embedding a Proof-of-Concept Test in an At-Scale National Policy Experiment: Greater Policy Learning but at What Cost to Statistical Power? The Social Security Administration's Benefit Offset National Demonstration (BOND)

Autor/inn/en	Bell, Stephen H.; Stapleton, David C.; Wood, Michelle; Gubits, Daniel
Titel	Embedding a Proof-of-Concept Test in an At-Scale National Policy Experiment: Greater Policy Learning but at What Cost to Statistical Power? The Social Security Administration's Benefit Offset National Demonstration (BOND)
Quelle	In: American Journal of Evaluation, 44 (2023) 1, S.118-132 (15 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Bell, Stephen H.) ORCID (Stapleton, David C.)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1098-2140
DOI	10.1177/10982140211006786
Schlagwörter	Public Policy; Policy Formation; Federal Programs; Social Services; Disabilities; Insurance; Research Design; Statistical Analysis; Costs; Volunteers; Program Effectiveness; Evidence; Program Evaluation + Suchen Sie Ihr Suchwort? Öffentliche Ordnung; Politische Betätigung; Social service; Soziale Dienstleistung; Soziale Dienste; Handicap; Behinderung; Forschungsdesign; Statistische Analyse; Cost; Kosten; Freiwilliger; Evidenz; Programme evaluation; Programmevaluation
Abstract	A randomized experiment that measures the impact of a social policy in a sample of the population reveals whether the policy will work on average with universal application. An experiment that includes only the subset of the population that volunteers for the intervention generates narrower "proof-of-concept" evidence of whether the policy can work for motivated individuals. Both forms of learning carry value, yet evaluations rarely combine the two designs. The U.S. Social Security Administration conducted an exception, the Benefit Offset National Demonstration (BOND). This article uses BOND to examine the statistical power implications and potential gains in policy learning--relative to costs--from combining volunteer and population-representative experiments. It finds that minimum detectable effects of volunteer experiments rise little when one adds a population-representative experiment, but those of a population-representative experiment double or quadruple with the addition of a volunteer experiment. (As Provided).
Anmerkungen	SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01