Using Big Data to Sharpen Design-Based Inference in A/B Tests [Konferenzbericht] Paper presented at the International Conference on Educational Data Mining (EDM) (11th, Raleigh, NC, Jul 16-20, 2018).

Autor/inn/en	Sales, Adam C.; Botelho, Anthony; Patikorn, Thanaporn; Heffernan, Neil T.
Titel	Using Big Data to Sharpen Design-Based Inference in A/B Tests [Konferenzbericht] Paper presented at the International Conference on Educational Data Mining (EDM) (11th, Raleigh, NC, Jul 16-20, 2018).
Quelle	(2018), (7 Seiten) PDF als Volltext kostenfreie Datei Verfügbarkeit
Zusatzinformation	Weitere Informationen
Sprache	englisch
Dokumenttyp	gedruckt; online; Monographie
Schlagwörter	Courseware; Data Analysis; Causal Models; Prediction; Outcomes of Education; Evaluation Methods; Mastery Learning; Skill Development; Intelligent Tutoring Systems; Statistical Bias; Randomized Controlled Trials; Artificial Intelligence + Suchen Sie Ihr Suchwort? Lernsoftware; Auswertung; Kausalanalyse; Vorhersage; Lernleistung; Schulerfolg; Kompetenzentwicklung; Qualifikationsentwicklung; Intelligentes Tutorsystem; Künstliche Intelligenz
Abstract	Randomized A/B tests in educational software are not run in a vacuum: often, reams of historical data are available alongside the data from a randomized trial. This paper proposes a method to use this historical data--often highdimensional and longitudinal--to improve causal estimates from A/B tests. The method proceeds in two steps: first, fit a machine learning model to the historical data predicting students' outcomes as a function of their covariates. Then, use that model to predict the outcomes of the randomized students in the A/B test. Finally, use design-based methods to estimate the treatment effect in the A/B test, using prediction errors in place of outcomes. This method retains all of the advantages of design-based inference, while, under certain conditions, yielding more precise estimators. This paper will give a theoretical condition under which the method improves statistical precision, and demonstrates it using a deep learning algorithm to help estimate effects in a set of experiments run inside ASSISTments. [For the full proceedings, see ED593090.] (As Provided).
Anmerkungen	International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: http://www.educationaldatamining.org
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01