Replicating Studying Adaptive Learning Efficacy Using Propensity Score Matching and Inverse Probability of Treatment Weighting

Autor/inn/en	Mojarad, Shirin; Baker, Ryan S.; Essa, Alfred; Stalzer, Steve
Titel	Replicating Studying Adaptive Learning Efficacy Using Propensity Score Matching and Inverse Probability of Treatment Weighting
Quelle	In: Journal of Interactive Learning Research, 32 (2021) 3, S.169-203 (35 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1093-023X
Schlagwörter	Intelligent Tutoring Systems; Replication (Evaluation); Probability; Quasiexperimental Design; Statistical Analysis; Outcomes of Education; Two Year College Students; Community Colleges; Mathematics Instruction; Remedial Mathematics; College Mathematics; Instructional Effectiveness; Educational Research; ACCUPLACER + Suchen Sie Ihr Suchwort? Intelligentes Tutorsystem; Wahrscheinlichkeitsrechnung; Wahrscheinlichkeitstheorie; Statistische Analyse; Lernleistung; Schulerfolg; Community college; Community College; Mathematics lessons; Mathematikunterricht; Unterrichtserfolg; Bildungsforschung; Pädagogische Forschung
Abstract	Despite the importance of replication, it remains rare in the interactive learning research community. In this paper, we attempt to replicate recent quasi-experimental results suggesting that the ALEKS intelligent tutoring system is effective at improving student course outcomes in higher education (Mojarad et al., 2018). In this paper, we conduct a near replication, collecting a new data set of higher education students using ALEKS, at the same university as in that earlier paper. We investigate the robustness of the results found to the choice of quasi-experimental methodology. In the earlier work, the popular propensity score matching algorithm was used; a recent methodological paper challenges this method (King & Nielsen, 2019). We therefore investigate the impact of using another matching algorithm, inverse probability of treatment weighting (IPTW) instead of propensity score matching, and compare the results obtained by these two methods. We replicate the previous study: ALEKS is statistically significantly associated with better student course outcomes. The use of IPTW leads to the same qualitative result as PSM, but IPTW achieves superior matching, suggesting that this method should be preferred for future quasi-experiments within the interactive learning research community. (As Provided).
Anmerkungen	Association for the Advancement of Computing in Education. P.O. Box 719, Waynesville, NC 28786. Tel: 828-246-9558; Fax: 828-246-9557; e-mail: info@aace.org; Web site: http://www.aace.org
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01