When Probabilities Are Not Enough--A Framework for Causal Explanations of Student Success Models

Autor/in	Cohausz, Lea
Titel	When Probabilities Are Not Enough--A Framework for Causal Explanations of Student Success Models
Quelle	In: Journal of Educational Data Mining, 14 (2022) 3, S.52-75 (24 Seiten) PDF als Volltext kostenfreie Datei Verfügbarkeit
Zusatzinformation	ORCID (Cohausz, Lea)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
Schlagwörter	Guidelines; Academic Achievement; Dropouts; Prediction; Identification; Intervention; Accuracy; Learning Analytics; Causal Models; Artificial Intelligence; Social Science Research; Evaluation Methods; Probability; Undergraduate Students; Student Characteristics; Foreign Countries; Age Differences; Gender Differences; Foreign Students; Germany + Suchen Sie Ihr Suchwort? Richtlinien; Schulleistung; Drop-out; Drop-outs; Dropout; Early leavers; Schulversagen; Vorhersage; Identifikation; Identifizierung; Kausalanalyse; Künstliche Intelligenz; Social scientific research; Sozialwissenschaftliche Forschung; Wahrscheinlichkeitsrechnung; Wahrscheinlichkeitstheorie; Ausland; Age; Difference; Age difference; Altersunterschied; Geschlechterkonflikt; Deutschland
Abstract	Student success and drop-out predictions have gained increased attention in recent years, connected to the hope that by identifying struggling students, it is possible to intervene and provide early help and design programs based on patterns discovered by the models. Though by now many models exist achieving remarkable accuracy-values, models outputting simple probabilities are not enough to achieve these ambitious goals. In this paper, we argue that they can be a first exploratory step of a pipeline aiming to be capable of reaching the mentioned goals. By using Explainable Artificial Intelligence (XAI) methods, such as SHAP and LIME, we can understand what features matter for the model and make the assumption that features important for successful models are also important in real life. By then additionally connecting this with an analysis of counterfactuals and a theory-driven causal analysis, we can begin to reasonably understand not just "if" a student will struggle but "why" and provide fitting help. We evaluate the pipeline on an artificial dataset to show that it can, indeed, recover complex causal mechanisms and on a real-life dataset showing the method's applicability. We further argue that collaborations with social scientists are mutually beneficial in this area but also discuss the potential negative effects of personal intervention systems and call for careful designs. (As Provided).
Anmerkungen	International Educational Data Mining. e-mail: jedm.editor@gmail.com; Web site: https://jedm.educationaldatamining.org/index.php/JEDM
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01