Using Large Language Models to Provide Formative Feedback in Intelligent Textbooks

Autor/inn/en	Morris, Wesley; Crossley, Scott; Holmes, Langdon; Ou, Chaohua; McNamara, Danielle; Dascalu, Mihai
Titel	Using Large Language Models to Provide Formative Feedback in Intelligent Textbooks
Quelle	(2023), (7 Seiten) PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Morris, Wesley) ORCID (Crossley, Scott) ORCID (Holmes, Langdon) ORCID (Ou, Chaohua) ORCID (McNamara, Danielle) ORCID (Dascalu, Mihai) Weitere Informationen
Sprache	englisch
Dokumenttyp	gedruckt; online; Monographie
Schlagwörter	Textbooks; Electronic Publishing; Feedback (Response); Formative Evaluation; Scores; Comparative Analysis; Scoring Rubrics; Factor Analysis; Reading Comprehension; Computational Linguistics; Intelligent Tutoring Systems; Writing Evaluation; High School Students; College Students; Adult Education + Suchen Sie Ihr Suchwort? Textbook; Text book; Schulbuch; Lehrbuch; Elektronisches Publizieren; Scoring formulas; Auswertungsbogen; Faktorenanalyse; Leseverstehen; Linguistics; Computerlinguistik; Intelligentes Tutorsystem; High school; High schools; Student; Students; Oberschule; Schüler; Schülerin; Studentin; Collegestudent; Adult; Adults; Education; Adult basic education; Adult training; Erwachsenenbildung
Abstract	As intelligent textbooks become more ubiquitous in classrooms and educational settings, the need arises to automatically provide formative feedback to written responses provided by students in response to readings. This study develops models to automatically provide feedback to student summaries written at the end of intelligent textbook sections. The study builds on Botarleanu et al. (2022), who used the Longformer Large Language Model, a transformer Neural Network, to build a summary grading model. Their model explains around 55% of holistic summary score variance when compared to scores assigned by human raters on an analytic rubric. This study uses a principal component analysis to distill scores from the analytic rubric into two principal components -- content and wording. When training the models on the summaries and the sources using these principal components, we explained 79% and 66% of the score variance for content and wording, respectively. The developed models are freely available on HuggingFace and will allow formative feedback to users of intelligent textbooks to assess reading comprehension through summarization in real-time. The models can also be used for other summarization applications in learning systems. [This paper was published in: "AIED 2023, CCIS 1831," edited by N. Wang et al., Springer Nature Switzerland, 2023, pp. 484-489.] (As Provided).
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01