Validity Arguments for AI-Based Automated Scores: Essay Scoring as an Illustration

Autor/inn/en	Ferrara, Steve; Qunbar, Saed
Titel	Validity Arguments for AI-Based Automated Scores: Essay Scoring as an Illustration
Quelle	In: Journal of Educational Measurement, 59 (2022) 3, S.288-313 (26 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0022-0655
DOI	10.1111/jedm.12333
Schlagwörter	Artificial Intelligence; Scoring; Essays; Automation; Validity; Accuracy + Suchen Sie Ihr Suchwort? Künstliche Intelligenz; Bewertung; Essay; Aufsatzunterricht; Gültigkeit
Abstract	In this article, we argue that automated scoring engines should be transparent and construct relevant--that is, as much as is currently feasible. Many current automated scoring engines cannot achieve high degrees of scoring accuracy without allowing in some features that may not be easily explained and understood and may not be obviously and directly relevant to the target assessment construct. We address the current limitations on evidence and validity arguments for scores from automated scoring engines from the points of view of the Standards for Educational and Psychological Testing (i.e., construct relevance, construct representation, and fairness) and emerging principles in Artificial Intelligence (e.g., explainable AI, an examinee's right to explanations, and principled AI). We illustrate these concepts and arguments for automated essay scores. (As Provided).
Anmerkungen	Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01