Building "e-rater"® Scoring Models Using Machine Learning Methods. Research Report. ETS RR-16-04

Autor/inn/en	Chen, Jing; Fife, James H.; Bejar, Isaac I.; Rupp, André A.
Titel	Building "e-rater"® Scoring Models Using Machine Learning Methods. Research Report. ETS RR-16-04
Quelle	In: ETS Research Report Series, (2016), (14 Seiten)Infoseite zur Zeitschrift PDF als Volltext kostenfreie Datei Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	2330-8516
Schlagwörter	Scoring; Models; Artificial Intelligence; Automation; Multiple Regression Analysis; Correlation; Essay Tests + Suchen Sie Ihr Suchwort? Bewertung; Analogiemodell; Künstliche Intelligenz; Korrelation; Schriftlicher Sprachgebrauch
Abstract	The "e-rater"® automated scoring engine used at Educational Testing Service (ETS) scores the writing quality of essays. In the current practice, e-rater scores are generated via a multiple linear regression (MLR) model as a linear combination of various features evaluated for each essay and human scores as the outcome variable. This study evaluates alternative scoring models based on several additional machine learning algorithms, including support vector machines (SVM), random forests (RF), and "k"-nearest neighbor regression (k-NN). The results suggest that models based on the SVM algorithm outperform MLR models in predicting human scores. Specifically, SVM-based models yielded the highest agreement between human and e-rater scores. Furthermore, compared with MLR, SVM-based models improved the agreement between human and e-rater scores at the ends of the score scale. In addition, the high correlation between SVM-based e-rater scores with external measures such as examinee's scores on the other parts of the test provided some validity evidence for SVM-based e-rater scores. Future research is encouraged to explore the generalizability of these findings. (As Provided).
Anmerkungen	Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/research/policy_research_reports/ets
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01