Automated Scoring of Junior and Senior High Essays Using Coh-Metrix Features: Implications for Large-Scale Language Testing

Autor/inn/en	Latifi, Syed; Gierl, Mark
Titel	Automated Scoring of Junior and Senior High Essays Using Coh-Metrix Features: Implications for Large-Scale Language Testing
Quelle	In: Language Testing, 38 (2021) 1, S.62-85 (24 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Latifi, Syed)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0265-5322
DOI	10.1177/0265532220929918
Schlagwörter	Writing Evaluation; Computer Assisted Testing; Scoring; Essays; Junior High School Students; Computational Linguistics; Test Validity; Test Reliability + Suchen Sie Ihr Suchwort? Bewertung; Essay; Aufsatzunterricht; Junior High Schools; Student; Students; Sekundarstufe I; Schüler; Schülerin; Linguistics; Computerlinguistik; Testvalidität; Testreliabilität
Abstract	An automated essay scoring (AES) program is a software system that uses techniques from corpus and computational linguistics and machine learning to grade essays. In this study, we aimed to describe and evaluate particular language features of Coh-Metrix for a novel AES program that would score junior and senior high school students' essays from their large-scale assessments. Specifically, we studied nine categories of Coh-Metrix features for developing prompt-specific AES scoring models for our sample. We developed the models by capitalizing on the nine features' informativeness as a function of dimensionality reduction. We used a three-staged scoring framework. The machine scores were validated against a "gold standard" of ratings, that is, those assigned by two human raters. The nine language features reliably captured the construct of the students' writing quality. We performed a secondary analysis to see how the scoring models performed in relation to other, already established AES systems, and there was no systematic pattern of scoring discrepancy. However, for essays with widely divergent human ratings, the scoring models were disadvantaged owing to the inherent unreliability of the human scores. (As Provided).
Anmerkungen	SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01