Developing a Machine-Supported Coding System for Constructed-Response Items in PISA. Research Report. ETS RR-17-47

Autor/inn/en	Yamamoto, Kentaro; He, Qiwei; Shin, Hyo Jeong; von Davier, Mattias
Titel	Developing a Machine-Supported Coding System for Constructed-Response Items in PISA. Research Report. ETS RR-17-47
Quelle	In: ETS Research Report Series, (2017), (17 Seiten)Infoseite zur Zeitschrift PDF als Volltext kostenfreie Datei Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	2330-8516
Schlagwörter	Foreign Countries; Achievement Tests; International Assessment; Secondary School Students; Test Items; Responses; Scoring; Computer Assisted Testing; Test Scoring Machines; Reading Tests; Interrater Reliability; Accuracy; Efficiency; Australia; China; France; Germany; Japan; South Korea; Netherlands; Program for International Student Assessment + Suchen Sie Ihr Suchwort? Ausland; Achievement test; Achievement; Testing; Test; Tests; Leistungsbeurteilung; Leistungsüberprüfung; Leistung; Testdurchführung; Testen; Sekundarschüler; Test content; Testaufgabe; Bewertung; Lesetest; Interrater-Reliabilität; Effectiveness; Effektivität; Wirkungsgrad; Australien; Frankreich; Deutschland; Korea; Republik; Niederlande
Abstract	Approximately a third of the Programme for International Student Assessment (PISA) items in the core domains (math, reading, and science) are constructed-response items and require human coding (scoring). This process is time-consuming, expensive, and prone to error as often (a) humans code inconsistently, and (b) coding reliability in international large-scale assessments varies across items and countries. The shift in PISA 2015 from paper- to computer-based assessment digitized all responses and associated coding, thus providing opportunities to introduce technology and analytical methods to improve data processing and analyses in future cycles. In this computer-based setting, this research explains the framework and approach for improving the accuracy and efficiency of the coding process in constructed-response items for future PISA cycles. In the pilot study associated with this research, we demonstrate how the proposed machine-supported coding system can be utilized using sample reading items from the PISA 2015 cycle. (As Provided).
Anmerkungen	Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/research/policy_research_reports/ets
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01