Estimating Item Difficulty with Comparative Judgments. Research Report. ETS RR-14-39

Autor/inn/en	Attali, Yigal; Saldivia, Luis; Jackson, Carol; Schuppan, Fred; Wanamaker, Wilbur
Titel	Estimating Item Difficulty with Comparative Judgments. Research Report. ETS RR-14-39
Quelle	In: ETS Research Report Series, (2014), (10 Seiten)Infoseite zur Zeitschrift PDF als Volltext kostenfreie Datei Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	2330-8516
Schlagwörter	Test Items; Difficulty Level; Comparative Analysis; College Entrance Examinations; Mathematics Tests; Scaling; Interrater Reliability; Simulation; Correlation; SAT (College Admission Test) + Suchen Sie Ihr Suchwort? Test content; Testaufgabe; Schwierigkeitsgrad; Aufnahmeprüfung; Scale construction; Skalenkonstruktion; Interrater-Reliabilität; Simulation program; Simulationsprogramm; Korrelation
Abstract	Previous investigations of the ability of content experts and test developers to estimate item difficulty have, for themost part, produced disappointing results. These investigations were based on a noncomparative method of independently rating the difficulty of items. In this article, we argue that, by eliciting comparative judgments of difficulty, judges can more accurately estimate item difficulties. In this study, judges from different backgrounds rank ordered the difficulty of "SAT"® mathematics items in sets of 7 items. Results showed that judges are reasonably successful in rank ordering several items in terms of difficulty, with little variability across judges and content areas. Simulations of a possible implementation of comparative judgments for difficulty estimation show that it is possible to achieve high correlations between true and estimated difficulties with relatively few comparisons. Implications of these results for the test development process are discussed. (As Provided).
Anmerkungen	Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/research/policy_research_reports/ets
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01