Item Difficulty Estimation: An Auspicious Collaboration between Data and Judgment

Autor/inn/en	Wauters, Kelly; Desmet, Piet; Van Den Noortgate, Wim
Titel	Item Difficulty Estimation: An Auspicious Collaboration between Data and Judgment
Quelle	In: Computers & Education, 58 (2012) 4, S.1183-1193 (11 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0360-1315
DOI	10.1016/j.compedu.2011.11.020
Schlagwörter	Expertise; Electronic Learning; Feedback (Response); Sample Size; Difficulty Level; Item Response Theory; Computation; Educational Environment; Test Items; Comparative Analysis; Bayesian Statistics; Cooperation + Suchen Sie Ihr Suchwort? Expert appraisal; Schwierigkeitsgrad; Item-Response-Theorie; Lernumgebung; Pädagogische Umwelt; Schulumwelt; Test content; Testaufgabe; Co-operation; Kooperation
Abstract	The evolution from static to dynamic electronic learning environments has stimulated the research on adaptive item sequencing. A prerequisite for adaptive item sequencing, in which the difficulty of the item is constantly matched to the ability level of the learner, is to have items with a known difficulty level. The difficulty level can be estimated by means of the item response theory (IRT). However, the requirement of a large sample size for calibrating items based on IRT models is not easily met in many practical learning situations. The aim of this paper is to search for relatively simple and fast alternative estimation methods and to review the accuracy of these methods as compared to IRT-based calibration in one single setting, and this for various sample sizes. Using real data, six alternative estimation methods are compared next to IRT-based calibration: proportion correct, learner feedback, expert rating, one-to-many comparison (learner), one-to-many comparison (expert) and the Elo rating system. Results indicate that proportion correct has the strongest relation with IRT-based difficulty estimates, followed by learner feedback, the Elo rating system, expert rating and finally one-to-many comparison. Learner feedback and one-to-many comparison (learner) provide stable estimates even with a small sample size. IRT, proportion correct and the Elo rating system provide reliable estimates, especially with a sample size of 200-250 learners. The alternative estimation methods can be utilized for adaptive item sequencing when IRT-based calibration does not yet provide reliable estimates or can be used as a prior in a Bayesian estimation method. (Contains 5 tables and 1 figure.) (As Provided).
Anmerkungen	Elsevier. 6277 Sea Harbor Drive, Orlando, FL 32887-4800. Tel: 877-839-7126; Tel: 407-345-4020; Fax: 407-363-1354; e-mail: usjcs@elsevier.com; Web site: http://www.elsevier.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10