A Method for the Comparison of Item Selection Rules in Computerized Adaptive Testing

Autor/inn/en	Barrada, Juan Ramon; Olea, Julio; Ponsoda, Vicente; Abad, Francisco Jose
Titel	A Method for the Comparison of Item Selection Rules in Computerized Adaptive Testing
Quelle	In: Applied Psychological Measurement, 34 (2010) 6, S.438-452 (15 Seiten) PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0146-6216
DOI	10.1177/0146621610370152
Schlagwörter	Test Items; Simulation; Adaptive Testing; Item Analysis; Test Construction; Computer Assisted Testing; Comparative Analysis; Selection; Equations (Mathematics) + Suchen Sie Ihr Suchwort? Test content; Testaufgabe; Simulation program; Simulationsprogramm; Itemanalyse; Testaufbau; Auslese; Equations; Mathematics; Gleichungslehre
Abstract	In a typical study comparing the relative efficiency of two item selection rules in computerized adaptive testing, the common result is that they simultaneously differ in accuracy and security, making it difficult to reach a conclusion on which is the more appropriate rule. This study proposes a strategy to conduct a global comparison of two or more selection rules. A plot showing the performance of each selection rule for several maximum exposure rates is obtained and the whole plot is compared with other rule plots. The strategy was applied in a simulation study with fixed-length CATs for the comparison of six item selection rules: the point Fisher information, Fisher information weighted by likelihood, Kullback-Leibler weighted by likelihood, maximum information stratification with blocking, progressive and proportional methods. Our results show that there is no optimal rule for any overlap value or root mean square error (RMSE). The fact that a rule, for a given level of overlap, has lower RMSE than another does not imply that this pattern holds for another overlap rate. A fair comparison of the rules requires extensive manipulation of the maximum exposure rates. The best methods were the Kullback-Leibler weighted by likelihood, the proportional method, and the maximum information stratification method with blocking. (Contains 1 table and 3 figures.) (As Provided).
Anmerkungen	SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10