Optimizing the Length of Computerized Adaptive Testing for the Force Concept Inventory

Autor/inn/en	Yasuda, Jun-ichiro; Mae, Naohiro; Hull, Michael M.; Taniguchi, Masa-aki
Titel	Optimizing the Length of Computerized Adaptive Testing for the Force Concept Inventory
Quelle	In: Physical Review Physics Education Research, 17 (2021) 1, Artikel 010115 (15 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Yasuda, Jun-ichiro) ORCID (Hull, Michael M.)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	2469-9896
Schlagwörter	Foreign Countries; College Students; Student Evaluation; Computer Assisted Testing; Adaptive Testing; Science Tests; Mechanics (Physics); Scientific Concepts; Test Length; Accuracy; Test Bias; Error of Measurement; Item Response Theory; Monte Carlo Methods; Class Size; Japan + Suchen Sie Ihr Suchwort? Ausland; Collegestudent; Schulnote; Studentische Bewertung; Mechanik; Testkritik; Messfehler; Item-Response-Theorie; Monte-Carlo-Methode; Klassengröße
Abstract	As a method to shorten the test time of the Force Concept Inventory (FCI), we suggest the use of computerized adaptive testing (CAT). CAT is the process of administering a test on a computer, with items (i.e., questions) selected based upon the responses of the examinee to prior items. In so doing, the test length can be significantly shortened. As a step to develop a CAT-based version of the FCI (FCI-CAT), we examined the optimal test length of the FCI-CAT such that accuracy and precision [which we measure in terms of bias, standard error, and root-mean-square error (RMSE)] of Cohen's d would be comparable to that of the full FCI for a given class size. First, we estimated the item parameters of the FCI items based on the three-parameter logistic model of item response theory, which are used in the algorithm of CAT. For this estimation, we used 2882 responses of Japanese university students. Second, we conducted a Monte Carlo simulation to analyze how the bias, standard error, and RMSE of Cohen's "d" depend upon the test length. Third, we conducted a "post hoc" simulation to examine the consistency of the Monte Carlo results with what would have been obtained using empirical responses. For this comparison, we used 86 pairs of pre- and post- test responses of Japanese university students. As a result, we found that for a class size of 40, we may reduce the test length of the FCI-CAT to 15-19 items, thereby reducing the test time of the FCI to 50%-63%, with an accompanying decrease in accuracy and precision of only 5%-10%. The results of the Monte Carlo study and the "post hoc" simulation were consistent, which supports the adequacy of our Monte Carlo study and its relevance in terms of administering the FCI-CAT in real classrooms. (As Provided).
Anmerkungen	American Physical Society. One Physics Ellipse 4th Floor, College Park, MD 20740-3844. Tel: 301-209-3200; Fax: 301-209-0865; e-mail: assocpub@aps.org; Web site: http://prst-per.aps.org
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01