A Diagnostic Tree Model for Adaptive Assessment of Complex Cognitive Processes Using Multidimensional Response Options

Autor/inn/en	Davison, Mark L.; Weiss, David J.; DeWeese, Joseph N.; Ersan, Ozge; Biancarosa, Gina; Kennedy, Patrick C.
Titel	A Diagnostic Tree Model for Adaptive Assessment of Complex Cognitive Processes Using Multidimensional Response Options
Quelle	In: Journal of Educational and Behavioral Statistics, 48 (2023) 6, S.914-941 (28 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Davison, Mark L.) ORCID (Biancarosa, Gina) ORCID (Kennedy, Patrick C.) Weitere Informationen
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1076-9986
DOI	10.3102/10769986231158301
Schlagwörter	Cognitive Processes; Diagnostic Tests; Measurement; Accuracy; Reading Comprehension; Computer Assisted Testing; Item Response Theory; Models + Suchen Sie Ihr Suchwort? Cognitive process; Kognitiver Prozess; Diagnostic test; Diagnostischer Test; Messverfahren; Leseverstehen; Item-Response-Theorie; Analogiemodell
Abstract	A tree model for diagnostic educational testing is described along with Monte Carlo simulations designed to evaluate measurement accuracy based on the model. The model is implemented in an assessment of inferential reading comprehension, the Multiple-Choice Online Causal Comprehension Assessment (MOCCA), through a sequential, multidimensional, computerized adaptive testing (CAT) strategy. Assessment of the first dimension, reading comprehension (RC), is based on the three-parameter logistic model. For diagnostic and intervention purposes, the second dimension, called process propensity (PP), is used to classify struggling students based on their pattern of incorrect responses. In the simulation studies, CAT item selection rules and stopping rules were varied to evaluate their effect on measurement accuracy along dimension RC and classification accuracy along dimension PP. For dimension RC, methods that improved accuracy tended to increase test length. For dimension PP, however, item selection and stopping rules increased classification accuracy without materially increasing test length. A small live-testing pilot study confirmed some of the findings of the simulation studies. Development of the assessment has been guided by psychometric theory, Monte Carlo simulation results, and a theory of instruction and diagnosis. (As Provided).
Anmerkungen	SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01