Measures of Agreement to Assess Attribute-Level Classification Accuracy and Consistency for Cognitive Diagnostic Assessments

Autor/inn/en	Johnson, Matthew S.; Sinharay, Sandip
Titel	Measures of Agreement to Assess Attribute-Level Classification Accuracy and Consistency for Cognitive Diagnostic Assessments
Quelle	In: Journal of Educational Measurement, 55 (2018) 4, S.635-664 (30 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0022-0655
DOI	10.1111/jedm.12196
Schlagwörter	Classification; Accuracy; Test Reliability; Diagnostic Tests; Cognitive Tests + Suchen Sie Ihr Suchwort? Classification system; Klassifikation; Klassifikationssystem; Testreliabilität; Diagnostic test; Diagnostischer Test; Kognitiver Fähigkeitstest
Abstract	One of the proposed uses of cognitive diagnostic assessments is to classify the examinees as either masters or nonmasters on each of a number of skills being assessed. As with any test, it is important to report the quality of these binary classifications with measures of their reliability. Cui et al. and Wang et al. have suggested reliability measures that can be calculated from the model parameters of cognitive diagnosis models; these previously suggested indices are measures of agreement between either the estimated and true mastery classifications, or between the estimated classifications from parallel assessments. This article discusses the limitations of these existing methods and suggests the use of other measures of agreement. A simulation study demonstrates that the proposed measures are related to factors that would be expected to be associated with reliability; for example, reliability increases with variability in the population and with item discrimination, whereas the previously suggested measures do not show the same pattern. A real data example is also included. (As Provided).
Anmerkungen	Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: cs-journals@wiley.com; Web site: http://www.wiley.com/WileyCDA
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01