Is the Area under Curve Appropriate for Evaluating the Fit of Psychometric Models?

Autor/inn/en	Han, Yuting; Zhang, Jihong; Jiang, Zhehan; Shi, Dexin
Titel	Is the Area under Curve Appropriate for Evaluating the Fit of Psychometric Models?
Quelle	In: Educational and Psychological Measurement, 83 (2023) 3, S.586-608 (23 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Shi, Dexin)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0013-1644
DOI	10.1177/00131644221098182
Schlagwörter	Goodness of Fit; Psychometrics; Error of Measurement; Item Response Theory; Bayesian Statistics; Comparative Analysis; Evaluation Methods + Suchen Sie Ihr Suchwort? Psychometry; Psychometrie; Messfehler; Item-Response-Theorie
Abstract	In the literature of modern psychometric modeling, mostly related to item response theory (IRT), the fit of model is evaluated through known indices, such as X[superscript 2], M2, and root mean square error of approximation (RMSEA) for absolute assessments as well as Akaike information criterion (AIC), consistent AIC (CAIC), and Bayesian information criterion (BIC) for relative comparisons. Recent developments show a merging trend of psychometric and machine learnings, yet there remains a gap in the model fit evaluation, specifically the use of the area under curve (AUC). This study focuses on the behaviors of AUC in fitting IRT models. Rounds of simulations were conducted to investigate AUC's appropriateness (e.g., power and Type I error rate) under various conditions. The results show that AUC possessed certain advantages under certain conditions such as high-dimensional structure with two-parameter logistic (2PL) and some three-parameter logistic (3PL) models, while disadvantages were also obvious when the true model is unidimensional. It cautions researchers about the dangers of using AUC solely in evaluating psychometric models. (As Provided).
Anmerkungen	SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01