Impact of Differential Item Functioning on Group Score Reporting in the Context of Large-Scale Assessments

Autor/inn/en	Joo, Sean; Ali, Usama; Robin, Frederic; Shin, Hyo Jeong
Titel	Impact of Differential Item Functioning on Group Score Reporting in the Context of Large-Scale Assessments
Quelle	In: Large-scale Assessments in Education, 10 (2022), Artikel 18 (21 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Joo, Sean)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
DOI	10.1186/s40536-022-00135-7
Schlagwörter	Test Items; Item Response Theory; Scores; Student Evaluation; Sampling; Test Bias; Achievement Tests; Foreign Countries; Secondary School Students; International Assessment; Program for International Student Assessment + Suchen Sie Ihr Suchwort? Test content; Testaufgabe; Item-Response-Theorie; Schulnote; Studentische Bewertung; Testkritik; Achievement test; Achievement; Testing; Test; Tests; Leistungsbeurteilung; Leistungsüberprüfung; Leistung; Testdurchführung; Testen; Ausland; Sekundarschüler
Abstract	We investigated the potential impact of differential item functioning (DIF) on group-level mean and standard deviation estimates using empirical and simulated data in the context of large-scale assessment. For the empirical investigation, PISA 2018 cognitive domains (Reading, Mathematics, and Science) data were analyzed using Jackknife sampling to explore the impact of DIF on the country scores and their standard errors. We found that the countries that have a large number of DIF items tend to increase the difference of the country scores computed with and without the DIF adjustment. In addition, standard errors of the country score differences also increased with the number of DIF items. For the simulation study, we evaluated bias and root mean squared error (RMSE) of the group mean and standard deviation estimates using the multigroup item response theory (IRT) model to explore the extent to which DIF items create a bias of the group mean scores and how effectively the DIF adjustment corrects the bias under various conditions. We found that the DIF adjustment reduced the bias by 50% on average. The implications and limitations of the study are further discussed. (As Provided).
Anmerkungen	Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01