Indices of Subscore Utility for Individuals and Subgroups Based on Multivariate Generalizability Theory

Autor/inn/en	Raymond, Mark R.; Jiang, Zhehan
Titel	Indices of Subscore Utility for Individuals and Subgroups Based on Multivariate Generalizability Theory
Quelle	In: Educational and Psychological Measurement, 80 (2020) 1, S.67-90 (24 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Raymond, Mark R.) ORCID (Jiang, Zhehan)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0013-1644
DOI	10.1177/0013164419846936
Schlagwörter	Generalizability Theory; Multivariate Analysis; Scores; Reliability; Profiles + Suchen Sie Ihr Suchwort? Multivariate Analyse; Reliabilität; Charakterisierung; Profilanalyse
Abstract	Conventional methods for evaluating the utility of subscores rely on traditional indices of reliability and on correlations among subscores. One limitation of correlational methods is that they do not explicitly consider variation in subtest means. An exception is an index of score profile reliability designated as [G], which quantifies the ratio of true score profile variance to observed score profile variance. [G] has been shown to be more sensitive than correlational methods to group differences in score profile utility. However, it is a group average, representing the expected value over a population of examinees. Just as score reliability varies across individuals and subgroups, one can expect that the reliability of score profiles will vary across examinees. This article proposes two conditional indices of score profile utility grounded in multivariate generalizability theory. The first is based on the ratio of observed profile variance to the profile variance that can be attributed to random error. The second quantifies the proportion of observed variability in a score profile that can be attributed to true score profile variance. The article describes the indices, illustrates their use with two empirical examples, and evaluates their properties with simulated data. The results suggest that the proposed estimators of profile error variance are consistent with the known error in simulated score profiles and that they provide information beyond that provided by traditional measures of subscore utility. The simulation study suggests that artificially large values of the indices could occur for about 5% to 8% of examinees. The article concludes by suggesting possible applications of the indices and discusses avenues for further research. (As Provided).
Anmerkungen	SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01