Detecting Measurement Disturbances in Rater-Mediated Assessments

Autor/inn/en	Wind, Stefanie A.; Schumacker, Randall E.
Titel	Detecting Measurement Disturbances in Rater-Mediated Assessments
Quelle	In: Educational Measurement: Issues and Practice, 36 (2017) 4, S.44-51 (8 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0731-1745
DOI	10.1111/emip.12164
Schlagwörter	Measurement; Testing Problems; Writing Tests; Performance Based Assessment; Graphs; Evaluators; Item Response Theory; Goodness of Fit + Suchen Sie Ihr Suchwort? Messverfahren; Writing test; Schreibtest; Leistungsermittlung; Grafische Darstellung; Item-Response-Theorie
Abstract	The term measurement disturbance has been used to describe systematic conditions that affect a measurement process, resulting in a compromised interpretation of person or item estimates. Measurement disturbances have been discussed in relation to systematic response patterns associated with items and persons, such as start-up, plodding, boredom, or fatigue. An understanding of the different types of measurement disturbances can lead to a more complete understanding of persons or items in terms of the construct being measured. Although measurement disturbances have been explored in several contexts, they have not been explicitly considered in the context of performance assessments. The purpose of this study is to illustrate the use of graphical methods to explore measurement disturbances related to raters within the context of a writing assessment. Graphical displays that illustrate the alignment between expected and empirical rater response functions are considered as they relate to indicators of rating quality based on the Rasch model. Results suggest that graphical displays can be used to identify measurement disturbances for raters related to specific ranges of student achievement that suggest potential rater bias. Further, results highlight the added diagnostic value of graphical displays for detecting measurement disturbances that are not captured using Rasch model-data fit statistics. (As Provided).
Anmerkungen	Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: cs-journals@wiley.com; Web site: http://www.wiley.com/WileyCDA
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01