Uncovering Predictors of Disagreement: Ensuring the Quality of Expert Ratings

Autor/inn/en	Hoth, Jessica; Schwarz, Björn; Kaiser, Gabriele; Busse, Andreas; König, Johannes; Blömeke, Sigrid
Titel	Uncovering Predictors of Disagreement: Ensuring the Quality of Expert Ratings
Quelle	In: ZDM: The International Journal on Mathematics Education, 48 (2016) 1-2, S.83-95 (13 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1863-9690
DOI	10.1007/s11858-016-0758-z
Schlagwörter	Rating Scales; Predictor Variables; Expertise; Followup Studies; Teacher Competencies; Video Technology + Suchen Sie Ihr Suchwort? Rating-Skala; Prädiktor; Expert appraisal; Follow-up studies; Kontaktstudium; Lehrkunst
Abstract	Rating scales are a popular item format used in many types of assessments. Yet, defining which rating is correct often represents a challenge. Using expert ratings as benchmarks is one approach to ensuring the quality of a rating instrument. In this paper, such expert ratings are analyzed in detail taking a video-based test instrument of teachers' professional competencies from a follow-up study to TEDS-M (the so-called TEDS-FU study) as an example. The paper focuses on those items that did NOT reach sufficient consensus among the experts and analyzes in depth their features by coding the experts' comments on those items and additionally considering their rating outcome. The results revealed that the item-wording and the composition of the group of experts strengthened or weakened agreement among the experts. (As Provided).
Anmerkungen	Springer. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: service-ny@springer.com; Web site: http://www.springerlink.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01