Understanding Rater Behavior in Observations of Special Education Teachers

Autor/inn/en	Johnson, Evelyn S.; Zheng, Yuzhu; Moylan, Laura A.; Crawford, Angela
Titel	Understanding Rater Behavior in Observations of Special Education Teachers
Quelle	(2020), (36 Seiten) PDF als Volltext (1); PDF als Volltext kostenfreie Datei (2) Verfügbarkeit
Zusatzinformation	Weitere Informationen
Sprache	englisch
Dokumenttyp	gedruckt; online; Monographie
Schlagwörter	Interrater Reliability; Teacher Evaluation; Special Education Teachers; Item Response Theory; Evaluation Criteria; Direct Instruction; Scoring Rubrics; Protocol Analysis; Teacher Effectiveness + Suchen Sie Ihr Suchwort? Interrater-Reliabilität; Teacher appraisal; Lehrerbeurteilung; Special education; Teacher; Teachers; Sonderpädagoge; Sonderpädagogik; Sonderschulwesen; Lehrer; Lehrerin; Lehrende; Item-Response-Theorie; Direct instructional procedues; Direct instructional approach; Unterrichtsverfahren; Scoring formulas; Auswertungsbogen; Effectiveness of teaching; Instructional effectiveness; Lehrerleistung; Unterrichtserfolg
Abstract	In this study, we investigated factors that influence raters' application of the scoring criteria of an Explicit Instruction (EI) observation protocol using many-faceted Rasch measurement (MFRM) and think aloud analysis. Specifically, we investigated the extent to which raters are able to consistently represent the scoring criteria in the EI rubric, how raters discriminate among levels of performance on each instructional element, and the consistency with which the raters applied evidence to support their scoring decisions. Video observations of instruction from 30 special education teachers across three states were collected. External raters (n = 15) observed and scored videos, providing rationales for item level scores. MFRM analyses showed that raters differed in their severity, but each individual rater was able to apply scoring criteria in a consistent manner. Think aloud analyses showed that raters varied in how they interpreted items and in the evidence they used to support their scores. Implications for research are discussed. [This paper was published in "Studies in Educational Evaluation" v64 Article 100827 2020.] (As Provided).
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01