Using Anchoring Vignettes to Calibrate Teachers' Self-Assessment of Teaching

Autor/inn/en	Yuan, Kun; Engberg, John; Kaufman, Julia; Hamilton, Laura; Hill, Heather; Umland, Kristin; McCaffrey, Daniel
Institution	Society for Research on Educational Effectiveness (SREE)
Titel	Using Anchoring Vignettes to Calibrate Teachers' Self-Assessment of Teaching
Quelle	(2014), (6 Seiten) PDF als Volltext kostenfreie Datei Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Monographie
Schlagwörter	Self Evaluation (Individuals); Mathematics Teachers; Vignettes; Online Surveys; Teacher Surveys; Validity; Mathematics Instruction; Correlation; Comparative Analysis; Teacher Evaluation; Value Added Models; Student Surveys; Observation; Elementary School Teachers; Middle School Teachers; Secondary School Teachers; Grade 4; Grade 5; Grade 6; Grade 7; Grade 8; Grade 9; Statistical Analysis + Suchen Sie Ihr Suchwort? Mathematics; Teacher; Teachers; Mathematik; Lehrer; Lehrerin; Lehrende; Gültigkeit; Mathematics lessons; Mathematikunterricht; Korrelation; Teacher appraisal; Lehrerbeurteilung; Schülerbefragung; Beobachtung; Elementary school; Grundschule; Volksschule; Middle school; Middle schools; Mittelschule; Mittelstufenschule; School year 04; 4. Schuljahr; Schuljahr 04; School year 05; 5. Schuljahr; Schuljahr 05; School year 06; 6. Schuljahr; Schuljahr 06; School year 07; 7. Schuljahr; Schuljahr 07; School year 08; 8. Schuljahr; Schuljahr 08; School year 09; 9. Schuljahr; Schuljahr 09; Statistische Analyse
Abstract	High-quality measures of instructional practice are essential for research and evaluation of innovative instructional policies and programs, as well as for providing feedback to teachers and administrators. In this study, the authors examined whether using anchoring vignettes in web-based surveys improved the validity of teachers' self-assessments of their mathematics instruction. To investigate validity, they compared correlations between teachers' self-ratings and other measures of teaching including teachers' value-added scores, student surveys, and observation ratings of instruction before and after calibration to examine whether calibration improves the correlation between teachers' self-ratings and other teaching measures. Data came from 61 mathematics teachers in grades 4-9 participating in the Bill & Melinda Gates Foundation's Measures of Effective Teaching Extension project. The sample was roughly evenly distributed between elementary (grades 4-5) and middle school (grades 6-8) teachers, with only three percent teaching 9th grade. The authors used both non-parametric and parametric methods to calibrate teachers' self-ratings. Preliminary findings suggest that anchoring vignettes represent a promising innovation for measuring teachers' instruction through survey self-reports. Specifically, the authors found: (1) Teachers' survey responses that are calibrated through the use of anchoring vignettes have increased variation compared to teachers' raw survey responses, particularly for the cognitive challenge dimension; (2) Teachers' calibrated survey responses regarding mathematical vocabulary and cognitively challenging tasks are more strongly correlated with the composite measure of teacher performance compared to raw survey responses; and (3) If teachers gave their self-rating after rating the vignettes, rather than before, the entire collection of calibrated self-ratings are significantly correlated with the composite performance measure (p<0.05). (ERIC).
Anmerkungen	Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail: inquiries@sree.org; Web site: http://www.sree.org
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01