Validating Arguments for Observational Instruments: Attending to Multiple Sources of Variation

Autor/inn/en	Hill, Heather C.; Charalambous, Charalambos Y.; Blazar, David; McGinn, Daniel; Kraft, Matthew A.; Beisiegel, Mary; Humez, Andrea; Litke, Erica; Lynch, Kathleen
Titel	Validating Arguments for Observational Instruments: Attending to Multiple Sources of Variation
Quelle	In: Educational Assessment, 17 (2012) 2-3, S.88-106 (19 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	Weitere Informationen
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1062-7197
DOI	10.1080/10627197.2012.715019
Schlagwörter	Validity; Classroom Observation Techniques; Measures (Individuals); Teacher Effectiveness; Mathematics Instruction; Middle School Teachers; Mathematics Teachers; Teacher Evaluation; Scores; Interrater Reliability; Generalization; Accuracy; Correlation + Suchen Sie Ihr Suchwort? Gültigkeit; Messdaten; Effectiveness of teaching; Instructional effectiveness; Lehrerleistung; Unterrichtserfolg; Mathematics lessons; Mathematikunterricht; Middle school; Middle schools; Teacher; Teachers; Mittelschule; Mittelstufenschule; Lehrer; Lehrerin; Lehrende; Mathematics; Mathematik; Teacher appraisal; Lehrerbeurteilung; Interrater-Reliabilität; Korrelation
Abstract	Measurement scholars have recently constructed validity arguments in support of a variety of educational assessments, including classroom observation instruments. In this article, we note that users must examine the robustness of validity arguments to variation in the implementation of these instruments. We illustrate how such an analysis might be used to assess a validity argument constructed for the Mathematical Quality of Instruction instrument, focusing in particular on the effects of varying the rater pool, subject matter content, observation procedure, and district context. Variation in the subject matter content of lessons did not affect rater agreement with master scores, but the evaluation of other portions of the validity argument varied according to the composition of the rater pool, observation procedure, and district context. These results demonstrate the need for conducting such analyses, especially for classroom observation instruments that are subject to multiple sources of variation. (Contains 7 tables and 5 footnotes.) (As Provided).
Anmerkungen	Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10