IRR (Inter-Rater Reliability) of a COP (Classroom Observation Protocol)--A Critical Appraisal

Autor/inn/en	Rui, Ning; Feldman, Jill M.
Titel	IRR (Inter-Rater Reliability) of a COP (Classroom Observation Protocol)--A Critical Appraisal
Quelle	(2012), S.305-315 (11 Seiten)Infoseite zur Zeitschrift PDF als Volltext kostenfreie Datei Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1548-6613
Schlagwörter	Classroom Observation Techniques; Interrater Reliability; Correlation; Psychometrics; Physical Environment; Classroom Environment; Probability; Federal Programs; Learner Engagement; Criticism; Item Analysis; Program Evaluation; Literacy; Poverty; Urban Areas; Reading Programs + Suchen Sie Ihr Suchwort? Interrater-Reliabilität; Korrelation; Psychometry; Psychometrie; Natürliche Umwelt; Klassenklima; Unterrichtsklima; Wahrscheinlichkeitsrechnung; Wahrscheinlichkeitstheorie; Kritik; Itemanalyse; Programme evaluation; Programmevaluation; Alphabetisierung; Schreib- und Lesefähigkeit; Armut; Urban area; Stadtregion
Abstract	Notwithstanding broad utility of COPs (classroom observation protocols), there has been limited documentation of the psychometric properties of even the most popular COPs. This study attempted to fill this void by closely examining the item and domain-level IRR (inter-rater reliability) of a COP that was used in a federally funded striving readers program. A combination of reliability measures (e.g., joint-probability of agreement, Cohen's kappa, polychoric correlation and intra-class correlation coefficients) was selected dependent upon which were appropriate given the scale of each item set. Results indicate that most items in physical environment, cognitive demand and students' class engagement can be assessed with moderate reliability. Items in classroom climate and instructional modes yielded mixed estimates. Recommendations were provided for possible improvement of similar instruments. (Contains 12 tables.) (As Provided).
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10