Literaturnachweis - Detailanzeige
| Autor/in | Saenz, David Arron |
|---|---|
| Titel | Inter-Rater Reliability in Comprehensive Examination Scoring: The Case for Consistent and Collaborative Rater Training and Calibration |
| Quelle | (2023), (100 Seiten)
PDF als Volltext kostenfreie Datei |
| Sprache | englisch |
| Dokumenttyp | gedruckt; online; Monographie |
| Schlagwörter | Interrater Reliability; Scoring; Training; Scoring Rubrics; Graduate Students; Student Evaluation; Scores; Testing; Action Research |
| Abstract | There is a vast body of literature documenting the positive impacts that rater training and calibration sessions have on inter-rater reliability as research indicates several factors including frequency and timing play crucial roles towards ensuring inter-rater reliability. Additionally, increasing amounts research indicate possible links in rater-reliability and grade inflation pertaining to faculty status and length of employment. This research study analyzed the impact that consistent and collaborative training and rubric calibration, or the lack thereof, has on inter-rater reliability. Additionally, this study investigated whether faculty status and faculty length of employment has an influence on scoring reliability. Finally, participatory action research was planned to assist in the creation of a formal training policy and processed that could then be tested to explore the impact that such training would have on future examination rater-reliability. Five years of examination scoring data was utilized from a small Midwest region private graduate school. Intraclass correlation coefficient statistical analysis was employed to determine inter-rater reliability along with independent samples t-test to determine statistical significance the faculty groups. Mean scoring differences outputs were then tested utilizing a Likert-type scale to evaluate scoring gaps amongst faculty. The findings within this study indicate that inter-rater reliability is negatively impacted when no formal or consistent training. However, no significant differences between the mean scores were found based on faculty status nor between faculty length of employment. The hypothesis testing yielded support of the main hypothesis within this research study and the current literature in which inter-rater reliability is negatively impacted when no formal or consistent rater-training and rubric calibrations are performed for raters of examinations. The implications for practice resulting from this study display the need for formal training policy and processes be created, and consistently completed to ensure inter-rater reliability will be positively impacted. (As Provided). |
| Erfasst von | ERIC (Education Resources Information Center), Washington, DC |
| Update | 2024/1/01 |
Literaturbeschaffung und Bestandsnachweise in Bibliotheken prüfen
Permalink als QR-Code
Inhalt auf sozialen Plattformen teilen (nur vorhanden, wenn Javascript eingeschaltet ist)
Teile diese Seite:
