The longitudinal stability of rating characteristics in an EFL examination: Methodological and substantive considerations.

Autor/inn/en	Lamprianou, Iasonas; Tsagari, Dina; Kyriakou, Nansia
Titel	The longitudinal stability of rating characteristics in an EFL examination: Methodological and substantive considerations.
Quelle	In: Language testing, 38 (2021) 2, S. 273-301Infoseite zur Zeitschrift PDF als Volltext (1); PDF als Volltext (2) Link als defekt meldenVerfügbarkeit
Beigaben	Anhang; Abbildungen 4; Anmerkungen 2; Tabellen 5
Sprache	englisch
Dokumenttyp	online; gedruckt; Zeitschriftenaufsatz
ISSN	0265-5322; 1477-0946
Schlagwörter	Empirische Forschung; Langzeituntersuchung; Leistungsbeurteilung; Fremdsprachenunterricht; Englischunterricht; Bewertung
Abstract	This longitudinal study (2002-2014) investigates the stability of rating characteristics of a large group of raters over time in the context of the writing paper of a national high-stakes examination. The study uses one measure of rater severity and two measures of rater consistency. The results suggest that the rating characteristics of individual raters are not stable. Thus, predictions from one administration to the next are difficult, although not impossible. In fact, as the membership of the group of raters changes from year to year, past data on rating characteristics become less useful. When the membership of the group of raters is retained, the community of raters develops more stable characteristics. However, "cultural shocks" (low retention of raters and large numbers of newcomers) destabilize the rating characteristics of the community and predictions become more difficult. The authors propose practical measures to increase the stability of rating across time and offer methodological suggestions for more efficient rater effect-related research designs and analyses. (Verlag, adapt.).
Erfasst von	Informationszentrum für Fremdsprachenforschung, Marburg
Update	2022/2