Adapting CEF-Descriptors for Rating Purposes: Validation by a Combined Rater Training and Scale Revision Approach

Autor/inn/en	Harsch, Claudia; Martin, Guido
Titel	Adapting CEF-Descriptors for Rating Purposes: Validation by a Combined Rater Training and Scale Revision Approach
Quelle	In: Assessing Writing, 17 (2012) 4, S.228-250 (23 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1075-2935
DOI	10.1016/j.asw.2012.06.003
Schlagwörter	Rating Scales; Validity; Media Adaptation; Feedback (Response); Scoring; Alignment (Education); Evaluators; Training; Foreign Countries + Suchen Sie Ihr Suchwort? Rating-Skala; Gültigkeit; Medienverwendung; Bewertung; Ausbildung; Ausland
Abstract	We explore how a local rating scale can be based on the Common European Framework CEF-proficiency scales. As part of the scale validation (Alderson, 1991; Lumley, 2002), we examine which adaptations are needed to turn CEF-proficiency descriptors into a rating scale for a local context, and to establish a practicable method to revise the initial scale-draft while simultaneously training raters for trialling the rating scale. We propose a data-driven approach focusing on the descriptors, involving raters in the scale revision as part of the scale validation process (Barkaoui, 2010a; Cohen, 1994). Rating data were analysed during a two-month period, employing 19 writing tasks and 13 raters; individual descriptors and raters were monitored for consistency. Our approach facilitated identifying causes of low agreement, let it be raters or scale wording, which in turn allowed targeted feedback to raters and revisions of scale-wording where indicated. We conclude with a summary of necessary adaptations of CEF-descriptors for rating purposes, and we outline further steps to validate the revised rating scale. We aim to contribute to improve rater training and rating scale development based on the CEF, since the literature for "real" test development contexts seems rather scarce (Barkaoui, 2010b; Knoch, 2011a,b). (Contains 10 tables and 1 figure.) (As Provided).
Anmerkungen	Elsevier. 6277 Sea Harbor Drive, Orlando, FL 32887-4800. Tel: 877-839-7126; Tel: 407-345-4020; Fax: 407-363-1354; e-mail: usjcs@elsevier.com; Web site: http://www.elsevier.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10