Evaluating the Efficacy of Rater Self-Training.

Autor/inn/en	Kenyon, Dorry; Stansfield, Charles W.
Institution	Center for Applied Linguistics, Washington, DC.
Titel	Evaluating the Efficacy of Rater Self-Training.
Quelle	(1993), (27 Seiten) PDF als Volltext kostenfreie Datei Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Monographie
Schlagwörter	Bilingual Education; Comparative Analysis; Evaluators; Individual Characteristics; Interrater Reliability; Language Proficiency; Language Tests; Models; Motivation; Performance Based Assessment; Rating Scales; Testing + Suchen Sie Ihr Suchwort? Bilingual teaching; Bilingualer Unterricht; Personality characteristic; Personality traits; Persönlichkeitsmerkmal; Interrater-Reliabilität; Language skill; Language skills; Sprachkompetenz; Language test; Sprachtest; Analogiemodell; psychologische; Motivation (psychologisch); Leistungsermittlung; Rating-Skala; Testdurchführung; Testen
Abstract	This paper examines whether individuals who train themselves to score a performance assessment will rate acceptably when compared to known standards. Research on the efficacy of rater self-training materials developed by the Center for Applied Linguistics for the Texas Oral Proficiency Test (TOPT) is examined. Rater self-materials are described and analysis of data is reported from a study conducted during their development. Eight individuals worked through the materials on their own and submitted qualitative feedback on the materials and on their experience with them. They also participated in a calibration study in which they independently rated recorded segments from a TOPT administration. These ratings were analyzed by traditional approaches and by a multifaceted Rasch approach. Findings indicate that the raters as a group scored the TOPT consistently, although differences in rater severity led to some important disagreements with the rating key. The study illustrates the role of background characteristics and motivation in the success of rater self-training. (Contains 8 references.) (JL)
Erfasst von	ERIC (Education Resources Information Center), Washington, DC