Towards More Valid Scoring Criteria for Integrated Reading-Writing and Listening-Writing Summary Tasks

Autor/inn/en	Chan, Sathena; May, Lyn
Titel	Towards More Valid Scoring Criteria for Integrated Reading-Writing and Listening-Writing Summary Tasks
Quelle	In: Language Testing, 40 (2023) 2, S.410-439 (30 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Chan, Sathena) ORCID (May, Lyn)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0265-5322
DOI	10.1177/02655322221135025
Schlagwörter	Scoring; Writing Evaluation; Reading Tests; Listening Skills; Reading Skills; Writing Skills; Evaluation Methods; Validity; Documentation; Interdisciplinary Approach; Rating Scales; Evaluators; English for Academic Purposes; English (Second Language); Error Patterns; Written Language; Oral Language; Test of English as a Foreign Language + Suchen Sie Ihr Suchwort? Bewertung; Lesetest; Reading skill; Lesefertigkeit; Writing skill; Schreibfertigkeit; Gültigkeit; Dokumentation; Fächerübergreifender Unterricht; Fächerverbindender Unterricht; Interdisziplinarität; Rating-Skala; English as second language; English; Second Language; Englisch als Zweitsprache; Fehlertyp; Geschriebene Sprache; Oral interpretation; Mündlicher Sprachgebrauch
Abstract	Despite the increased use of integrated tasks in high-stakes academic writing assessment, research on rating criteria which reflect the unique construct of integrated summary writing skills is comparatively rare. Using a mixed-method approach of expert judgement, text analysis, and statistical analysis, this study examines writing features that discriminate summaries produced by 150 candidates at five levels of proficiency on integrated reading-writing (R-W) and listening-writing (L-W) tasks. The expert judgement revealed a wide range of features which discriminated R-W and L-W responses. When responses at five proficiency levels were coded by these features, significant differences were obtained in seven features, including relevance of ideas, paraphrasing skills, accuracy of source information, academic style, language control, coherence and cohesion, and task fulfilment across proficiency levels on the R-W task. The same features did not yield significant differences in L-W responses across proficiency levels. The findings have important implications for clarifying the construct of integrated summary writing in different modalities, indicating the possibility of expanding integrated rating categories with some potential for translating the identified criteria into automated rating systems. The results on the L-W indicate the need for developing descriptors which can more effectively discriminate L-W responses. (As Provided).
Anmerkungen	SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01