Developing an Innovative Elicited Imitation Task for Efficient English Proficiency Assessment. TOEFL® Research Report. RR-96. ETS RR-21-24

Autor/inn/en	Davis, Larry; Norris, John
Titel	Developing an Innovative Elicited Imitation Task for Efficient English Proficiency Assessment. TOEFL® Research Report. RR-96. ETS RR-21-24
Quelle	In: ETS Research Report Series, (2021), (30 Seiten)Infoseite zur Zeitschrift PDF als Volltext kostenfreie Datei Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	2330-8516
Schlagwörter	Task Analysis; Language Proficiency; Speech Communication; Language Tests; English (Second Language); Second Language Learning; Imitation; Holistic Approach; Test Format; Test Construction; Scoring; Test Validity; Linguistic Input; Interrater Reliability; Rating Scales; Correlation; Foreign Countries; Participant Characteristics; College Attendance; Animation; Colombia; United States; Turkey; China; South Korea; Japan; India; Germany; Jordan; Mexico; Test of English as a Foreign Language + Suchen Sie Ihr Suchwort? Aufgabenanalyse; Language skill; Language skills; Sprachkompetenz; Language test; Sprachtest; English as second language; English; Second Language; Englisch als Zweitsprache; Zweitsprachenerwerb; Holistischer Ansatz; Testentwicklung; Testaufbau; Bewertung; Testvalidität; Sprachbildung; Interrater-Reliabilität; Rating-Skala; Korrelation; Ausland; College; Colleges; Attendance; Hochschule; Fachhochschule; Anwesenheit; Kolumbien; USA; Türkei; Korea; Republik; Indien; Deutschland; Mexiko
Abstract	The elicited imitation task (EIT), in which language learners listen to a series of spoken sentences and repeat each one verbatim, is a commonly used measure of language proficiency in second language acquisition research. The "TOEFL® Essentials"™ test includes an EIT as a holistic measure of speaking proficiency, referred to as the "Listen and Repeat" task type. In this report, we describe the design considerations that informed the development of the EIT for TOEFL Essentials. We also report the results of a series of investigations conducted during the prototyping and pilot phases of test development, which were undertaken with the goal of confirming task design specifications, evaluating scoring performance, and obtaining initial validity evidence to support score interpretation and use of the EIT in the TOEFL Essentials test. We found that task design variables generally performed as expected. The length of input sentence was strongly associated with performance (Pearson r = 0.88), consistent with the construct measured by the EIT, while other task variables not directly related to the EIT construct did not impact performance (e.g., graphics, speaker accent, and response time). Scorers drawn from TOEFL iBT test raters were able to score responses consistently with over 98% exact or adjacent interrater agreement on a 6-point scale, and scores on the pilot version of the EIT were highly reliable (Cronbach's [alpha] = 0.93 on the 15-item pilot version). Correlations between EIT scores and other measures were generally as expected: Correlations with other speaking tasks were high (0.78-0.84) and slightly to somewhat lower for other language measures (0.73 for writing, 0.68 for listening, and 0.57 for reading). Correlation with an independent measure of holistic language proficiency (C-test) was moderately high (0.69), as expected. We discuss the study findings in terms of the TOEFL Essentials test validity argument and point out limitations to the current results along with future research needs. Overall, we believe that the findings provide initial support to warrant the use of the EIT as operationalized in the TOEFL Essentials test. (As Provided).
Anmerkungen	Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/research/policy_research_reports/ets
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01