The Development and Evaluation of Interactional Competence Elicitor for Oral Language Assessments. TOEFL® Research Report. RR-92. ETS RR-21-06

Autor/inn/en	Chukharev-Hudilainen, Evgeny; Ockey, Gary J.
Titel	The Development and Evaluation of Interactional Competence Elicitor for Oral Language Assessments. TOEFL® Research Report. RR-92. ETS RR-21-06
Quelle	In: ETS Research Report Series, (2021), (20 Seiten)Infoseite zur Zeitschrift PDF als Volltext kostenfreie Datei Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	2330-8516
Schlagwörter	Intercultural Communication; Oral Language; Communicative Competence (Languages); Error Analysis (Language); Natural Language Processing; Dialogs (Language); Test Construction; English (Second Language); Language Tests; Second Language Learning; Student Placement; Foreign Students; Undergraduate Students; Audio Equipment; Computer Software; Student Attitudes; Iowa; Test of English as a Foreign Language + Suchen Sie Ihr Suchwort? Interkulturelle Kommunikation; Oral interpretation; Mündlicher Sprachgebrauch; Communicative competence; Languages; Kommunikative Kompetenz; Sprache; Error analysis; Language; Fehleranalyse; Natürliche Sprache; Dialog; Dialogs; Dialogue; Dialogues; Testaufbau; English as second language; English; Second Language; Englisch als Zweitsprache; Language test; Sprachtest; Zweitsprachenerwerb; Schülerpraktikum; Audio-CD; Schülerverhalten
Abstract	This paper describes the development and evaluation of Interaction Competence Elicitor (ICE), a spoken dialog system (SDS) for the delivery of a paired oral discussion task in the context of language assessment. The purpose of ICE is to sustain a topic-specific conversation with a test taker in order to elicit discourse that can be later judged to assess the test taker's oral language ability, including interactional competence. The development of ICE is reported in detail to provide guidance for future developers of similar systems. The performance of ICE is evaluated on two aspects: (1) by analyzing system errors that occur at different stages in the natural language processing (NLP) pipeline in terms of both their preventability and their impact on the downstream stages of the pipeline; and (2) by analyzing questionnaire and semistructured interview data to establish the test takers' experience with the system. Findings suggest that ICE was robust in 90% of the dialog turns it produced, and test takers noted both positive and negative aspects of communicating with the system as opposed to a human interlocutor. We conclude that this prototype system lays important groundwork for the development and use of specialized SDSs in the assessment of oral communication, which includes interactional competence. (As Provided).
Anmerkungen	Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/research/policy_research_reports/ets
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01