Using Confidence Intervals to Determine Adequate Item Sample Sizes for Vocabulary Tests: An Essential but Overlooked Practice

Autor/inn/en	Gyllstad, Henrik; McLean, Stuart; Stewart, Jeffrey
Titel	Using Confidence Intervals to Determine Adequate Item Sample Sizes for Vocabulary Tests: An Essential but Overlooked Practice
Quelle	In: Language Testing, 38 (2021) 4, S.558-579 (22 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Gyllstad, Henrik) ORCID (McLean, Stuart) ORCID (Stewart, Jeffrey)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0265-5322
DOI	10.1177/0265532220979562
Schlagwörter	Vocabulary Development; Sample Size; Language Tests; Test Items; Word Frequency; Word Lists; Computational Linguistics; English (Second Language); Second Language Instruction; Second Language Learning; Test Format; Scores; Inferences; Native Language; Japanese; Intervals; Foreign Countries; Undergraduate Students; Test of English for International Communication + Suchen Sie Ihr Suchwort? Wortschatzarbeit; Language test; Sprachtest; Test content; Testaufgabe; Word analysis; Frequency; Wortanalyse; Häufigkeit; Wortliste; Linguistics; Computerlinguistik; English as second language; English; Second Language; Englisch als Zweitsprache; Fremdsprachenunterricht; Zweitsprachenerwerb; Testentwicklung; Inference; Inferenz; Japaner; Japanisch; Ausland
Abstract	The last three decades have seen an increase of tests aimed at measuring an individual's vocabulary level or size. The target words used in these tests are typically sampled from word frequency lists, which are in turn based on language corpora. Conventionally, test developers sample items from frequency bands of 1000 words; different tests employ different sampling ratios. Some have as few as 5 or 10 items representing the underlying population of words, whereas other tests feature a larger number of items, such as 24, 30, or 40. However, very rarely are the sampling size choices supported by clear empirical evidence. Here, using a bootstrapping approach, we illustrate the effect that a sample-size increase has on confidence intervals of individual learner vocabulary knowledge estimates, and on the inferences that can safely be made from test scores. We draw on a unique dataset consisting of adult L1 Japanese test takers' performance on two English vocabulary test formats, each featuring 1000 words. Our analysis shows that there are few purposes and settings where as few as 5 to 10 sampled items from a 1000-word frequency band (1K) are sufficient. The use of 30 or more items per 1000-word frequency band and tests consisting of fewer bands is recommended. (As Provided).
Anmerkungen	SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01