3K-LEx-MC
The creation and validation of a multiple-choice vocabulary placement test for Spanish language learners
This study presents the development and validation of a 132-item Spanish-English bilingual multiple-choice
vocabulary test based on the 3,000 most frequent lemmas that distinguishes between North American university students who satisfy
the Foreign Language requirement and those who need to complete coursework. 819 students were assigned to one of the two 144-item
forms of the preliminary test, which had 72 shared anchor items and other 72 form-specific items. Factor analysis was used to
evaluate dimensionality and the Rasch model was used to select the items that best differentiated between these two student
populations. This final form was administered to 213 students. Results showed high levels of unidimensionality, and the final form
provided a Rasch reliability coefficient of 0.97.
Article outline
- High frequency vocabulary in Spanish
- Spanish vocabulary tests
- The Rasch model
- Methods
- Participants
- Instruments
- Procedure
- Results
- Calibration and item bank development
- Unidimensionality
- Item level fit statistics
- Construction of final form
- Participants
- Explanatory IRT
- Conclusions
- Notes
-
References
References (65)
References
Beavers, A. S., Lounsbury, J. W., Richards, J. K., Huck, S. W., Skolits, G. J., & Esquivel, S. L. (2013). Practical
considerations for using exploratory factor analysis in educational research. Practical
Assessment, Research, and
Evaluation,
18
(6), 1–13. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Beglar, D. (2010). A
Rasch-based validation of the vocabulary size test. Language
Testing,
27
(1), 101–118. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Berendt, E., Kurosaki, E., Maeda, A., Matsui, K., & Ochi, N. (2006). English
loan words of Japanese elementary school children and their mental lexicon. Asian
Englishes,
8
(2), 26–45. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Biemiller, A. (2003). Vocabulary:
Needed if more children are to read well. Reading
Psychology,
24
(3–4), 323–335. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Bond, T. G., & Fox, C. M. (2015). Applying
the Rasch model: Fundamental measurement in the human
sciences. Routledge. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Bürkner, P. C. (2017). brms:
An R package for bayesian multilevel models using Stan. Journal of Statistical
Software,
80
(1), 1–28. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., & Riddel, A. (2017). Stan:
A probabilistic programming language. Journal of Statistical
Software,
76
(1), 1–32. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Casado, M. A., & Dereshiwsky, M. I. (2001). Foreign
language anxiety of university students. College Student
Journal,
35
(4), 539–552.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Chalmers, R. P. (2012). mirt:
A Multidimensional item response theory package for the R environment. Journal of Statistical
Software,
48
(6), 1–29. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Davies, M. (2005). Vocabulary
range and text coverage: Insights from the forthcoming Routledge frequency dictionary of
Spanish. In D. Eddington (Ed.), Selected
proceedings of the 7th Hispanic Linguistics
Symposium (pp. 106–115). Somerville, MA: Cascadilla Proceedings Project.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Davies, M., & Hayward Davies, K. (2017). A
frequency dictionary of Spanish: Core vocabulary for learners (2nd
ed.). Routledge. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
de Ayala, R. J. (2009). The
theory and practice of item response theory. The Guilford Press.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
De Boeck, P., & Wilson, M. (2004). Explanatory
item response models: A generalized linear and nonlinear
approach. Springer. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Desjardins, C. D., & Bulut, O. (2018). Handbook
of educational measurement and psychometrics using R. CRC Press. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Dorans, N. J., Moses, T. P., & Eignor, D. R. (2010). Principles
and practices of test score equating. ETS Research Report
Series, 2010(2), i–41. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Frantzen, D., & Magnan, S. S. (2005). Anxiety
and the true beginner – false beginner dynamic in beginning French and Spanish classes. Foreign
Language
Annals,
38
(2), 171–186. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Floyd, F. J., & Widaman, K. F. (1995). Factor
analysis in the development and refinement of clinical assessment instruments. Psychological
Assessment,
7
(3), 286–299. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Gairns, R., & Redman, S. (1986). Working
with words. A guide to teaching and learning vocabulary. Cambridge University Press.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013). Bayesian
data analysis (3rd ed.). CRC Press. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals
of item response theory. SAGE Publications, Inc.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Horn, J. L. (1965). A
rationale and test for the number of factors in factor
analysis. Psychometrika,
30
(2), 179–185. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Horwitz, E. (1990). Attending
to the affective domain in the foreign language classroom. In S. Magnan (Ed.), Shifting
the instructional focus to the
learner (pp. 15–33). Northeast
Conference on the Teaching of Foreign Languages.
Hu, M., & Nation, I. S. P. (2000). Vocabulary
density and reading comprehension. Reading in a Foreign
Language,
23
(1), 4031–430.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Institute for Test Research and Test
Development. (2013). Assessing evidence of validity of the ACTFL reading
proficiency test (RPT).![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Izura, C., Cuetos, F., & Brysbaert, M. (2014). Lextale-Esp:
A test to rapidly and efficiently assess the Spanish vocabulary
size. Psicológica,
35
(1), 49–66.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Jones, R., & Tschirner, E. (2006). A
frequency dictionary of German: Core vocabulary for
learners. Routledge.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kaplan, R. M., & Saccuzzo, D. P. (2009). Psychological
testing principles, applications, and issues (7th
ed.). Wadsworth.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kruschke, J. K. (2015). Doing
Bayesian data analysis: A tutorial with R, JAGS, and Stan (2nd
ed.). Elsevier.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Laufer, B., & Aviad-Levitzky, A. (2017). What
type of vocabulary knowledge predicts reading comprehension: Word meaning recall or word meaning
recognition? The Modern Language
Journal,
101
(4), 729–741. [URL]![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Laufer, B., & McLean, S. (2016). Loanwords
and vocabulary size test scores: A case of different estimates for different L1
learners. Language Assessment
Quarterly,
13
(3), 202–217. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Liu, N., & Nation, I. S. P. (1985). Factors
affecting guessing vocabulary in context. RELC
Journal,
16
(1), 33–42. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Looney, D., & Lusin, N. (2019). Enrollments
in languages other than English in United States institutions of higher education, summer 2016 and fall 2016: Final
report. Modern Language Association of America. [URL]
McElreath, R. (2020). Statistical
rethinking (2nd ed.). CRC Press. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
McLean, S., Kramer, B., & Beglar, D. (2015a). The
creation and validation of a listening vocabulary levels test. Language Teaching
Research,
19
(6), 741–760. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
McLean, S., Kramer, B., & Stewart, J. (2015b). An
empirical examination of the effect of guessing on vocabulary size test scores. Vocabulary
Learning and
Instruction,
4
(1), 26–35. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
McLean, S., Stewart, J., & Batty, A. O. (2020). Predicting
L2 reading proficiency with modalities of vocabulary knowledge: A bootstrapping
approach. Language
Testing,
37
(3), 389–411. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
McNamara, T., & Knoch, U. (2012). The
Rasch wars: The emergence of Rasch measurement in language testing. Language
Testing,
29
(4), 555–576. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Meara, P. (2010). EFL
vocabulary test (2nd ed.). Centre for Applied Language Studies.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Meara, P., & Milton, J. (2003). The
Swansea levels test. Express.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Milton, J. (2009). Measuring
second language vocabulary acquisition. Multilingual Matters. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Nation, I. S. P. (2006). How
large a vocabulary is needed for reading and listening? Canadian Modern Language
Review,
63
(1), 591–82. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Nation, I. S. P., & Beglar, D. (2007). A
vocabulary size test. The Language
Teacher,
31
(7), 96–13.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Nation, I. S. P., & Webb, S. A. (2011). Researching
and analyzing vocabulary. Cengage.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Nguyen, L. T. C., & Nation, I. S. P. (2011). A
bilingual vocabulary size test of English for Vietnamese learners. RELC
Journal,
42
(1), 86–99. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
O’Connor, B. P. (2000). SPSS
and BAS programs for determining the number of components using parallel analysis and Velicer’s MAP
test. Behavior Research Methods, Instruments, &
Computers,
32
(3), 396–402. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Pellicer-Sánchez, A., & Schmitt, N. (2012). Scoring
Yes–No vocabulary tests: Reaction time vs. nonword approaches. Language
Testing,
29
(4), 489–509. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
R Core Team. (2019). R: A language and
environment for statistical computing. R Foundation for Statistical Computing. [URL]
Revelle, W. (2020). Psych:
Procedures for psychological, psychometric, and personality research. Northwestern University. [URL]
Robles-García, P. (2020). 3K-LEx:
Desarrollo y validación de una prueba de amplitud Léxica en español. Journal of Spanish
Language
Teaching,
7
(1), 64–76. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Robles-García, P. (2022). Receptive
vocabulary knowledge in L2 learners of Spanish: The role of high-frequency words. Foreign
Language Annals, 1–22. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Schmith, R. (2002). Qualtrics [survey
software]. Utah, U.S. available at [URL]
Schmitt, N., Cobb, T., Horst, M., & Schmitt, D. (2017). How
much vocabulary is needed to use English? Replication of van Zeeland & Schmitt (2012),
Nation (2006) and Cobb (2007). Language
Teaching,
50
(2), 212–226. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Schmitt, N., Jiang, X., & Grabe, W. (2011). The
percentage of words known in a text and reading comprehension. Modern Language
Journal,
95
(1), 26–43. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Schmitt, N. & Schmitt, D. (2014). A
reassessment of frequency and vocabulary size in L2 vocabulary teaching. Language
Teaching,
47
(4), 484–503. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Schmitt, N., Schmitt, D., & Clapham, C. (2001). Developing
and exploring the behaviour of two new versions of the vocabulary levels test. Language
Testing,
18
(1), 55–88. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Stoeckel, T. (2018). High-frequency
and academic English vocabulary growth among first-year students at UNP. Journal of
International Studies and Regional
Development,
9
1, 15–30.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Stoeckel, T., McLean, S., & Nation, I. S. P. (2021). Limitations
of size and levels tests of written receptive vocabulary knowledge. Studies in Second Language
Acquisition,
43
(1), 181–203. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Van Zeeland, H., & Schmitt, N. (2012). Lexical
coverage in L1 and L2 listening comprehension: The same or different from reading
comprehension? Applied
Linguistics,
34
(4), 457–479. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Webb, S. A., & Chang, A. C.-S. (2012). Second
language vocabulary growth. RELC
Journal,
43
(1), 113–126. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Wilson, M., De Boeck, P., & Carstensen, C. H. (2008). Explanatory
item response models: A brief introduction. In M. Wilson & P. De Boeck (Eds.), Explanatory
Item Response Models (2nd
ed.). Springer.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Zhang, S., & Zhang, X. (2020). The
relationship between vocabulary knowledge and L2 reading/listening comprehension: A
meta-analysis. Language Teaching
Research,
26
(4), 696–725. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Zhang, X. (2013). The
I don’t know option in the vocabulary size test. TESOL
Quarterly,
47
(4), 790–811. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Zipf, G. (1935). The
psychobiology of language: An introduction to dynamic philology. MIT Press.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Cited by (2)
Cited by two other publications
Robles-García, Pablo, Stuart McLean, Jeffrey Stewart, Ji-young Shin & Claudia Helena Sánchez-Gutiérrez
2024.
The Development and Initial Validation of O-WSVLT, a Meaning-Recall Online L2 Spanish Vocabulary Levels Test.
Language Assessment Quarterly 21:2
► pp. 181 ff.
![DOI logo](//benjamins.com/logos/doi-logo.svg)
Robles-García, Pablo, Jeffrey Stewart, Christopher Nicklin, Joseph P. Vitta, Stuart McLean & Brandon Kramer
2023.
‘The wisdom of crowds’: When teacher judgments outperform word-frequency as a predictor of students’ vocabulary knowledge.
Language Teaching Research ![DOI logo](//benjamins.com/logos/doi-logo.svg)
This list is based on CrossRef data as of 4 july 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.