Article published In:
ITL - International Journal of Applied Linguistics: Online-First ArticlesThe effect of test format on productive recall of derivatives
This study investigated the extent to which two recall test formats – contextualized and decontextualized tests –
affected productive recall of derivatives, and how the effects of token frequencies of derivatives and L2 receptive vocabulary
knowledge on recalling derivatives was moderated by test format. Mixed effects logistic regression models examined the derivatives
elicited from L1 (n = 21) and L2 English speakers’ (n = 107) on the two recall tests. Results
indicated that contextual cues significantly facilitated recalling derivatives, while such facilitative effects were larger for
native speakers and L2 learners with greater vocabulary knowledge. Furthermore, token frequency affected the responses on the
decontextualized test to a greater degree compared to the contextualized test. Results suggest that test format influences
test-takers’ ability to recall knowledge to produce derivatives.
Keywords: testing vocabulary knowledge, test format effects, frequency effects, productive derivational knowledge, receptive vocabulary knowledge
Article outline
- Introduction
- The effect of context on productive recall
- Measuring derivational knowledge in context and isolation
- To what extent does context facilitate productive recall of derivatives?
- How does test format moderate the effects of derivative frequency and prior vocabulary knowledge?
- Rationale and research questions
- Method
- Participants
- Procedure
- Derivative form recall test format
- Contextualized form recall test
- Decontextualized form recall test
- Scoring
- Derivative frequency
- The Updated Vocabulary Levels Test (uVLT)
- Analysis
- Results
- To what extent does test format moderate productive recall of L1 and L2 derivatives?
- To what extent does test format affect learners at different vocabulary levels?
- To what extent does test format moderate the effects of derivative frequency and prior vocabulary knowledge?
- Discussion
- To what extent does test format moderate productive recall of derivatives?
- To what extent does test format affect productive recall of derivatives for learners at different vocabulary levels?
- To what extent does test format moderate the effects of derivative frequency and prior vocabulary knowledge?
- Methodological and pedagogical implications
- Conclusion
-
References
References (61)
Aviad-Levitzky, T., Laufer, B., & Goldstein, Z. (2019). The
new computer adaptive test of size and strength (CATSS): Development and validation. Language
Assessment
Quarterly,
16
(3), 345–368. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Bates, D., Mächler, M., Bolker, B. M., & Walker, S. C. (2015). Fitting
linear mixed-effects models using lme4. Journal of Statistical
Software,
67
(1), 1–48. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Barclay, J. R., Bransford, J. D., Franks, J. J., McCarrell, N. S., & Nitsch, K. (1974). Comprehension
and semantic flexibility. Journal of Verbal Learning and Verbal
Behavior,
13
(4), 471–481. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Brezina, V., & Gablasova, D. (2015). Is
there a core general vocabulary? Introducing the new general service list. Applied
Linguistics,
36
(1), 1–22. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Brysbaert, M., & Stevens, M. (2018). Power
analysis and effect size in mixed effects models: A tutorial. Journal of
Cognition,
1
(1):91, 1–20. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Cheng, L., & DeLuca, C. (2011). Voices
from test-takers: Further evidence for language assessment validation and use, Educational
Assessment,
16
(2), 104–122. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Clahsen, H., & Neubauer, K. (2020). Morphology,
frequency, and the processing of derived words in native and non-native
speakers. Lingua, 120(11), 2627–2637. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Cobb, T., & Laufer, B. (2021). The
nuclear word family list: A list of the most frequent family members, including base and affixed
words. Language
Learning.
71
(3), 834–871. publication. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Coxhead, A., Nation, I. S. P., & Sim, D. (2015). Measuring
the vocabulary size of native speakers of English in New Zealand secondary schools. New Zealand
Journal of Educational
Studies,
50
(1), 121–135. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Crossley, S. A., & Salsbury, T. (2010). Using
lexical indices to predict produced and not produced words in second language learners. The
Mental
Lexicon,
5
(1), 115–147. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Dang, T. N. Y., & Webb, S. (2016). Making
an essential word list for beginners. In I. S. P. Nation, Making and using word lists for
language learning and
testing (pp. 153–167, 188–195). John Benjamins. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Davies, M. (2009). The
385+ million word Corpus of Contemporary American English (1990– 2008+): Design, architecture, and linguistic
insights. International Journal of Corpus
Linguistics,
14
(2), 159–190. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Ellis, N. C. (2002). Frequency
effects in language processing: A review with implications for theories of implicit and explicit language
acquisition. Studies in Second Language
Acquisition,
24
(2), 143–188. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Fox, J., & Weisberg, S. (2019). An
R Companion to Applied Regression (3rd
ed.). Sage. [URL]
González-Fernández, B., & Schmitt, N. (2020). Word
knowledge: Exploring the relationships and order of acquisition of vocabulary knowledge
components. Applied
Linguistics,
41
(4) 1–26. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Gries, S. T. (2021). (Generalized
linear) mixed-effects modeling: A learner corpus example. Language
Learning,
71
(3), 757–798. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Hartig, F. (2020). DHARMa:
Residual diagnostics for hierarchical (multi-level/mixed) regression models. (R package version
0.4.1) [Computer software]. [URL]
Iwaizumi, E. & Webb, S. (2021). To
what extent does productive derivational knowledge of adult L1 speakers and L2 learners at two educational levels
differ? TESOL
Journal,
12
(4), 1–22. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Iwaizumi, E., & Webb, S. (2022). Measuring
L1 and L2 productive derivational knowledge: How many derivatives can L1 and L2 learners with differing vocabulary levels
produce? TESOL
Quarterly,
56
(1), 100–129. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Iwaizumi, E. & Webb, S. (2023). To
what extent do learner- and word-related variables affect production of derivatives? Language
Learning,
73
(1), 301–336. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Ishii, T., & Schmitt, N. (2009). Developing
an integrated diagnostic test of vocabulary size and depth. RELC
Journal,
40
(1), 5–22. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kremmel, B. (2021). Selling
the (word) family silver? A response to Webb’s lemma dilemma Studies in Second Language
Studies
,
43
(5), 962–964. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kremmel, B., & Schmitt, N. (2016). Interpreting
vocabulary test scores: What do various item formats tell us about learners’ ability to employ
words? Language Assessment
Quarterly,
13
(4), 377–392. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kyle, K., Crossley, S., & Berger, C. (2018). The
tool for the automatic analysis of lexical sophistication (TAALES): version 2.0. Behavior
Research
Methods,
50
(3), 1030–1046. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Laufer, B. (1998). The
development of passive and active vocabulary in a second language: Same or different. Applied
Linguistics,
19
(2), 255–271. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
(2023). Understanding
L2-derived words in context: Is complete receptive morphological knowledge necessary? Studies
in Second Language Acquisition. Advanced online publication. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Laufer, B., & Aviad-Levitzky, T. (2017). What
type of vocabulary knowledge predicts reading comprehension: Word meaning recall or word meaning
recognition? The Modern Language
Journal,
101
(4), 729–741. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Laufer, B., & Goldstein, Z. (2004). Testing
vocabulary knowledge: Size, strength, and computer adaptiveness. Language
Learning,
54
(3), 399–436. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Laufer, B., & Nation, P. (1995). Vocabulary
size and use: Lexical richness in L2 written production. Applied
Linguistics,
16
(3), 307–322. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Laufer, B., & Ravenhorst-Kalovski, G. C. (2010). Lexical
threshold revisited: Lexical text coverage, learners’ vocabulary size and reading
comprehension. Reading in a Foreign
Language,
22
(1), 15–30. [URL]
Laufer, B., Webb, S., Kim, S., & Yohanan, B. (2021). How
well do learners know derived words in a second language? The effect of proficiency, word frequency and type of
affix. ILT – International Journal of Applied
Linguistics,
172
(2), 229–258. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Lenth, R. V. (2022). emmeans:
Estimated marginal means, aka least-squares means. (R package version
1.7.3) [Computer software] [URL]
Leontjev, D., Huhta, A., & Mäntylä, K. (2016). Word
derivational knowledge and writing proficiency: How do they
link? System,
59
1, 73–89. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
McCutchen, D., & Stull, S. (2015). Morphological
awareness and children’s writing: accuracy, error, and invention. Reading and
Writing,
28
(2), 271–289. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Miralpeix, I., & Muñoz, C. (2018). Receptive
vocabulary size and its relationship to EFL language skills, International Review of Applied
Linguistics in Language
Teaching,
56
(1), 1–24. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Nation, I. S. P. (2006). How
large a vocabulary is needed for reading and listening? The Canadian Modern Language
Review,
63
(1), 59–82. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Nation, Nation, P., & Beglar, D. (2007). A
vocabulary size test. The Language
Teacher, 31(7), 9–13. [URL]![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Nation, I. S. P. (2012). The
BNC/COCA word family list. [URL]
Peters, E. (2018). The
effect of out-of-class exposure to English language media on learners’ vocabulary
knowledge. ITL–International Journal of Applied
Linguistics,
169
(1), 142–168. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Qian, D. D. (2005). Demystifying
lexical inferencing: The role of aspects of vocabulary knowledge. TESL Canada
Journal,
22
(2), 34–54. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
(2008). From
single words to passages: Contextual effects on predictive power of vocabulary measures for assessing reading
performance, Language Assessment
Quarterly,
5
(1), 1–19, ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
R Core Team. (2022). R: A language and
environment for statistical computing. R Foundation for Statistical Computing (Version
4.1.2) [Computer software]. [URL]
Schmitt, N., & Meara, P. (1997). Researching
vocabulary through a word knowledge framework. Studies in Second Language
Acquisition,
19
(1), 17–36. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Schmitt, N., & Zimmerman, C. B. (2002). Derivative
word forms: What do learners know? TESOL
Quarterly,
36
(2), 145–171. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Snoder, P., & Laufer, B. (2023). EFL
learners’ receptive knowledge of derived words: The case of Swedish Adolescents. TESOL
Quarterly,
56
(4), 1242–1265. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Tulving, E., & Thomson, D. M. (1973). Encoding
specificity and retrieval processes in episodic memory. Psychological
Review,
80
(5), 352 – 373. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Ullman, M. T. (2001). The
neural basis of lexicon and grammar in first and second language: The declarative/procedural model.
Bilingualism: Language and
Cognition,
4
(1), 105–122. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Watanabe, Y. (1997). Input,
intake, and retention: Effects of increased processing on incidental learning of foreign language
vocabulary. Studies in Second Language
Acquisition,
19
(3), 287–308. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Webb, S. (2005). Receptive
and productive vocabulary learning: The effects of reading and writing on word
knowledge. Studies in Second Language
Acquisition,
27
(1), 33–52. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
(2008). The
effects of context on incidental vocabulary learning. Reading in a Foreign
Language,
20
(2), 232–245. [URL]
(2018). Advanced
lexical development. In Malovrh, P. A. & A. G. Benati (Eds). The
handbook of advanced proficiency in second language
acquisition (pp. 401–418). John Wiley & Sons, Inc. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
(2021). The
lemma dilemma: How should words be operationalized in research and pedagogy? Studies in Second
Language
Acquisition,
43
(5), 941–949. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Webb, S. & Chang, A. C.-S. (2015). How
does prior word knowledge affect vocabulary learning progress in an extensive reading
program? Studies in Second Language
Acquisition,
43
(5), 941–949. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Webb, S., Sasao, Y., & Ballance, O. (2017). The
updated vocabulary levels test: Developing and validating two new forms of the VLT. ITL –
International Journal of Applied
Linguistics,
168
(1), 33–69. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)