Measuring the visual in audio-visual input
The effects of imagery in vocabulary learning through TV viewing
This exploratory study investigates the effects of imagery on word learning through audio-visual input. A total of 82 adolescent EFL learners were exposed to 8 episodes of a TV series under four conditions, depending on the language of the on-screen text (L1 or L2) and whether they were pre-taught target words or not. The effects of co-occurrence of the word with its image, and the image time on screen (ITOS) were explored, alongside frequency, proficiency, and learning condition variables. Results showed that both image-related variables and frequency predicted word-form learning, while only ITOS predicted word-meaning recall, with a longer exposure to image associated to higher gains, suggesting that, at this age and proficiency level, the images associated with the words can be conducive to learning.
Article outline
- Background
- TV series and vocabulary learning
- TV series and imagery
- Frequency of occurrence in viewing
- Aim and research questions
- Methodology
- Participants
- Audio-visual materials
- Instruments
- Imagery measures
- Procedure
- Scoring of vocabulary tests
- Preliminary analysis
- Results
- Word-form learning
- Word-meaning learning
- Discussion
- Imagery
- Frequency of encounters
- Learning conditions and learner variables
- Conclusions
- Notes
-
References
References (69)
References
Avello, D. & Muñoz, C. (forthcoming). The use of captioned-video viewing to support the development of L2 reading skills. Proceedings of the fifth ELTRIA Conference, Barcelona.
Baltova, I. (1994). The impact of video on the comprehension skills of core French students. Canadian Modern Language Review,
50
(3), 507–31.
Bianchi, F., & Ciabattoni, T. (2008). Captions and subtitles in EFL learning: an investigative study in a comprehensive computer environment. In Baldry, A., Pavesi, M., & Taylor Torsello, C. (Eds.) From didactas to ecolingua: An ongoing research project on translation and corpus linguistics (pp. 69–90). Trieste: Edizioni Università di Trieste. [URL]
Boers, F. (2018). Intentional versus incidental learning. The TESOL Encyclopedia of English Language Teaching, 1–6.
Brown, R., Waring, R., & Donkaewbua, S. (2008). Incidental vocabulary acquisition from reading, reading-while-listening, and listening to stories. Reading in a foreign language,
20
(2), 136–163.
Brysbaert, M., Warriner, A. B., & Kuperman, V. (2014). Concreteness ratings for 40 thousand generally known English word lemmas. Behavior research methods,
46
(3), 904–911.
Chai, J., & Erlam, R. (2008). The effect and the influence of the use of video and captions on second language learning. New Zealand Studies in Applied Linguistics,
14
(2), 25.
Charles, T., & Trenkic, D. (2015). Speech segmentation in a second language: The role of bimodal input. In Gambier, Y., Caimi, A., & Mariotti, C. (Eds.) Subtitles and language learning: Principles, strategies and practical experiences (pp. 173–198). Bern: Peter Lang.
Chen, Y. R., Liu, Y. T., & Todd, A. G. (2018). Transient but effective? Captioning and adolescent EFL learners’ spoken vocabulary acquisition. English Teaching & Learning, 421, 25–56.
Danan, M. (2004). Captioning and subtitling: undervalued language learning strategies. Meta: Journal Des Traducteurs,
49
(1), 67–77.
Deconinck, J., Boers, F., & Eyckmans, J. (2017). ‘Does the form of this word fit its meaning?’ The effect of learner-generated mapping elaborations on L2 word recall. Language teaching research,
21
(1), 31–53.
Douglas Fir Group. (2016). A transdisciplinary framework for SLA in a multilingual world. The Modern Language Journal,
100
(S1), 19–47.
Durbahn, M., Rodgers, M., & Peters, E. (2020). The relationship between vocabulary and viewing comprehension. System,
88
1.
Elley, W. B. (1989). Vocabulary acquisition from listening to stories. Reading Research Quarterly
24
(2), 174–187. [URL].
Feng, Y., & Webb, S. (2020). Learning vocabulary through reading, listening, and viewing: Which mode of input is most effective? Studies in Second Language Acquisition,
42
(3), 499–523.
Hasan, A. (2000). Learners’ perceptions of listening comprehension problems. Language, Culture and Curriculum,
13
(2), 137–53.
Horst, M., Cobb, T., & Meara, P. (1998). Beyond a clockwork orange: Acquiring second language vocabulary through reading. Reading in a Foreign Language
11
(2), 207–23.
Hulstijn, J. H. (2003). Incidental and Intentional Learning. In Doughty, C. & Long, M. H. (Eds.), The handbook of second language acquisition (Vol. 191, pp. 349–381). MA: Blackwell.
Hulstijn, J. H. (2013). Incidental learning in second language acquisition. In Chapelle, C. A. (Ed.) The encyclopedia of applied linguistics (Vol. 51, pp. 2632–40). Chichester: Wiley-Blackwell.
Jones, L. (2004). Testing L2 vocabulary recognition and recall using pictorial and written test items. Language learning & technology
8
(3), 122–143. ISSN 1094-3501
Jones, L., & Plass, J. (2002). Supporting listening comprehension and vocabulary acquisition in French with multimedia annotations. The Modern Language Journal,
86
(4), 546–61.
Khan, N., Kasdar, J., Melvin, M., Blomquist, R., Huang, E., & McEwen, J. (2015). Fresh off the Boat [TV series] Los Angeles, CA: ABC.
Krashen, S. D. (2003). Explorations in language acquisition and use. Portsmouth, NH: Heinemann. ISBN: 0-325-00554-0
Lee, S. (2007). Effects of textual enhancement and topic familiarity on Korean EFL students’ reading comprehension and learning of passive form. Language Learning,
57
(1), 87–118.
Lindgren, E., & Muñoz, C. (2013). The influence of exposure, parents, and linguistic distance on young European learners’ foreign language comprehension. International Journal of Multilingualism,
10
(1), 105–129.
Maleki, A., & Safaee Rad, M. (2011). The effect of visual and textual accompaniments to verbal stimuli on the listening comprehension test performance of Iranian high and low proficient EFL learners. Theory and Practice in Language Studies,
1
(1), 28–36.
Markham, P. (1999). Captioned videotapes and second-language listening word recognition. Foreign Language Annals,
32
(3), 321–28.
Mayer, R. (2014). The Cambridge handbook of multimedia learning. Cambridge: Cambridge University Press.
Meara, P., & Milton, J. (2003). X_Lex: the Swansea levels test. Berkshire: Express Publishing.
Miralpeix, I. (2012, March). X_Lex and Y_Lex: A validation study. Paper presented at the 22nd Lexical Studies Conference, Newtown, UK.
Montero Perez, M. (2019). Pre-learning vocabulary before viewing captioned video: an eye-tracking study. The Language Learning Journal,
47
(4), 460–478.
Montero Perez, M., Peters, E., & Desmet, P. (2018). Vocabulary learning through viewing video: the effect of two enhancement techniques. Computer Assisted Language Learning,
31
(1–2), 1–26.
Montero Perez, M., Peters, E., Clarebout, G., & Desmet, P. (2014). Effects of captioning on video comprehension and incidental vocabulary learning. Language Learning and Technology,
18
(1), 118–41.
Muñoz, C., Pujadas, G., & Pattemore, A. (2023). Audio-visual input for learning L2 vocabulary and grammatical constructions. Second Language Research
39
(
1
), 13–38.
Naghizadeh, M., & Darabi, T. (2015). The impact of bimodal, Persian and no-subtitle movies on Iranian EFL learners’ L2 vocabulary learning. Journal of Applied Linguistics and Language Research,
2
(2), 66–79.
Nation, P. (2015). Principles guiding vocabulary learning through extensive reading. Reading in a Foreign Language
27
(1): 136–145.
Nation, P., & Heatley, A. (2002). Range: A program for the analysis of vocabulary in texts [software]. Retrieved from: [URL]
Pellicer-Sánchez, A., & Schmitt, N. (2010). Incidental Vocabulary Acquisition from an Authentic Novel: Do “Things Fall Apart”? Reading in a Foreign Language,
22
(1), 31–55.
Peters, E. (2019). The effect of imagery and on-screen text on foreign language vocabulary learning from audiovisual input. TESOL Quarterly,
53
(4), 1008–1032.
Peters, E., & Webb, S. (2018). Incidental vocabulary acquisition through viewing L2 television and factors that affect learning. Studies in Second Language Acquisition,
40
(3), 551–77.
Peters, E., Heynen, E., & Puimège, E. (2016). Learning Vocabulary through Audiovisual Input: The Differential Effect of L1 Subtitles and Captions. System
63
1, 134–48.
Pujadas, G. (2019). Language learning through extensive TV viewing: A study with adolescent EFL learners (Unpublished doctoral dissertation). University of Barcelona.
Pujadas, G., & Muñoz, C. (2018). What words do we learn better through TV series? The effects of age, proficiency and type of instruction. Paper presented at the Second Language Research Forum, Montreal, Canada.
Pujadas, G., & Muñoz, C. (2019). Extensive viewing of captioned and subtitled TV series: A study of L2 vocabulary learning by adolescents. The Language Learning Journal,
47
(4), 479–496.
Pujadas, G., & Muñoz, C. (2020). Examining Adolescent EFL leaners’ TV viewing comprehension through captions and subtitles. Studies in Second Language Acquisition,
42
(3), 551–575.
Robinson, P., Mackey, A., Gass, S. M., & Schmidt, R. (2012). Attention and awareness in second language acquisition. In Gass, S. M., & Mackey, A. (Eds.) The Routledge Handbook of Second Language Acquisition (pp. 247–267).
Rodgers, M. P. H. (2013). English language learning through viewing television: An investigation of comprehension, incidental vocabulary acquisition, lexical coverage, attitudes, and captions (Unpublished doctoral dissertation). Victoria University of Wellington.
Rodgers, M. P. H. (2016). Extensive listening and viewing: The benefits of audiobooks and television. The European Journal of Applied Linguistics and TEFL,
5
(2), 43–57.
Rodgers, M. P. H., & Webb, S. (2011). Narrow viewing: The vocabulary in related television programs. TESOL Quarterly,
45
(4), 689–717.
Suárez, M. D. M., Gilabert, R., & Moskvina, N. (2021). The mediating role of vocabulary size, working memory, attention and inhibition in early vocabulary learning under different TV genres: An exploratory study. TESOL Journal,
12
(4), e637.
Sydorenko, T. (2010). Modality of input and vocabulary acquisition. Language Learning and Technology,
14
(2), 50–73.
Uchihara, T., Webb, S., & Yanagisawa, A. (2019). The Effects of Repetition on Incidental Vocabulary Learning: A Meta-Analysis of Correlational Studies. Language Learning,
69
(3), 559–99.
van Zeeland, H., & Schmitt, N. (2013). Incidental vocabulary acquisition through L2 listening: A dimensions approach. System,
41
(3), 609–624.
Vandergrift, L. (2007). Recent developments in second and foreign language listening comprehension research. Language Teaching,
40
(3), 191–210.
Vanderplank, R. (2010). Déjà vu? A decade of research on language laboratories, television and video in language learning. Language Teaching
43
(1), 1–37.
Vidal, K. (2011). A comparison of the effects of reading and listening on incidental vocabulary acquisition. Language Learning,
61
(1), 219–258.
Webb, S. (2007). The effects of repetition on vocabulary knowledge. Applied Linguistics,
28
(1), 46–65.
Webb, S. (2010a). Pre-learning low-frequency vocabulary in second language television programmes. Language Teaching Research,
14
(4), 501–15.
Webb, S. (2010b). Using Glossaries to Increase the Lexical Coverage of Television Programs. Reading in a Foreign Language,
22
(1), 201–221.
Webb, S., & Chang, A. C. (2015). Second language vocabulary learning through extensive reading with audio support: How do frequency and distribution of occurrence affect learning? Language Teaching Research,
19
(6), 667–686.
Webb, S., & Nation, P. (2017). How vocabulary is learned. Oxford University Press.
Webb, S., & Rodgers, M. (2009a). Vocabulary demands of television programs. Language Learning,
59
(2), 335–66.
Webb, S., & Rodgers, M. (2009b). The lexical coverage of movies. Applied Linguistics,
30
(3), 407–27.
Winke, P., Gass, S., & Sydorenko, T. (2010). The effects of captioning videos used for foreign language listening activities. Language Learning and Technology,
14
(1), 65–86.
Yang, H. Y. (2014). The effects of advance organizers and subtitles on EFL learners’ listening comprehension skills. Calico Journal,
31
(3), 345–373.