In this paper, we aim to promote the use of linear mixed models (LMMs) in eye-tracking research on subtitling. Using eye tracking to study viewers’ reading of subtitles often warrants controlling for many confounding variables. However, even assuming that these variables are known to researchers, such control may not be possible or desired. Traditional statistical methods such as t-tests or ANOVAs exacerbate the problem due to the use of aggregated data: each participant has one data point per dependent variable. As a solution, we propose the use of LMMs, which are better suited to account for a number of subtitle and participant characteristics, thus explaining more variance. We introduce essential theoretical aspects of LMMs and highlight some of their advantages over traditional statistical methods. To illustrate our point, we compare two analyses of the same dataset: one using a t-test; another using LMMs.
Baayen, Harald. 2008a. Analyzing Linguistic Data: A Practical Introduction to Statistics Using R. Cambridge: Cambridge University Press.
Baayen, Harald. 2008b. Exploratory Data Analysis: An Introduction to R for the Language Sciences. Cambridge: Cambridge University Press.
Balling, Laura Winther. 2008. “A Brief Introduction to Regression Designs and Mixed-Effects Modelling by a Recent Convert.” Copenhagen Studies in Language, 175–92.
Barr, Dale. J., Roger Levy, Christoph Scheepers, and Harry J. Tily. 2013. “Random Effects Structure for Confirmatory Hypothesis Testing: Keep It Maximal.” J Mem Lang 68 (3), 255–278.
Bates, Douglas, Reinhold Kliegl, Shravan Vasishth, and R. Harald Baayen. 2018. “Parsimonius Mixed Models.” arXiv:1506.04967v2 [stat.ME].
Bisson, Marie-Josée, Walter J. B. Heuven, Kathy Conklin, and Richard J. Tunney. 2014. “The Role of Repeated Exposure to Multimodal Input in Incidental Acquisition of Foreign Language Vocabulary.” Language Learning 64 (4): 855–77.
Brown, Violet A.2021. An Introduction to Linear Mixed-Effects Modelling in R. Advances in Methods and Practices in Psychological Science 4 (1): 1–19.
Brysbaert, Marc, and Boris New. 2009. “Moving beyond Kučera and Francis: A Critical Evaluation of Current Word Frequency Norms and the Introduction of a New and Improved Word Frequency Measure for American English.” Behavior Research Methods 41 (4): 977–90.
Burnham, Denis, Kaoru Sekiyama, Gerard Bailly, Pascal Perrier, and Eric Vatikiotis-Bateson. 2012. “Investigating Auditory-Visual Speech Perception Development.” In Audiovisual Speech Processing, edited by Gérard Bailly, Pascal Perrier and Eric Vatikiotis-Bateson, 62–75. Cambridge: Cambridge University Press.
Cambra, Cristina, Olivier Penacchio, Núria Silvestre, and Aurora Leal. 2014. “Visual Attention to Subtitles When Viewing a Cartoon by Deaf and Hearing Children: An Eye-Tracking Pilot Study.” Perspectives 22 (4): 607–17.
Carson, Robyn J., and Christina M. L. Beeson. 2013. “Crossing Language Barriers: Using Crossed Random Effects Modelling in Psycholinguistics Research.” Tutorials in Quantitative Methods for Psychology 9 (1): 25–41.
Crossley, Scott A., Tom Cobb, and Danielle S. McNamara. 2013. “Comparing Count-Based and Band-Based Indices of Word Frequency: Implications for Active Vocabulary Research and Pedagogical Applications.” System 41 (4): 965–81.
Cunnings, Ian, and Ian Finlayson. 2015. “Mixed Effects Modeling and Longitudinal Data Analysis.” In Advancing Quantitative Methods in Second Language Research, edited by Luke Plonsky, 159–81. New York: Routledge.
De Bruycker, Wim, and Gery d’Ydewalle. 2003. “Reading Native and Foreign Language Television Subtitles in Children and Adults.” In The Mind’s Eye: Cognitive and Applied Aspects of Eye Movement Research, edited by Jukka Hyönä, Ralph Radach, and Heiner Deubel, 671–84. Amsterdam: North-Holland.
Dijkstra, Ton, Koji Miwa, Bianca Brummelhuis, Maya Sappelli, and Harald Baayen. 2010. “How Cross-Language Similarity and Task Demands Affect Cognate Recognition.” Journal of Memory and Language 62 (3): 284–301.
Doherty, Stephen, and Jan-Louis Kruger. 2018. “The Development of Eye Tracking in Empirical Research on Subtitling and Captioning.” In Seeing into Screens: Eye Tracking and the Moving Imaging, edited by Jodi Sita, Tessa Dwyer, Sean Redmond, and Claire Perkins, 46–64. London: Bloomsbury.
Duyck, Wouter, Eva Van Assche, Denis Drieghe, and Robert J. Hartsuiker. 2007. “Visual Word Recognition by Bilinguals in a Sentence Context: Evidence for Nonselective Lexical Access.” J Exp Psychol Learn Mem Cogn 33 (4): 663–79.
Díaz-Cintas, Jorge and Aline Remael (2021). Subtitling. Concepts and Practices. London and New York, Routledge.
Ellis, Nick C., and Alan Beaton. 1993. “Psycholinguistic Determinants of Foreign Language Vocabulary Learning.” Language Learning 43 (4): 559–617.
Field, Andy. 2017. Discovering Statistics Using IBM SPSS Statistics. 5th ed. London: SAGE Publications, Inc.
Garson, G. David. 2020. Multilevel modelling: Applications in Stata®, IBM® SPSS®, SAS®, R, & HLMTM. New York: Sage.
Gerber-Morón, Olivia, and Agnieszka Szarkowska. 2018. “Line Breaks in Subtitling: An Eye Tracking Study on Viewer Preferences.” Journal of Eye Movement Research 11 (3): 1–22.
Gerber-Morón, Olivia, Agnieszka Szarkowska, and Bencie Woll. 2018. “The Impact of Text Segmentation on Subtitle Reading.” Journal of Eye Movement Research 11 (4): 1–18.
Ghia, Elisa. 2012. “The Impact of Translation Strategies on Subtitle Reading.” In Eye Tracking in Audiovisual Translation, edited by Elisa Perego, 155–82. Roma: Aracne Editrice.
Gile, Daniel. 2016. “Experimental research.” In Researching Translation and Interpreting, edited by Claudia V. Angelelli and Brian James Baer, 220–228. New York: Routledge.
Hajduk, Gabriela K.2019. “Introduction to Linear Mixed Models.” [URL]
Harrison, Xavier. A., Lynda Donaldson, Maria E. Correa-Cano, Julian Evans, David N. Fisher, Cecily E. D. Goodwin, Beth. S. Robinson, David J. Hodgson, and Richard Inger. 2018. “A Brief Introduction to Mixed Effects Modelling and Multi-Model Inference in Ecology.” PeerJ 61: e4794.
Heck, Ronald H., Scott L. Thomas, and Lynn N. Tabata. 2012. Multilevel Modelling of Categorical Outcomes Using IBM SPSS. Quantitative Methodology Series. New York / London: Routledge.
Hedeker, Donald. 2003. “A Mixed-Effects Multinomial Logistic Regression Model.” Statistics in Medicine 22 (9): 1433–46.
Holmqvist, Kenneth, Marcus Nyström, Richard Andersson, Richard Dewhurst, Halszka Jarodzka, and Joost van de Weijer. 2011. Eye Tracking: A Comprehensive Guide to Methods and Measures. Oxford: Oxford University Press.
Inhoff, Albrecht W., and Keith Rayner. 1986. “Parafoveal Word Processing during Eye Fixations in Reading: Effects of Word Frequency.” Perception & Psychophysics 40 (6): 431–39.
Jensema, Carl. 1998. “Viewer Reaction to Different Television Captioning Speeds.” American Annals of the Deaf 143 (4): 318–24.
Koolstra, Cees M., Tom H. A. Van Der Voort, and Gery d’Ydewalle. 1999. “Lengthening the Presentation Time of Subtitles on Television: Effects on Children’s Reading Time and Recognition.” Communications 24 (4): 407–22.
Krejtz, Izabela, Agnieszka Szarkowska, and Maria Łogińska. 2016. “Reading Function and Content Words in Subtitled Videos.” Journal Of Deaf Studies And Deaf Education 21 (2): 222–32.
Kruger, Jan-Louis. 2018. “Eye tracking in audiovisual translation research”. In The Routledge Handbook of Audiovisual Translation, edited by L. Pérez-González, 350–366. Routledge.
Kruger, Jan-Louis, Esté Hefer, and Gordon Matthew. 2014. “Attention Distribution and Cognitive Load in a Subtitled Academic Lecture: L1 vs. L2.” Journal of Eye Movement Research 7 (5): 1–15.
Kruger, Jan-Louis, Stephen Doherty, and María T. Soto-Sanfiel. 2017. “Original Language Subtitles: Their Effects on the Native and Foreign Viewer.” Comunicar, 501 (January): 23–32.
Lemhöfer, Kristin, and Ton Dijkstra. 2004. “Recognizing Cognates and Interlingual Homographs: Effects of Code Similarity in Language-Specific and Generalized Lexical Decision.” Memory & Cognition 32 (4): 533–50.
Liao, Sixin, Jan-Louis Kruger, and Stephen Doherty. 2020. “The Impact of Monolingual and Bilingual Subtitles on Visual Attention, Cognitive Load, and Comprehension.” Journal of Specialised Translation 331: 70–98.
Liao, Sixin, Lili Yu, Erik D. Reichle, and Jan-Louis Kruger. 2020. “Using Eye Movements to Study the Reading of Subtitles in Video.” Scientific Studies of Reading 25 (5): 417–435.
Łuczak, Krzysztof. 2017. “The Effects of the Language of the Soundtrack on Film Comprehension, Cognitive Load and Subtitle Reading Patterns. An Eye-Tracking Study.” Institite of Applied Linguistics. Warsaw: University of Warsaw.
Mellinger, Christopher D., and Thomas A. Hanson. (2017). Quantitative Research Methods in Translation and Interpreting Studies. New York: Routledge.
Meteyard, Lotte, and Robert A. I. Davies. 2020. “Best Practice Guidance for Linear Mixed-Effects Models in Psychological Science.” Journal of Memory and Language 1121.
Moran, Siobhan. 2009. “The Effect of Linguistic Variation on Subtitle Reception.” Toronto: York University.
Muñoz, Carmen. 2017. “The Role of Age and Proficiency in Subtitle Reading. An Eye-Tracking Study.” System 671: 77–86.
Orrego-Carmona, David. 2015. “The Reception of (Non)Professional Subtitling.” Department of English and German Studies. Tarragona: Universitat Rovira i Virgili.
Peters, Elke. 2019. “Factors Affecting the Learning of Single-Word Items.” In The Routledge Handbook of Vocabulary Studies, edited by Stuart Webb, 125–42. New York: Routledge.
Ragni, Valentina. 2020. “More than Meets the Eye: An Eye-Tracking Study of the Effects of Translation on the Processing and Memorisation of Reversed Subtitles.” Journal of Specialised Translation 331: 99–128.
Rajendran, Dhevi J., Andrew T. Duchowski, Pilar Orero, Juan Martínez, and Pablo Romero-Fresco. 2013. “Effects of Text Chunking on Subtitling: A Quantitative and Qualitative Examination.” Perspectives 21 (1): 5–21.
Rayner, Keith. 2009. “Eye movements and attention in reading, scene perception, and visual search”. The Quarterly Journal of Experimental Psychology 62(8); 1457–1506.
Rayner, Keith, and Susan A. Duffy. 1986. “Lexical Complexity and Fixation Times in Reading: Effects of Word Frequency, Verb Complexity, and Lexical Ambiguity.” Memory & Cognition 14 (3): 191–201.
Rayner, Keith, Tessa Warren, Barbara J. Juhasz, and Simon P. Liversedge. 2004. “The Effect of Plausibility on Eye Movements in Reading.” J Exp Psychol Learn Mem Cogn 30 (6): 1290–1301.
Rayner, K., Timothy J. Slattery, Denis Drieghe, and Simon P. Liversedge. 2011. “Eye Movements and Word Skipping during Reading: Effects of Word Length and Predictability.” J Exp Psychol Hum Percept Perform 37 (2): 514–28.
Romero-Fresco, Pablo. 2015. “Final Thoughts: Viewing Speed in Subtitling.” In The Reception of Subtitles for the Deaf and Hard of Hearing in Europe, edited by Pablo Romero-Fresco, 335–41. Bern: Peter Lang.
RStudio Team. 2021. Studio: Integrated Development for R. RStudio. R. Boston. [URL]
Saldanha, Gabriela, and Sharon O’Brien. 2014. Research Methodologies in Translation Studies. New York: Routledge.
Schwartz, Ana I., and Judith F. Kroll. 2006. “Bilingual Lexical Activation in Sentence Context.” Journal of Memory and Language 55 (2): 197–212.
Singmann, Henrik, and David Kellen. 2019. “An Introduction to Mixed Models for Experimental Psychology.” In New Methods in Cognitive Psychology, edited by Daniel Spieler and Eric Schumacher, 1st ed., 4–31. Routledge.
Silva, Breno B., David Orrego-Carmona, and Agnieszka Szarkowska. 2021a. Data for paper entitled ‘Using linear mixed models to analyze data from eye-tracking research on subtitling’ Figshare. Available at: (accessed March 2022).
Szarkowska, Agnieszka, Izabela Krejtz, Zuzanna Kłyszejko, and Anna Wieczorek. 2011. “Verbatim, Standard, or Edited? Reading Patterns of Different Captioning Styles among Deaf, Hard of Hearing, and Hearing Viewers.” American Annals of the Deaf 156 (4): 363–78.
Szarkowska, Agnieszka, and Olivia Gerber-Morón. 2018. “Viewers Can Keep up with Fast Subtitles: Evidence from Eye Movements.” Plos One 13 (6).
Szarkowska, Agnieszka, and Olivia Gerber-Morón. 2019. “Two or Three Lines: A Mixed-Methods Study on Subtitle Processing and Preferences.” Perspectives 27 (1): 144–64.
“UCLA Institute of Digital Research.” 2021. FAQ How Do I Interpret a Regression Model When Some Variables Are Log Transformed? 2021. [URL]
Winter, Bodo. 2020. Statistics for Linguists: An Introduction Using R. New York / London: Routledge.
Ydewalle, Géry d’, Johan van Rensbergen, and Joris Pollet. 1987. “Reading a Message When the Same Message Is Available Auditorily in Another Language: The Case of Subtitling.” In Eye Movements: From Physiology to Cognition, edited by J. K. O’Regan and A. Levy-Schoen, 313–21. Amsterdam/New York: Elsevier.
Ydewalle, Géry d’, Caroline Praet, Karl Verfaillie, and Johan Van Rensbergen. 1991. “Watching Subtitled Television: Automatic Reading Behavior.” Communication Research 18 (5): 650–66.
Ydewalle, Géry d’, and Wim De Bruycker. 2007. “Eye Movements of Children and Adults While Reading Television Subtitles.” European Psychologist 12 (3): 196–205.
Cited by (9)
Cited by nine other publications
Balaskas, Stefanos, Maria Koutroumani, Maria Rigou & Spiros Sirmakessis
2024. From Eye Movements to Personality Traits: A Machine Learning Approach in Blood Donation Advertising. AI 5:2 ► pp. 635 ff.
Berlin Khenis, A.A., A.N. Puchkova, E.S. Kashchenko & M.Yu. Lebedeva
2024. Metacognitive Reading Strategies: Analysis of Self-report Data and Oculomotor Behaviour of Russian Schoolchildren. Psychological Science and Education 29:2 ► pp. 15 ff.
Das, Anwesha, Zekun Wu, Iza Skrjanec & Anna Maria Feit
2024. Shifting Focus with HCEye: Exploring the Dynamics of Visual Highlighting and Cognitive Load on User Attention and Saliency Prediction. Proceedings of the ACM on Human-Computer Interaction 8:ETRA ► pp. 1 ff.
Szarkowska, Agnieszka, Valentina Ragni, Sonia Szkriba, Sharon Black, David Orrego-Carmona, Jan-Louis Kruger & Antonio Peña-Fernández
2024. Watching subtitled videos with the sound off affects viewers’ comprehension, cognitive load, immersion, enjoyment, and gaze patterns: A mixed-methods eye-tracking study. PLOS ONE 19:10 ► pp. e0306251 ff.
2024. AI-Powered Eye Tracking for Bias Detection in Online Course Reviews: A Udemy Case Study. Big Data and Cognitive Computing 8:11 ► pp. 144 ff.
Hrabčáková, Adriána
2023. Some Current Trends and Research Possibilities in the Audiovisual Translation of the 2020s. Language Culture Politics International Journal 1:1/2023 ► pp. 49 ff.
Hrabčáková, Adriána
2023. Some Current Trends and Research Possibilities in the Audiovisual Translation of the 2020s. Language Culture Politics International Journal 1:1/2023 ► pp. 49 ff.
Greco, Gian Maria, Anna Jankowska & Agnieszka Szarkowska
This list is based on CrossRef data as of 8 november 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.