Cover not available
Part of
Reassessing Dubbing: Historical approaches and current trends
Edited by Irene Ranzato and Serenella Zanotti
[Benjamins Translation Library 148] 2019
► pp. 125–144

Chapter 6
Are we all together across languages?

An eye tracking study of original and dubbed films

In one of the first attempts to apply eye-tracking technology to the area of dubbing, this chapter reports on an experiment conducted in 2016 with viewers of the original English and dubbed Italian version of Wes Anderson’s The Grand Budapest Hotel. The study analysed the gaze behaviour of participants watching different scenes, focusing on variations in attentional synchrony and visual momentum. The results show less attentional synchrony and a higher visual momentum for the dubbing viewers. They also support the existence of the so-called dubbing effect, an unconscious eye movement strategy performed by dubbing viewers to avoid looking at mouths in dubbing, which prevails over the natural and idiosyncratic way in which they watch original films and real-life scenes.
Keywords: attentional synchrony, dubbing, dubbing effect, eye tracking, perception, reception, visual momentum
Article outline
  • 1.Introduction
  • 2.Key concepts in eye-tracking research and beyond
  • 3.Eye tracking and audiovisual translation
  • 4. The Grand Budapest Hotel: An eye-tracking study
    • 4.1Research questions and material
    • 4.2Experiment set up
    • 4.3Close ups and medium close ups; the dubbing effect
    • 4.4Written text in dubbed films
    • 4.5Action scenes and long shots
  • 5.Conclusion
  • References
  • Filmography
Published online: 6 August 2019
DOI logo
https://doi.org/10.1075/btl.148.06di
References (35)
References
Alton, John. 1949. Painting with Light. California: University of California Press.Google Scholar
Berger, John. 1972. Ways of Seeing. London: Penguin Books.Google Scholar
Bisson, Marie-Josée, Walter Van Heuven, Kathy Conklin, and Richard Tunney. 2014. “Processing of Native and Foreign Language Subtitles in Films: An Eye Tracking Study.” Applied Psycholinguistics 35 (2): 399–418. DOI logoGoogle Scholar
Chion, Michael. 1994. Audio-Vision: Sound on Screen. New York: Columbia University Press.Google Scholar
Coleridge, Samuel T. 1871. Biographia Literaria, ed. by J. Shawcross, Oxford Clarendon Press.Google Scholar
Coutrot, Antoine, Nathalie Guyader, Gelu Ionescu, and Alice Caplier. 2012. “Influence of Soundtrack on Eye Movements During Video Exploration”. Journal of Eye Movement Research 5(5): 1–10.Google Scholar
Crouzet, Sebastien M., Holle Kirchner, and Simon J. Thorpe. 2010. “Fast Saccades Toward Faces: Face Detection in Just 100 ms”. Journal of Vision, 10 (4): 1–17. DOI logoGoogle Scholar
De Linde, Zoé, and Kay, Neil. 1999. The Semiotics of Subtitling. Manchester, UK: St. Jerome Publishing.Google Scholar
d’Ydewalle, Géry, Johan Van Rensbergen, and Joris Pollet. 1987. “Reading a Message when the Same Message Is Available Auditorily in Another Language: The Case of Subtitling”. In Eye Movements: From Physiology to Cognition, ed. by J.K O’Reagan and A. Lévy Schoen, 313–321. Amsterdam: Elsevier Science Publishers B.V. (North-Holland). DOI logoGoogle Scholar
d’Ydewalle, Géry, and Ingrid Gielen. 1992. “Attention Allocation with Overlapping Sound, Image, and Text.” In Eye Movements and Visual Cognition: Scene Perception and Reading, ed. by Keith Rayner, 415–427. New York: Springer-Verlag. DOI logoGoogle Scholar
d’Ydewalle, Géry, and Wim De Bruycker. 2007. “Eye Movements of Children and Adults while Reading Television Subtitles”. European Psychologist 12: 196–205. DOI logoGoogle Scholar
Dyer, Adrian G., and Sarah Pink. 2015. “Movement, Attention and Movies: The Possibilities and Limitations of Eye Tracking?”. Refractory 25: online at [URL].Google Scholar
Dwyer, Tessa, Claire Perkins, Sean Redmond, and Jodi Sita (eds.) 2018. Seeing into Screens: Eye Tracking and the Moving Image, Bloomsbury.Google Scholar
Foulsham, Tom, and Lucy Anne Sanderson. 2013. “Look Who’s Talking? Sound Changes Gaze Behaviour in a Dynamic Social Scene. ” Visual Cognition 21 (7): 922–944. DOI logoGoogle Scholar
Hochberg, Jullian. 1962. “The Psychophysics of Pictorial Perception.” Audio-Visual Communications Review 10: 22–54.Google Scholar
Hochberg, Julian, and Virginia Brooks. 1978. “Film Cutting and Visual Momentum.” In Eye Movements and the Higher Psychological Functions, ed. by John W. Senders, Dennis F. Fisher, and Richard A. Monty, 293–313. Hillsdale: Laurence Erlbaum.Google Scholar
Hvelplund, Kristian Tangsgaard 2017. “Eye tracking and the process of dubbing translation.” In Fast-forwarding with Audiovisual Translation, ed. by Jorge Díaz Cintas, and Kristijan Nikolić, 110–125. London: Multilingual Matters.Google Scholar
Jensema, Carl, J. Sameh el Sharkawy, Ramalinga S. Danturthi, Robert Burch, and David Hsu. 2000. “Eye Movement Patterns of Captioned Television Viewers”. American Annals of the Deaf 145 (3): 275–285. DOI logoGoogle Scholar
Land, Michael F., and David N. Lee. 1994. “Where We Look when We Steer.” Nature 369: 742–744. DOI logoGoogle Scholar
Mercado, Gustavo. 2010. The Filmmaker’s Eye: Learning (and Breaking) the Rules of Cinematic Composition. Amsterdam: Focal Press.Google Scholar
Perego, Elisa, Fabio del Missier, Marco Porta, and Mauro Mosconi. 2010. “The Cognitive Effectiveness of Subtitle Processing”. Media Psychology 13: 243–272. DOI logoGoogle Scholar
Perego, Elisa, David Orrego-Carmona and Sara Bottiroli. 2016. “An Empirical Take on the Dubbing vs Subtitling Debate. An Eye Movement Study”. In Lingue e Linguaggi 19: 255–274.Google Scholar
Rassell, Andrea, Redmond, Sean, et al.. 2016. “Seeing, Sensing Sound: Eye Tracking Soundscapes in Saving Private Ryan and Monster Inc.” In Making Sense of Cinema. Empirical Studies into Film Spectators and Spectatorship, ed. by CarrieLynn D. Reinhard, and Christopher J. Olson, 139–164. New York/London: Bloomsbury.Google Scholar
Redmond, Sean, and Craig Batty. 2015. “Seeing into Things: Eye Tracking the Moving Image.” Refractory 25: online at [URL].Google Scholar
Robinson, Jennifer, Jane Stadler and Andrea Rassell. 2015. “Sound and Sight: An Exploratory Look at Saving Private Ryan through the Eye Tracking Lens.” Refractory. A Journal of Entertainment Media 25. Available online: [URL] (accessed 1 October 2016).Google Scholar
(2015). “Sound and Sight: An Exploratory Look at Saving Private Ryan through the Eye-tracking Lens”. Refractory: a Journal of Entertainment Media 25. [URL] (consulted 20.09.2018).Google Scholar
Romero-Fresco, Pablo. 2009. “Naturalness in the Spanish Dubbing Language: a Case of Not-so-close Friends.”. Meta 54 (1): 49–72.Google Scholar
Romero-Fresco, P. (forthcoming) “The dubbing effect: An eye-tracking study on how viewers make dubbing work”. Journal of Audiovisual Translation, Jostrans, 33, Special Issue: Experimental Research and Cognition in AVT.
Smith, Tim J., and John M. Henderson. 2008. “Edit Blindness: The Relationship Between Attention and Global Change Blindness in Dynamic Scenes.” Journal of Eye Movement Research 2(2): 1–17.Google Scholar
Smith, Tim J. 2011. “Watching You Watch THERE WILL BE BLOOD.” Observations on Film Art. ed. by Kristin Thompson and David Bordwell. Online at: [URL].Google Scholar
, 2015. “Read, Watch, Listen. A Commentary on Eye Tracking and Moving Images.” Refractory 25: online at [URL].Google Scholar
Treuting, Jennifer. 2006. “Eye Tracking and Cinema: A Study of Film Theory and Visual Perception”. Society of Motion Picture and Television Engineers 115 (1): 31–40.Google Scholar
Vilarò, Anna, and Tim J. Smith. (2011) “Subtitle Reading Effects on Visual and Verbal Information Processing in Films.” Published abstract In Perception. ECVP abstract supplement. 40:153. European Conference on Visual Perception. Toulouse, France.Google Scholar
Filmography
Casablanca. Michael Curtiz. USA. 1942.Google Scholar
The Grand Budapest Hotel. Wes Anderson. UK/Germany. 2014.Google Scholar
Cited by (7)

Cited by seven other publications

Wang, Ruibing & Bin Liu
2024. Knowledge Mapping of Audiovisual Translation (2005–2024): A Bibliometric Analysis. Frontiers in Humanities and Social Sciences 4:8 ► pp. 90 ff. DOI logo
Brannon, William, Yogesh Virkar & Brian Thompson
2023. Dubbing in Practice: A Large Scale Study of Human Localization With Insights for Automatic Dubbing. Transactions of the Association for Computational Linguistics 11 ► pp. 419 ff. DOI logo
Chen, Xuemei
2023.  Danmu-assisted learning through back translation. Babel. Revue internationale de la traduction / International Journal of Translation / Revista Internacional de Traducción 69:5 ► pp. 598 ff. DOI logo
Hayes, Lydia
2023. English Dubs: Why are Anglophone Viewers Receptive to English Dubbing on Streaming Platforms and to Foreign-Accent Strategies?. Íkala, Revista de Lenguaje y Cultura 28:2 ► pp. 1 ff. DOI logo
Sánchez-Mompeán, Sofía
2023. Engaging English Audiences in the Dubbing Experience: A Matter of Quality or Habituation?. Íkala, Revista de Lenguaje y Cultura 28:2 ► pp. 1 ff. DOI logo
Di Giovanni, Elena
2020. Reception Studies and Audiovisual Translation. In The Palgrave Handbook of Audiovisual Translation and Media Accessibility [Palgrave Studies in Translating and Interpreting, ], ► pp. 397 ff. DOI logo
Raffi, Francesca
2020. The Impact of Italian Dubbing on Viewers’ Immersive Experience: An Audience Reception Study. Online Journal of Communication and Media Technologies 10:3 ► pp. e202019 ff. DOI logo

This list is based on CrossRef data as of 3 september 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.