This paper will explore the multimodal approach to audiovisual translation (AVT). It must first be stressed, however, that most research on multimodality has not as yet focused on questions of translation. The Routledge Handbook of Multimodal Analysis (Jewitt 2009), which contains articles by most of the leading figures in the field, while representing a major step forward in multimodal studies, does not tackle translation head on. The word ‘translation’ does not even appear in the index. Over a relatively short time span, most of the major contributions to the field have been more purely linguistically based and intent on providing keys to the understanding of the interplay of semiotic resources such as words, images, gesture, music, light, etc. (see O’Toole 1994; Kress and van Leeuwen 1996; Martinec 2000; Unsworth 2001; Baldry and Thibault 2006, etc.). The work of these scholars, however, has provided an impetus to developing ideas on how to exploit multimodal analyses in the area of AVT. Thibault’s work, for example, on the ‘multimodal transcription’ provided this author with the basis for investigating how the integration of semiotic modalities in a film text could assist the subtitler in making those all-important decisions on what to retain and what to discard when faced with time constraints. Other scholars have studied the co-articulation of words and image in their discussion of how different modalities realize social functions and make meaning (O’Halloran 2008; Bednarek 2010), emphasising the importance of supplementing purely linguistic analyses with studies of all the other semiotic resources that make up a multimodal text. Findings will inevitably be reported verbally but the analyses need to explore the concept of integration and how other resources can interact with language and, crucially, how translators can be made sensitive to the entire semiotic impact of a multimodal text.
Baldry, Anthony, ed. 2000. Multimodality and Multimediality in the Distance Learning Age. Campobasso: Palladino Editore.
Baldry, Anthony. 2002. “Subtitling Dynamic Multimodal Texts: The Development of Interactive Self-access Tools.” In The Economy Principle in English: Linguistic, Literary and Cultural Perspectives, ed. by Giovanni Iamartino, Maria L. Bignami, and Carlo Pagetti, 54–62. Milan: Edizioni Unicopi.
Baldry, Anthony. 2005. A Multimodal Approach to Text Studies in English. Campobasso: Palladino Editore.
Baldry, Anthony, and Paul Thibault. 2006. Multimodal Transcription and Text Analysis. London: Equinox.
Bateman, John. 2009. “Film and Representation: Making Filmic Meaning.” In Metarepresentation, Self-Organisation and Art, ed. by Wolfgang Wildgen, and Barend van Heusden, 137–162. Bern: Peter Lang.
Bednarek, Monica. 2010. The Language of Fictional Television: Drama and Identity. London: Continuum.
Bordwell, David. 2008. “Three Dimensions of Film Narrative.” In Poetics of Cinema, 85–134. London: Routledge.
Bruti, Silvia. 2006. “Cross-cultural Pragmatics: The Translation of Implicit Compliments in Subtitles.” Jostrans 61: 185–197.
Cattrysse, Patrick, and Yves Gambier. 2008. “Screenwriting and Translating Screenplays.” In The Didactics of Audiovisual Translation, ed. by Jorge Díz-Cintas, 39–55. New York: Routledge.
Fillmore, Charles. 1982 (2006). “Frame Semantics.” In Cognitive Linguistics: Basic Readings, ed. by Dirk Geeraerts, 373–400. Berlin: Mouton de Gruyter.
Flynn, Gillian. 2013. Gone Girl. London: Phoenix.
Fryer, Louise, and Jonathan Freeman. 2012. “Presence in Those with and without Sight: Audio Description and its Potential for Visual Reality Applications.” Journal of CyberTherapy and Rehabilitation 5 (1): 15–23.
Gambier, Yves. 2006. “Multimodality and Audiovisual Translation.” In Audiovisual Translation Scenarios: Proceedings of the Marie Curie Euroconferences MuTra: Audiovisual Translation Scenarios, Copenhagen 1–5 May 2006, ed. by Mary Carroll, Heidrun Gerzymisch-Arbogast, and Sandra Nauert. [URL]. Accessed December 22, 2015.
Gambier, Yves. 2013. “Moves towards Multimodality and Language Representation in Movies.” In Readings in Intersemiosis and Multimedia. Proceedings of the 3rd TICOM, Pavia, May 2006, ed. by Elena Montagna, 1–10. Pavia: Como.
Gottlieb, Henrik. 1992. “Subtitling: A New University Discipline.” In Teaching Translation and Interpreting: Training Talent and Experience. Papers from the First Language International Conference, Elsinore, Denmark 1991, ed. by Cay Dollerup, and Anne Loddegaard, 161–170. Amsterdam: John Benjamins.
Gregory, Michael. 2002. “Phasal Analysis within Communication Linguistics: Two Contrastive Discourses.” In Relations and Functions within and around Language, ed. by Peter H. Fries, Michael Cumming, David Lockwood, and William Spruiell, 316–345. London: Continuum.
Ivarsson, Jan. 1992. Subtitling for the Media. A Handbook of an Art. Stockholm: Transedit.
Jewitt, Carey, ed. 2009. The Routledge Handbook of Multimodal Analysis. Abingdon: Routledge.
Kress, Gunther, and Theo van Leeuwen. 1996. Reading Images: the Grammar of Visual Design. London: Routledge.
Kress, Gunther, and Theo van Leeuwen. 2001. Multimodal Discourse: the Modes and Media of Contemporary Communication. London: Edward Arnold.
Lomheim, Sylfest. 1999. “The Writing on the Screen. Subtitling: A Case Study from Norwegian Broadcasting (NRK), Oslo.” In Word, Text, Translation, ed. by Gunilla Anderman, and Margaret Rogers, 190–207. Clevedon: Multilingual Matters.
Malcolm, Karen. 2010. Phasal Analysis: Analysing Discourse through Communication Linguistics. London: Continuum.
Mazur, Iwona. 2014. “Gestures and Facial Expressions in Audio Description.” In Audio Description: New Perspectives Illustrated, ed, by Anna Maszerowska, Anna Matamala, and Pilar Orero, 179–198. Amsterdam: John Benjamins.
Minsky, Marvin. 1986. The Society of Mind. New York: Simon and Schuster.
Perego, Elisa. 2009. “The Codification of Non-Verbal Information in Subtitled Texts.” In New Trends in Audiovisual Translation, ed. by Jorge D. Cintas, 58–69. Bristol: Multilingual Matters.
Pettit, Zoe. 2013. “Language and Image: Translating the Audiovisual Text in Subtitling and Dubbing.” In Readings in Intersemiosis and Multimedia. Proceedings of the 3rd TICOM, Pavia, May 2006, ed. by Elena Montagna, 213–232. Pavia: Como.
Ramière, Nathalie. 2006. “Reaching a Foreign Audience: Cultural Transfers in Audiovisual Translation.” Jostrans 61: 152–166.
Snyder, Joel. 2005. “Audio Description. The Visual made Verbal across Arts Disciplines across the Globe.” Translating Today 41: 15–17.
Taylor, Christopher. 2003. “Multimodal Transcription in the Analysis, Translation and Subtitling of Italian Films.” The Translator 9 (2): 191–206.
Taylor, Christopher. 2004. “Multimodal Text Analysis and Subtitling.” In Perspectives on Multimodality, ed. by Eija Ventola, Cassily Charles, and Martin Kaltenbacher, 153–172. Amsterdam: John Benjamins.
Taylor Christopher. 2013. “Multimodality and Audiovisual Translation.” In Handbook of Translation Studies, ed. by Yves Gambier, and Luc van Doorslaer, vol. 41, 98–104. Amsterdam: John Benjamins.
Taylor, Christopher, ed. Forthcoming (2016). A Text of Many Colours – Translating the Television Series ‘The West Wing’. Special issue of Intralinea 18.
Thibault, Paul. 2000. “The Multimodal Transcription of a Television Advertisement: Theory and Practice.” In Baldry 2000, 311–385.
Unsworth, Len. 2001. Teaching Multiliteracies across the Curriculum: Changing Contexts of Text and Image in Classroom Practice. Buckingham: Open University Press.
Wilken, Nicole, and Jan-Louis Kruger. Forthcoming (2016). “Putting the Audience in the Picture: Mise-en-shot and Psychological Immersion in Audio Described Film.” In Empirical Approaches to Audiovisual Translation, ed. by Elisa Perego, special issue of Across Languages and Cultures 17(2).
Cited by (18)
Cited by 18 other publications
Abu-Rayyash, Hussein & Raeda Al-Ramadan
2024. Quality Assessment of Creating Interlingual Pre-Recorded Subtitles for the Deaf and Hard-of-Hearing (SDH): The LOT Model. Studies in Linguistics, Culture, and FLT 12:2 ► pp. 219 ff.
Liu, Lisheng
2024. Multimodal approaches to Chinese-English translation and interpreting. Social Semiotics 34:4 ► pp. 726 ff.
Lu, Siwen
2024. More than words: a multimodal analytical framework for studying the subtitling of swearwords. Perspectives 32:1 ► pp. 14 ff.
Van Meerbergen, Sara & Reglindis De Ridder
2024. Global or glocal heroes in PJs? A multimodal analysis of female superhero and villain depiction in dubbed children’s animation. Perspectives► pp. 1 ff.
2023. Chinese Subtitling of Participants in an English Film Mission Impossible: Ghost Protocol: A Narrative-Informed Visual-Verbal Reference. Sage Open 13:3
Gao, Fei
2023. Getting the message in ‘Sound’ across at conference interpreting: a case study on rendering prosodic emphasis. Perspectives 31:6 ► pp. 1033 ff.
Đorđević, Jasmina P. & Dušan Stamenković
2023. Classification of multimodal translation errors in the entertainment industry: a proposal. The Translator 29:3 ► pp. 265 ff.
2021. Im/Politeness, Fiction and AVT. In Understanding Im/politeness Through Translation [Advances in (Im)politeness Studies, ], ► pp. 137 ff.
Ulvydienė Huber, Loreta & Viktorija Lideikytė
2021. Audiovisual Communication and Subtitling from the Perspective of Semiotic Cohesion: A Case Study of “Garden of Eden”. Respectus Philologicus :39 (44)
Messerli, Thomas C.
2020. Ocean's Eleven scene 12 – lost in transcription. Perspectives 28:6 ► pp. 837 ff.
Li, Long, Xi Li & Jun Miao
2019. A translated volume and its many covers – a multimodal analysis of the influence of ideology. Social Semiotics 29:2 ► pp. 261 ff.
2024. The translation of multimodal texts: challenges and theoretical approaches. Perspectives 32:1 ► pp. 1 ff.
This list is based on CrossRef data as of 7 november 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.