Cover not available
Article published In:
Computer-mediated communication in class: Fostering access to digital mediascapes
Edited by Stefania Maci and Marianna Lya Zummo
[Translation and Translanguaging in Multilingual Contexts 10:1] 2024
► pp. 28–53

Towards online audiovisual translation of videos

| University of Messina
The paper reports on the progress made by second-year language degree university students as regards their use of online corpus construction, annotation and search tools when exploring video genres (Baldry and Thibault 2010, 2020; Jablonkai and Csomay 2022). Through first-hand experiences undertaken with students using the OpenMWS platform (Taibi 2020; http://​mws​.pa​.itd​.cnr​.it/ ), the paper describes the ways in which, through the creation of new interactive communities, participation in the Messina OVP (Online Video Project) has empowered these students with regard to their acquisition of textual competences and, above all, their preparation for more intensive AVT studies in their third and final year. As such, the paper is a first-step towards online audiovisual translation that lays the bases for subsequent translation work and which is thus concerned with the underlying cohesion and coherence of the overall text as well as the analysis of specific textual formations in these videos.
Keywords: multimodality, AVT (audiovisual translation), cohesion, coherence, video corpora
Article outline
  • 1.Introduction
  • 2.Materials and methodology
    • 2.1Stage 1: Video selection
    • 2.2Stage 2: The transcript
    • 2.3Stage 3: Sequencing
    • 2.4Stage 4: Synchronising oral discourse-written discourse
    • 2.5Stage 5: Language analysis
    • 2.6Stage 6: Translation and subtitling
  • 3.Findings
  • 4.Discussion
  • 5.Conclusion
  • Notes
  • References
Published online: 9 February 2024
DOI logo
https://doi.org/10.1075/ttmc.00125.cam
References (58)
References
Ackerley, Katherine, and Francesca Coccetta. 2007. “Enriching Language Learning through a Multimedia Corpus.” ReCALL 19(3): 351–370. DOI logoGoogle Scholar
Aston, Guy, Silvia Bernardini, and Dominic Stewart, eds. 2004. Corpora and Language Learners. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Baldry, Anthony. 2021. “Multimodality and Genre Evolution. A Decade-by-decade Approach to Online Video Genre Analysis.” In Mediation and Multimodal Meaning Making in Digital Environments, ed. by Ilaria Moschini, and Maria Grazia Sindoni, 151–167. London: Routledge. DOI logoGoogle Scholar
Baldry, Anthony, and Paul Thibault. 2001. “Towards Multimodal Corpora.” In Corpora in the Description and Teaching of English, ed. by Guy Aston, and Lou Burnard, 87–102. Bologna: CLUEB.Google Scholar
. 2006. “Multimodal Corpus Linguistics.” In System and Corpus: Exploring Connections, ed. by Geoff Thompson, and Susan Hunston, 164–183. London: Equinox.Google Scholar
. 2010 [2006]. Multimodal Transcription and Text Analysis. A Multimedia Toolkit with Associated On-line Course. London: Equinox.Google Scholar
Baldry, Anthony, Alessandro Gaggia, and Marco Porta. 2011. “Multimodal Web Concordancing and Annotation: An Overview of the MCAWEB System.” In Identities in Transition in the English-speaking World, ed. by Nicoletta Vasta, Antonella Riem Natale, Maria Bortoluzzi, and Debora Saidero, 39–60. Udine: Forum Editrice.Google Scholar
Baldry, Anthony, and Paul Thibault. 2020. “Analysis A: A Model for Multimodal Corpus Construction of Video Genres: Phasal Analysis and Resource Integration Using OpenMWS.” In Multiliteracy Advances and Multimodal Challenges in ELT Environments, ed. by Nicoletta Vasta, and Anthony Baldry, 159–173. Udine: Forum Editrice.Google Scholar
Baldry, Anthony, Deirdre Kantz, Anna Loiacono, Ivana Marenzi, Davide Taibi, and Francesca Tursi. 2020. “THE MWSWEB PROJECT: Accessing Medical Discourse in Video-hosting Websites.” Lingue e Linguaggi 401: 433–472.Google Scholar
Baldry, Anthony, and Deirdre Kantz. 2022. “Corpus-assisted Approaches to Online Multimodal Discourse Analysis of Videos.” In Analyzing Multimodality in Specialized Discourse Settings. Innovative Research Methods and Applications, ed. by Veronica Bonsignori, Belinda Crawford Camiciottoli, and Denise Filmer, 1–22. Wilmington: Vernon Press.Google Scholar
Baldry, Anthony, Francesca Coccetta, and Deirdre Kantz. 2022. “What if? Healthcare Simulations, Online Searchable Video Corpora and Formulating Hypotheses.” In Analysing Health Discourse in Digital Environments, ed. by Anna F. Plastina, 126–146. Newcastle upon Tyne: Cambridge Scholars Publishing.Google Scholar
Bateman, John. 2017. “Triangulating Transmediality: A Multimodal Semiotic Framework Relating Media, Modes and Genres.” Discourse, Context Media 201: 160–174. DOI logoGoogle Scholar
Bianchi, Francesca, Davide Taibi, Philipp Kemkes, and Ivana Marenzi. 2022. “Learning Analytics at the Service of Interpreter Training in Academic Curricula.” In Corpus Linguistics and Translation Tools for Digital Humanities, ed. by Stefania Maci, 153–176. London: Bloomsbury. DOI logoGoogle Scholar
Bogucki, Łukasz, and Jorge Díaz-Cintas. 2020. “An Excursus on Audiovisual Translation.” In The Palgrave Handbook of Audiovisual Translation and Media Accessibility, ed. by Łukasz Bogucki, and Mikołaj Deckert, 11–32. London: Palgrave Macmillan. DOI logoGoogle Scholar
Böhme, Gernot. 2012. Invasive Technification. Critical Essays in the Philosophy of Technology. London: Bloomsbury.Google Scholar
Bolter, David, and Richard Grusin. 1998. Remediation. Understanding New Media. Boston: MIT Press.Google Scholar
Boulton, Alex. 2017. “Data-Driven Learning and Language Pedagogy.” In Language, Education and Technology, ed. by Steven Thorne, and Stephen May, 181–192. New York: Springer. DOI logoGoogle Scholar
Boulton, Alex, and Nina Vyatkina. 2021. “Thirty Years of Data-driven Learning: Taking Stock and Charting New Directions.” Language Learning & Technology 25(3): 66–89.Google Scholar
Brown, Gillian, and George Yule. 1983. Discourse Analysis. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Bruti, Silvia. 2018. “Spoken Discourse and Conversational Interaction in Audiovisual Translation.” In The Routledge Handbook of Audiovisual Translation, ed. by Luis Pérez-González, 192–208. London: Routledge. DOI logoGoogle Scholar
Cambria, Mariavita. 2012. ““I’ll Never Say I’m sorry”: A Former Italian Prime Minister in a Corpus of English Online Newspapers.” Web Genres and Web Tools, ed. by Mariavita Cambria, Cristina Arizzi, and Francesca Coccetta, 277–294. Como: Ibis.Google Scholar
Cambria, Mariavita, Cristina Arizzi, and Francesca Coccetta, eds. 2012. Web Genres and Web Tools. Como: Ibis.Google Scholar
Chaume, Frederic. 2018. “An Overview of Audiovisual Translation: Four Methodological Turns in a Mature Discipline.” Journal of Audiovisual Translation 1(1): 40–63. DOI logoGoogle Scholar
Coccetta, Francesca. 2022. “Multimodal Corpora and Concordancing in Data-driven Learning.” In The Routledge Handbook of Corpora in English Language Teaching and Learning, ed. by Reka Jablonkai, and Eniko Csomay. London: Routledge. DOI logoGoogle Scholar
Cronin, Martin. 2013. “Translation and Globalization.” In The Routledge Handbook of Translation Studies, ed. by Carmen Millán, and Francesca Bartrina, 491–502. London: Routledge.Google Scholar
De Beaugrande, Robert-Alain, and Wolfang Dressler. 1981. Introduction to Text Linguistics. Harlow: Longman. DOI logoGoogle Scholar
Díaz-Cintas, Jorge. 2018. “Subtitling’s a Carnival’: New Practices in Cyberspace.” The Journal of Specialised Translation 301: 127–149.Google Scholar
Díaz-Cintas, Jorge, and Serenella Massidda. 2019. “Technological Advances in Audiovisual Translation.” In The Routledge Handbook of Translation and Technology, ed. by Minako O’Hagan, 255–270. London: Routledge. DOI logoGoogle Scholar
Dontcheva-Navratilova, Olga, and Renata Povolná, eds. 2009. Coherence and Cohesion in Spoken and Written Discourse. Newcastle upon Tyne: Cambridge Scholars Publishing.Google Scholar
Greco, Gian Maria. 2018. “The Nature of Accessibility Studies.” Journal of Audiovisual Translation 1(1): 205–232. DOI logoGoogle Scholar
Halliday, Michael A. K., and Ruqaiya Hasan. 1976. Cohesion in English. London: Arnold.Google Scholar
Hansen-Schirra, Silvia, Stella Neumann, and Eric Steiner. 2007. “Cohesive Explicitness and Explicitation in an English-German Translation Corpus.” Language Contrast 7 (2): 241–265. DOI logoGoogle Scholar
Herring, Susan C. 2013. “Discourse in Web 2.0: Familiar, Reconfigured, and Emergent.” In Discourse 2.0: Language and New Media, ed. by Deborah Tannen, and Anne M. Trester, 1–25. Washington, DC: Georgetown University Press.Google Scholar
Hoffmann, Christian R. 2012. Cohesive Profiling: Meaning and Interaction in Personal Weblogs. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Hunston, Susan, and Gill Francis. 1995. Pattern Grammar. A Corpus-Driven Approach to the Lexical Grammar of English. Amsterdam: John Benjamins.DOI logoGoogle Scholar
Jablonkai, Reka, and Eniko Csomay, eds. 2022. The Routledge Handbook of Corpora in English Language Teaching and Learning. London: Routledge. DOI logoGoogle Scholar
Johns, Tim, and Florence Davies. 1983. “Text as a Vehicle for Information: The Classroom Teaching of Reading in ESP.” Reading in a Foreign Language 1(1): 1–19.Google Scholar
Johns, Tim. 1993. “Data-driven Learning: An Update.” TELL&CALL 21: 4–10.Google Scholar
. 1997. “Contexts: The Background, Development and Trialling of a Concordance-based CALL Program.” In Teaching and Language Corpora, ed. by Anne Wichmann, Steven Fligelstone, Tom McEnery, and Gerry Knowles, 100–115. Harlow: Longman.Google Scholar
. 2002. “Data-driven Learning: The Perpetual Challenge.” In Teaching and Learning by Doing Corpus Analysis, ed. by Bernhard Kettemann, and George Marko, 107–117. Amsterdam: Rodopi. DOI logoGoogle Scholar
Klein, Yvonne. 2012. “Cohesion in English and German.” In Cross-Linguistic Corpora for the Study of Translations: Insights from the Language Pair English-German, ed. by Silvia Hansen-Schirra, Stella Neumann, and Eric Steiner, 161–172. Berlin: De Gruyter Mouton. DOI logoGoogle Scholar
Liu, Yu, and Kate O’Halloran. 2009. “Intersemiotic Texture: Analyzing Cohesive Devices Between Language and Images.” Social Semiotics 19(4): 367–388. DOI logoGoogle Scholar
O’Hagan, Minako. 2016. “Massively Open Translation: Unpacking the Relationship Between Technology and Translation in the 21st Century.” International Journal of Communication 101(2016): 929–946.Google Scholar
, ed. 2019. The Routledge Handbook of Translation and Technology. London: Routledge. DOI logoGoogle Scholar
Pavesi, Maria, and Elisa Perego. 2008. “Tailor-made Interlingual Subtitling as a Means to Enhance Second Language Acquisition.” In The Didactics of Audiovisual Translation, ed. by Jorge Díaz-Cintas, 215–225. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Pavesi, Maria. 2018. “Corpus-based Audiovisual Translation Studies: Ample Room for Development.” The Routledge Handbook of Audiovisual Translation, ed. by Luis Pérez-González, 315–333. London: Routledge. DOI logoGoogle Scholar
Perego, Elisa, and Christopher Taylor, eds. 2022. The Routledge Handbook of Audio Description. London: Routledge.Google Scholar
Sinclair, John. 1991. Corpus, Concordance, Collocation. Oxford: Oxford University Press.Google Scholar
Sindoni, Maria Grazia. 2013. Spoken and Written Discourse in Online Interactions: A Multimodal Approach. London: Routledge.Google Scholar
Sindoni, Maria Grazia, Elisabetta Adami, Styliani Karatza, Ivana Marenzi, Ilaria Moschini, and Sandra Petroni. 2019. The Common Framework of Reference for Intercultural Digital Literacies. [URL]
Steiner, Erich. 2017. “Contrastive Studies of Cohesion and their Impact on our Knowledge of Translation.” In Discourse Analysis in Translation Studies, ed. by Jeremy Munday, and Meifan Zhang, 27–45. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Taibi, Davide. 2020. “Analysis D: Shaping Digital Identities through Group Study: A Simulated Case Study Using Online Tools and Videos to Explore Animal-human Interactions.” In Multiliteracy Advances and Multimodal Challenges in ELT Environments, ed. by Nicoletta Vasta, and Anthony Baldry, 190–195. Udine: Forum Editrice.Google Scholar
Taibi, Davide, Deirdre Kantz, and Giovanni Fulantelli. 2014. “Supporting Formative Assessment in Content and Language Integrated Learning: The MWS-Web Platform.” International Journal of Technology Enhanced Learning 6(4): 361–379. DOI logoGoogle Scholar
Taibi, Davide, Ivana Marenzi, Deirdre Kantz, and Giovanni Fulantelli. 2015. “MWS-WEB: A System to Support a Pedagogical Approach Based on the Concept of Multiliteracies.” Italian Journal of Educational Technology 23(3): 164–171.Google Scholar
Taibi, Davide, Ivana Marenzi, and Qazi Asim Ijaz Ahmad. 2019. “Ain’t That Sweet: Reflections on Scene Level Indexing and Annotation in the House Corpus Project.” In Representing and Redefining Specialised Knowledge: Medical Discourse, ed. by Anthony Baldry, Francesca Bianchi, and Anna Loiacono, 151–181. Lecce: ESE University of Salento Publishing.Google Scholar
Tanskanen, Sanna-Kaisa. 2006. Collaborating towards Coherence: Lexical Cohesion in English Discourse. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Taylor, Christopher. 2003. “Multimodal Transcription in the Analysis: Translation and Subtitling of Italian Films.” The Translator 9(2): 191–206. DOI logoGoogle Scholar
. 2004. “Multimodal Text Analysis and Subtitling.” Perspectives on Multimodality, ed. by Eija Ventola, Cassily Charles, and Martin Kaltenbacher, 153–172. Amsterdam: John Benjamins. DOI logoGoogle Scholar