Cover not available
Part of
Exploring Discourse Strategies in Social and Cognitive Interaction: Multimodal and cross-linguistic perspectives
Edited by Manuela Romano and M. Dolores Porto
[Pragmatics & Beyond New Series 262] 2016
► pp. 111–135

The construction of meaning in multimodal discourse

A digital story as a case study

| Technical University of Madrid
This chapter’s main goal is to shed light onto the most characteristic meaning making processes in digital storytelling by thoroughly analyzing a single digital narrative from a functional-cognitive perspective. This mixed approach for a case study allows the analysts to unveil the macro and micro discourse strategies and the cognitive processes developed by both the narrator and the audience to produce and interpret the multiple meanings conveyed by the different modes in a digital story. We believe findings presented here can be of interest for discourse analysts, cognitivists and researchers working in multimodality.
Keywords: digital story, discourse strategies, functional-cognitive approach, multimodality
Published online: 31 March 2016
DOI logo
https://doi.org/10.1075/pbns.262.05mol
References (43)
References
Alexander, Brian and Alan Levine. 2008. “Web 2.0 Storytelling: Emergence of a New Genre”. EDUCAUSE Review 43 (6): 40–48.Google Scholar
Alonso Belmonte, Isabel, Silvia Molina Plaza and M. Dolores Porto Requejo. 2013. “Multimodal digital storytelling: Integrating information, emotion and social cognition”. In The Annual Review of Cognitive Linguistics, Special issue “Multimodality and Cognitive Linguistics”, 11 (2): 369–385. Amsterdam/Philadelphia: John Benjamins. DOI logoGoogle Scholar
Baldry, Anthony P. and Paul J. Thibault. 2006. Multimodal Transcription and Text Analysis. London and New York: Equinox.Google Scholar
Barthes, Roland. 1977. Image, Music, Text. London: Fontana.Google Scholar
Bateman, John A. 2008. Multimodality and Genre. A Foundation for the Systematic Analysis of Multimodal Documents. Basingstoke: Palgrave MacMillan.Google Scholar
. 2014. Text and Image: A Critical Introduction to the Visual/Verbal Divide. London and New York: Routledge.Google Scholar
Bateman, John A. and Janina Wildfeuer. 2014. “A Multimodal Discourse Theory of Visual Narrative.” Journal of Pragmatics 74: 180–208. DOI logoGoogle Scholar
Burgess, Jean. 2006. “Hearing Ordinary Voices”. Continuum: Journal of Media & Culture 20 (2): 201–214. DOI logoGoogle Scholar
Cohn, Neil. 2013. Visual Narrative Structure. Cognitive Science 37 (3): 413–452. DOI logoGoogle Scholar
Dancygier, Barbara. 2008. “The text and the story: levels of blending in fictional narratives.” In Mental Spaces in Discourse and Interaction, ed. by Todd Oakley and Anders Hougaard, 51–78. Amsterdam: John Benjamins. DOI logoGoogle Scholar
. 2012. The Language of Stories: A Cognitive Approach. Cambridge: 
Cambridge University Press.Google Scholar
Fauconnier, Giles and Mark Turner. 2002. The Way We Think: Conceptual Blending and the Mind’s Hidden Complexities. New York: Basic Books.Google Scholar
Forceville, Charles and Eduardo Urios-Aparisi (eds). 2009. Multimodal Metaphor. Berlin: 
Mouton de Gruyter. DOI logoGoogle Scholar
Gleason, Christina. 2012. “5 Brands Taking Digital Storytelling to the Next Level”. Ignite Social Media at [URL] (last accessed, January 2015).
Handler Miller, Carolyn. 2014 [2004]. Digital Storytelling: A Creator’s Guide to Interactive Entertainment. New York and London: Focal Press.Google Scholar
Hühn, Peter, Wolf Schmid and Jörg Schönert (eds). 2009. Point of View, Perspective, and Focalization: Modeling Mediation in Narrative. Berlin: Mouton de Gruyter. DOI logoGoogle Scholar
Jewitt, Carey. (ed). 2009. The Routledge Handbook of Multimodal Analysis. London: Routledge.Google Scholar
Kövecses, Zoltan. 2002. Metaphor: A Practical Introduction. Oxford: Oxford University Press.Google Scholar
Kress, Gunther. 2010. Multimodality. A Social Semiotic Approach to Contemporary Communication. London: Routledge.Google Scholar
Kress, Gunther and Theo J. van Leeuwen. 2001. Multimodal Discourse: The Modes and Media of Contemporary Communication. London: Arnold.Google Scholar
. 2006 [1996]. Reading Images: The Grammar of Visual Design. London: Routledge.Google Scholar
Lakoff, George and Mark Turner. 1989. More than Cool Reason: A Field Guide to Poetic Metaphor. Chicago: University of Chicago Press. DOI logoGoogle Scholar
Lambert, Joe. 2013. Digital Storytelling: Capturing Lives, Creating Community. London and New York: Routledge.Google Scholar
Lundby, Knut. (ed). 2008. Digital Storytelling, Mediatized Stories: Self-Representations in New Media. New York: Peter Lang Publishing.Google Scholar
McLellan, Hilary. 2007. “Digital Storytelling in Higher Education.” Journal of Computing in Higher Education 19 (1): 65–79. DOI logoGoogle Scholar
McNeill, David. 2005. Gesture and Thought. Chicago: Chicago University Press. DOI logoGoogle Scholar
Oakley, Todd and Anders Hougaard (eds). 2008. Mental Spaces in Discourse and Interaction. Amsterdam: John Benjamins. DOI logoGoogle Scholar
O’Halloran, Kay L. (ed). 2004. Multimodal Discourse Analysis: Systemic Functional Perspectives. London: Bloomsbury Academic.Google Scholar
. 2008. “Systemic Functional Multimodal Discourse Analysis (SFMDA): Constructing Ideational Meaning Using Language and Visual Imagery.” Visual Communication 7 (4): 443–475. DOI logoGoogle Scholar
O’Halloran, Kay L. and Bradley A. Smith. 2011. “Multimodal Text Analysis.” In The Encyclopedia of Applied Linguistics, ed. by Carol A. Chapelle. Oxford, UK: Wiley-Blackwell.Google Scholar
Porto Requejo, M. Dolores. 2016. “Music in Multimodal Narratives: The Role of the Soundtrack in Digital Stories”. In Audionarratology: Interfaces of Sound and Narrative, ed. by Jarmila Mildorf and Till Kinzel, 29–46. Berlin: De Gruyter.
Porto, M. Dolores and Manuela Romano. 2010. “Conceptual Integration in Natural Oral Narratives”. Actes des Journées D’étude “Narratology and the New Social Dimension of Narrative. [URL] (last accessed January 2015).Google Scholar
Porto Requejo, M. Dolores and Isabel Alonso Belmonte. 2014. “From Local to Global: Visual Strategies of Glocalisation in Digital Storytelling.“ Language and Communication 39: 14–23. DOI logoGoogle Scholar
Roland, Craig. 2006. Digital Stories in the Classroom. School Art 105 (7): 26.Google Scholar
Rossiter, Marsha and Penny A. Garcia. 2010. “Digital Storytelling: A New Player on the Narrative Field.” New Directions for Adult and Continuing Education 126: 37–48. DOI logoGoogle Scholar
Tara La Rose, Janice. 2012. “Digital Media Stories through Multimodal Analysis: A Case Study of Erahoneybee’s Song about a Child Welfare Agency.” Journal of Technology in Human Services 30 (3–4): 299–311. DOI logoGoogle Scholar
Tseng, Chiao. 2013. Cohesion in film: Tracking film elements. Basingstoke: Palgrave Macmillan. DOI logoGoogle Scholar
Unsworth, Len and Chris Cleirigh. 2009. “Multimodality and Reading: The Construction of Meaning through Image-Text Interaction.” In The Routledge Handbook of Multimodal Analysis, ed. by Carey Jewitt, 151–163. London: Routledge.Google Scholar
Velasco, Olga I. 2001. “Metaphor, Metonymy, and Image-Schemas: an Analysis of Conceptual Interaction Patterns.” Journal of English Studies 3: 47–63.Google Scholar
Zacks, Jeffrey M. and Barbara Tversky. 2001. “Event Structure in Perception and Conception.” Psychological Bulletin 127 (1): 3–21 DOI logoGoogle Scholar
Zwaan, Rolf A. 1999. “Five Dimensions of Narrative Comprehension: The Event-Indexing Model.” In Narrative Comprehension, Causality, and Coherence: Essays in Honor of Tom Trabasso, ed. by Susan R. Goldman, Arthur C. Graesser and Paul van den Broek, 93–110. Mahwah, NJ, US: Lawrence Erlbaum.Google Scholar
Zwaan, Rolf A., Mark C. Langston and Arthur C. Graesser. 1995. “The Construction of Situation Models in Narrative Comprehension: An Event-Indexing Model.” Psychological Science 6 (5): 292–297. DOI logoGoogle Scholar
Zwaan, Rolf A. and Gabriel A. Radvansky. 1998. “Situation Models in Language Comprehension and Memory.” Psychological Bulletin 123 (2): 162–185. DOI logoGoogle Scholar