Cover not available
Part of
The Pragmatics of Discourse Coherence: Theories and applications
Edited by Helmut Gruber and Gisela Redeker
[Pragmatics & Beyond New Series 254] 2014
► pp. 179–206

Coherence in multimodal arts installations

Multimodal and process-oriented design in digital installation art is pushing for the resolution of physical, spatial and temporal borders as well as coherent structures, which may make visitor perception substantially difficult. Hence, this form of art is well suited for the discussion of the concept of coherence in terms of applicability and constraints. In order to analyse and discuss coherence in multimodal and process-oriented design, I have developed a five-layer framework, based on systemic functional and multimodal theory and capable of analysing coherence relations in the digital installation art that this paper will look at.
Published online: 26 November 2014
DOI logo
https://doi.org/10.1075/pbns.254.07hue
References (51)
Arnheim, Rudolf. 1969. Visual Thinking. London: University of California Press.Google Scholar
Bateman, John A., and Karl-Heinrich Schmidt. 2012. Multimodal Film Analysis: How Films Mean. New York, London: Routledge.Google Scholar
Bateman, John A., and Chiaoi Tseng. 2012. “Multimodal Narrative Construction in Christopher Nolan’s Memento a Description of Analytic Method.” Visual Communication 11 (1): 91–119. DOI logoGoogle Scholar
Bobrow, Daniel G., and Donald A. Norman. 1975. “Some Principles of Memory Schemata.” In Representation and Understanding: Studies in Cognitive Sciences, ed. by Daniel G. Bobrow, and Allan Collins, 131–149. New York: Academic Press. DOI logoGoogle Scholar
Carrell, Patricia L. 1982. “Cohesion is not Coherence.” TESOL Quarterly 16 (4): 479–488. DOI logoGoogle Scholar
Day, Jon. 2013 [online]. “Please Do Touch.” The Free Library (December, 1), [URL], [visited on 19.02.2014].Google Scholar
De Beaugrande, Robert-Alain, and Wolfgang Dressler. 1981. Einführung in die Textlinguistik. Tübingen: Niemeyer. DOI logoGoogle Scholar
Ehlich, Konrad. 2007. Sprache und sprachliches Handeln. Diskurs-Narration-Text-Schrift. Vol. 3. Berlin, New York: Walter de Gruyter. DOI logoGoogle Scholar
Halliday, Michael A. K. 1978. Language as Social Semiotic. The Interpretation of Language and Meaning. London: Arnold.Google Scholar
. 2002. “The De-Automatization of Grammar: From Priestly’s An Inspector Calls.” In The Collected Works of M. A. K. Halliday, Vol. 2, 126–148. London: Continuum.Google Scholar
Halliday, Michael A. K., and Ruqaiya Hasan. 1976. Cohesion in English. London: Longman.Google Scholar
Hasan, Ruqaiya. 1989. Linguistics, Language and Verbal Art. Oxford: Oxford University Press.Google Scholar
Huemer, Birgit. 2008a. “Representations of Individual and Mass: Modelling Experience through Multiple Modes in Digital Art.” In From Language to Multimodality. New Developments in the Study of Ideational Meaning, ed. by Carys Jones, and Eija Ventola, 255–274. London: Equinox.Google Scholar
. 2008b. “A Semiotic Approach to Visitor Interaction, Participation and Perspective in Digital Arts.” In New Realities: Being Syncretic. Consciousness Reframed: The Planetary Collegium’s IXth International Research Conference Series: Edition Angewandte, ed. by Roy Ascott, Gerald Bast, and Wolfgang Fiel, 144–147. Wien: Springer.Google Scholar
. 2014. Semiotik der digitalen Medienkunst. Göttingen: V&R unipress.Google Scholar
Jakobson, Roman. 1985. “Poetry of Grammar and Grammar of Poetry.” In Jakobson, Roman: Verbal Art, Verbal Sign, Verbal Time, ed. by Krystyna Pomorska, and Stephen Rudy, 37–46. Oxford: Blackwell.Google Scholar
Jewitt, Carey. 2005. “Multimodality, ‘‘Reading’’, and ‘‘Writing’’ for the 21st Century.” Discourse: Studies in the Cultural Politics of Education 26 (3): 315–331. DOI logoGoogle Scholar
. 2006. Technology, Literacy and Learning: A Multimodal Approach. Abingdon: Routledge.Google Scholar
Kahn, Douglas. 2001. Noise, Water, Meet. A History of Sound in the Arts. Cambridge: MIT Press.Google Scholar
Kamalski, Judith, Ted Sanders, and Leo Lentz. 2008. “Coherence Marking, Prior Knowledge, and Comprehension of Informative and Persuasive Texts: Sorting Things Out.” Discourse Processes 45: 323–345. DOI logoGoogle Scholar
Kress, Gunther. 2003. Literacy in the New Media Age. London: Routledge. DOI logoGoogle Scholar
Kress, Gunther, and Theo van Leeuwen. 2001. Multimodal Discourse. The Modes and Media of Contemporary Communication. London: Arnold.Google Scholar
. 2006. Reading Images. The Grammar of Visual Design, 2nd ed. London: Routledge.Google Scholar
Martin, James R. 1992. English Text: System and Structure. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Martin, James R., and David Rose. 2003. Working with Discourse. Meaning beyond the Clause. London: Continuum.Google Scholar
Martinec, Radan. 1998. “Cohesion in Action.” Semiotica 120 (1/2): 161–180. DOI logoGoogle Scholar
. 2000. “Rhythm in Multimodal Texts.” Leonardo 33 (4): 289–297. DOI logoGoogle Scholar
Martinec, Radan, and Andrew Salway. 2005. “A System for Image-Text Relations in New (and Old) Media.” Visual Communication 4 (3): 337–371. DOI logoGoogle Scholar
Mills, Kathy A. 2010. “A Review of the “Digital Turn” in the New Literacy Studies.” Review of Educational Research 80 (2): 246–271. DOI logoGoogle Scholar
Morgan, Jerry L., and Manfred Sellner. 1980. “Discourse and Linguistic Theory.” In Theoretical Issues in Reading Comprehension: Perspectives from Cognitive Psychology, Linguistics, Artificial Intelligence, and Education, ed. by Rand J. Spiro, Bertram C. Bruce, and William F. Brewer, 165–200. Hillsdale: Lawrence Erlbaum.Google Scholar
Norris, Sigrid. 2004. Analyzing Multimodal Interaction: A Methodological Framework. New York: Routledge.Google Scholar
Norris, Sigrid, and Rodney H. Jones (eds). 2005. Discourse in Action. Introducing Mediated Discourse Analysis. London: Routledge.Google Scholar
O’Toole, Michael. 1994. The Language of Displayed Art, new edition. London: Leicester ­University Press.Google Scholar
Royce, Terry. 1999. Visual-Verbal Intersemiotic Complementarity in the Economist Magazine. PhD. Dissertation, University of Reading.Google Scholar
Royce, Terry D., and Wendy L. Bowcher (eds). 2007. New Directions in the Analysis of Multimodal Discourse. New Jersey: Lawrence Erlbaum.Google Scholar
Sanders, Ted, Jentine Land, and Gerben Mulder. 2007. “Linguistic Markers of Coherence Improve Text Comprehension in Functional Contexts.” Information Design Journal 15 (3): 219–235. DOI logoGoogle Scholar
Schaefer, Murray R. 1986. The Thinking Ear. Toronto: Arcana.Google Scholar
Sperber, Dan, and Deirdre Wilson. 1986. Relevance: Communication and Cognition. Oxford: Blackwell.Google Scholar
Spooren, Wilbert, and Ted Sanders. 2008. “The Acquisition Order of Coherence Relations: On Cognitive Complexity in Discourse.” Journal of Pragmatics 40 (12): 2003–2026. DOI logoGoogle Scholar
Tseng, Chiaoi, and John A. Bateman. 2012. “Multimodal Narrative Construction in Christopher Nolan’s Memento a Description of Analytic Method.” Visual Communication 11 (1): 91–119. DOI logoGoogle Scholar
Van Leeuwen, Theo. 1991. “Conjunctive Structure in Documentary Film and Television.” Journal of Media and Cultural Studies 5 (1): 76–114. DOI logoGoogle Scholar
. 1999. Speech, Music, Sound. London: Palgrave.Google Scholar
. 2005. Introducing Social Semiotics. London: Routledge.Google Scholar
Data samples in this paper
Bystander (2005): Ross Gibson and Kate Richards: [Online]: [URL], [visited on 06.08.2012].Google Scholar
Emotional Traffic (e-traffic), (2005): Maurice Benayoun and Jean-Baptiste Barrière, music and software by Jean-Baptiste Barrière, software in co-production with V2 Lab, Birgit ­Lichtenegger, Artem Baguinsky, Marloes de Valk: [Online]: [URL], [visited on 06.08.2012].Google Scholar
Listening Post (2001): Ben Rubin and Mark Hansen: [Online]: [URL], [visited on 06.08.2012].Google Scholar
Pussy Weevil (2003): Julian Bleecker and Marina Zurkow: [Online]: [URL], [visited on 06.08.2012].Google Scholar
Telematic Dreaming (1992): Paul Sermon: [Online]: [URL], [visited on 06.08.2012].Google Scholar
Tempest (2004): Erich Berger: [Online]:[URL], [visited on 06.08.2012].Google Scholar
The Legible City (1989): Jeffrey Shaw: [Online]: [URL], [visited on 06.08.2012].Google Scholar
The Messenger (1998, new version 2005): Paul De Marinis: [Online]: [URL], [visited on 06.08.2012].Google Scholar