Article In: Pragmatics & Cognition: Online-First Articles
The role of visual perception in narrative scene construction
A pilot study with visually impaired children
This content is being prepared for publication; it may be subject to changes.
Abstract
Previous research suggests that imagery processes grounded in visual experience are crucial for narrative comprehension, particularly for creating mental models of story events. Visual impairments offer a unique lens to explore this issue, but research is limited. This pilot study investigates the role of visual perception in narrative processing by comparing 20 children with varying visual impairments to 20 sighted peers. Participants listened to stories in either an auditory format or a multisensory condition. Comprehension was assessed via story retellings, analyzing micro- (narrative length, syntactic complexity) and macro-linguistic features (core story details, situational dimensions, internal states). Results show that visually impaired children matched sighted peers in identifying core story elements and internal states but exhibited reduced structural language and difficulty constructing coherent event and scene representations. These findings suggest that visual experience supports critical aspects of narrative processing, which are not fully compensated by alternative mechanisms.
Article outline
- 1.Introduction
- 1.1Visual imagery and narrative processing
- 1.2The case of visual impairments
- 2.The present study
- 3.Materials and methods
- 3.1Participants
- 3.2Procedure
- 3.3Cognitive assessment
- 3.4Narrative assessment
- 3.4.1Narrative stimuli
- 3.4.2TUI-based methodology
- 3.4.3Narrative analysis
- 4.Results
- 4.1Statistical analyses
- 4.2Results
- 5.Discussion
- 6.Conclusions
- Acknowledgments
- Note
- Author queries
References
References (99)
Abu Talib, Ruhiyati I., Predrag Nikolic, Mohd Sunar & Rui Prada. 2020. In-visible island: Inclusive storytelling platform for visually impaired children. Mobile Networks and Applications 251. 913–924.
Adornetti, Ines, Alessandra Chiera, Daniela Altavilla, Valentina Deriu, Andrea Marini, Marika Gobbo, Giovanni Valeri, Rita Magni & Francesco Ferretti. 2023. Defining the characteristics of story production of autistic children: A multilevel analysis. Journal of Autism and Developmental Disorders 541. 3759–3776.
Argyropoulos, Vassilios, Elvira Masoura, Thomai Tsiakali, Magda Nikolaraizi & Christina Lappa. 2017. Verbal working memory and reading abilities among students with visual impairment. Research in Developmental Disabilities 641. 87–95.
Bal, Matthijs & Martijn Veltkamp. 2013. How does fiction reading influence empathy? An experimental investigation on the role of emotional transportation. PloS One 8(1). e55341.
Baldassano, Christopher, Janice Chen, Asieh Zadbood, Jonathan Pillow, Uri Hasson & Kenneth Norman. 2017. Discovering event structure in continuous narrative perception and memory. Neuron 95(3). 709–721.
Bara, Florence, Edouard Gentaz & Dannyelle Valente. 2018. The effect of tactile illustrations on comprehension of storybooks by three children with visual impairments: An exploratory study. Journal of Visual Impairment & Blindness 112(6). 759–765.
Barry, Daniel, Gareth Barnes, Ian Clark & Eleanor Maguire. 2019. The neural dynamics of novel scene imagery. Journal of Neuroscience 39(22). 4375–4386.
Bartoli, Gloriana, Daniela Bulgarelli, Paola Molina. 2019. Theory of mind development in children with visual impairment: The contribution of the adapted comprehensive test ToM storybooks. Journal of Autism and Developmental Disorders 49(9). 3494–3503.
Belacchi, Carmen, Teresa Scalisi, Eleonora Cannoni & Cesare Cornoldi. 2008. Manuale CPM coloured progressive matrices: Standardizzazione italiana. Florence: Giunti.
Boyd, Brian. 2018. The evolution of stories: From mimesis to language, from fact to fiction. Wiley Interdisciplinary Reviews: Cognitive Science 9(1). e1444.
Bowler, Dermot, Sebastian Gaigg & Sophie Lind. 2011. Memory in autism: Binding, self and brain. In Ilona Roth & Payam Rezaie (eds.), Researching the autism spectrum: Contemporary perspectives, 316–346. Cambridge: Cambridge University Press.
Brambring, Michael & Doreen Asbrock. 2010. Validity of false belief tasks in blind children. Journal of Autism and Developmental Disorders 401. 1471–1484.
Brosch, Renate. 2018. What we ‘see’ when we read: Visualization and vividness in reading fictional narratives. Cortex 1051. 135–143.
Brunyé, Tad, Tali Ditman, Caroline Mahoney, Jason Augustyn & Holly Taylor. 2009. When you and I share perspectives: Pronouns modulate perspective taking during narrative comprehension. Psychological Science 20(1). 27–32.
Busselle, Rick & Helena Bilandzic. 2008. Fictionality and perceived realism in experiencing stories: A model of narrative comprehension and engagement. Communication Theory 18(2). 255–280.
Caracciolo, Marco. 2013. Blind reading: Toward an enactivist theory of the reader’s imagination. In Lars Bernaerts, Dirk De Geest, Luc Herman & Bart Vervaeck (eds.), Stories and minds: Cognitive approaches to literary narrative, 81–106. Lincoln & London: University of Nebraska.
Cattaneo, Zaira & Tomaso Vecchi. 2011. Blind vision: The neuroscience of visual impairment. Boston: MIT Press.
Cheng, Sen, Markus Werning & Thomas Suddendorf. 2016. Dissociating memory traces and scenario construction in mental time travel. Neuroscience & Biobehavioral Reviews 601. 82–89.
Coady, Jeffry Julia & Evans. 2008. Uses and interpretations of non-word repetition tasks in children with and without SLI. International Journal of Language & Communication Disorders 43(1). 1–40.
Collignon, Olivier, Giulia Dormal, Geneviève Albouy, Gilles Vandewalle, Patrice Voss, Christophe Phillips & Franco Lepore. 2013. Impact of blindness onset on the functional organization and the connectivity of the occipital cortex. Brain 136(9). 2769–2783.
Cornoldi, Cesare, Federica Rigoni, Annalena Venneri & Tomaso Vecchi. 2000. Passive and active processes in visuo-spatial memory: Double dissociation in developmental learning disabilities. Brain and Cognition 43(1–3). 117–120.
D’Angiulli, Amedeo, Matthew Runge, Andrew Faulkner, Jila Zakizadeh, Aldrich Chan & Selvana Morcos. 2013. Vividness of visual imagery and incidental recall of verbal cues, when phenomenological availability reflects long-term memory accessibility. Frontiers in Psychology 41. 1.
Davis, Albyn & Carl Coelho. 2004. Referential cohesion and logical coherence of narration after closed head injury. Brain & Language 89(3). 508–523.
de Graaf, Anneke, Hans Hoeken, Josè Sanders, Hans Beentjes. 2009. The role of dimensions of narrative engagement in narrative persuasion. Communications 34(4). 385–405.
de Villiers, Jill & Jennie Pyers. 2002. Complements to cognition: A longitudinal study of the relationship between complex syntax and false-belief understanding. Cognitive Development 171. 1037–1060.
Dravida, Swethasri, Rebecca Saxe & Marina Bedny. 2013. People can understand descriptions of motion without activating visual motion brain regions. Frontiers in Psychology 41. 537.
Duff, Melissa & Sarah Brown-Schmidt. 2012. The hippocampus and the flexible use and processing of language. Frontiers in Human Neuroscience 61. 69.
Edirisinghe, Chamari, Norhidayati Podari & Adrian Cheok. 2018. A multi-sensory interactive reading experience for visually impaired children: A user evaluation. Personal and Ubiquitous Computing 261. 807–819.
Edmonds, Caroline & Linda Pring. 2006. Generating inferences from written and spoken language: A comparison of children with visual impairment and children with sight. British Journal of Developmental Psychology 24(2). 337–351.
Farrar, Jeffrey, Joann Benigno, Virginia Tompkins & Nicholas Gage. 2017. Are there different pathways to explicit false belief understanding? General language and complementation in typical and atypical children. Cognitive Development 431. 49–66.
Ferretti, Francesco. 2022. Narrative persuasion: A cognitive perspective on language evolution. Cham: Springer Nature.
Ferretti, Francesco, Ines Adornetti, Alessandra Chiera, Serena Nicchiarelli, Rita Magni, Giovanni Valeri & Andrea Marini. 2017. Mental time travel and language evolution: A narrative account of the origins of human communication. Language Sciences 631. 105–118.
Galiano, Anna & Serge Portalier. 2011. Language and visual impairment: Literature review. International Psychology, Practice & Research 21. 1–10.
Gazella, Jamie & Ida Stockman. 2003. Children’s story retelling under different modality and task conditions. American Journal of Speech-Language Pathology 12(1). 61–72.
Goldman, Alvin. 2006. Imagination and simulation in audience responses to fiction. In Shaun Nichols (ed.), The architecture of the imagination, 41–56. Oxford: Oxford University Press.
González, Luis & Miguel Pérez Pereira. 2006. Comprensión de textos y modalidades de acceso a la información: comparación de rendimientos entre personas ciegas y videntes. Integración. Revista Sobre Ceguera y Deficiencia Visual 481. 7–24.
Green, Sarah, Linda Pring & John Swettenham. 2004. An investigation of first-order false belief understanding of children with congenital profound visual impairment. British Journal of Developmental Psychology 22(1). 1–17.
Green, Melanie, Sheryl Kass, Jana Carrey, Benjamin Herzig, Ryan Feeney & John Sabini. 2008. Transportation across media: Repeated exposure to print and film. Media Psychology 11(4). 512–539.
Hanning-Zwanenburg, Aline, Inger Rødbroe, Anne Nafstad & Jacques Souriau. 2015. Narrative-based conversations with children who are congenitally deaf-blind. Journal of Deafblind Studies on Communication 1(1). 40–53.
Hassabis, Demis & Eleanor Maguire. 2007. Deconstructing episodic memory with construction. Trends in Cognitive Sciences 11(7). 299–306.
Hassabis, Demis, Dharshan Kumaran & Eleanor Maguire. 2007. Using imagination to understand the neural basis of episodic memory. Journal of Neuroscience 27(52). 14365–14374.
Hobson, Peter, Anthony Lee & Rachel Brown. 1999. Autism and congenital blindness. Journal of Autism and Developmental Disorders 291. 45–56.
Iverson, Jana & Susan Goldin-Meadow. 1997. What’s communication got to do with it? Gesture in children blind from birth. Developmental Psychology 33(3). 453.
James, Deborah & Vesna Stojanovik. 2007. Communication skills in blind children: A preliminary investigation. Child: Care, Health & Development 33(1). 4–10.
Kamei-Hannan, Cheryl, Ya-Chih Chang & Mitch Fryling. 2022. Using a multisensory storytelling approach to improve language and comprehension: A pilot study. British Journal of Visual Impairment 40(2). 175–186.
Kim, Judy, Giulia Elli & Marina Bedny. 2019. Knowledge of animal appearance among sighted and blind adults. Proceedings of the National Academy of Sciences 116(23). 11213–11222.
Koushik, Varsha, Darren Guinness & Shaun Kane. 2019. StoryBlocks: A tangible programming game to create accessible audio stories. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 1–12.
Kran, Barry, Linda Lawrence, Luisa Mayer & Gena Heidary. 2019. Cerebral/cortical visual impairment: A need to reassess current definitions of visual impairment and blindness. Seminars in Pediatric Neurology 311. 25–29.
Kuijpers, Moniek, Frank Hakemulder, Ed Tan & Miruna Doicaru. 2014. Exploring absorbing reading experiences. Scientific Study of Literature 4(1). 89–122.
Kuzmičová, Anežka. 2012. Presence in the reading of literary narrative: A case for motor enactment. Semiotica 189(1). 23–48.
. 2014. Literary narrative and mental imagery: A view from embodied cognition. Style 48(3). 275–293.
Landau, Barbara & Lila Gleitman. 1985. Language and experience: Evidence from the blind child. London: Harvard University Press.
Leinonen, Eeva, Carolyn Letts & Benita Smith. 2000. Children’s pragmatic communication difficulties. London: Whurr.
Loiotile, Rita, Rhodri Cusack & Marina Bedny. 2019. Naturalistic audio-movies and narrative synchronize “visual” cortices across congenitally blind but not sighted individuals. Journal of Neuroscience 39(45). 8940–8948.
Maguire, Eleanor & Sinead Mullally. 2013. The hippocampus: A manifesto for change. Journal of Experimental Psychology: General 142(4). 1180.
Mar, Raymond. 2011. The neural bases of social cognition and story comprehension. Annual Review of Psychology 62(1). 103–134.
Marini, Andrea & Sergio Carlomagno. 2004. Analisi del discorso e patologia del linguaggio. Milano: Springer Verlag.
Marini, Andrea, Sara Andreetta, Silvana Del Tin & Sergio Carlomagno. 2011. A multi-level approach to the analysis of narrative language in aphasia. Aphasiology 25(11). 1372–1392.
Marini, Andrea, Martina Ozbič, Rita Magni & Giovanni Valeri. 2020. Toward a definition of the linguistic profile of children with autism spectrum disorder. Frontiers in Psychology 111. 808.
Miglino, Orazio, Andrea Di Ferdinando, Massimiliano Schembri, Massimiliano Caretti, Angelo Rega & Carlo Ricci. 2013. STELT: Una piattaforma per realizzare ambienti di realtà aumentata per apprendere, insegnare e giocare. Sistemi Intelligenti 25(2). 397–404.
Miller, George & Philip Johnson-Laird. 1976. Language and perception. Cambridge: Cambridge University Press.
Milligan, Karen, Janet Astington & Lisa Dack. 2007. Language and theory of mind: Meta-analysis of the relation between language ability and false-belief understanding. Child Development 78(2). 622–646.
Nelson, Katherine. 2003. Narrative and the emergence of a consciousness of self. In Gary Fireman, Ted McVay & Owen Flanagan (eds.), Narrative and consciousness, 17–36. Oxford: Oxford University Press.
Papastergiou, Athanasia & Vasileios Pappas. 2019. A comparison of sighted and visually impaired children’s text comprehension. Research in Developmental Disabilities 851. 8–19.
Pérez-Pereira, Miguel. 2014. Contrasting views on the pragmatic abilities of blind children. Enfance 1(1). 73–88.
Pérez-Pereira, Miguel & Gina Conti-Ramsden. 2019. Language development and social interaction in blind children. London: Routledge.
Peristeri, Eleni, Maria Andreou & Ianthi Tsimpli. 2017. Syntactic and story structure complexity in the narratives of high- and low-language ability children with autism spectrum disorder. Frontiers in Psychology 81. 2027.
Pijnacker, Judith, Mathijs Vervloed & Bert Steenbergen. 2012. Pragmatic abilities in children with congenital visual impairment: An exploration of non-literal language and advanced theory of mind understanding. Journal of Autism and Developmental Disorders 42(11). 2440–2449.
Pring, Linda, Hazel Dewart & Margaret Brockbank. 1998. Social cognition in children with visual impairments. Journal of Visual Impairment & Blindness 92(11). 754–768.
Race, Elizabeth, Margaret Keane & Mieke Verfaellie. 2015. Sharing mental simulations and stories: Hippocampal contributions to discourse integration. Cortex 631. 271–281.
Radvansky, Gabriel & David Copeland. 2001. Working memory and situation model updating. Memory & Cognition 29(8). 1073–1080.
Renzi, Chiara, Zaira Cattaneo, Tomaso Vecchi & Cesare Cornoldi. 2013. Mental imagery and blindness. In Simon Lacey & Rebecca Lawson (eds.), Multisensory imagery, 115–130. New York: Springer.
Rice, Heather & David Rubin. 2009. I can see it both ways: First-and third-person visual perspectives at retrieval. Consciousness & Cognition 18(4). 877–890.
Richardson, Hilary, Rebecca Saxe & Marina Bedny. 2023. Neural correlates of theory of mind reasoning in congenitally blind children. Developmental Cognitive Neuroscience 631. 101285.
Röder, Brigitte, Frank Rösler & Helen Neville. 2000. Event-related potentials during auditory language processing in congenitally blind and sighted people. Neuropsychologia 38(11). 1482–1502.
Röder, Brigitte, Oliver Stock, Siegfried Bien, Helen Neville & Frank Rösler. 2002. Speech processing activates visual cortex in congenitally blind humans. European Journal of Neuroscience 16(5). 930–936.
Rosel, Jesùs, Antonio Caballer, Pilar Jara & Juan Oliver. 2005. Verbalism in the narrative language of children who are blind and sighted. Journal of Visual Impairment & Blindness 99(7). 413–425.
Şafak, Pinar, Hatice Yılmaz & Pinar Demiryürek. 2016. Using multisensory storytelling to increasing listening comprehension for students with multiple disabilities include visual impairment. In Jiri Vopava et al. (eds), Proceedings of the Multidisciplinary Academic Conference in Prague, 731. Prague.
Shaer, Orit & Eva Hornecker. 2010. Tangible user interfaces: Past, present, and future directions. Foundations and Trends in Human-Computer Interaction 3(1–2). 4–137.
Simony, Erez, Christopher Honey, Janice Chen, Olga Lositsky, Yaara Yeshurun, Ami Wiesel & Uri Hasson. 2016. Dynamic reconfiguration of the default mode network during narrative comprehension. Nature Communications 7(1). 12141.
Somma, Federica, Raffaele Di Fuccio, Lavinia Lattanzio & Francesco Ferretti. 2021. Multisensorial tangible user interface for immersive storytelling: A usability pilot study with a visually impaired child. In Proceedings of the CEUR Workshop Proceedings, 7–8 January 2021.
Speer, Nicole, Jeffrey Zacks & Jeremy Reynolds. 2007. Human brain activity time-locked to narrative event boundaries. Psychological Science 18(5). 449–455.
Sullivan, Susan & Jane Oakhill. 2020. Inference making skill in children with visual impairments. Research in Developmental Disabilities 104(2). 103713.
Tadić, Val, Linda Pring & Naomi Dale. 2010. Are language and social communication intact in children with congenital visual impairment at school age? Journal of Child Psychology and Psychiatry 51(6). 696–705.
Tinti, Carla, Mauro Adenzato, Marco Tamietto & Cesare Cornoldi. 2006. Visual experience is not necessary for efficient survey spatial cognition: Evidence from blindness. Quarterly Journal of Experimental Psychology 59(7). 1306–1328.
van Dijk, Teun & Walter Kintsch. 1978. Cognitive psychology and discourse: Recalling and summarizing stories. In Wolfgang Dressler (ed.), Current Trends in Text Linguistics, 61–80. Berlin: Mouton de Gruyter.
Vanlierde, Annick & Marie-Chantal Wanet-Defalque. 2004. Abilities and strategies of blind and sighted subjects in visuo-spatial imagery. Acta Psychologica 116(2). 205–222.
Wechsler, David. 1993. Manual for the Wechsler intelligence scale for children — III. San Antonio: Psychological Corporation.
Weikle, Bobbie & Azar Hadadian. 2003. Emergent literacy practices among parents of preschool children with and without disabilities. International Journal of Special Education 18(1). 80–99.
Williams, Marian, Cassandra Fink, Irina Zamora & Mark Borchert. 2014. Autism assessment in children with optic nerve hypoplasia and other vision impairments. Developmental Medicine and Child Neurology 56(1). 66–72.
Withagen, Ans, Astrid Kappers, Mathijs Vervloed, Harry Knoors & Ludo Verhoeven. 2013. Short term memory and working memory in blind versus sighted children. Research in Developmental Disabilities 34(7). 2161–2172.
Zacks, Jeffrey, Nicole Speer & Jeremy Reynolds. 2009. Segmentation in reading and film comprehension. Journal of Experimental Psychology: General 138(2). 307–327.