Article published In:
GestureVol. 19:1 (2020) ► pp.41–71
Emotion matters
The effect of hand gesture on emotionally valenced sentences
Recent theories and neural models of co-speech gesture have extensively considered its cognitive role in language comprehension but have ignored the emotional function. We investigated the integration of speech and co-speech gestures in memory for verbal information with different emotional connotations (either positive, negative, or neutral). In a surprise cued-recall task, gesture boosted memory for speech with all three emotional valences. Interestingly, gesture was more likely to become integrated into memory of neutrally and positively valenced speech than negatively valenced speech. The results suggest that gesture-speech integration is modulated by emotional valence of speech, which has implications for the emotional function of gesture in language comprehension.
Article outline
- Cognitive function of gesture: Production
- Cognitive function of gesture: Comprehension
- Emotional valence and gesture
- Present study
- Method
- Participants
- Materials
- Video stimulus
- Stimulus norming
- Filler task
- Memory tests
- Procedure
- Coding
- Design
- Results
- Cued-recall responses
- Remembering of speech
- Addition of information from gesture
- Remembering speech and adding from gesture
- Discussion
- Gesture and memory
- Emotional valence and memory
- Conclusion
- Acknowledgements
- Notes
- Cued-recall response coding manual
-
References
References (54)
References
Adelman, James S. & Estes, Zachary. (2013). Emotion and memory: A recognition advantage for positive and negative words independent of arousal. Cognition, 1291, 530–535. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Baumeister, Roy F., Bratslavsky, Ellen, Finkenauer, Catrin, & Vohs, Kathleen D. (2001). Bad is stronger than good. Review of General Psychology, 5 (4), 323–370. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Bohn, Annette & Berntsen, Dorthe. (2007). Pleasantness bias in flashbulb memories: Positive and negative flashbulb memories of the fall of the Berlin Wall among east and west Germans. Memory & Cognition, 35 (3), 565–577. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Borg, Céline, Bouazza, Zineb, Godeau, Marielle, Getenet, Jean-Claude, & Chainay, Hanna. (2018). Effect of emotion and type of encoding on memory for actions: Verbal and subjected-performed tasks. Dementia and Geriatric Cognitive Disorders, 45 (3/4), 162–179. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Bowen, Holly J., Kark, Sarah M., & Kensinger, Elizabeth A. (2018). NEVER forget: negative emotional valence enhances recapitulation. Psychonomic Bulletin & Review, 25 (3), 870–891. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Broaders, Sara C. & Goldin-Meadow, Susan. (2010). Truth is at hand: How gesture adds information during investigative interviews. Psychological Science, 21 (5), 623–628. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Charles, Susan T., Mather, Mara, & Carstensen, Laura L. (2003). Aging and emotional memory: The forgettable nature of negative images for older adults. Journal of Experimental Psychology: General, 132 (2), 310–324. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Cohen, Ronald L. (1989). Memory for action events: The power of enactment. Educational Psychology Review, 1 (1), 57–80. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Darwin, Charles. (1872). The expression of the emotions in man and animals. London, England: John Murray. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
De Stefani, Elisa & De Marco, Doriana. (2019). Gesture, language and emotional communication: An embodied view of social interaction. Frontiers in Psychology, 101, 2063. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Dewhurst, Stephen A. & Parry, Lisa A. (2000). Emotionality, distinctiveness, and recollective experience. European Journal of Cognitive Psychology, 12 (4), 541–551. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Ekman, Paul & Friesen, Wallace V. (1969). The repertoire of nonverbal behavior: Categories, origins, usage, and coding. Semiotica, 1 (1), 49–98. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Ekman, Paul & Friesen, Wallace V. (1972). Hand movements. Journal of Communication, 22 (4), 353–374. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Feyereisen, Pierre. (2009). Enactment effects and integration processes in younger and older adults’ memory for actions. Memory, 17 (4), 374–385. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Gallese, Vittorio. (2018). Embodied simulation and its role in cognition. Reti, saperi, linguaggi, 7 (13), 31–46.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Goldin-Meadow, Susan & Morford, Marolyn. (1985). Gesture in early child language: Studies of deaf and hearing children. Merrill-Palmer Quarterly, 31 (2), 145–176.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Hirata, Yukari, Kelly, Spencer D., Huang, Jessica, & Manansala, Michael. (2014). Effects of hand gestures on auditory learning of second-language vowel length contrasts. Journal of Speech, Language, and Hearing Research, 57 (6), 2090–2101. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Holland, Alisha C. & Kensinger, Elizabeth A. (2012). Younger, middle-aged, and older adults’ memories for the 2008 Presidential election. Journal of Applied Research on Memory and Cognition, 1 (3), 163–170. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Hostetter, Autumn B. (2011). When do gestures communicate? A meta-analysis. Psychological Bulletin, 137 (2), 297–315. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Hostetter, Autumn B. & Alibali, Martha W. (2008). Visible embodiment: Gestures as simulated action. Psychonomic Bulletin & Review, 15 (3), 495–514. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Hostetter, Autumn B. & Alibali, Martha W. (2018). Gesture as simulated action: Revisiting the framework. Psychonomic Bulletin & Review, 261, 721–752. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kelly, Spencer D., Barr, Dale J., Church, R. Breckinridge, & Lynch, Katheryn. (1999). Offering a hand to pragmatic understanding: The role of speech and gesture in comprehension and memory. Journal of Memory and Language, 401, 577–592. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kelly, Spencer D., Hirata, Yukari, Manansala, Michael, & Huang, Jessica. (2014). Exploring the role of hand gestures in learning novel phoneme contrasts and vocabulary in a second language. Frontiers in Psychology, 5, 673, 1–11. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kelly, Spencer D. & Lee, Angela L. (2012). When actions speak too much louder than words: Hand gestures disrupt word learning when phonetic demands are high. Language and Cognitive Processes, 27 (6), 793–807. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kendon, Adam. (1972). Some relationships between body motion and speech. In Aron Wolfe Siegman & Benjamin Pope (Eds.), Studies in dyadic communication (pp. 177–210). New York: Pergamon Press. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kendon, Adam. (1980). Gesticulation and speech: Two Aspects of the. In Mary R. Key (Ed.), The relationship of verbal and nonverbal communication (pp. 207–227). The Hague: Mouton. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kensinger, Elizabeth A. (2009). Remembering the details: Effects of emotion. Emotion Review, 1 (2), 99–113. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kensinger, Elizabeth A. & Choi, Elizabeth S. (2009). When side matters: Hemispheric processing and the visual specificity of emotional memories. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35 (1), 247–253.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kensinger, Elizabeth A. & Corkin, Suzanne. (2004). Two routes to emotional memory: Distinct neural processes for valence and arousal. Proceedings of the National Academy of Sciences, 101 (9), 3310–3315. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kensinger, Elizabeth A., Garoff-Eaton, Rachel J., & Schacter, Daniel L. (2006). Memory for specific visual details can be enhanced by negative arousing content. Journal of Memory and Language, 541, 99–112. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kensinger, Elizabeth A. & Schacter, Daniel L. (2006). When the red sox shocked the yankees: Comparing negative and positive memories. Psychonomic Bulletin & Review, 13 (5), 757–763. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Krahmer, Emiel & Swerts, Marc. (2007). The effects of visual beats on prosodic prominence: Acoustic analyses, auditory perception and visual perception. Journal of Memory and Language, 57 (3), 396–414. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Levine, Linda J. & Bluck, Susan. (2004). How emotions fade: valence, appraisals, and the emotional impact of remembered events. In S. P. Shohov (Ed.), Advances in psychology research (Vol. 301, pp. 3–20). Hauppauge, NY: Nova Science Publishers.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Macedonia, Manuela, Müller, Karsten, & Friederici, Angela D. (2011). The impact of iconic gestures on foreign language word learning and its neural substrate. Human Brain Mapping, 32 (6), 982–998. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Matheson, Heath E. & Barsalou, Lawrence W. (2018). Embodiment and grounding in cognitive neuroscience. Stevens’ handbook of experimental psychology and cognitive neuroscience, 31, 1–27. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
McNeill, David. (1992). Hand and mind. Chicago: The University of Chicago Press.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
McNeill, David & Levy, Elena. (1982). Conceptual representations in language activity and gesture. In R. J. Jarvella & W. Klein (Eds.), Speech, place, and action: Studies in deixis and related topics (pp. 271–295). New York: John Wiley & Sons.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
McNeill, David, Pedelty, Laura L., & Levy, Elena T. (1990). Speech and gesture. Advances in Psychology, 701, 203–256. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Nascimento, Juliana Maria Steffen do, Grüdtner Buratto, Luciano, Schaefer, Alexandre, & Milnitsky Stein, Lilian. (2016). Emotional valence and perceived event frequency affect memory accuracy for a personally relevant life event. Applied Cognitive Psychology, 301, 1020–1029. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Nilsson, Lars-Göran & Bäckman, Lars. (1989). Implicit memory and the enactment of verbal instructions. In Stephan Lewandowsky, John C. Dunn, & Kim Kirsner (Eds.), Implicit memory: Theoretical issues (pp. 173–183). Hillsdale, NJ: Erlbaum.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Ochsner, Kevin N. (2000). Are affective events richly recollected or simply familiar? The experience and process of recognizing feelings past. Journal of Experimental Psychology: General, 129 (2), 242–261. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Post, Lysanne S., van Gog, Tamara, Paas, Fred, & Zwaan, Rolf A. (2013). Effects of simultaneously observing and making gestures while studying grammar animations on cognitive load and learning. Computers in Human Behavior, 29 (4), 1450–1455. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Pouw, Wim T. J. L., de Nooijer, Jaqueline A., van Gog, Tamara, Zwaan, Rolf A., & Paas, Fred. (2014). Toward a more embedded/extended perspective on the cognitive function of gestures. Frontiers in Psychology, 5, 359, 1–14. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Rozin, Paul & Royzman, Edward B. (2001). Negativity bias, negativity dominance, and contagion. Personality and social psychology review, 5 (4), 296–320. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Sekine, Kazuki, Sowden, Hannah, & Kita, Sotaro. (2015). The development of the ability to semantically integrate information in speech and iconic gesture in comprehension. Cognitive Science, 391, 1855–1880. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Skipper, Jeremy I. (2015). The NOLB model: A model of the natural organization of language and the brain. In Roel M. Willems (Ed.), Cognitive neuroscience of natural language use (pp. 101–134). Cambridge: Cambridge University Press. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Spranger, Tina, Schatz, Tanja R., & Knopf, Monika. (2008). Does action make you faster? A retrieval-based approach to investigating the origins of the enactment effect. Scandinavian Journal of Psychology, 49 (6), 487–495. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Straube, Benjamin, Green, Antonia, Weis, Susanne, Chatterjee, Anjan, & Kircher, Tilo. (2009). Memory effects of speech and gesture binding: cortical and hippocampal activation in relation to subsequent memory performance. Journal of Cognitive Neuroscience, 21 (4), 821–836. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Wang, Bo. (2013). Facial expression influences recognition memory for faces: Robust enhancement effect of fearful expression. Memory, 21 (3), 301–314. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Wilson, Margaret. (2002). Six views of embodied cognition. Psychonomic Bulletin & Review, 9 (4), 625–636. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Yeo, Amelia, Ledesma, Iasmine, Nathan, Mitchell J., Alibali, Martha W., & Church, Ruth Breckinridge. (2017). Teachers’ gestures and students’ learning: Sometimes “hands off” is better. Cognitive Research: Principles and Implications, 2 (1), 411, 1–11. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Cited by (2)
Cited by two other publications
Tian, Lixian, Steven G. McCafferty & Man Zhu
2024.
The cognitive-emotional dialectics in second language development: speech and co-speech gestures of Chinese learners of English.
International Review of Applied Linguistics in Language Teaching 0:0
![DOI logo](//benjamins.com/logos/doi-logo.svg)
Kelly, Spencer D. & Quang‐Anh Ngo Tran
2023.
Exploring the Emotional Functions of Co‐Speech Hand Gesture in Language and Communication.
Topics in Cognitive Science ![DOI logo](//benjamins.com/logos/doi-logo.svg)
This list is based on CrossRef data as of 4 july 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.
video
video 1
video 2
Video clips