Cover not available
Part of
From Gesture in Conversation to Visible Action as Utterance: Essays in honor of Adam Kendon
Edited by Mandana Seyfeddinipur and Marianne Gullberg
[Not in series 188] 2014
► pp. 15–34

Including facial gestures in gesture-speech ensembles

| Department of Psychology, University of Victoria
| Department of Psychology, University of Victoria
| Department of Psychology, University of Victoria

Conversational facial gestures fit Kendon’s (2004) specifications of the functions of hand gestures. We illustrate how facial gestures in dialogue, like hand gestures, convey referential content as well as serving pragmatic, interpersonal and interactive functions. Hand and facial gestures often occur together, creating an integrated visual image in gesture–speech ensembles. A semantic features analysis demonstrates how speakers adjust their use of these visible versus audible expressive resources according to context. Speakers who were interacting face-to-face (compared to speakers who could not see their addressee) were significantly more likely to rely on their hand and facial gestures than on their words when describing key semantic features, and their gestures were more likely to convey information that was not in their words.

Published online: 6 August 2014
DOI logo
https://doi.org/10.1075/z.188.02bav
References
Bavelas, J., and Chovil, N
1997 “Faces in dialogue.” In The Psychology of Facial Expression, J.A. Russell and J.M. Fernandez-Dols (eds), 334–346. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Bavelas, J., Chovil, N., Lawrie, D.A., and Wade, A
1992 “Interactive gestures.” Discourse Processes 15: 469–489. DOI logoGoogle Scholar
Bavelas, J., Gerwing, J., and Healing, S
2014 “Hand and facial gestures in conversational interaction.” In Handbook of Language and Social Psychology, T. Holtgraves (ed.), 111–130. Oxford: Oxford University Press.Google Scholar
in press “The effect of dialogue on demonstrations: direct quotations, facial portrayals, hand gestures, and figurative references.” (accepted for publication in Discourse Processes)
Bavelas, J., Gerwing, J., Sutton, C., and Prevost, D
2008 “Gesturing on the telephone: Independent effects of dialogue and visibility.” Journal of Memory and Language 58: 495–520. DOI logoGoogle Scholar
Bavelas, J., and Healing, S
2013 “Reconciling the effects of mutual visibility on gesturing. A review.” Gesture 13: 63–92. DOI logoGoogle Scholar
Beattie, G., and Shovelton, H
1999 “Do iconic hand gestures really contribute anything to the semantic information conveyed by speech? An experimental investigation.” Semiotica 123 (1–2): 1–30. DOI logoGoogle Scholar
2002 “An experimental investigation of some properties of individual iconic gestures that mediate their communicative power.” British Journal of Psychology 93: 179–192. DOI logoGoogle Scholar
Birdwhistell, R.L
.1970 “Masculinity and femininity as display.” Kinesics and Context: Essays on Body Motion. Philadelphia, PA: University of Pennsylvania.Google Scholar
Brugman, H., and Russel, A
2004 “Annotating multimedia/multimodal resources with ELAN.” In Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC 2004) , M.T. Lino, M.F. Xavier, F. Ferreira, R. Costa, and R. Silva (eds), 2065–2068. Paris: European Language Resources Association.Google Scholar
Brunner, L.J
1979 “Smiles can be back channels.” Journal of Personality and Social Psychology 37: 728–734. DOI logoGoogle Scholar
Chovil, N
1989 Communicative Functions of Facial Displays in Conversation. Ph.D. dissertation, University of Victoria, BC.
1991/1992 “Discourse-oriented facial displays in conversation.” Research on Language and Social Interaction 25: 163–192. DOI logoGoogle Scholar
1991 Social determinants of facial displays. Journal of Nonverbal Behavior 15: 141–153. DOI logoGoogle Scholar
1997 “Facing others. A social communicative perspective on facial displays.” In The Psychology of Facial Expression, J.A. Russell and J.M. Fernandez-Dols (eds), 321–333. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Coates, L.J
1991 A Collaborative Theory of Inversion: Irony in Dialogue. M.A. thesis, University of Victoria, BC.
de Ruiter, J.P., Bangerter, A., and Dings, P
2012 “The interplay between gesture and speech in the production of referring expressions: Investigating the tradeoff hypothesis.” Topics in Cognitive Science 4 (2): 232–248. DOI logoGoogle Scholar
Ekman, P
1979 “About brows: Emotional and conversational signals.” In Human Ethology, M. von Cranach, K. Foppa, W. Lepenies and D. Ploog (eds), 169–249. Cambridge: ­Cambridge University Press.Google Scholar
1985 Telling Lies. New York: Norton.Google Scholar
1997 “Should we call it expression or communication?” European Journal of Social Sciences 10: 333–359.Google Scholar
Ekman, P., and Friesen, W.V
1969 “The repertoire of nonverbal behavior: Categories, origins, usage, and coding.” Semiotica I: 49–98.Google Scholar
Fridlund, A.J., Ekman, P., and Oster, H
1987 “Facial expressions of emotion: Review of literature, 1970–1983.” In Nonverbal Communication and Behavior (2nd ed.), A.W. Siegman and S. Feldstein, eds. (143–224). Hillsdale, NJ: Erlbaum.Google Scholar
Holler, J., and Beattie, G
2002 “A micro-analytic investigation of how iconic gestures and speech represent core semantic features in talk.” Semiotica 142: 31–69.Google Scholar
2003 “How iconic gestures and speech interact in the representation of meaning: Are both aspects really integral to the process?” Semiotica 146: 81–116.Google Scholar
2004 “The interaction of iconic gesture and speech.” In Lecture Notes in Computer Science, 5th International Gesture Workshop, Genova, Italy, 2003: Selected Revised Papers , A. Cammurri and G. Volpe (eds), 63–69. Heidelberg: Springer Verlag.Google Scholar
Holler, J., and Stevens, R
2007 “An experimental investigation into the effect of common ground on how speakers use gesture and speech to represent size information in referential communication.” Journal of Language and Social Psychology 26: 4–27. DOI logoGoogle Scholar
Holler, J., and Wilkin, K
2009 “Communicating common ground: How mutually shared knowledge influences the representation of semantic information in speech and gesture in a narrative task.” Language and Cognitive Processes 24: 267–289. DOI logoGoogle Scholar
Holler, J., Tutton, M., and Wilkin, K
2011 “Co-speech gestures in the process of meaning coordination.” In Proceedings of the 2nd GESPIN – Gesture in Speech and Interaction Conference , Bielefeld, 5–7, September 2011.
Kendon, A
2004 Gesture: Visible Action as Utterance. Cambridge: Cambridge University Press.Google Scholar
Kraut, R.E., and Johnston, R.E
1979 “Social and emotional messages of smiling: An ethological approach.” Journal of Personality and Social Psychology 61: 743–745.Google Scholar
Pine, K.J., Burney, D.J., and Fletcher, B
2010 “The semantic specificity hypothesis: When gestures do not depend upon the presence of a listener.” Journal of Nonverbal Behavior 34: 169–178. DOI logoGoogle Scholar
Rimé, B
1982 “The elimination of visible behavior from social interactions: Effects on verbal, nonverbal and interpersonal variables.” European Journal of Social Psychology 12: 113–129. DOI logoGoogle Scholar
Sherzer, J
1973 Verbal and Nonverbal Deixis: The Pointed Lip Gesture among the San Blas Cuna. University of Texas at Austin: Institute of Latin American Studies.Google Scholar
Tomasello, M., Hare, B., Lehmann, H., and Call, J
2007 “Reliance on head versus eyes in the gaze following of great apes and human infants: the cooperative eye hypothesis.” Journal of Human Evolution 52 (3): 314–320. DOI logoGoogle Scholar
Wittenburg, P., Brugman, H., Russel, A., Klassman, A., and Sloetjes, H
2006 “ELAN: A professional framework for multimodality research.” In Proceedings of LREC 2006, Fifth International Conference on Language Resources and Evaluation , Genoa.
Cited by

Cited by 6 other publications

Armstrong, Meghan, Núria Esteve Gibert, Iris Hübscher, Alfonso Igualada & Pilar Prieto
2018. Developmental and cognitive aspects of children’s disbelief comprehension through intonation and facial gesture. First Language 38:6 ► pp. 596 ff. DOI logo
Clift, Rebecca
2021. Embodiment in Dissent: The Eye Roll as an Interactional Practice. Research on Language and Social Interaction 54:3 ► pp. 261 ff. DOI logo
Edman, Kristina
2024. Doing mutual understanding in child and family therapy sessions: How three interlocutors calibrate new information. Discourse Studies 26:2 ► pp. 199 ff. DOI logo
Imamura, Mai, Ayane Tashiro, Shiro Kumano & Kazuhiro Otsuka
2023. INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION, ► pp. 42 ff. DOI logo
Sharkov, F. I., V. V. Silkin & O. F. Kireeva
2022. Non-verbal signs of personality: Communicative meanings of facial expressions. RUDN Journal of Sociology 22:2 ► pp. 387 ff. DOI logo
Trujillo, James P. & Judith Holler
2023. Interactionally Embedded Gestalt Principles of Multimodal Human Communication. Perspectives on Psychological Science 18:5 ► pp. 1136 ff. DOI logo

This list is based on CrossRef data as of 19 april 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.