Chapter 10
Gaze as a predictor for lexical and gestural alignment
This chapter provides evidence for the role gaze might have on behaviour at another multimodal level of behaviour, viz. alignment (or copying behaviour) of hand gestures and lexical items. In a corpus of dyadic interactions we demonstrate that gaze behaviour affects alignment behaviour differently at the lexical than at the gestural level: if a speaker is looking at an addressee’s face while uttering a target word, this significantly increases the probability that the addressee will use that same word later in the conversation. If a speaker is looking at an addressee’s face while performing a target gesture, there is no correlation with subsequent gesture production by that addressee. However, if an addressee looked at a gesture made by a speaker, this gesture was significantly more often used by that addressee later in the conversation (compared to situations in which the addressee was not looking at that gesture). We argue that the difference in gaze behaviour between lexical and gestural alignment cases might be explained by the dual function of eye gaze in interaction, viz. gaze for visual perception and gaze for signalling meaning.
Article outline
- 1.Introduction
- 1.1Eye gaze during conversation
- 1.2Alignment during conversation
- 2.Case study 1: Gaze and lexical alignment
- 2.1Method
- 2.2Analysis
- 2.3Results
- 2.4Discussion
- 3.Case study 2: Gaze and gestural alignment
- 3.1Method & analysis
- 3.2Results
- 3.3Discussion
- 4.Conclusion & future work
-
Note
-
References
References (70)
References
Argyle, M. & Cook, M. (1976). Gaze and Eye contact. London: Cambridge University Press.
Argyle, M. & Ingham, R. (1972). Gaze, mutual gaze, and proximity. Semiotica, 6, 32–49.

Bateson, M., Nettle, D. & Roberts, G. (2006). Cues of being watched enhance cooperation in real-world setting. Biology Letters, 2, 412–414.

Beckner, C., Blythe, R., Bybee, J., Christiansen, M., Croft, W., Ellis, N., Holland, J., Ke, J., Larsen-Freeman, D. & Schoenemann, T. (2009). Language Is a Complex Adaptive System: Position Paper. Language Learning, 59, 1–26.

Bergmann, K. & Kopp, S. (2012). Gestural alignment in natural dialogue. Proceedings of the 34th Annual Conference of the Cognitive Science Society, 1326–1331.
Branigan, H., Pickering, M. & Cleland, A. (2000). Syntactic co-ordination in dialogue. Cognition, 75, 13–25.

Brennan S., Chen X., Dickinson C., Neider M. & Zelinsky G. (2008). Coordinating cognition: The costs and benefits of shared gaze during collaborative search. Cognition, 106, 1465–1477.

Brennan, S. & Clark, H. (1996). Conceptual pacts and lexical choice in conversation. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22, 1482–1493.
Brône, G., Oben, B. (2015). InSight interaction: A multimodal and multifocal dialogue corpus. Language Resources and Evaluation, 49(1), 195–214.

Clark, H. (1996). Using Language. Cambridge: Cambridge University Press.

Chartrand, T. & Bargh, J. (1999). The chameleon effect: The perception-behavior link and social interaction. Journal of Personality and Social Psychology, 76, 893–910.

Cummins, F. (2011). Gaze and blinking in dyadic conversation: A study in coordinated behaviour among individuals. Language and Cognitive Processes, 27, 1525–1549.

Dale, R., Fusaroli, R., Duran, N. & Richardson, D. (2014). The self-organization of human interaction. In B. Ross (Ed.), Psychology of Learning and Motivation (pp.43–95). Academic Press.
Dale, R., Kirkham, N. & Richardson, D. (2011). How two people become a tangram recognition system. Proceedings of the European Conference on Computer-Supported Cooperative Work. Berlin: Springer Verlag.
Dale, R. & Spivey, M. (2006). Unraveling the dyad: Using recurrence analysis to explore patterns of syntactic coordination between children and caregivers in conversation. Language Learning, 56, 391–430.

Dominey, P. (2004). Situation alignment and routinization in language acquisition. Behavioral and Brain Sciences, 27, 195.

Duchowsky, A. (2007). Eye Tracking Methodology.
Theory and Practice. Berlin: Springer.
Emery, N. (2000). The eyes have it: The neuroethology, function and evolution of social gaze. Neuroscience and Biobehavioral Reviews, 24, 581–604.

Frischen, A., Bayliss, A., & Tipper, S. (2007). Gaze cueing of attention: Visual attention, social cognition, and individual differences. Psychological Bulletin, 133, 694–724.

Fusaroli, R., Bahrami, B., Olsen, K., Roepstorff, A., Rees, G., Frith, C. & Tylén, K. (2012).
[URL]
. Psychological Science, 23, 931–939.

Fusaroli, R., Konvalinka, I. & Wallot, S. (2014). Analyzing social interactions: The promises and challenges of using cross recurrence quantification analysis. Springer Proceedings in Mathematics and Statistics, 103, 137–155.

Fusaroli, R. & Tylén, K. (2015). Investigating conversational dynamics: interactive alignment, interpersonal synergy and collective task performance. Cognitive Science, 40, 145–171.

Garrod, S. & Anderson, A. (1987). Saying what you mean in dialogue: A study in conceptual and semantic co-ordination. Cognition, 27, 181–218.

Giles, H. & Powesland, P. (1975). Speech styles and social evaluation. New York: Academic Press.
Gries, S. (2005). Syntactic Priming: A Corpus-based Approach. Journal of Psycholinguistic Research, 34, 365–399.

Gullberg, M. & Kita, S. (2009). Attention to speech-accompanying gestures: Eye movements and information uptake. Journal of Nonverbal Behaviour, 33, 251–277.

Hadelich, K. & Crocker, M. (2006). Gaze alignment of interlocutors in conversational dialogues. Proceedings of the 2006 Symposium on Eye Tracking Research and Applications, 38.

Jakobson, R. (1960). Linguistics and Poetics. In T. Sebeok (Ed.), Style in Language (pp.350–377). Cambridge MA: M.I.T. Press.
James, W. (1878). Brute and human intellect. The Journal of Speculative Philosphy, 12, 236–276.
Jones, B., DeBruine, L., Little, A., Conway, C. & Feinberg, D. (2006). Integrating gaze direction and expression in preferences for attractive faces. Psychological Science, 17, 588–591.

Kendon, A. (1967) Some functions of gaze-direction in social interaction. Acta Psychologica, 26, 22–63.

Kimbara, I. (2008). Gesture Form Convergence in Joint Description. Journal of Nonverbal Behavior, 32, 123–131.

Lachat, F., Conty, L., Hugueville, L. & George, N. (2012). Gaze cueing effect in a face-to-face situation. Journal of Nonverbal Behaviour, 36, 177–190.

Langton, S., Watt, R. & Bruce, V. (2000). Do the eyes have it? Cues to the direction of social attention. Trends in cognitive sciences, 4, 50–58.

Lewandowski, N. (2012). Talent in nonnative phonetic convergence (Unpublished doctoral dissertation). Universität Stuttgart, Stuttgart.
Louwerse, M., Dale, R., Bard, E. & Jeuniaux, P. (2012). Behavior matching in multimodal communication is synchronized. Cognitive Science, 36, 1404–1426.

Manson, J., Bryant, G., Gervais, M. & Kline, M. (2013). Convergence of speech rate in conversation predicts cooperation. Evolution and Human Behavior, 34, 419–426.

McNeill, D. (2006). Gesture, Gaze, and Ground. Lecture Notes in Computer Science 3869, 1-14.

Mathews, A., Fox, E., Yiend, J. & Calder, A. (2003). The face of fear: Effects of eye gaze and emotion on visual attention. Visual Cognition, 10, 823–835.

Mol, L., Krahmer, E., Maes, A. & Swerts, M. (2012). Adaptation in gesture: Converging hands or converging minds? Journal of Memory and Language, 66, 249–264.

Muir, L. & Richardson, I. (2005). Perception of sign language and its application to visual communications for deaf people. Journal of Deaf Studies and Deaf Education, 10, 390–340.

Nielsen, G. (1962). Studies in self confrontation. Copenhagen: Munksgaard.
Neider, M., Chen, X., Dickinson, C., Brennan, S. & Zelinsky, G. (2010). Coordinating spatial referencing using shared gaze. Psychonomic Bulletin & Review, 17, 718–724.

Oben, B. (2015). Modelling interactive alignment: a multimodal and temporal account. [unpublished doctoral thesis]
Oben B., Brône G. (2015). What you see is what you do: On the relation between gaze and gesture in multimodal alignment. Language and Cognition, 7(4), 546–562.

Paxton, A. & Dale, R. (2013). Frame-differencing methods for measuring bodily synchrony in conversation. Behavior Research Methods, 45, 329–343.

Paxton, A., Abney, D, Kello, C. & Dale, R. (2014). Network analysis of multimodal, multiscale coordination in dyadic problem solving. In P. Bello, M. Guarini, M. McShane & B. Scassellati (Eds.), Proceedings of the 36th Annual Meeting of the Cognitive Science Society (pp.2735–2740). Austin: Cognitive Science Society.
Piaget, J. (1932). The moral judgment of the child. Glencoe: The Free Press.
Piazza, J., Bering, J. & Ingram, G. (2011). Princess Alice is watching you: Children’s belief in an invisible person inhibits cheating. Journal of Experimental Child Psychology, 109, 311–320.

Pickering, M. & Garrod, S. (2004). Towards a Mechanistic Psychology of Dialogue. Behavioural and Brain Sciences, 27, 169–225.

Pickering, M. & Garrod, S. (2006). Alignment as the Basis for Successful Communication. Research on Language and Communication, 4, 203–288.

Costa, A., Pickering, M. & Sorace, A. (2008) Alignment in second language dialogue. Language and Cognitive Processes, 23, 528-556.

Posner, M., Snyder, C. & Davidson, B. (1980). Attention and the detection of signals. Journal of Experimental Psychology, 109, 160–174.

Postma, M., Brunninkhuis, N. & Postma, E. (2013). Eye Gaze Affects Vocal Intonation Mimicry. Proceedings of the 35th Annual Conference of the Cognitive Science Society, 1139–1144.
Powell, L., Roberts, G. & Nettle, D. (2012). Eye images increase charitable donations: Evidence from an opportunistic field experiment in a supermarket. Ethology, 118, 1–6.

Reitter, D., Moore, D. & Keller, F. (2006). Priming of syntactic rules in task-oriented dialogue and spontaneous conversation. Proceedings of the 28th Annual Conference of the Cognitive Science Society, 685–690.
Richardson, D. & Dale, R. (2005). Looking to understand: The coupling between speakers’ and listeners’ eye movements and its relationship to discourse comprehension. Cognitive Science, 29, 1045–1060.

Richardson, D., Dale, R., & Kirkham, N. (2007). The art of conversation is coordination. Common ground and the coupling of eye movements during dialogue. Psychological Science, 18, 407–413.

Richardson, D., Dale, R. & Tomlinson, J. (2009). Conversation, gaze coordination, and beliefs about visual context. Cognitive Science, 33, 1468–1482.

Roche, J., Dale, R. & Caucci, G. (2010). Doubling up on double meanings: Pragmatic alignment. Language and Cognitive Processes, 27, 1–24.

Shannon, C. (1948). A mathematical theory of communication. Bell System Technical Journal, 27, 379–423.

Shockley, K., Santana, M. & Fowler, C. (2003). Mutual interpersonal postural constraints are involved in cooperative conversation. Journal of Experimental Psychology: Human Perception and Performance, 29, 326–332.
Szczepek Reed, B. (2010). Prosody and alignment: A sequential perspective. Cultural Studies of Science Education, 5, 859–867.

Tollefsen, D. & Dale, R. (2012). Naturalizing joint action: A process-based approach. Philosophical Psychology, 25, 385–407.

Van Engen, K., Baese-Berk, M., Baker, R., Choi, A., Kim, M. & Bradlow, A. (2010). The Wildcat corpus of native- and foreign-accented English: Communicative efficiency across conversational dyads with varying language alignment profiles. Language and speech, 53, 510–540.

Wang, Y., Newport, R. & Hamilton, A. (2011). Eye contact enhances mimicry of intransitive hand movements. Biology Letters, 7, 7–10.

Wang, Y. & Hamilton, A. (2014). Why does gaze enhance mimicry? Placing gaze-mimicry effects in relation to other gaze phenomena. The Quarterly Journal of Experimental Psychology, 67, 747–762.

Zima, E. (2013). Kognition in der Interaktion. Eine kognitiv-linguistische und gesprächsanalytische Studie dialogischer Resonanz in österreichischen Parlamentsdebatten. Heidelberg: Universitätsverlag Winter.
Cited by (2)
Cited by two other publications
Pleyer, Michael
2023.
The role of interactional and cognitive mechanisms in the evolution of (proto)language(s).
Lingua 282
► pp. 103458 ff.

Rasenberg, Marlou, Asli Özyürek & Mark Dingemanse
2020.
Alignment in Multimodal Interaction: An Integrative Framework.
Cognitive Science 44:11

This list is based on CrossRef data as of 11 january 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.