Chapter 5
On the role of gaze for successful and efficient communication
Speakers tend to fixate objects they are about to mention, while listeners inspect those objects that they believe to be intended referents of the speaker. These production- and comprehension-contingent gaze behaviors may form an integral part of the signal itself, making it inherently reciprocal. Here, we present work that has investigated the interplay of gaze and language and assessed the role of speaker gaze for language comprehension as well as the utility of listener gaze for an instruction giver. Both lines of research make use of artificial interaction partners which increases experimental control while maintaining a dynamic interactive setting. Thus, the reciprocal nature of situated dialogue becomes a tractable aspect in the enterprise of dealing with human (gaze) behavior.
Article outline
- 1.Introduction
- 2.The benefit of speaker gaze
- 3.Mechanisms underlying gaze benefit
- 4.The benefit of listener gaze
- 5.Discussion
-
References
References
Allopenna, P., Magnuson, J., & Tanenhaus, M.
(
1998)
Tracking the time course of spoken word recognition using eye movements: Evidence for continuous mapping models.
Journal of Memory and Language, 38, 419–439.
Altmann, G., & Kamide, Y.
(
1999)
Incremental interpretation at verbs: restricting the domain of subsequent reference.
Cognition, 73, 247–264.
Bayliss, A., Paul, M., Cannon, P., & Tipper, S.
(
2006)
Gaze cueing and affective judgments of objects: I like what you look at.
Psychonomic Bulletin & Review, 13, 1061–1066.
Becchio, C., Bertone, C., & Castiello, U.
(
2008)
How the gaze of others influences object processing.
Trends in Cognitive Science, 12, 254–258.
Brown-Schmidt, S., & Hanna, J. E.
(
2011)
Talking in another person’s shoes: Incremental perspective-taking in language processing.
Dialogue and Discourse, 2, 11–33.
Brown-Schmidt, S., & Tanenhaus, M. K.
(
2008)
Real-time investigation of referential domains in unscripted conversation: A targeted language game approach.
Cognitive Science, 32(4), 643–684.
Castiello, U.
(
2003)
Understanding Other People’s Actions: Intention and Attention.
Journal of Experimental Psychology, 29, 416–430.
Clark, H. H., & Krych, M. A.
(
2004)
Speaking while monitoring addressees for understanding.
Journal of Memory and Language, 50, 62–81.
Driver, J., Davis, G., Ricciardelli, P., Kidd, P., Maxwell, E., & Baron-Cohen, S.
(
1999)
Gaze Perception Triggers Reflexive Visuospatial Orienting.
Visual Cognition, 6(5), 509–540.
Friesen, C., & Kingstone, A.
(
1998)
The eyes have it! Reflexive orienting is triggered by nonpredictive gaze.
Psychonomic Bulletin & Review, 5(3), 490–495.
Garoufi, K., Staudte, M., Koller, A. & Crocker, M. W.
(
2016)
Exploiting listener gaze to improve situated communication in dynamic virtual environments.
Cognitive Science, 40:1671–1703.
Gleitman, L. R., January, D., Nappa, R., & Trueswell, J. C.
(
2007)
On the give and take between event apprehension and utterance formulation.
Journal of Memory and Language, 57, 544–569.
Griffin, Z. M., & Bock, K.
(
2000)
What the eyes say about speaking.
Psychological Science, 11, 274–279.
Hanna, J., & Brennan, S.
(
2007)
Speakers’ eye gaze disambiguates referring expressions early during face-to-face conversation.
Journal of Memory and Language, 57(4), 596–615.
Hanna, J. E., Tanenhaus, M. K., & Trueswell, J. C.
(
2003)
The effects of common ground and perspective on domains of referential interpretation.
Journal of Memory and Language, 49(1).
Keysar, B., Barr, D. J., Balin, J. A., & Brauner, J. S.
(
2000)
Taking perspective in conversation: The role of mutual knowledge in comprehension.
Psychological Science, 11, 32–38.
Koller, A., Garoufi, K., Staudte, M., & Crocker, M. W.
(
2012)
Enhancing referential success by tracking hearer gaze. In Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue. Seoul, South Korea.
Knoeferle, P. & Crocker, M. W.
(
2006)
The coordinated interplay of scene, utterance, and world knowledge: evidence from eye tracking.
Cognitive Science, 30(3):481–529.
Knoeferle, P. & Crocker, M. W.
(
2007)
The influence of recent scene events on spoken comprehension: evidence from eye-movements.
Journal of Memory and Language (Special issue: Language-Vision Interaction); 57(2):519–543.
Knoeferle, P., Pyykkönen-Klauck, P. & Crocker, M. W.
Langton, S. R. & Bruce, V.
(
1999)
Reflexive Visual Orienting in Response to the Social Attention of Others.
Visual Cognition, 6(5), 541–567.
Meyer, A., Sleiderink, A., & Levelt, W.
(
1998)
Viewing and naming objects: Eye movements during noun phrase production.
Cognition, 66, B25–B33.
Meltzoff, A. N., Brooks, R., Shon, A. P., & Rao, R. P.
(
2010)
Social robots are psychological agents for infants: A test of gaze following.
Neural Networks, 23(8–9), 966–972. (Social Cognition: From Babies to Robots)
Metzing, C., & Brennan, S. E.
(
2003)
When conceptual pacts are broken: Partner-specific effects on the comprehension of referring expressions.
Journal of Memory and Language, 49(2), 201–213.
Nappa, R. & Arnold, J.
(
2014)
The road to understanding is paved with the speaker’s intentions: Cues to the speaker’s attention and intentions affect pronoun comprehension.
Cognitive Psychology, 70, 58–81.
Nappa, R., Wessel, A., McEldoon, K. L., Gleitman, L. R., & Trueswell, J. C.
(
2009)
Use of speaker’s gaze and syntax in verb learning.
Language Learning and Development, 5(4), 203–234.
Nass, C., & Moon, Y.
(
2000)
Machines and mindlessness: Social responses to computers.
Journal of Social Issues, 56, 81–103.
Rayner, K.
(
1998)
Eye movements in reading and information processing: 20 years of research.
Psychological Bulletin & Review, 124 (3), 372–422.
Staudte, M., & Crocker, M. W.
(
2011)
Investigating joint attention mechanisms through spoken human-robot interaction.
Cognition, 120, 268–291.
Staudte, M., Crocker, M. W., Heloir, A. & Michael Kipp
(
2014)
The influence of speaker gaze on listener comprehension: Contrasting visual versus intentional accounts.
Cognition, 133(1):317–328.
Tanenhaus, M. K., Spivey-Knowlton, M., Eberhard, K., & Sedivy, J.
(
1995)
Integration of visual and linguistic information in spoken language comprehension.
Science, 268, 1632–1634.
Tomlin, R. S.
(
1997)
Mapping conceptual representations into linguistic representations: The role of attention in grammar. In
J. Nuyts, &
E. Pederson (Eds.),
Language and conceptualization (pp.162–189), Cambridge: Cambridge University Press.
Yoon, S. O., Koh, S., & Brown-Schmidt, S.
(
2012)
Influence of perspective and goals on reference production in conversation.
Psychonomic Bulletin and Review, 19, 699–707.
Yu, C., & Smith, L. B.
(
2012)
Embodied attention and word learning by toddlers.
Cognition, 125, 244–262.
Cited by
Cited by 1 other publications
Härkki, Tellervo
2023.
Mobile gaze tracking and an extended linkography for collaborative sketching and designing.
International Journal of Technology and Design Education 33:2
► pp. 379 ff.
This list is based on CrossRef data as of 22 april 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.