Cover not available
Part of
Eye-tracking in Interaction: Studies on the role of eye gaze in dialogue
Edited by Geert Brône and Bert Oben
[Advances in Interaction Studies 10] 2018
► pp. 67–90

Chapter 4
Weaving oneself into others

Coordination in conversational systems

We review a range of findings that show how eye movements (and other body movements) exhibit correlated behavior across two or more people during natural interactions. We then synthesize these different results into a more general account of how people’s cognitive, sensory and motor systems become coordinated with one another during natural dialogue. We argue that treating conversants as parts of one integrated system is a useful explanatory strategy for understanding interaction. We end by describing explicit quantitative conditions for seeking “systemhood” in human interaction. These conditions motivate future research questions on social eye movements and other behaviors.

Keywords: behavioral synchrony, interaction as coupled multi-person system, dense-sampling measures, social gaze
Article outline
  • 1.Introduction: The eyes of social systems
    • 1.1Three types of system
  • 2.Collecting samples from the temporal dynamics of a cognitive process
    • 2.1Devise a cyclic motor behavior
    • 2.2A series of experimental trials as dynamics
    • 2.3Dense-sampling methods
  • 3.Eyetracking as a dense-sampling measure of human interaction
    • 3.1Human interaction as systemic coupling
  • 4.Concluding discussion: Criteria for systemhood
    • 4.1Implications for research: Weak and strong conditions for systemhood
    • 4.2Conclusion
  • References
Published online: 13 November 2018
DOI logo
https://doi.org/10.1075/ais.10.04dal
References
Adams, F., & Aizawa, K.
(2001) The bounds of cognition. Philosophical psychology, 14(1), 43–64.DOI logoGoogle Scholar
Allopenna, P. D., Magnuson, J. S., & Tanenhaus, M. K.
(1998) Tracking the time course of spoken word recognition using eye movements: Evidence for continuous mapping models. Journal of memory and language, 38(4), 419–439.DOI logoGoogle Scholar
Altmann, G. T., & Kamide, Y.
(1999) Incremental interpretation at verbs: Restricting the domain of subsequent reference. Cognition, 73(3), 247–264.DOI logoGoogle Scholar
Anderson, J. R.
(1996) ACT: A simple theory of complex cognition. American Psychologist, 51(4), 355.DOI logoGoogle Scholar
Ashby, W. R.
(1956) An introduction to cybernetics. London: Chapman & Hall.DOI logoGoogle Scholar
Bakeman, R., & Quera, V.
(2011) Sequential analysis and observational methods for the behavioral sciences. Cambridge: Cambridge University Press.DOI logoGoogle Scholar
Bechtel, W., & Abrahamsen, A.
(1991) Connectionism and the Mind. Backwell, Cambridge, MA.Google Scholar
(2010) Dynamic mechanistic explanation: Computational modeling of circadian rhythms as an exemplar for cognitive science. Studies in History and Philosophy of Science Part A, 41(3), 321–333.DOI logoGoogle Scholar
Beer, R. D., & Williams, P. L.
(2015) Information processing and dynamics in minimally cognitive agents. Cognitive science, 39(1), 1–38.DOI logoGoogle Scholar
Brennan, S. E., Galati, A., & Kuhlen, A. K.
(2010) Two minds, one dialog: Coordinating speaking and understanding. Psychology of Learning and Motivation, 53, 301–344.DOI logoGoogle Scholar
Brown-Schmidt, S., Campana, E., & Tanenhaus, M. K.
(2005) Real-time reference resolution by naïve participants during a task-based unscripted conversation. In J. Trueswell & M. Tanenhaus (Eds.), World Situated Language Use: Psycholinguistic, Linguistic and Computational Perspectives on Bridging the Product and Action Traditions, (pp.153–171) Cambridge, MA: MIT Press.Google Scholar
Brown-Schmidt, S., Yoon, S. O., & Ryskin, R. A.
(2015) People as contexts in conversation. Psychology of Learning and Motivation, 62, 59–99.DOI logoGoogle Scholar
Brown-Schmidt, S., & Tanenhaus, M. K.
(2008) Real-time investigation of referential domains in unscripted conversation: A targeted language game approach. Cognitive Science, 32(4), 643–684.DOI logoGoogle Scholar
Chemero, A.
(2011) Radical embodied cognitive science. MIT press.Google Scholar
Chemero, A., & Turvey, M. T.
(2008) Autonomy and hypersets. Biosystems,91(2), 320–330.DOI logoGoogle Scholar
Clark, H. H., & Wilkes-Gibbs, D.
(1986) Referring as a collaborative process. Cognition, 22(1), 1–39.DOI logoGoogle Scholar
Coupland, N.
(1985) ‘Hark, hark, the lark’: social motivations for phonological style-shifting. Language & Communication, 5(3), 153–171.DOI logoGoogle Scholar
Crosby, J. R., Monin, B. & Richardson, D. C.
(2008) Where Do We Look During Potentially Offensive Behavior? Psychological Science, 19 (3), 226–228.DOI logoGoogle Scholar
Crutchfield, J. P.
(1994) The calculi of emergence: computation, dynamics and induction. Physica D: Nonlinear Phenomena, 75(1), 11–54.DOI logoGoogle Scholar
Dale, R., Kirkham, N. Z. & Richardson, D. C.
(2011) The dynamics of reference and shared visual attention. Frontiers in Psychology, 2.DOI logoGoogle Scholar
Dotov, D. G., Nie, L., & Chemero, A.
(2010) A demonstration of the transition from ready-to-hand to unready-to-hand. PLoS One, 5(3), e9433.DOI logoGoogle Scholar
Eberhard, K. M., Spivey-Knowlton, M. J., Sedivy, J. C., & Tanenhaus, M. K.
(1995) Eye movements as a window into real-time spoken language comprehension in natural contexts. Journal of psycholinguistic research, 24(6), 409–436.DOI logoGoogle Scholar
Edelman, S.
(2008) On the nature of minds, or: truth and consequences.Journal of Experimental & Theoretical Artificial Intelligence, 20(3), 181–196.DOI logoGoogle Scholar
Eliasmith, C.
(1996) The third contender: A critical examination of the dynamicist theory of cognition. Philosophical Psychology, 9(4), 441–463.DOI logoGoogle Scholar
(2012) The complex systems approach: rhetoric or revolution.Topics in cognitive science, 4(1), 72–77.DOI logoGoogle Scholar
Enfield, N. J.
(2013) Relationship thinking: Agency, enchrony, and human sociality. Oxford University Press.DOI logoGoogle Scholar
Farmer, T. A., Cargill, S. A., Hindy, N. C., Dale, R., & Spivey, M. J.
(2007) Tracking the continuity of language comprehension: Computer mouse trajectories suggest parallel syntactic processing. Cognitive Science, 31(5), 889–909.DOI logoGoogle Scholar
Ferreira, F., & Clifton, C.
(1986) The independence of syntactic processing. Journal of memory and language, 25(3), 348–368.DOI logoGoogle Scholar
Fitneva, S. A. & Spivey, M. J.
(2005) Context and language processing: The effect of authorship. In J. Trueswell & M. Tanenhaus (Eds.), World Situated Language Use: Psycholinguistic, Linguistic and Computational Perspectives on Bridging the Product and Action Traditions, (pp.317–328) Cambridge, MA: MIT Press.Google Scholar
Frazier, L.
(1995) Constraint satisfaction as a theory of sentence processing. Journal of Psycholinguistic Research, 24(6), 437–468.DOI logoGoogle Scholar
Fusaroli, R., & Tylén, K.
(2015) Investigating conversational dynamics: Interactive alignment, Interpersonal synergy, and collective task performance. Cognitive Science, 40, 145–171.DOI logoGoogle Scholar
Fusaroli, R., Bahrami, B., Olsen, K., Roepstorff, A., Rees, G., Frith, C., & Tylén, K.
(2012) Coming to terms quantifying the benefits of linguistic coordination. Psychological Science, 23(8), 931–939.DOI logoGoogle Scholar
Fusaroli, R., Bjørndahl, J. S., Roepstorff, A., & Tylén, K.
(2016) A Heart for Interaction: Shared Physiological Dynamics and Behavioral Coordination in a Collective, Creative Construction Task. Journal of Experimental Psychology: Human Perception & Performance, 42(9), 1297–1310.Google Scholar
Fusaroli, R., Rączaszek-Leonardi, J., & Tylén, K.
(2014) Dialog as interpersonal synergy. New Ideas in Psychology, 32, 147–157.DOI logoGoogle Scholar
Gallagher, S., & Crisafi, A.
(2009) Mental institutions. Topoi, 28(1), 45–51.DOI logoGoogle Scholar
Gilden, D. L.
(2001) Cognitive emissions of 1/f noise. Psychological review, 108(1), 33–56.DOI logoGoogle Scholar
Gow, D. W. & Olson, B. B.
(2015): Sentential influences on acoustic- phonetic processing: a Granger causality analysis of multimodal imaging data, Language, Cognition and Neuroscience. DOI logoGoogle Scholar
Haken, H.
(1977) Synergetics. Physics Bulletin , 28(9), 412.DOI logoGoogle Scholar
Haken, H., Kelso, J. S., & Bunz, H.
(1985) A theoretical model of phase transitions in human hand movements. Biological cybernetics, 51(5), 347–356.DOI logoGoogle Scholar
Hasson, U., Nir, Y., Levy, I., Fuhrmann, G., & Malach, R.
(2004) Intersubject synchronization of cortical activity during natural vision. science, 303(5664), 1634-1640.DOI logoGoogle Scholar
Hove, M. J., & Keller, P. E.
(2015) Impaired movement timing in neurological disorders: rehabilitation and treatment strategies. Annals of the New York Academy of Sciences, 1337(1), 111–117.DOI logoGoogle Scholar
Hove, M. J., & Risen, J. L.
(2009) It’s all in the timing: Interpersonal synchrony increases affiliation. Social Cognition, 27(6), 949–960.DOI logoGoogle Scholar
Hove, M. J., Spivey, M. J., & Krumhansl, C. L.
(2010) Compatibility of motion facilitates visuomotor synchronization. Journal of Experimental Psychology: Human Perception and Performance, 36(6), 1525–1534.Google Scholar
Hutchins, E.
(1995) Cognition in the Wild. MIT press.Google Scholar
(1995) How a cockpit remembers its speeds. Cognitive science,19(3), 265–288.DOI logoGoogle Scholar
Jaeger, T. F.
(2010) Redundancy and reduction: Speakers manage syntactic information density. Cognitive psychology, 61(1), 23–62.DOI logoGoogle Scholar
Jordan, J. S., & Knoblich, G.
(2004) Spatial perception and control. Psychonomic Bulletin & Review, 11(1), 54–59.DOI logoGoogle Scholar
Kello, C. T., Anderson, G. G., Holden, J. G., & Van Orden, G. C.
(2008) The pervasiveness of 1/f scaling in speech reflects the metastable basis of cognition. Cognitive Science, 32(7), 1217–1231.DOI logoGoogle Scholar
Kello, C. T., Brown, G. D., Ferrer-i-Cancho, R., Holden, J. G., Linkenkaer-Hansen, K., Rhodes, T., & Van Orden, G. C.
(2010) Scaling laws in cognitive sciences. Trends in Cognitive Sciences, 14(5), 223–232.DOI logoGoogle Scholar
Kelso, J. A. S.
(1984) Phase transitions and critical behavior in human bimanual coordination. American Journal of Physiology-Regulatory, Integrative and Comparative Physiology, 246(6), R1000–R1004.DOI logoGoogle Scholar
(1997) Dynamic patterns: The self-organization of brain and behavior. MIT press.Google Scholar
Kelso, J. S., Tuller, B., Vatikiotis-Bateson, E., & Fowler, C. A.
(1984) Functionally specific articulatory cooperation following jaw perturbations during speech: evidence for coordinative structures. Journal of Experimental Psychology: Human Perception and Performance, 10(6), 812.Google Scholar
Kloos, H., & Van Orden, G.
(2010) Voluntary behavior in cognitive and motor tasks. Mind and Matter, 8(1), 19–43.Google Scholar
Knoblich, G., & Jordan, J. S.
(2003) Action coordination in groups and individuals: learning anticipatory control. Journal of Experimental Psychology: Learning, Memory, and Cognition, 29(5), 1006–1016.Google Scholar
Knoeferle, P., & Crocker, M. W.
(2006) The coordinated interplay of scene, utterance, and world knowledge: Evidence from eye tracking. Cognitive Science, 30(3), 481–529.DOI logoGoogle Scholar
Konvalinka, I., Xygalatas, D., Bulbulia, J., Schjødt, U., Jegindø, E. M., Wallot, S., Van Orden, G., & Roepstorff, A.
(2011) Synchronized arousal between performers and related spectators in a fire-walking ritual. Proceedings of the National Academy of Sciences, 108(20), 8514–8519.DOI logoGoogle Scholar
Kuhlen, A. K., Allefeld, C., & Haynes, J. D.
(2012) Content-specific coordination of listeners’ to speakers’ EEG during communication. Frontiers in Human Neuroscience, 6 , Article 266.Google Scholar
Lakin, J. L., & Chartrand, T. L.
(2003) Using nonconscious behavioral mimicry to create affiliation and rapport. Psychological science, 14(4), 334–339.DOI logoGoogle Scholar
Latash, M. L., Scholz, J. P., & Schöner, G.
(2002) Motor control strategies revealed in the structure of motor variability. Exercise and sport sciences reviews, 30(1), 26–31.DOI logoGoogle Scholar
Louwerse, M. M., Dale, R., Bard, E. G., & Jeuniaux, P.
(2012) Behavior matching in multimodal communication is synchronized. Cognitive Science,36(8), 1404–1426.DOI logoGoogle Scholar
Manson, J. H., Bryant, G. A., Gervais, M. M., & Kline, M. A.
(2013) Convergence of speech rate in conversation predicts cooperation. Evolution and Human Behavior, 34(6), 419–426.DOI logoGoogle Scholar
Mitchell, M.
(2009) Complexity: A guided tour. Oxford University Press.Google Scholar
Mønster, D., Håkonsson, D. D., Eskildsen, J. K., & Wallot, S.
(2016) Physiological evidence of interpersonal dynamics in a cooperative production task. Physiology & Behavior, 156, 24–34.DOI logoGoogle Scholar
Neisser, U.
(1976) Cognition and reality: Principles and implications of cognitive psychology. San Francisco, California: W. H. Freeman.Google Scholar
Newell, A.
(1980) Physical symbol systems*. Cognitive science, 4(2), 135–183.DOI logoGoogle Scholar
Noë, A.
(2009) Out of our heads: Why you are not your brain, and other lessons from the biology of consciousness. Macmillan.Google Scholar
Paxton, A., & Dale, R.
(2013) Argument disrupts interpersonal synchrony. The Quarterly Journal of Experimental Psychology, 66(11), 2092–2102.DOI logoGoogle Scholar
(2013a) Frame-differencing methods for measuring bodily synchrony in conversation. Behavior Research Methods, 45(2), 329–343.DOI logoGoogle Scholar
(2013b) Argument disrupts interpersonal synchrony. The Quarterly Journal of Experimental Psychology, 66(11), 2092–2102.DOI logoGoogle Scholar
Reali, F., Spivey, M. J., Tyler, M. J., & Terranova, J.
(2006) Inefficient conjunction search made efficient by concurrent spoken delivery of target identity. Perception & psychophysics, 68(6), 959–974.DOI logoGoogle Scholar
Repp, B. H.
(2005) Sensorimotor synchronization: A review of the tapping literature. Psychonomic bulletin & review, 12(6), 969–992.DOI logoGoogle Scholar
Richardson, D. C., & Dale, R.
(2005) Looking to understand: The coupling between speakers’ and listeners’ eye movements and its relationship to discourse comprehension. Cognitive science, 29(6), 1045–1060.DOI logoGoogle Scholar
Richardson, D. C., Dale, R. & Kirkham, N.
(2007) The art of conversation is coordination: common ground and the coupling of eye movements during dialogue. Psychological Science, 18, 407–413.DOI logoGoogle Scholar
Richardson, D. C., Street, C. N. H., Tan, J. Y. M., Kirkham, N. Z., Hoover, M. A., & Ghane Cavanaugh, A.
(2012).Joint perception: gaze and social context. Frontiers in Human Neuroscience, 6. DOI logoGoogle Scholar
Richardson, M. J., Dale, R. & Marsh, K.
(2014) Complex dynamical systems in social and personality psychology: theory, modeling and analysis. In H. T. Reis & C. M. Judd (Eds.), Handbook of Research Methods in Social and Personality Psychology (pp.253–282). Cambridge University Press.Google Scholar
Richardson, M. J., Marsh, K. L., & Schmidt, R. C.
(2005) Effects of visual and verbal interaction on unintentional interpersonal coordination. Journal of Experimental Psychology: Human Perception and Performance, 31(1), 62–79.DOI logoGoogle Scholar
Riley, M. A., Richardson, M. J., Shockley, K., & Ramenzoni, V. C.
(2011) Interpersonal synergies. Frontiers in psychology, 2, 38.DOI logoGoogle Scholar
Risko, E. F., & Kingstone, A.
(2011) Eyes wide shut: implied social presence, eye tracking and attention. Attention, Perception, & Psychophysics, 73(2), 291–296.DOI logoGoogle Scholar
Risko, E. F., Richardson, D. C., & Kingstone, A.
(2016) Breaking the Fourth Wall of Cognitive Science Real-World Social Attention and the Dual Function of Gaze. Current Directions in Psychological Science, 25(1), 70–74.DOI logoGoogle Scholar
Rupert, R. D.
(2011) Cognitive systems and the supersized mind. Philosophical Studies, 152(3), 427–436.DOI logoGoogle Scholar
Schmidt, R. C., Carello, C., & Turvey, M. T.
(1990) Phase transitions and critical fluctuations in the visual coordination of rhythmic movements between people. Journal of experimental psychology: human perception and performance, 16(2), 227–247.Google Scholar
Sebanz, N., Knoblich, G., & Prinz, W.
(2003) Representing others’ actions: just like one’s own?. Cognition, 88(3), B11–B21.DOI logoGoogle Scholar
Sebanz, N., Knoblich, G., Prinz, W., & Wascher, E.
(2006) Twin peaks: An ERP study of action planning and control in coacting individuals. Journal of Cognitive Neuroscience, 18(5), 859–870.DOI logoGoogle Scholar
Shenhar, A.
(1990) On system properties and systemhood. International Journal Of General System, 18(2), 167–174.DOI logoGoogle Scholar
Shockley, K., Baker, A. A., Richardson, M. J., & Fowler, C. A.
(2007) Articulatory constraints on interpersonal postural coordination. Journal of Experimental Psychology: Human Perception and Performance, 33(1), 201.Google Scholar
Shockley, K., Santana, M. V., & Fowler, C. A.
(2003) Mutual interpersonal postural constraints are involved in cooperative conversation. Journal of Experimental Psychology: Human Perception and Performance, 29(2), 326–332.Google Scholar
Simon, H. A.
(1992) What is an “explanation” of behavior?. Psychological Science, 3(3), 150–161.DOI logoGoogle Scholar
Spivey-Knowlton, M. J.
(1996) Integration of visual and linguistic information: Human data and model simulations. Ph.D. Dissertation, U. Rochester, NY.Google Scholar
Spivey, M.
(2008) The continuity of mind. Oxford University Press.Google Scholar
Spivey, M. J.
(2012) The spatial intersection of minds. Cognitive Processing, 13, 343–346.DOI logoGoogle Scholar
(2016) Semantics influences speech perception: commentary on Gow and Olson (2015). Language, Cognition, and Neuroscience, 31(7), 856-859.DOI logoGoogle Scholar
Strogatz, S.
(2003) Sync: The emerging science of spontaneous order. New York, NY: Hyperion.Google Scholar
Taleb, N. N.
(2012) Antifragile: Things that gain from disorder (Vol. 3). Random House Incorporated.Google Scholar
Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C.
(1995) Integration of visual and linguistic information in spoken language comprehension. Science, 268(5217), 1632–1634.DOI logoGoogle Scholar
Tollefsen, D. P.
(2006) From extended mind to collective mind. Cognitive systems research, 7(2), 140–150.DOI logoGoogle Scholar
Tollefsen, D. & Dale, R.
(2012) Naturalizing joint action: a process-based approach. Philosophical Psychology, 25, 385–407.DOI logoGoogle Scholar
Van Gelder, T.
(1998) The dynamical hypothesis in cognitive science.Behavioral and brain sciences, 21(05), 615–628.DOI logoGoogle Scholar
Van Orden, G. C., & Holden, J. G.
(2002) Intentional contents and self-control. Ecological Psychology, 14(1–2), 87–109.DOI logoGoogle Scholar
Van Orden, G. C., Holden, J. G., & Turvey, M. T.
(2003) Self-organization of cognitive performance. Journal of Experimental Psychology: General, 132(3), 331.DOI logoGoogle Scholar
Van Orden, G., & Stephen, D. G.
(2012) Is cognitive science usefully cast as complexity science?. Topics in cognitive science, 4(1), 3–6.DOI logoGoogle Scholar
Wegner, D. M.
(1987) Transactive memory: A contemporary analysis of the group mind. In Theories of group behavior (pp.185–208). Springer New York.DOI logoGoogle Scholar
Wiener, N.
(1961) Cybernetics or Control and Communication in the Animal and the Machine (Vol. 25). MIT press.Google Scholar
Cited by

Cited by 2 other publications

Dale, Rick, Anne S. Warlaumont & Kerri L. Johnson
2022. The fundamental importance of method to theory. Nature Reviews Psychology 2:1 ► pp. 55 ff. DOI logo
Spivey, Michael J.
2023. Cognitive Science Progresses Toward Interactive Frameworks. Topics in Cognitive Science 15:2 ► pp. 219 ff. DOI logo

This list is based on CrossRef data as of 22 april 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.