Cover not available
Article published In:
Interaction Studies
Vol. 24:3 (2023) ► pp.380–421

A matter of consequences

Understanding the effects of robot errors on people’s trust in HRI

ORCID logo | University of Naples Federico IINaples | University of HertfordshireHatfield
 | University of WaterlooWaterloo
 | University of HertfordshireHatfield
 | University of HertfordshireHatfield
On reviewing the literature regarding acceptance and trust in human-robot interaction (HRI), there are a number of open questions that needed to be addressed in order to establish effective collaborations between humans and robots in real-world applications. In particular, we identified four principal open areas that should be investigated to create guidelines for the successful deployment of robots in the wild. These areas are focused on: (1) the robot’s abilities and limitations; in particular when it makes errors with different severity of consequences, (2) individual differences, (3) the dynamics of human-robot trust, and (4) the interaction between humans and robots over time. In this paper, we present two very similar studies, one with a virtual robot with human-like abilities, and one with a Care-O-bot 4 robot. In the first study, we create an immersive narrative using an interactive storyboard to collect responses of 154 participants. In the second study, 6 participants had repeated interactions over three weeks with a physical robot. We summarise and discuss the findings of our investigations of the effects of robots’ errors on people’s trust in robots for designing mechanisms that allow robots to recover from a breach of trust. In particular, we observed that robots’ errors had greater impact on people’s trust in the robot when the errors were made at the beginning of the interaction and had severe consequences.
Our results also provided insights on how these errors vary according to the individuals’ personalities, expectations and previous experiences.
Keywords: trust, social robotics, previous experiences, antecedents of trust, robots’ errors, faulty robots
Article outline
  • 1.Introduction
  • 2.Background & related work
    • 2.1Robots’ errors in HRI
    • 2.2Antecedents of trust
    • 2.3Trust in long-term human-robot interactions
  • 3.Methodology
    • 3.1Study 1: Interactive storyboard
      • 3.1.1The robot
      • 3.1.2Motion picture generation
      • 3.1.3Experimental design
      • 3.1.4Participants
    • 3.2Study 2: A repeated-interactions study
    • 3.3The tasks
  • 4.Study 1: Trust and robot’s errors
    • 4.1Participants’ trust in the robot Jace in relation to the robot errors
    • 4.2Analysis of the explanations participants gave for decision-making in the emergency scenario
  • 5.Study 1: Antecedents of trust and robot errors
    • 5.1Effects of people’s personality on trust
    • 5.2Effects of people’s past experiences on trust
    • 5.3Effects of perception of robots
      • 5.3.1Perception of a robot as a companion
      • 5.3.2Expectation of a robot’s capabilities
      • 5.3.3Perception of a robot’s role
    • 5.4Effects of perception of the robot Jace in relation to the magnitude of consequences of the errors
      • 5.4.1Perceived companionship
      • 5.4.2Perceived reliability and faith in the ability of the robot
      • 5.4.3Perception of the robot’s role
  • 6.Study 2: Evolution of trust and erroneous robot behaviours
    • 6.1Trust in Care-O-bot 4 in relation to the robot’s errors
    • 6.2Antecedents of trust
    • 6.3Perception of Care-O-bot 4
    • 6.4Evaluation of robot errors
  • 7.Limitations
  • 8.Conclusion & future work
    • 8.1Original contributions to knowledge
    • 8.2Future works
  • Notes
  • References
Published online: 15 February 2024
DOI logo
https://doi.org/10.1075/is.21025.ros
References (83)
References
Adams, B. D., Bruyn, L. E., Houde, S., Angelopoulos, P., Iwasa-Madge, K., and McCann, C. (2003). Trust in automated systems. Ministry of National Defence.Google Scholar
Agresti, A. (2002). Categorical data analysis. Wiley-Interscience, Chichester;New York;, 2nd edition. DOI logoGoogle Scholar
Ambady, N., Bernieri, F. J., and Richeson, J. A. (2000). Toward a histology of social behavior: Judgmental accuracy from thin slices of the behavioral stream. Advances in Experimental Social Psychology, 321:201–271. DOI logoGoogle Scholar
Aroyo, A. M., Pasquali, D., Kothig, A., Rea, F., Sandini, G., and Sciutti, A. (2021). Expectations vs. reality: Unreliability and transparency in a treasure hunt game with icub. IEEE Robotics and Automation Letters, 6(3):5681–5688. DOI logoGoogle Scholar
Atkinson, D. J., Clancey, W. J., and Clark, M. H. (2014). Shared awareness, autonomy and trust in human-robot teamwork. In In Artificial Intelligence and Human-Computer Interaction: Papers from the 2014 AAAI Spring Symposium on.Google Scholar
Bainbridge, W. A., Hart, J. W., Kim, E. S., and Scassellati, B. (2011). The benefits of interactions with physically present robots over video-displayed agents. International Journal of Social Robotics, 3(1):41–52. DOI logoGoogle Scholar
Barrick, M. R., Stewart, G. L., Neubert, M. J., and Mount, M. K. (1998). Relating member ability and personality to work-team processes and team effectiveness. Journal of Applied Psychology, 83(3):377–391. DOI logoGoogle Scholar
Bottom, W. P., Gibson, K., Daniels, S. E., and Murnighan, J. K. (2002). When talk is not cheap: Substantive penance and expressions of intent in rebuilding cooperation. Organization Science, 13(5):497–513. DOI logoGoogle Scholar
Buhrmester, M., Kwang, T., and Gosling, S. (2011). Amazon’s mechanical turk: A new source of inexpensive, yet high-quality data? Perspectives on Psychological Science, 61:3–5. DOI logoGoogle Scholar
Chekroun, P. and Brauer, M. (2002). The bystander effect and social control behavior: the effect of the presence of others on people’s reactions to norm violations. European Journal of Social Psychology, 32(6):853–867. DOI logoGoogle Scholar
Colquitt, J. A., Scott, B. A., and LePine, J. A. (2007). Trust, trustworthiness, and trust propensity: A meta-analytic test of their unique relationships with risk taking and job performance. Journal of Applied Psychology, 92(4):909–927. DOI logoGoogle Scholar
Corritore, C. L., Kracher, B., and Wiedenbeck, S. (2003). On-line trust: concepts, evolving themes, a model. International Journal of Human – Computer Studies, 58(6):737–758. DOI logoGoogle Scholar
Costa, A. C., Roe, R. A., and Taillieu, T. C. B. (2001). Trust implications for performance and effectiveness. European Journal of Work and Organizational Psychology, 10(3):225–244. DOI logoGoogle Scholar
Daniela, C. E. C., and S., B. (2017). Personality factors and acceptability of socially assistive robotics in teachers with and without specialized training for children with disability. Life Span and Disability, 20(2):251–272.Google Scholar
Dautenhahn, K., Woods, S., Kaouri, C., Walters, M. L., Koay, Kheng Lee and Werry, I. (2005). “What is a robot companion – friend, assistant or butler?,” 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, Edmonton, AB, Canada, 2005, pp. 1192–1197, DOI logoGoogle Scholar
de Graaf, M. M. and Ben Allouch, S. (2014). Expectation setting and personality attribution in hri. In Proceedings of the 2014 ACM/IEEE International Conference on Human-Robot Interaction, HRI ’14, page 144–145, New York, NY, USA. Association for Computing Machinery. DOI logoGoogle Scholar
de Visser, E. J., Peeters, M. M. M., Jung, M. F., Kohn, S., Shaw, T. H., Pak, R., and Neerincx, M. A. (2020). Towards a theory of longitudinal trust calibration in human-robot teams. International Journal of Social Robotics, page 459–478. DOI logoGoogle Scholar
DeNeve, K. M. and Cooper, H. (1998). The happy personality: A meta-analysis of 137 personality traits and subjective well-being. Psychological Bulletin, 1241:197–229. DOI logoGoogle Scholar
Desai, M., Kaniarasu, P., Medvedev, M., Steinfeld, A., and Yanco, H. (2013). Impact of robot failures and feedback on real-time trust. In ACM/IEEE International Conference on HumanRobot Interaction, pages 251–258. DOI logoGoogle Scholar
Deutsch, M. (1958). Trust and suspicion. The Journal of Conflict Resolution, 2(4):265–279. DOI logoGoogle Scholar
Drury, J. L., Scholtz, J., and Yanco, H. A. (2003). Awareness in human-robot interactions. In Systems, Man and Cybernetics, 2003. IEEE International Conference on, volume 11, pages 912–918. DOI logoGoogle Scholar
Elson, J. S., Derrick, D. C., and Ligon, G. S. (2018). Examining trust and reliance in collaborations between humans and automated agents. In HICSS. DOI logoGoogle Scholar
Freedy, A., DeVisser, E., Weltman, G., and Coeyman, N. (2007). Measurement of trust in human-robot collaboration. In Proceedings of the 2007 International Symposium on Collaborative Technologies and Systems, CTS, pages 106–114. DOI logoGoogle Scholar
Furlough, C., Stokes, T., and Gillan, D. J. (2019). Attributing blame to robots: I. the influence of robot autonomy. Human Factors, 63(4):592–602. DOI logoGoogle Scholar
Gockley, R. and Matariundefined, M. J. (2006). Encouraging physical therapy compliance with a hands-off mobile robot. In Proceedings of the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction, page 150–155, New York, NY, USA. Association for Computing Machinery. DOI logoGoogle Scholar
Gosling, S. D., Rentfrow, P. J., and Swann, W. B. J. (2003). A very brief measure of the big five personality domains. Journal of Research in Personality, 371:504–528. DOI logoGoogle Scholar
Haberlandt, K. (1982). Reader expectations in text comprehension. In Ny], J.-F. L. and Kintsch, W., editors, Language and Comprehension, volume 9 of Advances in Psychology, pages 239–249. North-Holland. DOI logoGoogle Scholar
Hancock, P. A., Billings, D. R., and Schaefer, K. E. (2011a). Can you trust your robot? Ergonomics in Design, 19(3):24–29. DOI logoGoogle Scholar
Hancock, P. A., Billings, D. R., Schaefer, K. E., Chen, J. Y. C., de Visser, E. J., and Parasuraman, R. (2011b). A meta-analysis of factors affecting trust in human-robot interaction. Human Factors: The Journal of Human Factors and Ergonomics Society, 53(5):517–527. DOI logoGoogle Scholar
Haring, K. S., Matsumoto, Y., and Watanabe, K. (2013). How do people perceive and trust a lifelike robot. Lecture Notes in Engineering and Computer Science, 11:425–430. 9789881925169Google Scholar
Haselhuhn, M. P., Schweitzer, M. E., and Wood, A. M. (2010). How implicit beliefs influence trust recovery. Psychological Science, 51:645–648. DOI logoGoogle Scholar
Honig, S. and Oron-Gilad, T. (2018). Understanding and resolving failures in human-robot interaction: Literature review and model development. Frontiers in Psychology, 91. DOI logoGoogle Scholar
Koay, K., Walters, M., May, A., Dumitriu, A., Christianson, B., Burke, N., and Dautenhahn, K. (2013). Exploring robot etiquette: Refining a hri home companion scenario based on feedback from two artists who lived with robots in the uh robot house. In Social Robotics, Lecture Notes in Computer Science, pages 290–300. Springer. 5th Int Conf on Social Robotics, ICSR 2013; Conference date: 27-10-2013 Through 29-10-2013. DOI logo
Kudryavtsev, A. and Pavlodsky, J. (2012). Description-based and experience-based decisions: Individual analysis. Judgment and Decision Making, 7(3):316–331. DOI logoGoogle Scholar
Lee, A. Y. (2001). The mere exposure effect: An uncertainty reduction explanation revisited. Personality and Social Psychology Bulletin, 27(10):1255–1266. DOI logoGoogle Scholar
Lee, J. and Moray, N. (1992). Trust, control strategies and allocation of function in humanmachine systems. Ergonomics, 35(10):1243–1270. DOI logoGoogle Scholar
Lee, J. D. and See, K. A. (2004). Trust in automation: Designing for appropriate reliance. Human Factors: The Journal of the Human Factors and Ergonomics Society, 46(1):50–80. DOI logoGoogle Scholar
Lee, M. K., Forlizzi, J., Kiesler, S., Rybski, P., Antanitis, J., and Savetsila, S. (2012). Personalization in hri: A longitudinal field experiment. In Proceedings of the Seventh Annual ACM/IEEE International Conference on Human-Robot Interaction, page 319–326, New York, NY, USA. Association for Computing Machinery. DOI logoGoogle Scholar
Lemaignan, S., Fink, J., Mondada, F., and Dillenbourg, P. (2015). You’re doing it wrong! studying unexpected behaviors in child-robot interaction. In Tapus, A., André, E., Martin, J.-C., Ferland, F., and Ammi, M., editors, Social Robotics, pages 390–400, Cham. Springer International Publishing. DOI logoGoogle Scholar
Lewis, J. D. and Weigert, A. (1985). Trust as a social reality. Social Forces, 63(4):967–985. DOI logoGoogle Scholar
MacDorman, K. F. and Ishiguro, H. (2006). The uncanny advantage of using androids in cognitive and social science research. Interaction Studies, 7(3):297–337. DOI logoGoogle Scholar
Macias, W. (2003). A beginning look at the effects of interactivity, product involvement and web experience on comprehension: Brand web sites as interactive advertising. Journal of Current Issues & Research in Advertising, 25(2):31–44. DOI logoGoogle Scholar
Mayer, R. C., Davis, J. H., and Schoorman, F. D. (1995). An integrative model of organizational trust. Academy of Management Review, pages 709–734. DOI logoGoogle Scholar
McAllister, D. J. (1995). Affect- and cognition-based trust as foundations for interpersonal cooperation in organizations. Academy of Management Journal, 38(1):24–59. DOI logoGoogle Scholar
McKnight, D. H., Choudhury, V., and Kacmar, C. (2001). Developing and validating trust measures for e-commerce: An integrative typology. Information Systems Research, 13(3):334–359. DOI logoGoogle Scholar
Mori, M., MacDorman, K. F., and Kageki, N. (2012). The uncanny valley [from the field]. IEEE Robotics Automation Magazine, 19(2):98–100. DOI logoGoogle Scholar
Muir, B. M. and Moray, N. (1996). Trust in automation: Part ii. experimental studies of trust and human intervention in a process control simulation. Ergonomics, 391:429–460. DOI logoGoogle Scholar
Paetzel, M., Perugia, G., and Castellano, G. (2020). The persistence of first impressions: The effect of repeated interactions on the perception of a social robot. In Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction, HRI 20, page 73–82, New York, NY, USA. Association for Computing Machinery. DOI logoGoogle Scholar
Pixar (2011). The moon, short movie. [URL]. last accessed in May 6th, 2020.
Rae, I., Takayama, L., and Mutlu, B. (2013). In-body experiences: Embodiment, control, and trust in robot-mediated communication. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, page 1921–1930, New York, NY, USA. Association for Computing Machinery. DOI logoGoogle Scholar
Reber, R., Winkielman, P., and Schwarz, N. (1998). Effects of perceptual fluency on affective judgments. Psychological Science, 9(1):45–48. DOI logoGoogle Scholar
Reis, H.T., Maniaci, M.R., Caprariello, P.A., Eastwick, P.W., and Finkel, E.J. (2011). Familiarity does indeed promote attraction in live interaction. Journal of Personality and Social Psychology, 101(3):557–570. DOI logoGoogle Scholar
Robert, L. P. (2018). Personality in the human robot interaction literature: A review and brief critique. In Proceedings of the 24th Americas Conference on Information Systems.Google Scholar
Robinette, P., Howard, A. M., and Wagner, A. R. (2015). Timing is key for robot trust repair. Social Robotics. Lecture Notes in Computer Science, 93881:574–583. DOI logoGoogle Scholar
Robinette, P., Li, W., Allen, R., Howard, A. M., and Wagner, A. R. (2016). Overtrust of robots in emergency evacuation scenarios. In Proceeding HRI ’16 The Eleventh ACM/IEEE International Conference on Human Robot Interation, pages 101–108. IEEE Press Piscataway. DOI logoGoogle Scholar
Roccas, S., Sagiv, L., Schwartz, S. H., and Knafo, A. (2002). The big five personality factors and personal values. Personality and Social Psychology Bulletin, 28(6):789–801. DOI logoGoogle Scholar
Ross, J. M. (2008). Moderators of trust and reliance across multiple decision aids (doctoral dissertation), university of central florida, orlando.
Rossi, A., Dautenhahn, K., Koay, K. L., and Walters, M. L. (2017a). How the timing and magnitude of robot errors influence peoples’ trust of robots in an emergency scenario. In Kheddar, A., Yoshida, E., Ge, S. S., Suzuki, K., Cabibihan, J.-J., Eyssel, F., and He, H., editors, Social Robotics, pages 42–52, Cham. Springer International Publishing. DOI logoGoogle Scholar
(2017b). Human perceptions of the severity of domestic robot errors. In Kheddar, A., Yoshida, E., Ge, S. S., Suzuki, K., Cabibihan, J.-J., Eyssel, F., and He, H., editors, Social Robotics, pages 647–656, Cham. Springer International Publishing. DOI logoGoogle Scholar
(2018a). The impact of peoples’ personal dispositions and personalities on their trust of robots in an emergency scenario. Paladyn Journal of Behavioral Robotics, 91. DOI logoGoogle Scholar
(2020a). How social robots influence people’s trust in critical situations. In 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pages 1020–1025. DOI logoGoogle Scholar
Rossi, A., Holthaus, P., Dautenhahn, K., Koay, K. L., and Walters, M. L. (2018b). Getting to know pepper: Effects of people’s awareness of a robot’s capabilities on their trust in the robot. In Proceedings of the 6th International Conference on Human-Agent Interaction, HAI ’18, pages 246–252, New York, NY, USA. ACM. DOI logoGoogle Scholar
Rossi, A., Moros, S., Dautenhahn, K., Koay, K. L., and Walters, M. L. (2019). Getting to know kaspar : Effects of people’s awareness of a robot’s capabilities on their trust in the robot. In 2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pages 1–6. DOI logoGoogle Scholar
Rossi, A. and Rossi, S. (2021). Engaged by a bartender robot: Recommendation and personalisation in human-robot interaction. In Adjunct Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization, UMAP ’21, page 115–119, New York, NY, USA. Association for Computing Machinery. DOI logoGoogle Scholar
Rossi, S., Rossi, A., and Dautenhahn, K. (2020b). The secret life of robots: Perspectives and challenges for robot’s behaviours during non-interactive tasks. International Journal of Social Robotics. DOI logoGoogle Scholar
Rotter, J. B. (1967). A new scale for the measurement of interpersonal trust1. Journal of Personality, 35(4):651–665. DOI logoGoogle Scholar
Sadeghi, N., Kasim, Z. M., Tan, B. H., and Abdullah, F. S. (2012). Learning styles, personality types and reading comprehension performance. Psychology. DOI logoGoogle Scholar
Salem, M. and Dautenhahn, K. (2015). Evaluating Trust and Safety in HRI: Practical Issues and Ethical Challenges. In Emerging Policy and Ethics of Human-Robot Interaction: A Workshop at 10th ACM/IEEE Int Conf on Human-Robot Interaction (HRI 2015). ACM Press, 10th ACM/IEEE Int Conf on Human-Robot Interaction, Portland, United States, 2/03/15.Google Scholar
Salem, M., Lakatos, G., Amirabdollahian, F., and Dautenhahn, K. (2015). Would you trust a (faulty) robot? effects of error, task type and personality on human-robot cooperation and trust. In Proceedings of the Tenth Annual ACM/IEEE International Conference on HumanRobot Interaction, HRI 15, page 141–148, New York, NY, USA. Association for Computing Machinery. DOI logoGoogle Scholar
Schilke, O., Reimann, M., and Cook, K. S. (2013). Effect of relationship experience on trust recovery following a breach. Proceedings of the National Academy of Sciences, 110(38):15236–15241. DOI logoGoogle Scholar
Seo, S. H., Geiskkovitch, D., Nakane, M., King, C., and Young, J. E. (2015). Poor thing! would you feel sorry for a simulated robot? a comparison of empathy toward a physical and a simulated robot. In Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction, page 125–132, New York, NY, USA. Association for Computing Machinery. DOI logoGoogle Scholar
Sheehan, K. and Pittman, M. (2016). The academic’s guide to using amazon’s mechanical turk: The hit handbook for social science research. Irving: Melvin & Leigh.Google Scholar
Short, E., Hart, J., Vu, M., and Scassellati, B. (2010). No fair!! an interaction with a cheating robot. In 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pages 219–226. DOI logoGoogle Scholar
Takayama, L. and Pantofaru, C. (2009). Influences on proxemic behaviors in human-robot interaction. In Proceedings of the 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 09, page 5495–5502. IEEE Press. DOI logoGoogle Scholar
Tannenbaum, K. R., Torgesen, J. K., and Wagner, R. K. (2006). Relationships between word knowledge and reading comprehension in third-grade children. Scientific Studies of Reading, 10(4):381–398. DOI logoGoogle Scholar
Tseng, S. H., Hua, J. H., Ma, S. P., and e. Fu, L. (2013). Human awareness based robot performance learning in a social environment. In 2013 IEEE International Conference on Robotics and Automation, pages 4291–4296. DOI logoGoogle Scholar
van Maris, A., Lehmann, H., Natale, L., and Grzyb, B. (2017). The influence of a robot’s embodiment on trust: A longitudinal study. In Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction, page 313–314, New York, NY, USA. Association for Computing Machinery. DOI logoGoogle Scholar
Voelpel, S. C., Eckhoff, R. A., and Forster, J. (2008). David against goliath? group size and bystander effects in virtual knowledge sharing. Human Relations, 61(2):271–295. DOI logoGoogle Scholar
Walters, M. L., Oskoei, M. A., Syrdal, D. S., and Dautenhahn, K. (2011). A long-term humanrobot proxemic study. In IEEE RO-MAN, pages 137–142.Google Scholar
Williamson, J. M. (2018). Chapter 1 – individual differences. In Williamson, J. M., editor, Teaching to Individual Differences in Science and Engineering Librarianship, pages 1–10. Chandos Publishing. DOI logoGoogle Scholar
Wood, T. (2014). Exploring the role of first impressions in rater-based assessments. Adv Health Sci Educ Theory Pract, 19(3):409–427. DOI logoGoogle Scholar
Yu, K., Berkovsky, S., Taib, R., Conway, D., Zhou, J., and Chen, F. (2017). User trust dynamics: An investigation driven by differences in system performance. ACM, 1267451:307–317. DOI logoGoogle Scholar
Zajonc, R. B. (1968). Attitudinal effects of mere exposure. Journal of Personality and Social Psychology, 9(2, Pt.2):1–27. DOI logoGoogle Scholar