Cover not available
Article published In:
Translation, Cognition & Behavior
Vol. 5:1 (2022) ► pp.1–26

Considerations of ecological validity in cognitive translation and interpreting studies

ORCID logo | University of North Carolina at Charlotte
ORCID logo | Butler University
Renewed interest in research methods used in cognitive translation and interpreting studies (CTIS) has led to increased reflection on the nature of research and experimental design, internal and external validity, and the type and nature of experimental tasks. Of particular concern is the extent to which valid generalization can be made from empirical studies of translation, interpreting, and their associated cognitive behaviors as objects of study. This article traces the definition and history of ecological validity in the extant literature on research methods and philosophy of science, emphasizing three considerations: the experimental setting, the stimuli under investigation, and the behavioral response of participants. Subsequently, we discuss potential misunderstandings or misapplications of appeals to ecological validity, including mundane realism, naturalistic tasks, and various data collection methods. A final section describes key points to consider for research in CTIS. Throughout, the argument considers tradeoffs among various categories of validity as well as the importance of aligning experimental design with research goals. This extensive engagement with a singular aspect of validity is provided to encourage deeper reflection and better communication around the topic of ecological validity.
Keywords: ecological validity, generalizability, mundane realism, experimental control, naturalistic tasks, data collection methods
Article outline
  • 1.Introduction
  • 2.Definitions of ecological validity
    • 2.1Alternative conception of ecological validity
  • 3.Balancing risks to ecological validity
    • 3.1Naturalistic tasks
    • 3.2Mundane realism
    • 3.3Data collection methods
  • 4.Establishing ecological validity
    • 4.1Define the relevant environment for experimentation and generalization
    • 4.2Align the stimuli, variables, and measurement with the research purpose
    • 4.3Consider influences and constraints on participant responses
  • 5.Conclusion
  • References
Published online: 8 July 2022
DOI logo
https://doi.org/10.1075/tcb.00061.mel
References (89)
References
Alves, Fabio. 2015. “Translation Process Research at the Interface: Paradigmatic, theoretical, and methodological issues in dialogue with cognitive science, expertise studies, and psycholinguistics.” In Psycholinguistic and Cognitive Inquiries into Translation and Interpreting. Edited by A. Ferreira, and J. W. Schwieter, 17–40. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Araújo, Duarte, Keith Davids, and Pedro Passos. 2007. “Ecological Validity, Representative Design, and Correspondence between Experimental Task Constraints and Behavioral Setting: Comment on Rogers, Kadar, and Costall (2005).” Ecological Psychology 19 (1), 69–78. DOI logoGoogle Scholar
Banaji, Mahzarin R., and Robert G. Crowder. 1989. “The Bankruptcy of Everyday Memory.” American Psychologist 44 (9), 1185–1193. DOI logoGoogle Scholar
Baraldi, Claudio, and Christopher D. Mellinger. 2016. “Observations.” In Researching Translation and Interpreting. Edited by C. V. Angelelli, and B. J. Baer, 257–268. New York: Routledge.Google Scholar
Berkowitz, Leonard, and Edward Donnerstein. 1982. “External Validity is More than Skin Deep: Some answers to criticisms of laboratory experiments.” American Psychologist 37 (3), 245–257. DOI logoGoogle Scholar
Blanco-Elorrieta, Esti, and Liina Pylkkänen. 2018. “Ecological Validity in Bilingualism Research and the Bilingual Advantage.” Trends in Cognitive Sciences 22 (12), 1117–1126. DOI logoGoogle Scholar
Bolaños Medina, Alicia. 2016. “Translation Psychology within the Framework of Translator Studies: New research perspectives.” In From the Lab to the Classroom and Back Again: Perspectives on Translation and Interpreting Training. Edited by C. Martín de León, and V. González-Ruíz, 59–100. Peter Lang.Google Scholar
Bracht, Glenn H., and Gene V. Glass. 1968. “The External Validity of Experiments.” American Educational Research Journal 5 (4), 437–474. DOI logoGoogle Scholar
Bronfenbrenner, Urie. 1977. “Toward an Experimental Ecology of Human Development.” American Psychologist 32 (7), 513–531. DOI logoGoogle Scholar
Brunswik, Egon. 1943. “Organismic Achievement and Environmental Probability.” Psychological Review 50 (3), 255–272. DOI logoGoogle Scholar
. 1949. “Remarks on Functionalism in Perception.” Journal of Personality 18 (1), 56–65. DOI logoGoogle Scholar
. 1955. “Representative Design and Probabilistic Theory in a Functional Psychology.” Psychological Review 62 (3), 193–217. DOI logoGoogle Scholar
Calder, Bobby J., Lynn W. Phillips, and Alice M. Tybout. 1981. “Designing Research for Application.” Journal of Consumer Research 81, 197–207. DOI logoGoogle Scholar
. 1983. “Beyond External Validity.” Journal of Consumer Research 10 (1), 112–114. DOI logoGoogle Scholar
Campbell, Donald T., and Julian C. Stanely. 1963. Experimental and Quasi-experimental Designs for Research. Cengage.Google Scholar
Carlsmith, J. Merrill, Phoebe C. Ellsworth, and Eliot Aronson. 1976. Methods of Research in Social Psychology. Addison-Wesley.Google Scholar
Chaytor, Naomi, and Maureen Schmitter-Edgecombe. 2003. “The Ecological Validity of Neuropsychological Tests: A review of the literature on everyday cognitive skills.” Neuropsychology Review 13 (4), 181–197. DOI logoGoogle Scholar
Chow, Siu L. 1987. “Science, Ecological Validity, and Experimentation.” Journal for the Theory of Social Behavior 17 (2), 181–194. DOI logoGoogle Scholar
Cook, Thomas D., and Donald T. Campbell. 1979. Quasi-experimentation: Design & analysis issues for field settings. Houghton Mifflin.Google Scholar
Dipboye, Robert L., and Michael F. Flanagan. 1980. “Reply to Willems and Howard.” American Psychologist 351, 388–390. DOI logoGoogle Scholar
Dobbins, Gregory H., Irving M. Lane, and Dirk D. Steiner. 1988. “A Note on the Role of Laboratory Methodologies in Applied Behavioural Research: Don’t throw out the baby with the bath water.” Journal of Organizational Behavior 9 (3), 281–286. DOI logoGoogle Scholar
Downie, Jonathan. 2021. “Interpreting is Interpreting: Why we need to leave behind interpreting settings to discover comparative interpreting studies.” Translation and Interpreting Studies 16 (3), 325–346. DOI logoGoogle Scholar
Dunlosky, John, Sara Bottiroli, and Marissa Hartwig. 2009. “Sins Committed in the Name of Ecological Validity: A call for representative design in education science.” In Handbook of Metacognition in Education. Edited by D. J. Hacker, J. Dunlosky, and A. C. Graesser, 430–440. New York: Routledge.Google Scholar
Ehrensberger-Dow, Maureen, and Gary Massey. 2020. “Translation Workplace-based Research.” In The Routledge Handbook of Translation and Technology. Edited by M. O’Hagan, 354–369. New York: Routledge. DOI logoGoogle Scholar
Franzen, Michael D., and Karen L. Wilhelm. 1996. “Conceptual Foundations of Ecological Validity in Neuropsychological Assessment.” In Ecological Validity of Neuropsychological Testing. Edited by R. J. Sbordone, and C. J. Long, 91–112. New York: St. Lucie Press.Google Scholar
García, Adolfo M. 2019. The Neurocognition of Translation and Interpreting. John Benjamins. DOI logoGoogle Scholar
Gerber, Leah, Jim Hlavac, Irwyn Shepherd, Paul McIntosh, Alex Avella Archila, and Hyein Cho. 2021. “Stepping into the Future: Virtual reality training for community interpreters working in the area of family violence.” Journal of Specialised Translation 361, 252–275.Google Scholar
Gibson, James J. 1957. “Survival in a World of Probable Objects. Review of perception and the representative design of psychological experiment by E. Brunswik.” Contemporary Psychology 21, 33–35. DOI logoGoogle Scholar
1986. The Ecological Approach to Visual Perception. Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
Gile, Daniel. 2001. “Consecutive vs. Simultaneous: Which is more accurate?” Interpretation Studies 11, 8–20.Google Scholar
. 2009. “Interpreting Studies: A critical view from within.” MonTI 11, 135–155. DOI logoGoogle Scholar
. 2016. “Experimental Research.” In Researching Translation and Interpreting. Edited by C. V. Angelelli, and B. J. Baer, 220–228. New York: Routledge.Google Scholar
. 2018. “Research into Translation as a Specialism: An analysis and recommendations.” Journal of Specialised Translation 301, 23–39.Google Scholar
Hale, Sandra, Jane Goodman-Delahunty, Natalie Martschuk, and Stephen Doherty. 2021. “The Effects of Mode on Interpreting Performance in a Simulated Police Interview.” Translation and Interpreting Studies. DOI logoGoogle Scholar
Hammond, Kenneth R. 1998. “ Ecological Validity: Then and Now .” Available at [URL]
Hammond, Kenneth R., and Thomas R. Stewart. 2001. “Introduction.” In The Essential Brunswik: Beginnings, Explications, Applications. Edited by K. R. Hammond, and T. R. Stewart, 3–11. New York: Oxford University Press. DOI logoGoogle Scholar
Heilmann, Arndt, Tatiana Serbina, Daniel Couto Vale, and Stella Neumann. 2019. “Shorter than a Text, Longer than a Sentence: Source text length for ecologically valid translation experiments.” Target 31 (1), 98–124. DOI logoGoogle Scholar
Highhouse, Scott. 2009. “Designing Experiments that Generalize.” Organizational Research Methods 12 (3), 554–566. DOI logoGoogle Scholar
Hoc, Jean-Michel. 2001. “Towards Ecological Validity of Research in Cognitive Ergonomics.” Theoretical Issues in Ergonomic Science 2 (3), 278–288. DOI logoGoogle Scholar
Holleman, Gijs A., Ignace T. C. Hooge, Chantal Kemner, and Roy S. Hessels. 2020. “The “Real-World Approach” and its Problems: A critique of the term ecological validity.” Frontiers in Psychology 111, 721. DOI logoGoogle Scholar
House, Juliane. 2013. “Towards a New Linguistic-Cognitive Orientation in Translation Studies.” Target 25 (1), 46–60. DOI logoGoogle Scholar
Jääskeläinen, Riitta. 2012. “Translation Psychology.” In Handbook of Translation Studies. Edited by Y. Gambier, and L. van Doorslaer, 191–197. Amsterdam: John Benjamins. DOI logoGoogle Scholar
. 2017. “Verbal Reports.” In Handbook of Translation and Cognition. Edited by J. W. Schwieter, and A. Ferreira, 213–231. Hoboken, NJ: Wiley. DOI logoGoogle Scholar
Jakobsen, Arnt L. 2020. “Translation Technology Research with Eye Tracking.” In The Routledge Handbook of Translation and Technology. Edited by M. O’Hagan, 398–416. New York: Routledge. DOI logoGoogle Scholar
Kaiser, Henry F. 1960. “Directional Statistical Hypotheses.” Psychological Review 67 (3), 160–167. DOI logoGoogle Scholar
Kihlstrom, John F. 2021. “Ecological Validity and ‘Ecological Validity.’” Perspectives on Psychological Science 16 (2), 466–471. DOI logoGoogle Scholar
Korpal, Pawel. 2015. “Eye-tracking in Translation and Interpreting Studies: The growing popularity and methodological problems.” In Accessing Audiovisual Translation. Edited by L. Bogucki, and M. Deckert, 199–212. Frankfurt: Peter Lang.Google Scholar
Kothgassner, Oswald D., and Anna Felnhofer. 2020. “Does Virtual Reality Help to Cut the Gordian Knot between Ecological Validity and Experimental Control?” Annals of the International Communication Association 44 (3), 210–218. DOI logoGoogle Scholar
Kvavilashvili, Lia, and Judi A. Ellis. 2004. “Ecological Validity and the Real-life/Laboratory Controversy in Memory Research: A critical (and historical) review.” History and Philosophy of Psychology 6 (1), 59–80.Google Scholar
Ladouce, Simon, David I. Donaldson, Paul A. Dudchenko, and Magdalena Ietswaart. 2017. “Understanding Minds in Real-world Environments: Toward a mobile cognition approach.” Frontiers in Human Neuroscience 101, 694. DOI logoGoogle Scholar
Lewkowicz, David J. 2001. “The Concept of Ecological Validity: What are its limitations and is it bad to be invalid?” Infancy 2 (4), 437–450. DOI logoGoogle Scholar
Lucas, Jeffrey W. 2003. “Theory-testing, Generalization, and the Problem of External Validity.” Sociological Theory 21 (3), 236–253. DOI logoGoogle Scholar
Lund, Thorleif. 2021. “A Revision of the Campbellian Validity System.” Scandinavian Journal of Educational Research 65 (3), 523–535. DOI logoGoogle Scholar
Lynch, John G., Jr. 1999. “Theory and External Validity.” Journal of the Academy of Marketing Science 27 (3), 367–376. DOI logoGoogle Scholar
Malamatidou, Sofia. 2018. Corpus Triangulation: Combining Data and Methods in Corpus-based Translation Studies. London: Routledge.Google Scholar
Marín García, Álvaro. 2019. “The Opportunities of Epistemic Pluralism for Cognitive Translation Studies.” Translation, Cognition & Behavior 2 (2), 165–185. DOI logoGoogle Scholar
Matt, G. E., A. Brewer, and M. Sklar. 2010. “External validity.” In International Encyclopedia of Education, 3rd Ed. Edited by P. Peterson, E. Baker, and B. McGaw, 521–527. Elsevier Science. DOI logoGoogle Scholar
McGrath, Joseph E., and David Brinberg. 1983. “External Validity and the Research Process: A comment on the Calder/Lynch dialogue.” Journal of Consumer Research 10 (1), 115–124. DOI logoGoogle Scholar
Mellinger, Christopher D. 2015. “On the Applicability of Internet-mediated Research Methods to Investigate Translators’ Cognitive Behavior.” Translation & Interpreting 7 (1), 59–71.Google Scholar
2020. “Positionality in Public Service Interpreting Research.” FITISPos International Journal 71, 92–109. DOI logoGoogle Scholar
Mellinger, Christopher D., and Thomas A. Hanson. 2017. Quantitative Research Methods in Translation and Interpreting Studies. New York: Routledge.Google Scholar
. 2020. “Methodological Considerations for Survey Research: Validity, Reliability, and Quantitative Analysis.” Linguistica Antverpiensia, New Series – Themes in Translation Studies 191, 172–190. DOI logoGoogle Scholar
Mellinger, Christopher D., Nicoletta Spinolo, Maureen Ehrensberger-Dow, and Sharon O’Brien. Forthcoming. “Designing Naturalistic Tasks.” In Innovative Data Collection Methods in Cognitive Translation and Interpreting Studies. Edited by A. Rojo and R. Muñoz Martín. Amsterdam: John Benjamins.
Mook, Douglas G. 1983. “In Defense of External Invalidity.” American Psychologist 38 (4), 379–387. DOI logoGoogle Scholar
Muñoz Martín, Ricardo. 2010. “On Paradigms and Cognitive Translatology.” In Translation and Cognition. Edited by G. M. Shreve and E. Angelone, 169–188. Amsterdam: John Benjamins. DOI logoGoogle Scholar
. 2016. “Of Minds and Men–Computers and Translators.” Poznań Studies in Contemporary Linguistics 52 (2), 351–381. DOI logoGoogle Scholar
. 2017. “Looking Toward the Future of Cognitive Translation Studies.” In The Handbook of Translation and Cognition. Edited by J. W. Schwieter, and A. Ferreira, 554–572. Hoboken, NJ: Wiley. DOI logoGoogle Scholar
Neguţ, Alexandra, Silviu-Andrei Matu, Florin Alin Sava, and Daniel David. 2016. “Virtual Reality Measures in Neuropsychological Assessment: A meta-analytic review.” The Clinical Neuropsychologist 30 (2), 165–184. DOI logoGoogle Scholar
O’Brien, Sharon. 2013. “The Borrowers: researching the cognitive aspects of translation.” Target 25 (1), 5–17. DOI logoGoogle Scholar
Orne, Martin T. 1962. “On the Social Psychology of the Psychological Experiment: With particular reference to demand characteristics and their implications.” American Psychologist 17 (11), 776–783. DOI logoGoogle Scholar
Parsons, Thomas D. 2016. Clinical Neuropsychology and Technology: What’s New and How We Can Use It. Cham, Switzerland: Springer. DOI logoGoogle Scholar
Peecher, Mark E., and Ira Solomon. 2001. “Theory and Experimentation in Studies of Audit Judgments and Decisions: Avoiding common research traps.” International Journal of Auditing 5 (3), 193–203. DOI logoGoogle Scholar
Risku, Hanna, Regina Rogl, and Jelena Milošević. 2020. “Researching Workplaces.” In The Bloomsbury Companion to Language Industry Studies. Edited by E. Angelone, M. Ehrensberger-Dow, and G. Massey, 37–62. London: Bloomsbury. DOI logoGoogle Scholar
Rojo López, Ana M., and Pawel Korpal. 2020. “Through your Skin to your Heart and Brain: A Critical Evaluation of Physiological Methods in Cognitive Translation and Interpreting Studies.” Linguistica Antverpiensia, New Series: Themes in Translation Studies 191, 191–217. DOI logoGoogle Scholar
Saldanha, Gabriela, and Sharon O’Brien. 2013. Research Methodologies in Translation Studies. London: Routledge.Google Scholar
Schmuckler, Mark A. 2001. “What is Ecological Validity? A dimensional analysis.” Infancy, 2 (4), 419–436. DOI logoGoogle Scholar
Shadish, William R., Thomas D. Cook, and Donald T. Campbell. 2002. Experimental and Quasi-experimental Designs for Generalized Causal Inference. Cengage.Google Scholar
Shreve, Gregory M., and Erik Angelone. 2010. “Translation and Cognition: Recent developments.” In Translation and Cognition. Edited by G. M. Shreve, and E. Angelone, 1–13. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Spooner, Donna M., and Nancy A. Pachana. 2006. “Ecological Validity in Neuropsychological assessment: A case for greater consideration in research with neurologically intact populations.” Archives of Clinical Neuropsychology 211, 327–337. DOI logoGoogle Scholar
Steiner, Erich. 2021. “Translation, Equivalence, and Cognition.” In The Routledge Handbook of Translation and Cognition. Edited by F. Alves, and A. L. Jakobsen, 344–359. New York: Routledge.Google Scholar
Stoffregen, Thomas A. 1993. “‘Natural,’ ‘Real,’ and the Use of Non-physical Displays in Perception-Action Research.” ISEP Newsletter 61, 4–9.Google Scholar
Stoffregen, Thomas A., Benoit Bardy, L. J. Smart, and Randy J. Pagulayan. 2003. “On the Nature and Evaluation of Fidelity in Virtual Environments.” In Virtual and Adaptive Environments: Applications, Implications, and Human Performance Issues. Edited by L. J. Hettinger, and M. W. Haas, 111–128. Mahwah, NJ: Lawrence Erlbaum. DOI logoGoogle Scholar
Sun, Sanjun, Tian Li, and Xiaoyan Zhou. 2020. “Effects of Thinking Aloud on Cognitive Effort in Translation.” Linguistica Antverpiensia, New Series – Themes in Translation Studies 191, 132–151. DOI logoGoogle Scholar
Tiselius, Elisabet. 2021. “Informed Consent: An Overlooked Part of Ethical Research in Interpreting Studies.” InContext 1 (1), 83–100. DOI logoGoogle Scholar
Wachtel, Paul L. 1980. “Investigation and its Discontents: Some Constraints on Progress in Psychological Research.” American Psychologist 35 (5), 399–408. DOI logoGoogle Scholar
Weng, Yu, and Binghan Zheng. 2020. “A Multi-Methodological Approach to Studying Time-Pressure in Written Translation: Manipulation and measurement.” Linguistica Antverpiensia, New Series: Themes in Translation Studies 191, 218–236. DOI logoGoogle Scholar
Winer, Russell S. 1999. “Experimentation in the 21st Century: The importance of external validity.” Journal of the Academy of Marketing Science 27 (3), 349–358. DOI logoGoogle Scholar
Xiao, Kairong, and Ricardo Muñoz Martín. 2020. “Cognitive Translation Studies: Models and methods at the cutting edge.” Linguistica Antverpiensia, New Series: Themes in Translation Studies 191, 1–24. DOI logoGoogle Scholar
Ziegler, Klaus, and Sebastiano Gigliobianco. 2018. “Present? Remote? Remotely Present! New technological approaches to remote simultaneous conference interpreting.” In Interpreting and Technology. Edited by C. Fantinuoli, 119–139. Berlin: Language Science Press.Google Scholar
Cited by (13)

Cited by 13 other publications

Freiwald, Jonas, Zoë Miljanović, Arndt Heilmann & Stella Neumann
2024. Ecological validity in corpus-based and experimental translation research. Ampersand 12 ► pp. 100155 ff. DOI logo
Ma, Xingcheng & Dechao Li
2024. Effect of word order asymmetry on the cognitive load of English–Chinese sight translation. Translation and Interpreting Studies 19:1 ► pp. 105 ff. DOI logo
Zou, Deyan & Jiahao Guo
2024. Parallel translation process in consecutive interpreting: Differences between beginning and advanced interpreting students. Acta Psychologica 248 ► pp. 104358 ff. DOI logo
Barnett, Michael D., Danielle R. Hardesty, Rebekah A. Griffin & Thomas D. Parsons
2023. Performance on a virtual environment shopping task and adaptive functioning among older adults. Journal of Clinical and Experimental Neuropsychology 45:5 ► pp. 464 ff. DOI logo
Božović, Petar
2023. How should metaphors be rendered in audiovisual translation?. Translation and Interpreting Studies 18:3 ► pp. 471 ff. DOI logo
Cai, Rendong, Jiexuan Lin & Yanping Dong
2023. Psychological factors and interpreting competence in interpreting students: a developmental study. The Interpreter and Translator Trainer 17:2 ► pp. 246 ff. DOI logo
Chang, Vincent Chieh-Ying & I-Fei Chen
2023. Translation directionality and the Inhibitory Control Model: a machine learning approach to an eye-tracking study. Frontiers in Psychology 14 DOI logo
Gu, Chonglong & Binhua Wang
2023. From “Within” to “Beyond” in interpreting studies. Babel. Revue internationale de la traduction / International Journal of Translation / Revista Internacional de Traducción DOI logo
Jia, Huihuang
2023. Audiovisual translation studies. Translation and Interpreting Studies 18:1 ► pp. 159 ff. DOI logo
Michael, Carl
2023. Embodiment in Cognitive Translation Studies . In Human Translation and Natural Language Processing Towards a New Consensus? , DOI logo
Su, Wenchao
2023. Eye-voice span in sight interpreting: an eye-tracking investigation. Perspectives 31:5 ► pp. 969 ff. DOI logo
Zou, Longhui, Michael Carl & Devin Gilbert
2023. Integrating Trados-Qualitivity Data to the CRITT TPR-DB: Measuring Post-editing Process Data in an Ecologically Valid Setting. In Corpora and Translation Education [New Frontiers in Translation Studies, ], ► pp. 63 ff. DOI logo
Chmiel, Agnieszka & Nicoletta Spinolo
2022. Testing the impact of remote interpreting settings on interpreter experience and performance. Translation, Cognition & Behavior 5:2 ► pp. 250 ff. DOI logo

This list is based on CrossRef data as of 5 july 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.