Cover not available

Article In: Digital Translation: Online-First Articles

Taming the stochastic parrot

A translation studies approach to variability in LLM-based machine translation

ORCiD logo with link | Universidad Complutense de Madrid
This content is being prepared for publication; it may be subject to changes.
Abstract
This article proposes a Translation Studies–oriented approach to understanding and managing variability in machine translation outputs generated by large language models (LLMs). Drawing on the metaphor of the “stochastic parrot,” the study introduces the concept of temperature as a means for controlling stochasticity in LLM-based translation. Through a practical and replicable experiment conducted in Google Colab, technical texts are translated from English to Spanish under varying temperature conditions. Although the dataset is intentionally limited, the study’s primary contribution lies in establishing a replicable methodological pathway rather than in producing generalizable quantitative results. By combining computational experimentation with reflection on concepts relevant to translation theory, the study can inform both future research and practical approaches.
Article outline
  • 1.Introduction
  • 2.Exploring stochasticity and variability in LLMs
  • 3.Controlling output variability in language models: An experimental setup
    • 3.1The corpus
    • 3.2The evaluation criteria
  • 4.Exploring variability
    • 4.1Segment 1
    • 4.2Segment 2
    • 4.3Segment 3
    • 4.4Segment 4
    • 4.5Segment 5
    • 4.6Segment 6
    • 4.7Segment 7
    • 4.8Segment 8
  • 5.Discussion
  • 6.Limitations of the study and future work
  • 7.Conclusion
  • Notes
  • Author queries
  • References
References (33)
References
Abad, Paloma, Susana Pérez-Benavente, Sara Pérez-Luz, Julius N. Fobil, Batisseur Kitenge Luyenga, André Kazadi Mukendi, Antonio Puyet, Amalia Diez, Lucio Luzzatto, Isabel G. Azcárate, and José M. Bautista. 2025. “Coinheritance of Polymorphic Alleles of PIEZO1, G6PD and HBB Enhances Protection against Malaria.” One Health 201: 101051. Google Scholar logo with link to Google Scholar
Asscher, Omri, and Ella Glikson. 2021. “Human Evaluations of Machine Translation in an Ethically Charged Situation.” New Media & Society 25 (5): 1087–1107. Google Scholar logo with link to Google Scholar
Ayvazyan, Nune. 2025. “Human-Centered Pedagogies in the Age of Generative AI: Examining Student Perceptions of Augmentation, Progress, and Agency”. INContext: Studies in Translation and Interculturalism 5 (1): 167–193. Google Scholar logo with link to Google Scholar
Bender, Emily M., Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” FAccT 2021 — Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 610–23. Google Scholar logo with link to Google Scholar
Bowker, Lynne, and Jairo Buitrago Ciro. 2019. Machine Translation and Global Research: Towards Improved Machine Translation Literacy in the Scholarly Community. Bingley: Emerald Publishing. Google Scholar logo with link to Google Scholar
Briva-Iglesias, Vicent. 2025. “Are AI Agents the New Machine Translation Frontier? Challenges and Opportunities of Single-and Multi-Agent Systems for Multilingual Digital Communication.” Proceedings of Machine Translation Summit XX (Vol. 1), 365–377. Geneva: EAMT. [URL]
Castilho, Sheila, Stephen Doherty, Federico Gaspari, and Joss Moorkens. 2018. “Approaches to Human and Machine Translation Quality Assessment,” In Translation Quality Assessment, edited by Joss Moorkens, Sheila Castilho, Federico Gaspari, and Stephen Doherty, 9–38. Cham: Springer. Google Scholar logo with link to Google Scholar
Cheng, Yuheng, Ceyao Zhang, Zhengwen Zhang, Xiangrui Meng, Sirui Hong, Wenhao Li, Zihao Wang, Zekai Wang, Feng Yin, Junhua Zhao, Xiuqiang He. 2024. “Exploring Large Language Model Based Intelligent Agents: Definitions, Methods, and Prospects.” arXiv. [URL]
Du, Shuxiang, Ana Guerberof Arenas, Antonio Toral, Kyo Gerrits, Josep Marco Borillo. 2025. “Optimising ChatGPT for creativity in literary translation: A case study from English into Dutch, Chinese, Catalan and Spanish.” Proceedings of Machine Translation Summit XX (Vol. 1), edited by Pierrette Bouillon, Johanna Gerlach, Sabrina Girletti, et al., 578–591. Geneva: EAMT. [URL]
Gao, Yuan, Ruili Wang, and Feng Hou. 2023. “How to Design Translation Prompts for ChatGPT: An Empirical Study.” In Proceedings of the 6th ACM International Conference on Multimedia in Asia Workshops (MMAsia ’24 Workshops). Association for Computing Machinery, New York, NY, USA, Article 9, 1–7. Google Scholar logo with link to Google Scholar
Google. n.d. “Colaboratory. Frequently Asked Questions.” [URL]
Holtzman, Ari, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. “The Curious Case of Neural Text Degeneration.” 8th International Conference on Learning Representations, ICLR 2020, April. Google Scholar logo with link to Google Scholar
House, Juliane. 2015. Translation Quality Assessment: Past and Present. London: Routledge. Google Scholar logo with link to Google Scholar
Guerreiro, Nuno M., Elena Voita, and André Martins. 2023. “Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translation.” In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, 1059–1075, Dubrovnik: Association for Computational Linguistics. [URL].
Jiménez-Crespo, Miguel A. 2025. “Human-Centeredness in Translation.” INContext: Studies in Translation and Interculturalism 5 (1): 7–17. Google Scholar logo with link to Google Scholar
Krüger, Ralph. 2022. “Using Jupyter Notebooks as Didactic Instruments in Translation Technology Teaching.” The Interpreter and Translator Trainer 16 (4): 503–23. Google Scholar logo with link to Google Scholar
Lai, Haohong. 2025. “Using Critical Posthumanist Methods to Navigate Human Translators’ Roles in the AI Era.” INContext: Studies in Translation and Interculturalism 5 (1): 65–86. Google Scholar logo with link to Google Scholar
Liu, Ting, Chi-kiu Lo, Elizabeth Marshman, and Rebecca Knowles. 2024. “Evaluation Briefs: Drawing on Translation Studies for Human Evaluation of MT.” Proceedings of the 16th Conference of the Association for Machine Translation in the Americas (Vol. 1), 190–208. Chicago: AMTA. [URL]
Monzó Nebot, Esther, and Vicenta Tasa-Fuster (eds). 2024. The Social Impact of Automating Translation: An Ethics of Care Perspective on Machine Translation. London: Routledge. Google Scholar logo with link to Google Scholar
Moorkens, Joss. 2024. “‘I Am Not a Number’: On Quantification and Algorithmic Norms in Translation.” Perspectives 32 (3): 477–492. Google Scholar logo with link to Google Scholar
Moorkens, Joss, Andy Way and Séamus Lankford. 2025. Automating Translation. London: Routledge. Google Scholar logo with link to Google Scholar
Ng, Andrew. 2025. translation-agent. GitHub. Accessed July 16, 2025. [URL]
Olohan, Maeve. 2016. Scientific and Technical Translation. London: Routledge. Google Scholar logo with link to Google Scholar
Peeperkorn, Max, Tom Kouwenhoven, Dan Brown, and Anna Jordanous. 2024. “Is Temperature the Creativity Parameter of Large Language Models?” Proceedings of the 15th International Conference on Computational Creativity (ICCC’24) Google Scholar logo with link to Google Scholar
Peng, Keqin, Liang Ding, Qihuang Zhong, Li Shen, Xuebo Liu, Min Zhang, Yuanxin Ouyang, and Dacheng Tao. 2023. “Towards Making the Most of ChatGPT for Machine Translation,” In Findings of the Association for Computational Linguistics: EMNLP 2023, 5622–5633. Singapore: Association for Computational Linguistics. [URL].
Reiss, Katharina, and Hans J. Vermeer. 2013. Towards a General Theory of Translational Action: Skopos Theory Explained (C. Nord, Trans.; 1st ed.). London: Routledge. Google Scholar logo with link to Google Scholar
Savoldi, Beatrice, Marco Gaido, Luisa Bentivogli, Matteo Negri, and Marco Turchi. 2021. “Gender Bias in Machine Translation.” Transactions of the Association for Computational Linguistics 91: 845–874. [URL].
Vilar, David, Markus Freitag, Colin Cherry, Jiaming Luo, Viresh Ratnakar, and George Foster. 2023. “Prompting PaLM for Translation: Assessing Strategies and Performance.” In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Vol. 1), 15406–15427. Toronto: Association for Computational Linguistics. [URL].
Way, Andy. 2018. “Quality Expectations of Machine Translation. In Translation Quality Assessment, edited by Joss Moorkens, Sheila Castilho, Federico Gaspari, and Stephen Doherty, 159–178. Cham: Springer. Google Scholar logo with link to Google Scholar
Wu, Minghao, Jiahao Xu, Yulin Yuan, Gholamreza Haffari, Longyue Wan, Weihua Luo, and Kaifu Zhang. 2025. “(Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Texts.” Transactions of the Association for Computational Linguistics 131: 901–922. Google Scholar logo with link to Google Scholar
Zhao, Yunpu, Rui Zhang, Wenyi Li, and Ling Li. 2025. “Assessing and Understanding Creativity in Large Language Models.” Machine Intelligence Research 22 (3): 417–436. Google Scholar logo with link to Google Scholar
Zhang, Jia, Xiaoyu Zhao and Stephen Doherty. 2025. “Prompt Engineering in Translation: How Do Student Translators Leverage GenAI Tools for Translation Tasks.” Proceedings of Machine Translation Summit XX (Vol. 1), edited by Pierrette Bouillon, Johanna Gerlach, Sabrina Girletti, et al., 420–431. Geneva: EAMT. [URL]
Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue