Article published In:
Translation and Interpreting Studies: Online-First Articles

MT error detection and correction by Chinese language learners

ORCID logo | Dublin City University
ORCID logo | University College Dublin
ORCID logo | Dublin City University
This article analyzes texts that have been generated by machine translation (MT) and post-edited by native English-speaking trainee translators (English>Chinese) who are also Chinese language learners enrolled in a four-year undergraduate translation program. The project examines the work product of trainee translators to categorize 122 errors that are (un)noticed and (un)corrected by them. MT errors in the Accuracy category were best identified and corrected, followed by those in the Lexicon and Fluency categories. Trainee translators who were advanced language learners outperformed the intermediate group in MT error detection and correction, especially in the Lexicon category. This study sheds light upon the use of raw MT output as meaningful input for trainee translators who are in the process of learning Chinese. Its findings provide information regarding the type of exercises needed in language learning and translation training for students with different levels of language proficiency.
Keywords: post-editing, machine translation, trainee translators, language learning, Chinese-English
Article outline
  • Introduction
  • Pedagogical uses of post-editing
  • Research design
    • Typology adopted in the current study
    • Participants
    • Choice of source text (ST)
    • Research procedure
  • Data analysis
    • Quantitative analysis
    • The factor of language proficiency
    • Text level
    • Qualitative analysis
    • Fluency
    • Accuracy
    • Lexicon
  • Discussion
  • Limitations
  • Conclusion
  • Acknowledgements
  • Notes
  • References
Published online: 22 March 2024
DOI logo
https://doi.org/10.1075/tis.22092.zha
References (56)
References
Baayen, R. H., D. J. Davidson, and D. M. Bates. 2008. “Mixed-effects modeling with crossed random effects for subjects and items.” Journal of Memory and Language 59(4): 390–412. DOI logoGoogle Scholar
Bates, Douglas, Martin Mächler, Ben Bolker, and Steve Walker. 2015. “Fitting linear mixed-effects models using Lme4.” Journal of Statistical Software 671: 1–48. DOI logoGoogle Scholar
Bellassen, J., and Pengpeng Zhang. 1997. A Key to Chinese Speech and Writing. Beijing: Sinolingua.Google Scholar
Bowker, Lynne, and Jairo Buitrago Ciro. 2019. Machine Translation and Global Research: Towards Improved Machine Translation Literacy in the Scholarly Community. Bingley, UK: Emerald. DOI logoGoogle Scholar
Chang, Chia-chien, and Masaru Yamada. 2021. “Translation tasks for learning collocations: Effects of machine translation plus post-editing and sight translation.” English Teaching & Learning 45(1): 27–44. DOI logoGoogle Scholar
Chang, Pi-Chuan, Michel Galley, and Christopher D. Manning. 2008. “Optimizing Chinese word segmentation for machine translation performance.” In Proceedings of the Third Workshop on Statistical Machine Translation – StatMT ’08, 224–32. Columbus, OH: Association for Computational Linguistics. DOI logoGoogle Scholar
Chen, Hsin-hsi. 1994. “Contextual analysis of Chinese sentences with punctuation marks.” Literary and Linguistic Computing 9(4): 281–89. DOI logoGoogle Scholar
Clifford, Joan, Lisa Merschel, and Joan Munné. 2013. “Surveying the landscape – What is the role of machine translation in language learning.” @tic. Revista d’innovació Educativa 101: 108–21.Google Scholar
Cohen, Jacob. 1988. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. New York: Routledge. DOI logoGoogle Scholar
Correa, Maite. 2014. “Leaving the “peer” out of peer-editing: Online translators as a pedagogical tool in the Spanish as a second language classroom.” Latin American Journal of Content and Language Integrated Learning 7(1): 1–20. DOI logoGoogle Scholar
Cui, Songren, and Kuo-ming Sung. 2022. “Directional complement.” In A Reference Grammar for Teaching Chinese: Syntax and Discourse, ed. by Songren Cui and Kuo-ming Sung, 447–89. Singapore: Springer. DOI logoGoogle Scholar
Depraetere, Ilse. 2010. “What counts as useful advice in a university post-editing training context? Report on a case study.” In Proceedings of the 14th Annual Conference of the European Association for Machine Translation. St Raphaël, France. [URL]
Dorst, Aletta G., Susana Valdez, and Heather Bouman. 2022. “Machine translation in the multilingual classroom: How, when and why do humanities students at a Dutch University use machine translation?’ Translation and Translanguaging in Multilingual Contexts 8(1): 49–66. DOI logoGoogle Scholar
ELIS Research. 2023. “European Language Industry Survey 2023: Trends, Expectations and Concerns of the European Language Industry.” 10. ELIS. ELIA, EMT, EUATC, FIT Europe, GALA, LIND, Women in Localization. [URL]
Fonseca, Norma, and Fabio Alves. 2016. “Assessing complexity and difficulty levels of machine-translated texts.” Letras & Letras 32(1): 306–337. DOI logoGoogle Scholar
García, Ignacio, and María Isabel Pena. 2011. “Machine translation-assisted language learning: Writing for beginners.” Computer Assisted Language Learning 24(5): 471–87. DOI logoGoogle Scholar
Groves, Mike, and Klaus Mundt. 2021. “A ghostwriter in the machine? Attitudes of academic staff towards machine translation use in internationalised higher education.” Journal of English for Academic Purposes 501: article 100957. DOI logoGoogle Scholar
Guerberof-Arenas, Ana, and Joss Moorkens. 2019. “Machine translation and post-editing training as part of a master’s programme.” The Journal of Specialised Translation 311: 217–38.Google Scholar
Hu, Ke, and Patrick Cadwell. 2016. “A comparative study of post-editing guidelines.” Baltic Journal of Modern Computing 4 (2): 346–53.Google Scholar
ISO 18587. 2017. Translation Services – Post-Editing of Machine Translation Output – Requirements. Geneva: ISO.Google Scholar
Jensen, Kristian T. H. 2009. “Indicators of text complexity.” In Behind the Mind: Methods, Models and Results in Translation Process Research, ed. by Susanne Göpferich, Arnt Lykke Jakobsen, and Inger M. Mees, 61–80. Copenhagen: Samfundslitteratur.Google Scholar
Jia, Yanfang, Michael Carl, and Xiangling Wang. 2019a. “How does the post-editing of neural machine translation compare with from-scratch translation.” The Journal of Specialised Translation 311: 60–86.Google Scholar
. 2019b. “Post-editing neural machine translation versus phrase-based machine translation for English–Chinese.” Machine Translation 33(1–2): 9–29. DOI logoGoogle Scholar
Jia, Yanfang, and Binghan Zheng. 2022. “The interaction effect between source text complexity and machine translation quality on the task difficulty of NMT post-editing from English to Chinese: A multi-method study.” Across Languages and Cultures 23(1): 36–55. DOI logoGoogle Scholar
Klimova, Blanka, Marcel Pikhart, Alice Delorme Benites, Caroline Lehr, and Christina Sanchez-Stockhammer. 2023. “Neural machine translation in foreign language teaching and learning: A systematic review.” Education and Information Technologies 281: 663–682. DOI logoGoogle Scholar
Läubli, Samuel, Chantal Amrhein, Patrick Düggelin, Beatriz Gonzalez, Alena Zwahlen, and Martin Volk. 2019. “Post-editing productivity with neural machine translation: An empirical assessment of speed and quality in the banking and finance domain.” arXiv. [URL]
Li, Xiaoshi. 2010. “Sociolinguistic variation in the speech of learners of Chinese as a second language.” Language Learning 60(2): 366–408. DOI logoGoogle Scholar
Liu, Baolin, Zhongning Wang, and Zhixing Jin. 2010. “The effects of punctuations in Chinese sentence comprehension: An ERP study.” Journal of Neurolinguistics 23(1): 66–80. DOI logoGoogle Scholar
Lommel, Arle. 2018. “Metrics for translation quality assessment: A case for standardising error typologies.” In Translation Quality Assessment: From Principles to Practice, ed. by Joss Moorkens, Sheila Castilho, Federico Gaspari, and Stephen Doherty, 109–27. Cham: Springer International. DOI logoGoogle Scholar
Mellinger, Christopher D. 2017. “Translators and machine translation: Knowledge and skills gaps in translator pedagogy.” The Interpreter and Translator Trainer 11(4): 280–93. DOI logoGoogle Scholar
Mellinger, Christopher D., and Thomas A. Hanson. 2017. Quantitative Research Methods in Translation and Interpreting Studies. London: Routledge.Google Scholar
Miyata, Rei, and Atsushi Fujita. 2021. “Understanding pre-editing for black-box neural machine translation.” In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, 1539–50. EACL. DOI logoGoogle Scholar
Moorkens, Joss. 2018. “What to expect from neural machine translation: A practical in-class translation evaluation exercise.” The Interpreter and Translator Trainer 12(4): 375–87. DOI logoGoogle Scholar
Niño, Ana. 2008. “Evaluating the use of machine translation post-editing in the foreign language class.” Computer Assisted Language Learning 21(1): 29–49. DOI logoGoogle Scholar
. 2009. “Machine translation in foreign language learning: Language learners’ and tutors’ perceptions of its advantages and disadvantages.” ReCALL 21(2): 241–58. DOI logoGoogle Scholar
. 2020. “Exploring the use of online machine translation for independent language learning.” Research in Learning Technology 281: article 2402. DOI logoGoogle Scholar
O’Brien, Sharon. 2021. “Post-editing.” In Handbook of Translation Studies, vol. 51, ed. by Yves Gambier and Luc van Doorslaer, 178–84. Amsterdam: John Benjamins. DOI logoGoogle Scholar
. 2022. “How to deal with errors in machine translation: Post-editing.” In Machine Translation for Everyone: Empowering Users in the Age of Artificial Intelligence, ed. by Dorothy Kenny, 105–20. Berlin: Language Science Press.Google Scholar
Plitt, Mirko, and François Masselot. 2010. “A productivity test of statistical machine translation post-editing in a typical localisation context.” The Prague Bulletin of Mathematical Linguistics 931: 7–16. DOI logoGoogle Scholar
R Core Team. 2023. “R: The R Project for Statistical Computing.” 2023. [URL]
Rico Pérez, Celia. 2024. “Re-thinking machine translation post-editing guidelines.” The Journal of Specialised Translation 411: 26–47. DOI logoGoogle Scholar
Rico, Celia, and Diana González Pastor. 2022. “The role of machine translation in translation education: A thematic analysis of translator educators’ beliefs.” Translation & Interpreting 14(1): 177–197. DOI logoGoogle Scholar
Sánchez-Gijón, Pilar, and Olga Torres-Hostench. 2014. “MT post-editing into the mother tongue or into a foreign language? Spanish-to-English MT translation output post-edited by translation trainees.” In Proceedings of the 11th Conference of the Association for Machine Translation in the Americas, 5–19. Vancouver, Canada: AMTA.Google Scholar
Screen, Ben. 2017. “Machine translation and Welsh: Analysing free statistical machine translation for the professional.” The Journal of Specialised Translation 281: 317–44.Google Scholar
Somers, H. L. 2000. “Machine translation in the classroom.” In Computers and Translation: A Translator’s Guide, ed. by H. L. Sommers, 319–340. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Sun, Chaofen. 2006. Chinese: A Linguistic Introduction. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
TAUS, and CNGL. 2010. “Machine translation postediting guidelines.” [URL]
Toral, Antonio, and Andy Way. 2018. “What level of quality can neural machine translation attain on literary text?’ In Translation Quality Assessment: From Principles to Practice, ed. by Joss Moorkens, Sheila Castilho, Federico Gaspari, and Stephen Doherty, 263–87. Cham: Springer International. DOI logoGoogle Scholar
Vardaro, Jennifer, Moritz Schaeffer, and Silvia Hansen-Schirra. 2019. “Translation quality and error recognition in professional neural machine translation post-editing.” Informatics 6(3): article 41. DOI logoGoogle Scholar
Wang, Xiangling, Tingting Wang, Ricardo Muñoz Martín, and Yanfang Jia. 2021. “Investigating usability in postediting neural machine translation: Evidence from translation trainees’ self-perception and performance.” Across Languages and Cultures 22(1): 100–123. DOI logoGoogle Scholar
Wang, Yining, Long Zhou, Jiajun Zhang, and Chengqing Zong. 2017. “Word, subword or character? An empirical study of granularity in Chinese-English NMT.” In Machine Translation, ed. by Derek F. Wong and Deyi Xiong, 30–42. Singapore: Springer. DOI logoGoogle Scholar
Yamada, Masaru. 2019. “The impact of google neural machine translation on post-editing by student translators.” The Journal of Specialised Translation 311: 81–106.Google Scholar
Yang, Jizhou, ed. 2016. Chinese Course. 3rd ed. Beijing: Beijing Language and Culture University Press.Google Scholar
Zhang, Hong, and Olga Torres-Hostench. 2022. “Training in machine translation post-editing for foreign language students.” Language Learning & Technology 26(1): 1–17.Google Scholar
Zhang, Qi, and Zhouxiang Lu. 2014. “The writing of Chinese characters by CFL learners: Can writing on Facebook and using machine translation help?’ Language Learning in Higher Education 4(2): 441–467. DOI logoGoogle Scholar
Zhen, Yu-Yao, Ya-Ning Wu, Guang-Ming Yu, and Chun-Ping Zheng. 2021. “A review study of the application of machine translation in education from 2011 to 2020.” In Proceedings of the 29th International Conference on Computers in Education, ed. by Maria Mercedes T. Rodrigo, Sridhar Iyer, and Antonija Mitrovic, 17–24. Bangkok, Thailand: Asia-Pacific Society for Computers in Education (APSCE).Google Scholar