Cover not available

Article In: Terminology: Online-First Articles

Artificial intelligence as a supportive agent in the crafting of terminology development

The case of estonian national defence terminology

 | Estonian Military Academy
 | Estonian Language Institute
This content is being prepared for publication; it may be subject to changes.
Abstract
This paper examines the role of artificial intelligence (AI) as a supportive agent in professional terminology work, using the Estonian national defence terminology as a case study. The study addresses the growing need to manage large volumes of multilingual defence-related documentation while simultaneously preserving expert-driven, concept-oriented terminology development practices. We present and evaluate a retrieval-augmented, large language model (LLM)-based prototype designed to extract and structure source-grounded (English) terminological raw material — such as definition candidates, related terms, usage contexts, and exact references — rather than to generate dictionary-ready entries.
The prototype was evaluated through an expert-based assessment involving both domain specialists and experienced terminologists. The evaluation focused on the practical usefulness, contextual adequacy, and completeness of the extracted information, rather than on formal definitional correctness alone. The results indicate that the prototype performs reliably in retrieving definition candidates, contextual evidence, and source references, and can meaningfully support the early stages of terminology work. At the same time, persistent limitations were identified, including occasional hallucinations under weak source constraints, inconsistent handling of semantic relations, and challenges at the boundary between general and specialised language.
The findings underscore the continued centrality of human expertise in terminology development and support a human-at-the-core model — as distinct from a human-in-the-loop model, where humans primarily correct machine output — in which AI functions as an assistive, context-sensitive tool rather than an autonomous terminological agent. The study contributes contextualised empirical evidence to ongoing debates on AI-assisted terminology management and offers methodological insights into prompt design, expert evaluation, and the integration of AI tools into institutional terminology workflows, particularly in small-language and national defence contexts.
Article outline
  • 1.Introduction
  • 2.Background and objectives of the prototype development
  • 3.Use of AI in terminology work
  • 4.Prototype architecture and technical implementation
  • 5.Methodology
    • 5.1Prototype design considerations
    • 5.2Study design and evaluation approach
    • 5.3Prompt optimisation
    • 5.4Expert-based evaluation setup
    • 5.5Term selection for evaluation
    • 5.6Methodological considerations and constraints in prototype evaluation
  • 6.Results
    • 6.1General findings
    • 6.2Results by output type
      • Definitions
      • Related terms
      • Source referencing
      • Usage examples
      • “See also” field
  • 7.Conclusion
  • Acknowledgements
  • Note
  • References
References (40)
References
Bender, Emily M., Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 610–623. Google Scholar logo with link to Google Scholar
Bischof, Stefan, Erwin Filtz, Josiane Xavier Parreira, and Simon Steyskal. 2024. “LLM-Based Guided Generation of Ontology Term Definitions.” In The Semantic Web: ESWC 2024 Satellite Events, 133–137. Cham: Springer Nature.Google Scholar logo with link to Google Scholar
Bradley, Michael. 2014. “Is It Possible to Revitalize a Dying Language? An Examination of Attempts to Halt the Decline of Irish.” Open Journal of Modern Linguistics 4: 537–543. Google Scholar logo with link to Google Scholar
Breton, Julien, Mokhtar Mokhtar Billami, Max Chevalier, Ha Thanh Nguyen, Ken Satoh, Cassia Trojahn, and May Myo Zin. 2025. “Leveraging LLMs for Legal Terms Extraction with Limited Annotated Data.” Artificial Intelligence and Law. Google Scholar logo with link to Google Scholar
Chun, Yongchan, Minhyuk Kim, Dongjun Kim, Chanjun Park, and Heui-Seok Lim. 2025. “Enhancing Automatic Term Extraction with Large Language Models via Syntactic Retrieval.” In Findings of the Association for Computational Linguistics: ACL 2025, 9916–9926. Google Scholar logo with link to Google Scholar
Crystal, David. 2010. “Language Planning.” In The Cambridge Encyclopedia of Language, 3rd ed., 382–387. Cambridge University Press.Google Scholar logo with link to Google Scholar
Dubuc, Robert. 2002. Manuel pratique de terminologie. 4th ed. Brossard: Linguatech. Google Scholar logo with link to Google Scholar
Eiche, Sandra, Reet Hendrikson, Eleri Aedmaa, Esta Prangel, and Mari-Liis Tikerperi. 2025. “Tehisintellekti rakendamine erialakeele arendamisel riigikaitseterminoloogia näitel [Applying artificial intelligence in specialised language development: The example of Estonian national defence terminology].” Eesti Rakenduslingvistika Ühingu aastaraamat = Estonian Papers in Applied Linguistics 21: 29–45. Google Scholar logo with link to Google Scholar
Fishman, Joshua A. 1991. Reversing Language Shift: Theory and Practice of Assistance to Threatened Languages. Clevedon: Multilingual Matters. Google Scholar logo with link to Google Scholar
Gross, Daniel P., and Bhaven N. Sampat. 2020. “Inventing the Endless Frontier: The Effects of the World War II Research Effort on Post-War Innovation.” Harvard Business School Working Paper, no. 20–126. Google Scholar logo with link to Google Scholar
Haberland, Hartmut. 2005. “Domains and Domain Loss.” In The Consequences of Mobility, ed. by Bent Preisler, Anne Fabricius, Hartmut Haberland, Susanne Kjærbeck, and Karen Risager, 227–237. Roskilde: Roskilde University.Google Scholar logo with link to Google Scholar
Heinisch, Barbara. 2025a. “Large Language Models for Terminology Work: A Question of the Right Prompt?” Journal for Language Technology and Computational Linguistics 38 (2): 13–30. Google Scholar logo with link to Google Scholar
. 2025b. “Next-Gen Terminology: Transforming Terminology Work with Large Language Models.” Across Languages and Cultures 26 (S): 64–80. Google Scholar logo with link to Google Scholar
Hendrikson, Reet. 2012. “Keelesõjas sõjakeel? Ohvitseride oskuskeelevaadetest [Military language at war? The views of officers about military terminology].” Kaitseväe Ühendatud Õppeasutuste toimetised (Proceedings of the Estonian National Defence College) 16: 15–39.Google Scholar logo with link to Google Scholar
. 2014. “Tõhususe tahud sõjandusterminoloogia näitel: ülediferentsimine või erinev tunnetusväärtus? [Aspects of specialised language efficiency: Over-differentiation or cognitive motivation?]” Eesti Rakenduslingvistika Ühingu aastaraamat = Estonian Papers in Applied Linguistics 10: 107–123. Google Scholar logo with link to Google Scholar
. 2018. Kas sõjasõna sünnib sõtta? Erialakeele tõhusus sõjandusterminoloogia näitel [The efficiency of specialised language based on military terminology]. Doctoral dissertation, University of Tartu. [URL]
. 2020. “Ühtse mõistmise ja terminikasutuse võimalikkusest laiapindses riigikaitses [Term work is teamwork: The shared concepts and term system in comprehensive national defence?]” Sõjateadlane (Estonian Journal of Military Studies) 15: 31–86. [URL]
Humbley, John, and Kara Warburton. 2025. “Terminology throughout History: An Introduction.” In Terminology throughout History: A Discipline in the Making, ed. by Kara Warburton and John Humbley, 1–24. Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
International Organization for Standardization. 2022. Terminology Work — Principles and Methods (ISO Standard No. 704:2022). Geneva: International Organization for Standardization.Google Scholar logo with link to Google Scholar
Kikkas, Toivo, and Reet Hendrikson. 2021. “Pilguheit eestikeelse sõjandussõnavara kujunemisele aastatel 1885–1913 [An overview of the development of Estonian military vocabulary in 1885–1913].” Sõjateadlane (Estonian Journal of Military Studies) 18: 292–319.Google Scholar logo with link to Google Scholar
Lewis, Patrick, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.” Advances in Neural Information Processing Systems 33: 9459–9474.Google Scholar logo with link to Google Scholar
Lincoln, Yvonna S., and Egon G. Guba. 2000. “Paradigmatic Controversies, Contradictions, and Emerging Confluences.” In Handbook of Qualitative Research, ed. by Norman K. Denzin and Yvonna S. Lincoln, 163–188. Thousand Oaks, CA: Sage.Google Scholar logo with link to Google Scholar
Massion, François. 2024. “Terminology in the Age of AI: The Transformation of Terminology Theory and Practice.” Journal of Translation Studies 4 (1): 67–94. Google Scholar logo with link to Google Scholar
Militerm. n.d. National Defence Terminology Database Estonian–English–Estonian. [URL]
Mills, M. Anthony, and Mark P. Mills. 2014. “The Invention of the War Machine: Science, Technology, and the First World War.” The New Atlantis 42: 3–23.Google Scholar logo with link to Google Scholar
NATO. 2018. NATO Terminology Manual AAP-77. Ed. 1, Ver. 1. North Atlantic Treaty Organization. [URL]. Accessed 31 July 2026.
. 2024. Summary of NATO’s Revised Artificial Intelligence (AI) Strategy. North Atlantic Treaty Organization. [URL]. Accessed 31 July 2026.
OpenAI. 2024. Hello GPT-4o. [URL]. Accessed 31 July 2026.
San Martín, Antonio. 2024. “What Generative Artificial Intelligence Means for Terminological Definitions.” In Proceedings of the 3rd International Conference on Multilingual Digital Terminology Today (MDTT 2024). CEUR Workshop Proceedings 3703. [URL]
. 2025. “Toward Human-Centered AI-Assisted Terminology Work.” arXiv. Google Scholar logo with link to Google Scholar
Schreiter, Dimitri. 2025. “Prompt Engineering: How Prompt Vocabulary Affects Domain Knowledge.” arXiv. Google Scholar logo with link to Google Scholar
Strickland, Jonathan. 2023. “Do Wars Drive Technological Advancement?” HowStuffWorks. [URL]
Wang, Liang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024. “Multilingual E5 Text Embeddings: A Technical Report.” arXiv. Google Scholar logo with link to Google Scholar
Warburton, Kara. 2021. The Corporate Terminologist. Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
Wright, Quincy. 1941. “Inventions and War.” The Scientific Monthly 53 (6): 526–541.Google Scholar logo with link to Google Scholar
Xu, Yan, Tao Wang, Yang Yuan, Ziyue Huang, Xi Chen, Bo Zhang, Xiaorong Zhang, and Zehua Wang. 2025. “LLM-Enhanced Framework for Building Domain-Specific Lexicon for Urban Power Grid Design.” Applied Sciences 15 (8): 4134. Google Scholar logo with link to Google Scholar
Mobile Menu Logo with link to supplementary files background Layer 1 jbe-platform.quadraat-sans-bold jbe-platform.quadraat-sans-bold-dark jbe-platform.quadraat-sans-bold-light jbe-platform.quadraat-sans-bold prag Twitter_Logo_Blue