1.Introduction

In translation quality assurance models, terminology tends to be treated as a given associated with domain-specific conventions. For instance, in the Multidimensional Quality Metrics (MQM) taxonomy of issues, “terminology issues relate to the use of domain- or organization-specific terminology (i.e., the use of words to relate to specific concepts not considered part of general language)” (Lommel, Burchardt, and Uszkoreit 2015Lommel, Arle, Aljoscham Burchardt, and Hans Uszkoreit eds. 2015 Multidimensional Quality Metrics (MQM) Definition. Accessed 10 August 2021. http://​www​.qt21​.eu​/mqm​-definition​/definition​-2015​-12​-30​.html). It presents “adherence to specified terminology” as “an issue of central concern in both translation and content authoring” (ibid.). Likewise, ISO 17100 refers to “compliance with specific domain and client terminology and/or any other reference material provided and ensuring terminological consistency during translation”. This double requirement is followed by that of “semantic accuracy of the target language content” (ISO 2015ISO 17100 2015 Translation services – Requirements for translation services. Geneva: International Organization for Standardization., 10).

However, it is unrealistic to believe that terminological issues can be limited to conforming to specifications or reference material. In the case of legal terminology, this is compounded by the high variability of communicative parameters and the multiple incongruities that exist between legal systems (e.g., Sandrini 1996Sandrini, Peter 1996 “Comparative Analysis of Legal Terms: Equivalence Revisited.” In TKE ‘96– Terminology and Knowledge Engineering. Proceedings Fourth International Congress on Terminology and Knowledge Engineering, edited by Christian Galinski and Klaus D. Schmitz, 1–10. Frankfurt: Ergon Verlag.; Terral 2004Terral, Florence 2004 “L’empreinte culturelle des termes juridiques.” Meta: Translators’ Journal 49(4):876–890. ). Rather than searching for “one-fits-all” solutions or “equivalents”, adequacy requirements may vary significantly for the translation of a same term for different legal systems and purposes (Prieto Ramos 2014Prieto Ramos, Fernando 2014 “Parameters for Problem-Solving in Legal Translation: Implications for Legal Lexicography and Institutional Terminology Management.” In The Ashgate Handbook of Legal Translation, edited by Anne Wagner, King-Kui Sin and Le Cheng, 121–134. Farnham: Ashgate.). In turn, this explains the limitations of traditional legal lexicographical resources, especially bilingual ones, for legal translation (see, e.g., Biel 2008Biel, Łucja 2008 “Legal Terminology in Translation Practice: Dictionaries, Googling or Discussion Forums?” SKASE Journal of Translation and Interpretation 3(1):22–38.; De Groot and van Laer 2008De Groot, René and Conrad J. P. van Laer 2008 The Quality of Legal Dictionaries: An Assessment. Maastricht Faculty of Law Working Paper no. 6. Maastricht: University of Maastricht. ).

The translation of legal terminology found in multilingual texts of international organizations is not immune to these issues. The institution- or domain-specific (most often authoritative) terminology established by and for their own multilingual systems co-exists with legal terms that refer to singular bodies or notions of national legal systems (to be identified as such in the target texts), and concepts that originated in specific legal traditions and have been adopted for broader international use. It is precisely culture-bound terms that tend to be more problematic and time-consuming for institutional translators. As hypothesized in the LETRINT project on institutional legal translation,11.This study is part of the same project, “Legal Translation in International Institutional Settings: Scope, Strategies and Quality Markers”, which is led by the first author with the support of a Consolidator Grant (https://​transius​.unige​.ch​/letrint). higher levels of legal singularity from the perspective of the target audience generally entail higher degrees of translation difficulty that can be associated to quality gaps both in institutional translation precedents and terminological resources (Prieto Ramos and Guzmán 2018Prieto Ramos, Fernando and Diego Guzmán 2018 “Legal Terminology Consistency and Adequacy as Quality Indicators in Institutional Translation: A Mixed-Method Comparative Study.” In Institutional Translation for International Governance: Enhancing Quality in Multilingual Legal Communication, edited by Fernando Prieto Ramos, 81–101. London: Bloomsbury. ; Prieto Ramos 2020Prieto Ramos, Fernando 2020 “Ensuring Consistency and Accuracy of Legal Terms in Institutional Translation: The Role of Terminological Resources in International Organizations.” In Institutional Translation and Interpreting Assessing Practices and Managing for Quality, edited by Fernando Prieto Ramos, 128–149. New York: Routledge. ). While these terms may not be among the most frequent in institutional documents, they are key to ensuring reliable communication of legal specificities, particularly in the context of compliance monitoring or adjudication procedures.

Building on the same premises and preliminary results, this paper presents a corpus-based methodology to identify patterns of legal terminological decision-making and measure their quality according to the needs of institutional translation (Section 2). It broadens the scope of previous studies by applying this methodology to translations of a diversity of illustrative terms into two target languages in several institutional settings. We will briefly delve into the nature of the selected terms and the adequacy requirements and associated difficulty in translating them into French and Spanish (Section 3), before presenting the comparative analysis of adequacy levels per term, setting and target language. We will examine translation adequacy levels in the light of the legal singularity of each term (Section 4), as well as the potential connection with consistency patterns and the impact of intratextual inconsistencies on quality (Section 5). The ultimate aim is to provide tools and data that can inform measures for translation quality improvements, for example, by refining translation guidelines or other resources for internal standardization, where appropriate, so as to avoid reproducing inaccurate or inconsistent renderings.

2.Approach and corpus

In the approach to quality applied in this study (and in the LETRINT project more broadly), the adequacy of terminology translation decisions is measured with regard to the overall translation strategy (based on the brief, legal contextualization and communicative situation of the translation) and the microtextual priorities for each segment translated. These considerations guide the analysis of acceptability of possible reformulations during the translation process, and ultimately the quality or adequacy assessment of the translation product. In this functionalist approach (Prieto Ramos 2014Prieto Ramos, Fernando 2014 “Parameters for Problem-Solving in Legal Translation: Implications for Legal Lexicography and Institutional Terminology Management.” In The Ashgate Handbook of Legal Translation, edited by Anne Wagner, King-Kui Sin and Le Cheng, 121–134. Farnham: Ashgate., 2015Prieto Ramos, Fernando 2015 “Quality Assurance in Legal Translation: Evaluating Process, Competence and Product in the Pursuit of Adequacy.” International Journal for the Semiotics of Law – Revue internationale de Sémiotique juridique 28(1):11–30. ), the concept of “adequacy” encompasses all the elements of the source and the target communicative situations relevant to translation decision-making. It differs from Toury’s (1995)Toury, Gideon 1995 Descriptive Translation Studies – and Beyond. Amsterdam: John Benjamins. understanding of “adequacy” as source norm adherence versus “acceptability” as target norm orientation. In our approach, “acceptability analysis” is understood as the process of examining potential reformulations and their communicative fitness at the microtextual level in order to determine the most adequate solution in each case. This concept is differentiated from the ex post “adequacy assessment” of terminological decisions, and from the verification of overall adequacy of the whole translation product as the central purpose of revision (Prieto Ramos 2015Prieto Ramos, Fernando 2015 “Quality Assurance in Legal Translation: Evaluating Process, Competence and Product in the Pursuit of Adequacy.” International Journal for the Semiotics of Law – Revue internationale de Sémiotique juridique 28(1):11–30. , 20).

In the case of institutional translation, semantic accuracy and consistency, as also established in ISO 17100, are considered key adequacy requirements embedded in the standard brief, particularly in the translation of legal terminology, due to the need for semantic univocity and legal certainty of multilingual law (e.g., Stefaniak 2017Stefaniak, Karolina 2017 “Terminology Work in the European Commission: Ensuring High-Quality Translation in a Multilingual Environment.” In Quality Aspects in Institutional Translation, edited by Tomáš Svoboda, Łucja Biel and Krzysztof Łoboda, 109–121. Language Processing 8. Berlin: Language Science Press. , Šarčević 2018Šarčević, Susan 2018 “Challenges to Legal Translators in Institutional Settings.” In Institutional Translation for International Governance: Enhancing Quality in Multilingual Legal Communication, edited by Fernando Prieto Ramos, 9–24. Bloomsbury Advances in Translation. London: Bloomsbury. ). These adequacy requirements are shared by the three settings selected for examination on the basis of the scope of their legal functions (Prieto Ramos 2019Prieto Ramos, Fernando 2019 “Implications of text categorisation for corpus-based legal translation research: The case of international institutional settings.” In Research Methods in Legal Translation and Interpreting: Crossing Methodological Boundaries, edited by Łucja Biel, Jan Engberg, Rosario Martín Ruano and Vilelmini Sosoni, 29–47. London: Routledge. ): the main European Union (EU) supranational institutions (more specifically, the European Commission, the European Parliament, the Council of the EU and the Court of Justice of the EU (CJEU)), the United Nations (UN), and the World Trade Organization (WTO).

In order to test the impact of legal singularity on translation adequacy, and explore the commonalities and differences in translation patterns, five legal procedural terms were selected, including three court denominations: court of appeal (CoA), high court (HC), magistrates’ court (MC), prima facie evidence (PFE) and due process (DP). The selection was based on quantitative and qualitative criteria. Our preliminary corpus analysis revealed that the five terms were found in all the settings and periods under examination. DP and PFE are examples of procedural concepts originally borrowed from common law but with a relatively low level of legal singularity due to the existence of similar concepts in civil law jurisdictions. The three court names (CoA, HC and MC) designate judicial bodies in common law countries, and illustrate three different singularity levels from the perspective of the reference target systems in French- and Spanish-speaking jurisdictions (see Section 3 for further details).

The terms were extracted from the LETRINT 0 corpus, a trilingual dataset of over 1.17 billion tokens including all the texts translated from English into French and Spanish and published in the repositories of the EU, the UN and the WTO in 2005, 2010 and 2015 (see further details in Prieto Ramos, Cerutti, and Guzmán 2019Prieto Ramos, Fernando, Giorgina Cerutti, and Diego Guzmán 2019 “Building representative multi-genre corpora for legal and institutional translation research: The LETRINT approach to text categorization and stratified sampling.” Translation Spaces 8(1):93–116. ). Each term was queried with Agent Ransack in the English language original texts of the LETRINT 0 corpus. 8,135 occurrences were identified in 2,592 documents, which underwent trilingual alignment with a custom-made version of LF Aligner. Subsequently, the trilingual segments were cleansed and verified. As a result, 2,124 occurrences were discarded for technical or semantic reasons (see Table 1), including: duplications22.Duplications generally resulted from the overlap between subsequent versions of the same document, particularly in the case of EU legal acts and WTO trade policy review procedures, or from the publication of yearly compilations, especially by UN bodies. (36.96%); untranslated sentences (24.95%); use of English translations of court names with reference to judicial bodies of third countries rather than common law systems (17.56%); and occurrences that departed from the procedural concepts examined (the main semantic reason for exclusion of DP occurrences) (8.05%). The French and Spanish translations of the remaining 6,011 occurrences were then compiled in their contexts for our assessment of adequacy.

The most recurrent term is DP, except for the EU, where the most frequently used court name in the three institutional settings, HC, registered the highest density of the five terms. Unsurprisingly, given the nature of the UN’s human rights monitoring procedures and the reporting obligations of its broad membership, the UN sub-corpus includes the largest number of occurrences of all court names and DP. However, PFE, often employed in trade investigations, was most frequently found in the EU sub-corpus, followed by the WTO’s. MC is the least recurrent of all the terms, except for the UN sub-corpus, where it is more common than PFE. In line with the nature of textual production in each setting (see Prieto Ramos and Guzmán 2021Prieto Ramos, Fernando and Diego Guzmán 2021 “Examining institutional translation through a legal lens: A comparative analysis of multilingual text production at international organizations.” Target 33(2):254–281. ), the highest frequencies of the term set as a whole were found in law-making and policy formulation texts in the EU (48% of occurrences, closely followed by CJEU documents with a frequency of 44%), monitoring procedures in the UN (72%, compared to 18% of soft law and policy documents) and dispute settlement texts at the WTO (50% of occurrences, followed by monitoring-related texts at 38%).

While adequacy considerations must be nuanced for each specific text and context, the analysis of occurrences revealed, as expected, comparable communicative conditions and priorities at the microtextual level for the selected terms and settings. In the case of court names, it is essential to identify the source system specificity of each judicial entity through formulations that may facilitate understanding of the body’s defining competence among target readers of a diversity of national jurisdictions, i.e., avoiding national target system singularities in the translation. As for DP and PFE, institutional terminological preferences in specific procedural contexts may play a determining role in conveying the intended meaning in the international legal order. Both in French and Spanish, the number of target language reference jurisdictions increases from very few (France, Belgium and Luxembourg) or just one (Spain) in the EU to many more in the two global intergovernmental organizations under examination, the UN and the WTO. For all the renderings, as mentioned above, semantic accuracy and univocity, as well as terminological consistency, are required to achieve maximum adequacy.

In order to systematically measure and compare translation patterns and adequacy levels, each rendering was assigned a first numerical value according to the following scale:

This initial adequacy value thus focuses on the microtextual accuracy and communicative fitness of the translation. It presupposes that more than one translation may be acceptable for each source term and setting. However, when a source term was translated differently in the same text and there were no contextual elements to justify this decision for each occurrence, the adequacy level of the least recurrent renderings was reduced by one point,33.In the case of target terms used with the same frequency, the reduction was applied to all their occurrences. No penalty was applied to 0 level renderings, as they were already considered inadequate. as semantic univocity and therefore overall adequacy were compromised by inconsistencies at a macrotextual level.

To ensure the systematic application of the approach and reinforce the reliability of its results, all values were verified by at least two translators per target language and by the project leader, and borderline cases were discussed to reach consensus.

3.The source terms and their translation

Before presenting the results of the corpus analysis, it is worth examining the nature of the selected terms in more detail. We will focus on their legal singularity and the difficulty associated with their translation into Spanish (ES) and French (FR). This will be essential to explore the potential connections with adequacy patterns.

Table 2 summarizes the legal singularity and translation difficulty levels per term as seen from the perspective of the target language translators. It is hypothesized that higher legal singularity normally entails significant incongruity between the reference legal cultures at hand and, therefore, more cognitive effort to achieve translation adequacy. However, this effort can be greatly reduced when a translation of the term in question is clearly established and accepted as reliable in a particular institutional context. This may affect the difficulty associated with a term. It can also vary depending on each translator’s experience.

For the purpose of this study, however, the difficulty levels were agreed based on the complexity of each term for reformulation, and regardless of the relevance or reliability of specific institutional terminological resources. This approach was decided because: (1) it was impossible to know which resources translators resorted to, if any, for their decisions in the years considered; and (2) overall, as in the previous LETRINT studies, our verification of institutional termbanks pointed to insufficient reliable information in the case of court names and a diversity of possible renderings in the case of the other two source terms, with the exception of PFE in the WTO (see overview in Table 3). In all instances, therefore, except for CoA (for the reasons explained below), further research would be required for decision-making; this commonality represented an effort in itself and was accounted for in the analysis of difficulty levels.

Regarding court names, given the search for interlinguistic concordance and the communicative priority of identification of the judicial body in the source system, testing the acceptability of literal renderings emerges as a common step in practice. Surface-level similarity between the source and the target languages may help to identify the judicial body, but at the risk of leading to semantically inaccurate or even misleading formulations. In such cases, the decision-making effort increases. While literal translations of CoA generally ensure lexical and conceptual correspondence between legal traditions and languages (low singularity and difficulty), HC requires more research on the powers of the court in the national system of reference (medium singularity and difficulty), and literal renderings of MC in FR and ES are inaccurate and must be avoided through borrowings, system-neutral “conceptual” or descriptive formulations, or a combination of techniques, depending on institutional preferences (higher singularity and difficulty). For the latter term, however, the preference for the borrowing of national court names seems established in EU institutions such as the CJEU, which expedites decision-making.

As for PFE and DP, the main cognitive effort expected is linked to the capacity to grasp the procedural specificity of the concepts, as adopted with a shared meaning in the international legal framework, and to the verification of translation precedents and recommendations in each institutional context.

4.Adequacy patterns

To shed light on the potential impact of legal singularity on adequacy patterns, we will first focus on the initial adequacy values obtained for each term, institutional dataset and target language, without considering inadequate intratextual variations (Table 4). The impact of these variations on the final adequacy levels will be described in the next section.

Overall, the initial results show average values above 1 (i.e., translations of acceptable quality) for all the terms and institutions, except for MC renderings in ES (0.73, half the 1.51 average in FR). The scores for ES and FR are strikingly similar in the case of the other court names, but differ slightly for PFE (0.11 difference) and more significantly for DP (1.86 in ES and 1.26 in FR).

The initial adequacy scores for translations of court names (Table 4) point to a clear correlation with legal singularity and difficulty levels (see Table 2), with CoA at the top (1.99 in ES and FR), followed by HC (1.85 in ES and 1.92 in FR). CoA averages per setting and language fluctuate between 1.98 and 2.00. Both in ES and FR, over 99% of renderings of the term are highly adequate (predominantly, tribunal de apelación in ES, cour d’appel in FR, or the borrowing in both languages). The only instances of inadequacy are three exceptional cases of unjustified omission in two UN texts (0.41% of total occurrences), while only one level 1 solution was found (cour suprême in FR) (see distribution of adequacy values in Table 5, and most frequent renderings per organization in Table 6).

Similarly, level 2 renderings of HC prevail in both languages: between 90.55% in ES (most often, tribunal superior or the borrowing) and 95.41% in FR (led by haute cour (de justice) and the borrowing). Translations of intermediate adequacy account for 4.01% of occurrences in ES and 2.01% in FR. Most of these translations use tribunal supremo / corte suprema or cour suprême to refer to high courts in judicial systems where these courts co-exist with higher (supreme) courts, thus resulting in confusing renderings. A slightly higher proportion of renderings (5.28% in ES and 2.89% in FR), though limited in number, can be considered too vague or even misleading, including alta corte in ES and tribunal de grande instance in FR. The latter rendering is found in eight UN texts and three WTO texts where HC refers to common law systems in which the position of each high court within the judicial hierarchy is superior to that of tribunal de grande instance. In France, until their replacement by the tribunaux judiciaires in 2020, tribunaux de grande instance served as trial courts for claims above 10,000 euros through several specialized divisions. The translation of HC as tribunal de première instance, found in five occurrences of a UN text with reference to New Zealand’s HC, is another example of 0 level rendering.

Average adequacy levels are much lower in the case of the most singular judicial body, MC, in line with our preliminary terminological analysis. Although the number of occurrences is the most limited of the three court names, with a significant difference between the UN (62 occurrences) and the other two settings (20 in the EU dataset and 17 in the WTO’s), the adequacy scores are systematically higher in the case of FR translations, as noted above. The ES results comprise the most marked interinstitutional fluctuation in a target language for any of the terms, ranging from an average of 0 in the WTO to 1.20 in the EU, with the UN’s middle score of 0.77. This compares to averages between 1.47 (UN) and 1.60 (EU) in FR.

These results are explained by the negative impact of the false friend magistrados within literal translations into ES, which are predominant in the WTO, while the frequent use of borrowings for court names of EU Member States in EU documents has a positive impact on the results for this setting. Excluding the borrowings, the most accurate rendering retrieved in ES (tribunal de primera instancia) only appears in UN documents, whereas the conceptual rendering tribunal de première instance in FR is the most recurrent in the WTO texts and is twice as recurrent in our corpus as tribunal de primera instancia. Overall, the trends in ES and FR are rather opposite: almost two thirds of FR translations can be considered very adequate, and only five misleading translations as cour des magistrats were found among the 12 cases of 0 level renderings (12.12% of the total). In contrast, level 0 translations represent 62.63% of the MC sub-corpus in ES, compared to 35.35% of level 2 renderings. As anticipated (see Section 3), the fact that magistrat is too broad (including officers who sit in court as judges, or magistrats du siege, or who act as prosecutors, or magistrats du parquet, in France) may have helped translators to detect the risk of using the false friend in FR. However, the distinctive seniority of magistrado, as opposed to “magistrate”, was often overlooked in ES.

With similar levels of legal singularity and translation difficulty (as per our preliminary analysis), the other two procedural terms, PFE and DP, registered adequacy scores that are comparable to those of HC in ES (1.84 and 1.86, respectively), but are lower in FR, only slightly in the case of PFE (1.73) and more significantly in the case of DP (1.26).

The breakdown of scores and renderings per institution reveals interesting trends. The adequacy patterns for PFE translations are similarly positive in both target languages, within a range of 1.76 to 1.85, with the exception of FR in the UN, where the average drops to 1.35. While the overall proportion of inaccurate solutions (e.g., pruebas de primera mano in ES or de bonnes raisons de penser que and unjustified omissions in FR) remains under 2% in both languages, this proportion is 12.24% in the UN in FR. Level 2 renderings reach between 81.01% (WTO) and 87.03% (EU) in ES in all the settings, whereas level 1 translations fluctuate between 10.95% (EU) and 18.99% (WTO) in the same language. These level 1 proportions double in the case of FR in the EU and UN datasets, and largely explain the overall divergent averages for the two target languages.

The translation of PFE in the ES datasets is highly consistent. The following level 2 renderings account for 71.37% of the total: indicio(s) razonable(s) (the most frequent translation in EU texts, with 71.47% of occurrences) and prueba(s) prima facie (predominant in the WTO and the UN datasets, with 77.22% and 44.90%, respectively). In the EU, level 2 solutions such as élément(s) de preuve dont il ressort à première vue and élément(s) de preuve attestant / démontrant / indiquant / montrant à première vue account for 77.23% of the total, boosting the average adequacy to 1.76. The most recurrent renderings in the WTO texts are also highly adequate, with a marked preference for élément(s) de preuve prima facie (53.16%) and, to a lesser extent, preuve prima facie (26.58%), which support the average score of 1.82. In the UN, however, level 1 translations that are acceptable but semantically less precise (e.g., éléments permettant de penser que and éléments donnant à penser que) are more frequent (40.82% versus 21.90% in the EU and 17.72% in the WTO) and diverse than in the other settings, with 26 different translations in 39 documents (as opposed to 16 in ES). As a result, cases of unjustified intratextual inconsistency also have an impact on the overall adequacy result, as will be discussed in Section 5.

This trend is particularly marked in the case of DP translations, which, as mentioned above, registered the second largest difference between initial adequacy averages per target language (0.60) after those obtained by MC. The score in FR (1.26) is actually the lowest of all the terms in this language, as opposed to the overall result of 1.86 in ES. This disparity applies to the three settings and is especially pronounced in the WTO dataset, with initial adequacy scores of 1.97 in ES and 1.02 in FR. Level 2 renderings (most often debido proceso or (debidas) garantías procesales) prevail in ES in all the settings, with shares between 60.63% in the EU and 98.32% in the WTO. In the latter context, the frequent use of the literal translation debido proceso, which is also semantically accurate, has a positive impact on the corresponding adequacy score. No similar literal rendering appears to be adequate in FR. In this language, level 2 translations, such as garantie(s) d’une procédure régulière or procès equitable, account for approximately one third of occurrences in the EU and the UN datasets, but fall below 5% in the case of WTO texts. Inversely, level 1 renderings, such as régularité de la procedure or respect de la régularité, are predominant in FR (between 58.12% of translations in the UN and 94.80% in the WTO).

Finally, inadequate renderings remain marginal in both languages in the WTO and under 4% in the UN, but reach 17.77% in ES and 6.62% in FR in the EU. Apart from unjustified omissions, many of these level 0 translations are too generic and suggest that the legal nuance of the source term was not fully grasped (e.g., procedimientos adecuados and procédures judiciaires). This appears to be reinforced by the high fragmentation of target reformulations, sometimes including more than one rendering within the same text (see next section). This lack of intratextual consistency is at odds with ensuring semantic univocity for a key procedural concept when its nuance is detected.

5.The impact of intratextual inconsistencies

As explained in Section 2, a penalty of one point was applied to the adequacy level of the least recurrent translations in a text whenever the same term was translated differently within that text and the inconsistency could not be justified. Anaphoric references (e.g., estas garantías after debidas garantías procesales) or other justified cross-references or ellipses did not count as inconsistencies. Our approach helps to better discern the impact of unjustified inconsistencies on adequacy values in institutional translation settings, and any potential correlations with difficulty levels.

Although intertextual terminological consistency, a general aspiration in institutional translation, was not considered when assessing the adequacy of terminological decisions in each text, a first overview of the total number of renderings per target language and setting confirms that variability is a common phenomenon. It increases when the difficulty in conveying the specificity of the term can be partly associated with the lack of an adequate literal rendering or a clearly established institutional recommendation, i.e., it is higher for MC, PFE and DP. While the density of distinct renderings per number of documents (Table 7) ranges between one translation variant in every 8.67 to 27 documents for CoA, and one in every 9 to 25.64 documents for HC, the ratio is systematically below one new rendering of MC per 4 documents in all the settings. In the case of the other two procedural terms, the ratio fluctuates between one new rendering of PFE per 1.50 documents in FR in the UN (the most significant intertextual variation in the entire series) and one of DP per 8.68 documents in FR in the WTO.

Court names tend to be repeated in a source document more frequently than the other terms (see Table 8, line 1). Between one and two thirds of the documents where these terms appear include two or more occurrences, as opposed to figures generally under one third in the case of PFE and DP, except for the proportion in the WTO dataset, where 51.83% of the documents compiled include more than one occurrence of DP. These data suggest that higher concentrations of occurrences may contribute to lower intertextual dispersion, but, interestingly, the higher exposure to intratextual variation does not seem to have a bearing on the incidence of unjustified inconsistencies.

If we focus on the texts where each source term occurs more than once (Table 8), we observe that the proportion of documents affected by unjustified intratextual variation increases gradually between CoA (10.16% in ES and 6.25% in FR) and DP (36.34% and 58.92%, respectively, the largest shares of the series in both languages), with the only exception of PFE in ES (7.89%). The impact of inconsistencies in the total number of documents and occurrences results in very low penalties for CoA and HC renderings in both target languages (see final adequacy values in Table 9), including slightly better results in FR in all the settings (except for HC in FR in the EU, due to two marginal cases of inconsistency). For the other terms, the proportion of inconsistencies and their impact on final adequacy values are more pronounced in FR than in ES.

As with CoA and HC, the final penalties globally align with the initial adequacy trends of PFE and DP translations, but diverge slightly from such trends in the case of MC renderings. It is worth noting that no intratextual variation is observed in several datasets, including one for CoA (EU in FR), one for HC (EU in ES) and three for MC (WTO in French and EU in both languages). However, in the EU and WTO datasets for MC, only eight documents contain more than one occurrence. It must also be noted that similar proportions of occurrences affected by intratextual inconsistency may result in different penalty values due to the initial adequacy score of each occurrence and the relative weight of the different renderings within a text.

Only between 2.04% (UN) and 2.53% (WTO) of PFE renderings in ES are affected by unjustified inconsistency, among the lowest figures for any term in ES, with penalties of −0.02 or −0.03, while the proportions range between 6.34% (and a −0.06 penalty for the EU) and 14.29% (and −0.15 for the UN) in FR. These impact levels contribute to widening the overall gap between the initial adequacy results for PFE in each target language (from 0.11 to 0.17). The trend is precisely more marked in the setting where the initial adequacy average was the lowest, the UN (final inter-language difference of 0.60). As mentioned above, the recurrence of unjustified inconsistency has the greatest impact for both languages in the DP datasets. It affects more occurrences in ES than in FR in the EU (11.50% versus 10.80%) and WTO texts (21.10% versus 17.89%), but more often level 2 renderings in the first language. However, inconsistencies are remarkably more frequent in FR in the UN texts (29.15% as opposed to 8.70% of occurrences in ES), with predominant level 1 translations. The impact of inconsistency penalties on DP’s adequacy values is similar in both languages in WTO texts, which yield the lowest final adequacy level for any term in FR (0.87). The average adequacy level for DP in the whole corpus is 0.66 points lower in FR (1.74, compared to 1.08 in ES), as opposed to a 0.60 difference in the initial adequacy scores.

The impact of translation inconsistency on MC adequacy levels departs from the initial adequacy trends in ES and FR. In the most statistically-significant sub-corpus, that of the UN (62 occurrences of the source term), 16.22% of documents and 17.74% of occurrences in FR are affected by intratextual variation, as opposed to 10.81% of documents and 6.45% of occurrences in ES, resulting in a higher impact in FR (−0.19 as opposed to −0.06 in ES). However, the impact in ES, as in all the instances of intratextual inconsistency in the WTO in this language, is limited by the fact that level 0 renderings are not affected by the penalty because they are already inadequate. Otherwise, the results for the UN suggest that translators into FR avoided inadequate renderings (including the false friend) more often than ES, but the search for adequate reformulations also led to a few more cases of intratextual variation.

Figure 1 shows the final adequacy levels and the proportion of occurrences affected by the intratextual inconsistency penalty for each term and setting. The correlation between adequacy and inconsistency trends is apparent in all cases, except for the unique pattern of MC translations.

6.Concluding remarks

This study illustrates the applicability and potential benefits of a corpus-based methodology to analyze translation patterns and measure their quality by using adequacy requirements as a yardstick for assessment, according to a functionalist approach to translation decision-making in legal translation (Prieto Ramos 2014Prieto Ramos, Fernando 2014 “Parameters for Problem-Solving in Legal Translation: Implications for Legal Lexicography and Institutional Terminology Management.” In The Ashgate Handbook of Legal Translation, edited by Anne Wagner, King-Kui Sin and Le Cheng, 121–134. Farnham: Ashgate.). The focus of the paper is on the translation of five procedural legal terms into two target languages, but the approach can be applied to any terminology or phraseology and in any language. It facilitates the systematic mapping of translation variability and the comparison of adequacy levels in different text types and settings with a view to informing translation guidelines and terminological standardization processes.

Particularly, the identification of quality strengths and deficiencies from an institutional perspective can reveal the need to improve translators’ resources or introduce other measures to address inconsistencies or competence gaps. More broadly, the study also highlights the implications of terminological decision-making when there is no established translation for a legal term (i.e., when institutional resources or project specifications do not provide all the information required for decision-making) and translation precedents are inconsistent or unreliable.

Regarding the more specific aims of the study, the findings show that higher legal singularity and the associated difficulty of dealing with legal system incongruities (in the case of MC in particular) or conveying legal procedural nuance (in the case of PFE and especially DP) are coupled with lower adequacy levels and more frequent intratextual inconsistencies. The great majority of quality gaps detected can be related to these issues, and often involve the use of literal renderings that are inaccurate or misleading, or the use of imprecise and/or inconsistent alternatives due to the lack of an adequate literal reformulation of the term at hand.

These trends underlie the most significant differences identified between ES and FR translations, i.e., lower adequacy scores of MC translations in ES (mostly due to a misleading false friend) and of DP translations in FR (partly due to the lack of an acceptable literal rendering and more inconsistent translations). The above patterns are mirrored in (and potentially reinforced by) institutional termbanks, which include insufficient information on MC and the widest diversity of recommendations in the case of PFE and especially DP. Overall, obtaining and comparing quantitative and qualitative data for five terms, three institutional settings and two target languages has broadened the scope of previous studies of terminology translation quality, and adds empirical value to the correlations established.