Article published In:
InterpretingVol. 17:2 (2015) ► pp.226–254
Developing an analytic scale for assessing undergraduate students’ consecutive interpreting performances
This article reports on the process of developing an analytic rating scale for assessing undergraduate students’ consecutive interpreting performances. The development process was divided into three phases. First, a total of 42 criteria for interpreter performance assessment were identified from the related literature and grouped into three categories: ‘content’, ‘form’ and ‘delivery’. Second, these criteria were rated by importance in a questionnaire survey of 31 interpreter trainers. In this phase a total of 20 criteria were removed due to statistical concerns, while 22 criteria — seven criteria for content, seven for form, and eight for delivery — were retained to construct a draft rating scale. Third, to determine the appropriate weighting for each category, two interpreter trainers used the 22-item draft scale to rate 33 consecutive interpretations by Korean undergraduate students. A statistical analysis of these assessments showed that the content category should be assigned an effective weight of 2, while the other categories need not be weighted (i.e., weighting value: 1).
Keywords: rating, criteria, Korean, consecutive interpreting, assessment
References (48)
Alderson, C. (1991). Bands and scores. In C. Alderson & B. North (Eds.), Language testing in the 1990s: The communicative legacy. London: Modern English Publications/British Council/Macmillan.
Alderson, C., Clapham, C. & Wall, D. (1995). Language test construction and evaluation. Cambridge: Cambridge University Press.
Bachman, L.F. & Palmer, A. (2010). Language assessment in practice: Developing language assessments and justifying their use in the real world. Oxford: Oxford University Press.
Bale, R. (2013). Undergraduate consecutive interpreting and lexical knowledge. The Interpreter and Translator Trainer 7 (1), 27–50. 
Bartłomiejczyk, M. (2007). Interpreting quality as perceived by trainee interpreters: Self-evaluation. The Interpreter and Translator Trainer 1 (2), 247–267.
Brace, N., Kemp, R. & Snelgar, R. (2012). SPSS for psychologists (5th ed.). London: Palgrave.
Brown, D. & Abeywickrama, P. (2010). Language assessment: Principles and classroom practices (2nd ed.). New York: Pearson Education.
Bühler, H. (1986). Linguistic (semantic) and extralinguistic (pragmatic) criteria for the evaluation of conference interpretation and interpreters. Multilingua 5 (4), 231–235.
Carroll, J.B. (1966). An experiment in evaluating the quality of translations. Mechanical Translations and Computational Linguistics 9 (3-4), 55–66.
Chiaro, D. & Nocella, G. (2004). Interpreters’ perception of linguistic and non-linguistic factors affecting quality: A survey through the World Wide Web. Meta 49 (2), 279–293.
Choi, J.Y. (2013). Assessing the impact of text length on consecutive interpreting. In D. Tsagari & R. van Deemter (Eds.), Assessment issues in language translation and interpreting. Frankfurt am Main: Peter Lang, 85–96.
Clifford, A. (2001). Discourse theory and performance-based assessment: Two tools for professional interpreting. Meta 46 (2), 365–378.
. (2005). Putting the exam to the test: Psychometric validation and interpreter certification. Interpreting 7 (1), 97–131.
Douglas, D. (2010). Understanding language testing. Abingdon: Hodder Education.
Fulcher, G. (2008). Criteria for evaluating language quality. In E. Shohamy & N.H. Hornberger (Eds.), Encyclopedia of language and education: Language testing and assessment. New York: Springer Science+Business Media LLC, 157–176.
. (2010). Practical language testing. London: Hodder Education.
Gile, D. (2001). L’evaluation de la qualité de l’interpretation en cours de formation. Meta 46 (2), 379–393.
Green, A. (2014). Exploring language assessment and testing. London and New York: Routledge.
Green, A. & Hawkey, R. (2010). Marking assessments: Rating scales and rubrics. In C. Commbe, P. Davidson, B. O’Sullivan & S. Storynoff (Eds.), The Cambridge guide to second language assessment. New York: Cambridge University Press, 299–306.
Green, R. (2013). Statistical analyses for language testers. New York: Palgrave Macmillan.
Hamidi, M. & Pöchhacker, F. (2007). Simultaneous consecutive interpreting: A new technique put to the test. Meta 52 (2), 276–289.
Hartley, A., Mason, I., Peng, G. & Perez, I. (2003). Peer and self-assessment in conference interpreter training. [URL] (accessed 7 January 2014).
Hodges, B. (n.d.). Assessment of competence of trainees in psychiatry. [URL] (accessed 7 August 2014).
Hunter, D., Jones, R. & Randhawa, B. (1996). The use of holistic versus analytic scoring for large-scale assessment of writing. The Canadian Journal of Program Evaluation 11 (2), 61–85.
Kim, Y.-H. (2011). Teachers’ perceptions of using the EDD checklist in ESL academic writing assessment: A qualitative study. English Language Teaching 23 (3), 17–39.
Knoch, U. (2009a). Diagnostic assessment of writing: A comparison of two rating scales. Language Testing 26 (2), 275–304.
. (2009b). Diagnostic writing assessment: The development and validation of a rating scale. Bern: Peter Lang.
Kopczyński, A. (1994). Quality in conference interpreting: Some pragmatic problems. In S. Lambert & B. Moser-Mercer (Eds.), Bridging the gap: Empirical research into simultaneous interpretation. Amsterdam: John Benjamins, 87–99.
Kurz, I. (2001). Conference interpreting: Quality in the ears of the user. Meta 46 (2), 394–409.
Lee, J. (2008). Rating scales for interpreting performance assessment. The Interpreter and Translator Trainer 2 (2), 165–184.
Liu, M. (2013). Design and analysis of Taiwan’s interpretation certification examination. In D. Tsagari & R. van Deemter (Eds.), Assessment issues in language translation and interpreting. Frankfurt am Main: Peter Lang, 163–178.
Liu, M. & Chiu, Y.-H. (2009). Assessing source material difficulty for consecutive interpreting: Quantifiable measures and holistic judgment. Interpreting 11 (2), 244–266.
Luoma, S. (2004). Assessing speaking. Cambridge: Cambridge University Press.
Marquardt, T. & Gillam, R. (1999). Assessment in communication disorders: Some observations on current issues. Language Testing 16 (3), 249–269.
Ng, B.C. (1992). End users’ subjective reaction to the performance of student interpreters. The Interpreters’ Newsletter (Special Issue 1), 35–41.
North, B. (2003). Scales for rating language performance: Descriptive models, formulation styles, and presentation formats, TOEFL Monograph No. MS-24. Princeton, NJ: Educational Testing Service.
Pöchhacker, F. (2001). Quality assessment in conference and community interpreting. Meta 46 (2), 410–425.
Pradas Macías, M. (2006). Probing quality criteria in simultaneous interpreting: The role of silent pauses in fluency. Interpreting 8 (1), 25–43.
Riccardi, A. (2002). Evaluation in interpretation: Macrocriteria and microcriteria. In E. Hung (Ed.), Teaching translation and interpreting 4: Building bridges. Amsterdam: John Benjamins, 115–126.
Roberts, R.P. (2000). Interpreter assessment tools for different settings. In R.P. Roberts, S.E. Carr, D. Abraham & A. Dufour (Eds.), Critical link 2. Amsterdam: John Benjamins, 103–130.
Russo, M. (2005). Simultaneous film interpreting and users’ feedback. Interpreting 7 (1), 1–26.
Schjoldager, A. (1995). Assessment of simultaneous interpreting. In C. Dollerup & V. Appel (Eds.), Teaching translation and interpreting 3: New horizons. Amsterdam: John Benjamins, 187–195.
Setton, R. & Motta, M. (2007). Syntacrobatics: Quality and reformulation in simultaneous-with-text. Interpreting 9 (2), 199–230.
Skaaden, H. (2013). Assessing interpreter aptitude in a variety of languages. In D. Tsagari & R. van Deemter (Eds.), Assessment issues in language translation and interpreting. Frankfurt am Main: Peter Lang, 35–50.
Weigle, S.C. (2002). Assessing writing. Cambridge: Cambridge University Press.
Wu, F.S. (2013). How do we assess students in the interpreting examinations? In D. Tsagari & R. van Deemter (Eds.), Assessment issues in language translation and interpreting. Frankfurt am Main: Peter Lang, 15–33.
Zannirato, A. (2008). Teaching interpreting and interpreting teaching: A conference interpreter’s overview of second language acquisition. In J. Kearns (Ed.), Translator and interpreter training: Issues, methods and debates. London and New York: Continuum, 19–38.
Zwischenberger, C. (2010). Quality criteria in simultaneous interpreting: An international vs. a national view. Interpreters’ Newsletter 151, 127–142.
Cited by (35)
Cited by 35 other publications
Guo, Wei, Xun Guo, Junkang Huang & Sha Tian
2024. Modeling listeners’ perceptions of quality in consecutive interpreting: a case study of a technology interpreting event. Humanities and Social Sciences Communications 11:1 
Li, Fengying & Linling Fu
2024. 2024 6th International Conference on Computer Science and Technologies in Education (CSTE), ► pp. 40 ff.
Shafiei, Shilan
2024. A proposed analytic rubric for consecutive interpreting assessment: implications for similar contexts. Language Testing in Asia 14:1
Yan, Hao, Yi Zhang, Yanqin Feng, Yang Li, Yueting Zhang, Yujun Lee, Maoqing Chen, Zijuan Shi, Yuan Liang, Yuqin Hei & Xu Duan
2024. Assessing mental demand in consecutive interpreting: Insights from an fNIRS study. Acta Psychologica 243 ► pp. 104132 ff.
Duan, Xu, Jie Zhang, Yi Zhang, Yuan Liang, Yingying Huang & Hao Yan
2023. The effect of speech–gesture asynchrony on the neural coupling of interlocutors in interpreter-mediated communication. Social Cognitive and Affective Neuroscience 18:1
Li, Heng & Bo Yang
2023. I misunderstand you because I worry about you: The relationship between neuroticism and ratings of linguistic interpreting. Personality and Individual Differences 207 ► pp. 112170 ff.
Maadla, Jekaterina
2023. Dialogue interpreting: The point of contact between Translation Studies, Foreign Language Teaching, and Translation for Other Learning Contexts. STRIDON: Studies in Translation and Interpreting 3:1 ► pp. 71 ff.
Moratto, Riccardo & Zhimiao Yang
Yan, Jackie Xiu & Kangte Luo
2023. Audio description and interpreting training: a comparison of assessment criteria from the perspective of learners. Perspectives 31:6 ► pp. 1049 ff.
Yan, Jackie Xiu & Kangte Luo
2024. Assessment criteria in audio describer training: An investigation of learner perceptions in a university interpreting programme in Hong Kong of China. Interpreting and Society 4:1 ► pp. 26 ff.
Chen, Jing, Huabo Yang & Chao Han
2022. Holistic versus analytic scoring of spoken-language interpreting: a multi-perspectival comparative analysis. The Interpreter and Translator Trainer 16:4 ► pp. 558 ff.
Kuang, Huolingxiao & Binghan Zheng
2022. How does interpreting performance correlate with note-taking process, note-taking product and note-reading process? An eye-tracking and pen-recording study. Across Languages and Cultures 23:2 ► pp. 167 ff.
Kuang, Huolingxiao & Binghan Zheng
2023. Note-taking effort in video remote interpreting: effects of source speech difficulty and interpreter work experience. Perspectives 31:4 ► pp. 724 ff.
Chen, Hua, Ying Wang & T. Pascal Brown
2021. The effects of topic familiarity on information completeness, fluency, and target language quality of student interpreters in Chinese–English consecutive interpreting. Across Languages and Cultures 22:2 ► pp. 176 ff.
Chen, Jing & Chao Han
2021. Testing and Assessment of Interpreting in China: An Overview. In Testing and Assessment of Interpreting [New Frontiers in Translation Studies, ], ► pp. 1 ff.
Chou, Isabelle, Kanglong Liu & Nan Zhao
2021. Effects of Directionality on Interpreting Performance: Evidence From Interpreting Between Chinese and English by Trainee Interpreters. Frontiers in Psychology 12
Han, Chao, Rui Xiao & Wei Su
2021. Assessing the fidelity of consecutive interpreting. Interpreting. International Journal of Research and Practice in Interpreting 23:2 ► pp. 245 ff.
Liu, Yanmeng
2021. Exploring a Corpus-Based Approach to Assessing Interpreting Quality. In Testing and Assessment of Interpreting [New Frontiers in Translation Studies, ], ► pp. 159 ff.
Pavez, Pedro
2021. Dialogic education in the interpreting classroom: action research for developing simultaneous interpreting quality assessment tools. The Interpreter and Translator Trainer 15:3 ► pp. 360 ff.
Shang, Xiaoqi
2021. Developing a Weighting Scheme for Assessing Chinese-to-English Interpreting: Evidence from Native English-Speaking Raters. In Testing and Assessment of Interpreting [New Frontiers in Translation Studies, ], ► pp. 45 ff.
Wang, Weiwei
2021. Introducing China’s Standards of English Language Ability (CSE)—Interpreting Competence Scales. In Testing and Assessment of Interpreting [New Frontiers in Translation Studies, ], ► pp. 15 ff.
Han, Chao, Sijia Chen, Rongbo Fu & Qin Fan
2020. Modeling the relationship between utterance fluency and raters’ perceived fluency of consecutive
interpreting. Interpreting. International Journal of Research and Practice in Interpreting 22:2 ► pp. 211 ff.
Wang, Weiwei, Yi Xu, Binhua Wang & Lei Mu
2020. Developing Interpreting Competence Scales in China. Frontiers in Psychology 11
Lee, Sang-Bin
2019. Scale-referenced, summative peer assessment in undergraduate interpreter training: self-reflection from an action researcher. Educational Action Research 27:2 ► pp. 152 ff.
Abdel Latif, Muhammad M. M.
2018. Towards a typology of pedagogy-oriented translation and interpreting research. The Interpreter and Translator Trainer 12:3 ► pp. 322 ff.
Abdel Latif, Muhammad M. M.
2020. Translation and Interpreting Assessment Research. In Translator and Interpreter Education Research [New Frontiers in Translation Studies, ], ► pp. 61 ff.
Chang, Chia-chien
2018. English Language Needs of Chinese/English Interpreting Students: an Error Analysis of the Chinese-to-English Short Consecutive Interpreting Test. English Teaching & Learning 42:3-4 ► pp. 207 ff.
Han, Chao
2018. Using rating scales to assess interpretation. Interpreting. International Journal of Research and Practice in Interpreting 20:1 ► pp. 63 ff.
Han, Chao
2018. A longitudinal quantitative investigation into the concurrent validity of self and peer assessment applied to English-Chinese bi-directional interpretation in an undergraduate interpreting course. Studies in Educational Evaluation 58 ► pp. 187 ff.
Han, Chao
2019. A generalizability theory study of optimal measurement design for a summative assessment of English/Chinese consecutive interpreting. Language Testing 36:3 ► pp. 419 ff.
Han, Chao
2021. Detecting and Measuring Rater Effects in Interpreting Assessment: A Methodological Comparison of Classical Test Theory, Generalizability Theory, and Many-Facet Rasch Measurement. In Testing and Assessment of Interpreting [New Frontiers in Translation Studies, ], ► pp. 85 ff.
Han, Chao
2021. Analytic rubric scoring versus comparative judgment: a comparison of two approaches to assessing spoken-language interpreting. Meta 66:2 ► pp. 337 ff.
Han, Chao
2022. Assessing spoken-language interpreting. Interpreting. International Journal of Research and Practice in Interpreting 24:1 ► pp. 59 ff.
Han, Chao
2022. Interpreting testing and assessment: A state-of-the-art review. Language Testing 39:1 ► pp. 30 ff.
Ouyang, Qianhua
2018. Assessing meaning-dimension quality in consecutive interpreting training. Perspectives 26:2 ► pp. 196 ff.
This list is based on CrossRef data as of 12 september 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.