Cover not available

Article In: AILA Review: Online-First Articles

Opportunities associated with emerging technologies and oral language assessment

 | Iowa State University
 | Najran University
This content is being prepared for publication; it may be subject to changes.
Abstract
In recent years, there has been a rapid increase in the use of technology to assist in the delivery of second language oral communication (OC) assessments. In particular, video mediation, virtual reality, and spoken dialogue systems (SDSs) have been employed for this purpose. These emerging technologies have helped in assessing aspects of the OC construct that have been previously neglected, such as interactional competence. Thus, the purpose of this paper is to showcase how these technologies have been incorporated into OC assessments, what aspects of the construct they can help to assess, areas where they are inadequate, and what new challenges they introduce. The paper concludes with directions for future research.
Article outline
  • Opportunities associated with emerging technologies and oral language assessment
    • Video-mediated OC assessments
    • Virtual environment OC assessments
  • Rating in video-mediated and virtual environments
    • Spoken dialogue systems for assessing oral communication
  • Rule-based SDSs
  • Large language model-driven SDSs
  • Conclusions and future research
  • Author queries
  • References
References (43)
References
Aseeri, F. M. (2025). Judging a book by its cover: Investigating the influence of test takers’ physical attractiveness in synchronous oral communication assessment. Unpublished doctoral dissertation, Iowa State University.
Brooks, L. (2009). Interacting in pairs in a test of oral proficiency: Co-constructing a better performance. Language Testing, 26, 342–366. Google Scholar logo with link to Google Scholar
Buunk, A. P., & Dijkstra, P. (2011). Does attractiveness sell? Women’s attitude toward a product as a function of model attractiveness, gender priming, and social comparison orientation. Psychology & Marketing, 28(9), 958–973. Google Scholar logo with link to Google Scholar
Chukharev-Hudilainen, E., & Ockey, G. J. (2021). The development and evaluation of interactional competence elicitor for oral language assessments. ETS Research Report Series, 2021(1), 1–20. Google Scholar logo with link to Google Scholar
Dai, D. W. (2024). Assessing interactional competence: Principles, test development and validation through an L2 Chinese IC test (p. 446). Peter Lang International Academic Publishers. Google Scholar logo with link to Google Scholar
Dombi, J., Sydorenko, T., & Timpe-Laughlin, V. (2024). Openings and closings in human-human versus human-spoken dialogue system conversations. Language Learning & Technology, 28(2), 32–61. [URL].
Ducasse, A. M., & Brown, A. (2009). Assessing paired orals: Raters’ orientation to interaction. Language Testing, 26, 423–443. Google Scholar logo with link to Google Scholar
Galaczi, E., & Taylor, L. (2018). Interactional competence: Conceptualisations, operationalisations, and outstanding questions. Language Assessment Quarterly, 15(3), 219–236. Google Scholar logo with link to Google Scholar
Gukturk, N., & Chukharev, E. (2024). Exploring the potential of a spoken dialog system-delivered paired discussion task for assessing interactional competence. Language Assessment Quarterly, 21(1), 60–99. Google Scholar logo with link to Google Scholar
Hosoda, M., Stone-Romero, E. F., & Coats, G. (2003). The effects of physical attractiveness on job-related outcomes: A meta-analysis of experimental studies. Personnel Psychology, 56(2), 431–462. Google Scholar logo with link to Google Scholar
Jamieson, J., Eignor, D. R., Grabe, W. and Kunnan, A. J. (2008). “Frameworks for a new TOEFL”. In C. A. Chapelle, J. Jamieson, and M. K. Enright (Eds.), Building a validity argument for the test of English as a foreign language (pp. 55–95). Routledge.Google Scholar logo with link to Google Scholar
Jenkins, J. (2016). International tests of English: Are they fit for purpose? In H. Liao (Ed.), Critical reflections on foreign language education: Globalization and local interventions. The Language Training and Testing Center.Google Scholar logo with link to Google Scholar
Karatay, Y. (2022). Development and validation of spoken dialog system-based oral communication tasks in an ESP context (publication No. 29165842). Doctoral dissertation, Iowa State University ProQuest Dissertations Publishing.
Karatay, Y., & Xu, J. (2025). Exploring the potential of conversational AI for assessing second language oral proficiency. TESOL Quarterly (first view). Google Scholar logo with link to Google Scholar
Kim, G. (2025). Developing a GPT-powered dynamic pragmatics assessment using the expanded evidence centered design. Language Assessment Research Conference. Fullerton, California, USA.
Kim, J., & Craig, D. A. (2012). Validation of a videoconferenced speaking test. Computer Assisted Language Learning, 25(3), 257–275. Google Scholar logo with link to Google Scholar
Kurt, S. (2025). Development and evaluation of a virtual reality environment for the Oral English Certification Test. Unpublished doctoral dissertation, Iowa State University.
Lee, H. W., Mullooly, A., Devine, A., & Galaczi, E. (2024). Exploring interaction in video-call paired speaking tests: A look at scores, language, and perceptions. Applied Linguistics, 45(3), 533–552. Google Scholar logo with link to Google Scholar
Levis, J. (2020). Revisiting the intelligibility and nativeness principles. Journal of Second Language Pronunciation, 6(3), 310–328. Google Scholar logo with link to Google Scholar
McNamara, T. (2018). A challenge for language testing: The assessment of English as a Lingua Franca. Language Assessment Research Conference, Ames, Iowa, March 21–23.
Mehan, H. (1979). Learning lessons: Social organization in the classroom. Harvard UP. Google Scholar logo with link to Google Scholar
Muhanna, M. A. (2015). Virtual reality and the CAVE: Taxonomy, interaction challenges and research directions. Journal of King Saud University-Computer and Information Sciences, 27(3), 344–361. Google Scholar logo with link to Google Scholar
Na, I. (2025, April). GenAI in assessing interactional competence: Examining differences between human-human and human-ai interactions. Language Assessment Research Conference. Fullerton, California, USA.
Nakatsuhara, F., Inoue, C., Berry, V., & Galaczi, E. (2017). Exploring the use of video-conferencing technology in the assessment of spoken language: A mixed-methods study. Language Assessment Quarterly, 14(1), 1–18. Google Scholar logo with link to Google Scholar
Neiriz, R. (2023). Eliciting interactive oral communication samples through a spoken dialogue system to measure interactional competence. Unpublished doctoral dissertation, Iowa State University.
Ockey, G. J. (2009). The effects of a test taker’s group members’ personalities on the test taker’s second language group oral discussion test scores. Language Testing, 26 (2), 161–186. Google Scholar logo with link to Google Scholar
(2014). The potential of the L2 group oral to elicit discourse with a mutual contingency pattern and afford equal speaking rights in an ESP context. English for Specific Purposes, 35, 17–29. Google Scholar logo with link to Google Scholar
(2025). Introducing second language assessment. Cambridge University Press.Google Scholar logo with link to Google Scholar
Ockey, G. J., & Chukharev-Hudilainen, E. (2021). Human versus computer partner in the paired oral discussion test. Applied Linguistics, 42(5), 924–944. Google Scholar logo with link to Google Scholar
Ockey, G. J., Chukharev-Hudilainen, E., & Hirch, R. (2023). Assessing interactional competence: Interactional Competence Elicitor (ICE) vs a human partner. Language Assessment Quarterly, 20(4–5), 377–398. Google Scholar logo with link to Google Scholar
Ockey, G. J., Gu, L., & Keehner, M. (2017). Web-based virtual environments for facilitating assessment of L2 Oral communication ability. Language Assessment Quarterly, 14(4), 346–359. Google Scholar logo with link to Google Scholar
Ockey, G. J., & Hirch, R. R. (2020). A step toward the assessment of English as a lingua franca. In G. J. Ockey & B. A. Green (Eds.), Another generation of fundamental considerations in language testing (pp. 9–28). Springer. Google Scholar logo with link to Google Scholar
Ockey, G. J., Koyama, D., Setoguchi, E., & Sun, A. (2015). The extent to which TOEFL iBT speaking scores are associated with performance on oral language tasks and oral ability components for Japanese university students. Language Testing, 32(1), 39–62. Google Scholar logo with link to Google Scholar
Ockey, G. J., & Li, Z. (2015). New and not so new methods for assessing oral communication. Language Value, 7(1), 1–21. Google Scholar logo with link to Google Scholar
Ockey, G. J., & Neiriz, R. (2021). Evaluating technology-mediated second language oral communication assessment delivery models. Assessment in Education: Principles, Policy & Practice, 28(4), 350–368. Google Scholar logo with link to Google Scholar
Ockey, G. J., Timpe-Laughlin, V., Davis, L., & Gu, L. (2019). Exploring the potential of a video-mediated interactive speaking assessment (Report No. ETS RR-19-05). Princeton, NJ: Educational Testing Service. Google Scholar logo with link to Google Scholar
Ockey, G. J. & Qi, Q. (in press). Why multiple speech varieties when assessing L2 Chinese and English listening. In Wagner, E., & In’nami, Y. (Eds.), The Theory and Practice of Second Language Listening Assessment.
Poehner, M., & Lantolf, J. (2023). Advancing L2 Dynamic Assessment: Innovations in Chinese contexts. Language Assessment Quarterly, 20(1), 1–19. Google Scholar logo with link to Google Scholar
Ramanarayanan, V., Evanini, K., & Tsuprun, E. (2019). Beyond monologues. In Zechner, K., & Evanini, K. (Eds.). Automated speaking assessment: Using language technologies to score spontaneous speech (pp. 176–191). Routledge. Google Scholar logo with link to Google Scholar
Ramanarayanan, V., Suendermann-Oeft, D., Lange, P., Mundkowsky, R., Ivanov, A. V., Yu, Z., Qian, Y., & Evanini, K. (2017). Assembling the jigsaw: How multiple open standards are synergistically combined in the HALEF multimodal dialog system. In Dahl, D. A. Multimodal interaction with W3C standards: Toward natural user interfaces to everything (pp. 295–310). Springer International Publishing. Google Scholar logo with link to Google Scholar
Song, J., & Hsu, W. L. (2021). Design and implementation of a classroom-based virtual reality assessment. In Salaberry, M. & Burch, A. (Eds.), Assessing speaking in context: Expanding the construct and its applications (pp. 265–282). Channel View Publications. Google Scholar logo with link to Google Scholar
Vo, S. (2019). Effects of task types on interactional competence in oral communication assessment (Publication No. 17345). Doctoral dissertation, Iowa State University. ProQuest Dissertation and Theses Global.
Zhang, L., & Jin, Y. (2021). Assessing interactional competence in the computer-based CET-SET: An investigation of the use of communication strategies. Assessment in education: Principles, policy & practice, 28(4), 389–410. Google Scholar logo with link to Google Scholar
Mobile Menu Logo with link to supplementary files background Layer 1 jbe-platform.quadraat-sans-bold jbe-platform.quadraat-sans-bold-dark jbe-platform.quadraat-sans-bold-light jbe-platform.quadraat-sans-bold prag Twitter_Logo_Blue