Cover not available
Article published In:
Applied Pragmatics
Vol. 3:1 (2021) ► pp.51–83

Assessing L2 learners’ pragmatic ability in problem-solving situations at English-medium university

 | University of Melbourne
This paper discusses the oral pragmatic ability of current and prospective university students (international students) in an English-speaking country by examining how they are differentiated in the assessed levels of this target ability. A total of 67 students, all of whom were second language (L2) speakers of English, completed a set of oral discourse production tasks simulating interpersonal settings encountered at university. Their task performances were analyzed qualitatively and quantitatively. The results show a substantial gap between the highest performing university students and the prospective students in terms of their measured pragmatic ability. However, some university students scored quite poorly and were actually outperformed by a certain number of prospective students who were preparing to satisfy the English language requirements for university admission. These findings suggest that being a university student is not necessarily an indicator of having sufficiently developed pragmatic ability to participate in interpersonal settings in an academic environment. The findings shed light on the importance of providing students learning and assessment opportunities for pragmatics so they can gain greater awareness of their own developing pragmatic ability in an English-medium university.
Keywords: oral pragmatics, performance assessment, English as a second language, higher education
Article outline
  • 1.Introduction
  • 2.Literature review
  • 3.The present study
    • 3.1Assessment tasks
    • 3.2Participants
      • 3.2.1Test-takers
      • 3.2.2Interlocutors for the dialogue tasks
      • 3.2.3Raters
    • 3.3Procedures
      • 3.3.1Test administration procedure
      • 3.3.2Rating
    • 3.4Data analysis
  • 4.Results
    • 4.1RQ1: Pragmatic features discriminating among test-takers
      • 4.1.1Actions accomplished in pursuit of the communicative goal
      • 4.1.2Use of linguistically oriented features
      • 4.1.3Use of interactionally oriented features
    • 4.2Differentiation of university-level and preentry test-takers
  • 5.Discussion
  • 6.Limitations and future directions
  • Acknowledgements
  • Note
  • References
Published online: 1 February 2021
DOI logo
https://doi.org/10.1075/ap.19039.ike
References (38)
References
Al-Gahtani, S., & Roever, C. (2012). Proficiency and sequential organization of L2 requests. Applied Linguistics, 33(1), 42–65. DOI logoGoogle Scholar
(2013). ‘Hi doctor, give me handouts’: Low-proficiency learners and requests. ETL Journal, 67(4), 413–424.Google Scholar
Atkinson, J., & Heritage, J. (Eds.) (1984). Structures of social action. Cambridge University Press.Google Scholar
Brown, P., & Levinson, S. C. (1987). Politeness: Some universals in language usage. Cambridge University Press. DOI logoGoogle Scholar
Cohen, A. D. (2018). Learning pragmatics from native and non-native language teachers. Multilingual Matters. DOI logoGoogle Scholar
Crystal, D. (1997). The Cambridge encyclopedia of language (2nd ed.). Cambridge University Press.Google Scholar
Galaczi, E. (2014). Interactional competence across proficiency levels: How do learners manage interaction in paired speaking tests? Applied Linguistics, 35(5), 553–574. DOI logoGoogle Scholar
Grabowski, K. (2013). Investigating the construct validity of a role-play test designed to measure grammatical and pragmatic knowledge at multiple proficiency levels. In S. Ross & G. Kasper (Eds.), Assessing second language pragmatics (pp. 149–171). Palgrave Macmillan. DOI logoGoogle Scholar
Heritage, J. (1984). Garfinkel and ethnomethodology. Policy Press.Google Scholar
Hudson, T., Detmer, E., & Brown, J. D. (1992). A framework for testing cross-cultural pragmatics (Technical report #2). University of Hawai‘i, Second Language Teaching and Curriculum Center.Google Scholar
Ikeda, N. (2017). Measuring L2 oral pragmatic abilities for use in social contexts: Development and validation of an assessment instrument for L2 pragmatic performance in university settings [Unpublished doctoral dissertation]. University of Melbourne.
Ishida, M. (2009). Development of interactional competence: Changes in the use of ne in L2 Japanese during study abroad. In H. T. Nguyen & G. Kasper (Eds.), Talk-in-interaction: Multilingual perspective (pp. 351–385). National Foreign Language Resource Center, University of Hawai‘i.Google Scholar
(2011). Engaging in another person’s telling as a recipient in L2 Japanese: Development of interactional competence during one-year study abroad. In G. Pallotti & J. Wanger (Eds.), L2 Learning as social practice: Conversation-analytic perspectives (pp. 45–85). National Foreign Language Resource Center, University of Hawai‘i.Google Scholar
Ishihara, N. (2014). Assessment of pragmatics in the classroom. In N. Ishihara & A. D. Cohen (Eds.), Teaching and learning pragmatics (pp. 287–317). Routledge. DOI logoGoogle Scholar
Kane, M. T. (2006). Validation. In R. L. Brennan (Ed.), Educational Measurement (4th ed., pp. 17–64). Greenwood Publishing.Google Scholar
Kasper, G. (2006). Beyond repair: Conversation analysis as an approach to SLA. AILA Review, 191, 83–99. DOI logoGoogle Scholar
Kasper, G., & Ross, S. (2013). Assessing second language pragmatics: An overview and introductions. In S. Ross & G. Kasper (Eds.), Assessing second language pragmatics (pp. 1–40). Palgrave Macmillan.Google Scholar
Kim, Y. (2009). The Korean discourse markers-nuntey and kuntey in native-nonnative conversation: An acquisitional perspective. In H. T. Nguyen & G. Kasper (Eds.), Talk-in-interaction: Multilingual perspective (pp. 317–350). National Foreign Language Resource Center, University of Hawai‘i.Google Scholar
Linacre, M. (2014a). A user’s guide to FACETS Rasch-Model computer programs. [Computer software]. Winsteps.com.Google Scholar
Linacre, J. M. (2014b). Facets computer program for many facet Rasch measurement (Version 3.71.4) [Computer software]. Winsteps.com.Google Scholar
McNamara, T. F. (1996). Measuring second language performance. Longman.Google Scholar
McNamara, T. F., & Roever, C. (2006). Language testing: The social dimension. Blackwell.Google Scholar
Okada, Y. (2010). Role play in oral proficiency interviews: Interactive footing and interactional competencies. Journal of Pragmatics, 42(16), 1647–1468. DOI logoGoogle Scholar
Pekarek Doehler, S., & Pochon-Berger, E. (2011). Developing ‘methods’ for interaction: A cross-sectional study of disagreement sequences in French L2. In J. K. Hall, J. Hellermann, & S. Pekarek Doehler (Eds.), L2 interactional competence and development (pp. 206–243). Multilingual Matters. DOI logoGoogle Scholar
Roever, C. (2005). Testing ESL pragmatics. Peter Lang. DOI logoGoogle Scholar
(2011). Testing of second language pragmatics: Past and future. Language Testing, 28(4), 463–481. DOI logoGoogle Scholar
(2013). Testing implicature under operational conditions. In S. Ross & G. Kasper (Eds.), Assessing second language pragmatics (pp. 43–64). Palgrave Macmillan. DOI logoGoogle Scholar
Roever, C., Fraser, C., & Elder, C. (2014). Testing ESL sociopragmatics: Development and validation of a web-based test battery. Peter Lang. DOI logoGoogle Scholar
Schegloff, E. A. (2007). Sequence organization in interaction: A primer in conversation analysis. Cambridge University Press. DOI logoGoogle Scholar
Searle, J. R. (1969). Speech acts: An essay in the philosophy of language. Cambridge University Press. DOI logoGoogle Scholar
Taguchi, N. (2012). Context, individual differences and pragmatic competence. Multilingual Matters. DOI logoGoogle Scholar
(2019). The Routledge handbook of second language acquisition and pragmatics. Routledge. DOI logoGoogle Scholar
Taguchi, N., & Roever, C. (2017). Second language pragmatics. Oxford University Press.Google Scholar
Timpe, V. (2013). Assessing intercultural language learning. Peter Lang.Google Scholar
Walters, F. S. (2007). A conversation-analytic hermeneutic rating protocol to assess L2 oral pragmatic competence. Language Testing, 24(2), 155–183. DOI logoGoogle Scholar
Youn, S. J. (2013). Validating task-based assessment of L2 pragmatics in interaction using mixed methods [Unpublished doctoral dissertation]. University of Hawai‘i at Manoa.
(2015). Validity argument for assessing L2 pragmatics in interaction using mixed methods. Language Testing, 32(2), 199–225. DOI logoGoogle Scholar
Youn, S.J., & Bogorevich, V. (2019). Assessment in L2 pragmatics. In N. Taguchi (Ed.), The Routledge handbook of second language acquisition and pragmatics (pp. 308–321). Routledge. DOI logoGoogle Scholar
Cited by (6)

Cited by six other publications

Dai, David Wei
2025. Assessment of Interactional Competence. In The Encyclopedia of Applied Linguistics, ► pp. 1 ff. DOI logo
Dai, David Wei & Michael Davey
2024. On the Promise of Using Membership Categorization Analysis to Investigate Interactional Competence. Applied Linguistics 45:4 ► pp. 573 ff. DOI logo
Neiriz, Reza
2023. Developing and evaluating a contextualized interactional competence rating scale based on a metaphorical conceptualization. Journal of Second Language Studies 6:1 ► pp. 61 ff. DOI logo
Sada, Siti Haluma, Eko Widdy Astuti & Indri Astutik
2023. Meningkatkan Kemampuan Berbicara Siswa Kelas XI PM 1 dengan Strategi Pembelajaran Berdiferensiasi. Pubmedia Jurnal Penelitian Tindakan Kelas Indonesia 1:2 ► pp. 8 ff. DOI logo
Roever, Carsten & Naoki Ikeda
2022. What scores from monologic speaking tests can(not) tell us about interactional competence. Language Testing 39:1 ► pp. 7 ff. DOI logo
Roever, Carsten & Naoki Ikeda
2024. The Relationship Between L2 Interactional Competence and Proficiency. Applied Linguistics 45:4 ► pp. 676 ff. DOI logo

This list is based on CrossRef data as of 10 january 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.