Cover not available
Part of
Task-Based Approaches to Teaching and Assessing Pragmatics
Edited by Naoko Taguchi and YouJin Kim
[Task-Based Language Teaching 10] 2018
► pp. 288–304

Chapter 12
Pragmatics in task-based language assessment

Opportunities and challenges

| Educational Testing Service

Several researchers have pointed out synergies between task-based language assessment (TBLA) and L2 pragmatics assessment, insofar as both domains are based on a concern for effective communication in context (Kim & Taguchi, 2015; Norris, 2016; Timpe-Laughlin, Wain, & Schmidgall, 2015). It is therefore surprising that very little research has been carried out on pragmatics in TBLA. Kim and Taguchi (2015), for example, highlighted that “[p]ragmatics has been a particularly underinvestigated area of task-based research” (p. 660), both in teaching and assessment – a paradox given that the task-based assessment paradigm provides a fitting framework for the assessment of L2 pragmatics. Bringing together the two lines of research – L2 pragmatics and TBLA – this chapter will highlight similarities between the basic tenets of both domains, while discussing opportunities and challenges for employing TBLA as a framework for designing assessments that measure L2 pragmatics. To that end, the chapter begins with a brief review of the fundamental concepts of task-based assessment, highlighting the role of the task as a basic unit of analysis in designing tests that include a focus on pragmatics. It discusses challenges related to generalizability, reliability, and validity as well as issues of task design and task difficulty. It then canvasses current uses of TBLA to assess L2 pragmatics, illustrating different types of task-based assessments that have included pragmatic phenomena, both in research and in operational testing. Ultimately, it argues that, as a foundation for assessing L2 pragmatics, tasks offer a number of benefits that, despite the challenges, are worth pursuing.

Article outline
  • Introduction
  • Tasks and task characteristics
  • Task selection and development in TBLA
  • Developing rating criteria to evaluate task performance: Different approaches and challenges
  • Task-based pragmatics assessments in practice
  • TBPrA in language for specific purpose assessments
  • TBPrA in educational assessments
  • TBPrA in assessment research
  • Concluding remarks
  • References
Published online: 15 August 2018
DOI logo
https://doi.org/10.1075/tblt.10.12tim
References
Bachman, L. F.
(2002) Some reflections on task-based language performance assessment. Language Testing, 19(4), 453–476. DOI logoGoogle Scholar
Brindley, G.
(1994) Task-centred assessment in language learning programs: The promise and the challenge. In N. Bird, P. Falvey, A. Tsui, D. Allison, & A. McNeill (Eds.), Language and learning (pp. 73–94). Hong Kong: Institute of Language in Education.Google Scholar
(2009) Task-centered language assessment in language learning. The promise and the challenge. In K. Van den Branden, M. Bygate, & J. M. Norris (Eds.), Task-based language teaching. A reader (pp. 435–454). Amsterdamphia: John Benjamins.Google Scholar
Brown, J. D., Hudson, T., Norris, J. M., & Bonk, W.
(2002) An investigation of second language task-based performance assessments (Technical Report #24). Honolulu, HI: University of Hawai‘i, Second Language Teaching & Curriculum Center.Google Scholar
Byrnes, H.
(2002) The role of task and task-based assessment in a content-oriented collegiate foreign language curriculum. Language Testing, 19(4), 419–437. DOI logoGoogle Scholar
Byrnes, H., Maxim, H., & Norris, J. M.
(2010) Realizing advanced FL writing development in collegiate education: Curricular design, pedagogy, assessment. Modern Language Journal, Monograph. Cambridge: Wiley-Blackwell.Google Scholar
Cho, Y., & Choi, I.
(2017) Writing from sources: Does audience matter? Manuscript submitted for publication.Google Scholar
Clyne, M.
(1994) Inter-cultural communication at work: Cultural values in discourse. Cambridge: Cambridge University Press.Google Scholar
Council of Europe
(2001) Common European framework of reference for languages: Learning, teaching, assessment. Cambridge: Cambridge University Press.Google Scholar
Crystal, D.
(Ed.) (1997) The Cambridge encyclopedia of language. Cambridge: Cambridge University Press.Google Scholar
Curriculum Development Institute
(2005) Task-based assessment for English language learning at secondary level. Retrieved from [URL].
Elder, C.
(2001) Assessing the language proficiency of teachers: Are there any border controls? Language Testing, 18(2), 149–170. DOI logoGoogle Scholar
(2016) Exploring the limits of authenticity in LSP testing: The case of a specific-purpose language test for health professionals. Language Testing, 33(2), 147–152.DOI logoGoogle Scholar
Elder, C., McNamara, T., Woodward-Kron, R., Manias, E., McColl, G., Webb, G., & Pill, J.
(2013) Towards improved healthcare communication. Development and validation of the language proficiency standards for non-native English speaking health professionals. (Final report for the occupational English test centre). Melbourne, AUS: The University of Melbourne. Retrieved from [URL].
Ellis, R.
(2003) Task-based language learning and teaching. Oxford: Oxford University Press.Google Scholar
Gilabert, R., & Barón, J.
(2013) The impact of increasing task complexity on L2 pragmatic moves. In A. Mackey & K. McDonough (Eds.), Second language interaction in diverse educational contexts (pp. 45–69). Amsterdam: John Benjamins. DOI logoGoogle Scholar
Grabowski, K.
(2009) Investigating the construct validity of a test designed to measure grammatical and pragmatic knowledge in the context of speaking. Unpublished PhD dissertation, Teachers College, Columbia University, New York, NY.Google Scholar
(2013) Investigating the construct validity of a role-play test designed to measure grammatical and pragmatic knowledge at multiple proficiency levels. In S. Ross & G. Kasper (Eds.), Assessing second language pragmatics (pp. 149–171). Houndmills: Palgrave MacMillan. DOI logoGoogle Scholar
Holmes, J.
(2000) Doing collegiality and keeping control at work: Small talk in government departments. In J. Coupland (Ed.), Small talk (pp. 32–61). London: Continuum.Google Scholar
Hudson, T., Detmer, E., & Brown, J. D.
(1995) Developing prototypic measures f cross-cultural pragmatics. Honolulu, HI: University of Hawai‘i, Second Language Teaching and Curriculum Center.Google Scholar
Kasper, G.
(2006) Beyond repair: Conversation analysis as an approach to SLA. AILA Review, 19, 83–99. DOI logoGoogle Scholar
Kasper, G., & Ross, S.
(2013) Assessing second language pragmatics: An overview and introductions. In S. Ross & G. Kasper (Eds.), Assessing second language pragmatics (pp.1–40). Houndmills: Palgrave Macmillan. DOI logoGoogle Scholar
Kim, Y., & Taguchi, N.
(2015) Promoting task-based pragmatics instruction in EFL classroom contexts: The role of task complexity. Modern Language Journal, 99(4), 656–677. DOI logoGoogle Scholar
Lockwood, J. E.
(2015) Language for specific purpose (LSP) performance assessment in Asian call centres: Strong and weak definitions. Language Testing in Asia, 5(3), DOI logo.Google Scholar
McNamara, T. F.
(1996) Measuring second language performance. London: Longman.Google Scholar
McNamara, T. F., & Roever, C.
(2006) Language testing: The social dimension. Malden, MA: Blackwell.Google Scholar
Messick, S. J.
(1994) The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher 23, 13–23. DOI logoGoogle Scholar
Mislevy, R. J., Steinberg, L. S., & Almond, R. G.
(2002) Design and analysis in task-based language assessment. Language Testing, 19(4), 477–496. DOI logoGoogle Scholar
National Standards in Foreign Language Education Project
(1996) Standards for foreign language learning: Preparing for the 21st century. Yonkers, NY: Author.Google Scholar
Norris, J. M.
(2000) Purposeful language assessment: Selecting the right alternative test. English Teaching Forum, 38(1), 18–23.Google Scholar
(2001) Identifying rating criteria for task-based EAP assessment. In T. Hudson & J. D. Brown, (Eds.), A focus on language test development: Expanding the language proficiency construct across a variety of tests (Technical Report #21, pp. 163–204). Honolulu, HI: University of Hawai‘i, Second Language Teaching and Curriculum Center.Google Scholar
(2009) Task-based teaching and testing. In M. Long & C. Doughty, (Eds.), The handbook of language teaching (pp. 578–594). Malden, MA: Blackwell-Wiley. DOI logoGoogle Scholar
(2016) Current uses for task-based language assessment. Annual Review of Applied Linguistics, 36, 230–244.DOI logoGoogle Scholar
Norris, J. M., Brown, J. D., Hudson, T., & Yoshioka, J.
(1998) Designing second language performance assessments (Technical Report #18). Honolulu, HI: University of Hawai‘i, Second Language Teaching and Curriculum Center.Google Scholar
Okada, Y., & Greer, T.
(2013) Pursuing a relevant response in oral proficiency interview role plays. In S. Ross & G. Kasper (Eds.), Assessing second language pragmatics (pp. 288–310). Houndmills: Palgrave Macmillan. DOI logoGoogle Scholar
Quellmalz, E.
(1991) Developing criteria for performance assessments: The missing link. Applied Measurement in Education, 4, 319–331. DOI logoGoogle Scholar
Roever, C.
(2006) Validation of a web-based test of ESL pragmalinguistics. Language Testing, 23(2), 229–256. DOI logoGoogle Scholar
(2011) Testing of second language pragmatics: Past and future. Language Testing, 28(4), 463–481. DOI logoGoogle Scholar
Roever, C., Fraser, C., & Elder, C.
(2014) Testing ESL sociopragmatics: Development and validation of a web-based test battery. Frankfurt: Peter Lang. DOI logoGoogle Scholar
Roever, C., & McNamara, T. F.
(2006) Preview article. Language testing: The social dimension. International Journal of Applied Linguistics, 16(2), 242–258. DOI logoGoogle Scholar
Taguchi, N.
(2007) Task difficulty in oral speech act production. Applied Linguistics, 28, 113–135. DOI logoGoogle Scholar
Taguchi, N., & Roever, C.
(2017) Second language pragmatics. Oxford: Oxford University Press.Google Scholar
Timpe, V.
(2013a) “The difficulty with difficulty:” The issue of determining task difficulty in TBLA. Journal of Linguistics and Language Teaching, 4(1), 13–28.Google Scholar
(2013b) Assessing intercultural communicative competence. The dependence of receptive sociopragmatic competence and discourse competence on learning opportunities and input. Frankfurt: Peter Lang.Google Scholar
Timpe-Laughlin, V., Wain, J., & Schmidgall, J.
(2015) Defining and operationalizing the construct of pragmatic competence: Review and recommendations. (Research Report No. RR-15-06). Princeton, NJ: Educational Testing Service.Google Scholar
Timpe-Laughlin, V., & Park, I.
(2017) “Are you into beer pong? – Exploring question-answer sequences in an L2 oral performance assessment. Unpublished manuscript.Google Scholar
Washburn, G.
(2001) Using situation comedies for pragmatic language teaching and learning. TESOL Journal, 10(4), 21–26.Google Scholar
Youn, S. J.
(2008) Developing a task-based assessment of EAP pragmatics. Retrieved from [URL].
(2013) Validating task-based assessment of L2 pragmatics in interaction using mixed methods. Unpublished doctoral dissertation, Graduate Division of the University of Hawai‘i at Mānoa, Honolulu, HI.Google Scholar
(2015) Validity argument for assessing L2 pragmatics in interaction using mixed methods. Language Testing, 32, 199–225. DOI logoGoogle Scholar
Cited by

Cited by 5 other publications

Chen, Tzu-Hua
2024. The effects of task complexity on L2 English rapport-building language use and its relationship with paired speaking test task performance. Applied Linguistics Review 15:2 ► pp. 737 ff. DOI logo
Del Bono, Federica & Elisabetta Bonvino
2021. Dalla valutazione in DaD alle scale di valutazione dell’adeguatezza. EuroAmerican Journal of Applied Linguistics and Languages 8:2 DOI logo
González-Lloret, Marta
2022. The present and future of functional adequacy. TASK. Journal on Task-Based Language Teaching and Learning 2:1 ► pp. 146 ff. DOI logo
Kuiken, Folkert & Ineke Vedder
2022. Measurement of functional adequacy in different learning contexts. TASK. Journal on Task-Based Language Teaching and Learning 2:1 ► pp. 8 ff. DOI logo
Kuiken, Folkert & Ineke Vedder
2022. Chapter 14. Speaking. In Instructed Second Language Acquisition Research Methods [Research Methods in Applied Linguistics, 3], ► pp. 329 ff. DOI logo

This list is based on CrossRef data as of 22 april 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.