Discourse Completion Tasks (DCTs) have been one of the most popular tools in pragmatics research. Yet, many have
criticized DCTs for their lack of authenticity (e.g., Culpeper, Mackey, & Taguchi,
2018; Nguyen, 2019). We propose that corpora can serve as resources in
designing and evaluating DCTs. We created a DCT using advice-seeking prompts from the Q+A corpus (Baker & Egbert, 2016). Then, we administered the DCT to 33 participants. We evaluated the DCT by (1)
comparing the linguistic form and the semantic content of the participants’ DCT responses (i.e., advice-giving expressions) with
authentic data from the corpus; and (2) interviewing the participants about the instrument quality. Chi-square tests between DCT
data and corpus data revealed no significant differences in advice-giving expressions in terms of both the overall level of
directness (χ2 [2, N = 660] = 6.94, p = .03, V = .10) and
linguistic realization (χ2 [8, N = 660] = 17.75, p = .02,
V = .16), and showed a significant difference but small effect size in terms of semantic content
(χ2 [6, N = 512] = 30.35, p < .01, V = .24). Taken
together with the interview data, our findings indicate that corpora are useful in designing DCTs.
Aijmer, K., & Rühlemann, C. (Eds.). (2015). Corpus pragmatics. Cambridge: Cambridge University Press.
Alcón, E., & Safont, P. (2001). Occurrence of exhortative speech acts in ELT materials and natural speech data: A focus on request, suggestion and advice realization strategies. Studies in English Language and Linguistics, 31, 5–22.
Bachman, L. F. (2002). Some reflections on task-based language performance assessment. Language Testing, 19(4), 453–476.
Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice: Designing and developing useful language tests. Oxford: Oxford University Press.
Baker, P., & Egbert, J. (Eds.). (2016). Triangulating methodological approaches in corpus linguistic research. New York, NY: Routledge.
Bardovi-Harlig, K. (2018). Matching modality in L2 pragmatics research design. System, 751, 13–22.
Bardovi-Harlig, K., Mossman, S., & Su, Y. (2017). The effect of corpus-based instruction on pragmatic routines. Language Learning & Technology, 21(3), 76–103.
Bardovi-Harlig, K., Mossman, S., & Vellenga, H. E. (2015). Developing corpus-based materials to teach pragmatic routines. TESOL Journal, 6(3), 499–526.
Beebe, L. M., & Cummings, M. C. (1996). Natural speech act data versus written questionnaire data: How data collection method affects speech act performance. In S. Gass & J. Neu (Eds.), Speech acts across cultures: Challenges to communication in a second language (pp. 65–86). Berlin: Mouton de Gruyter.
Biber, D., & Conrad, S. (2019). Register, genre, and style. Cambridge: Cambridge University Press.
Biber, D., Conrad, S., Reppen, R., Byrd, P., Helt, M., Clark, V., … Urzua, A. (2004). Representing language use in the university: Analysis of the TOEFL 2000 spoken and written academic language corpus. (ETS TOEFL Monograph Series, MS-25). Princeton, NJ: Educational Testing Service.
Billmyer, K., & Varghese, M. (2000). Investigating instrument-based pragmatic variability: effects of enhancing discourse completion tests. Applied Linguistics, 21(4), 517–552.
Blum-Kulka, S. (1980). Learning to say what you mean in a second language: A study of the speech act performance of learners of Hebrew as a second language. Applied Linguistics, 31, 29–59.
Boyatzis, R. E. (1998). Transforming qualitative information: Thematic analysis and code development. Thousand Oaks, CA: Sage.
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101.
Brown, P., & Levinson, S. C. (1987). Politeness: Some universals in language use. Cambridge: Cambridge University Press.
Canli, Z., & Canli, B. (2013). Keep calm and say sorry!: The use of apologies by EFL teachers in Turkish and English. Educational Process: International Journal, 2(1), 36–46.
Chapelle, C. A. (2012). Validity argument for language assessment: The framework is simple…. Language Testing, 29(1), 19–27.
Cohen, A. D., & Olshtain, E. (1994). Researching the production of second language speech acts. In E. T. Tarone, S. M. Gass, & A. D. Cohen (Eds.), Research methodology in second language acquisition (pp. 143–156). Hillsdale, NJ: Lawrence Erlbaum Associates.
Culpeper, J., Mackey, A., & Taguchi, N. (2018). Second language pragmatics: From theory to methods. New York, NY: Routledge.
Cutrona, C. E. & Suhr, J. A. (1994). Social support communication in the context of marriage: An analysis of couples’ supportive interactions. In B. R. Burleson, T. L. Albrecht, & I. G. Sarason (Eds.), Communication of social support: Messages, interactions, relationships, and community (pp. 113–135). Thousand Oaks, CA: Sage.
DeCapua, A., & Dunham, J. F. (2007). The pragmatics of advice giving: Cross-cultural perspectives. Intercultural Pragmatics, 4(3), 319–342.
Eslami-Rasekh, Z. (2005). Raising the pragmatic awareness of language learners. ELT Journal, 59(3), 199–208.
Flöck, I., & Geluykens, R. (2015). Speech acts in corpus pragmatics: A quantitative contrastive study of directives in spontaneous and elicited discourse. In J. Romero-Trillo (Ed.), Yearbook of corpus linguistics and pragmatics 2015: Current approaches to discourse and translation studies (pp. 7–37). Cham: Springer.
Golato, A. (2003). Studying compliment responses: A comparison of DCTs and recordings of naturally occurring talk. Applied Linguistics, 24(1), 90–121.
Goldsmith, D. J. (2004). Communicating social support. New York, NY: Cambridge University Press.
Grabowski, K. C. (2007). Reconsidering the measurement of pragmatic knowledge using a reciprocal written test format. Columbia University Working Papers in TESOL & Applied Linguistics, 7(1), 1–48.
Halenko, N., & Jones, C. (2011). Teaching pragmatic awareness of spoken requests to Chinese EAP learners in the UK: Is explicit instruction effective?System, 39(2), 240–250.
Hartford, B. S., & Bardovi-Harlig, K. (1992). Experimental and observational data in the study of interlanguage pragmatics. Pragmatics and Language Learning, 31, 33–52.
Hinkel, E. (1997). Appropriateness of advice: DCT and multiple choice data. Applied Linguistics, 18(1), 1–26.
Hong, C. Y., & Shih, S. C. (2013). Proficiency and complaints: Analyses of production and perceptions. Intergrams, 14(1), 1–20.
Johnston, B., Kasper, G., & Ross, S. (1998). Effect of rejoinders in production questionnaires. Applied Linguistics, 19(2), 157–182.
Kasper, G., & Dahl, M. (1991). Research methods in interlanguage pragmatics. Studies in Second Language Acquisition, 13(2), 215–247.
Kouper, I. (2010). The pragmatics of peer advice in a Live Journal community. Language@ internet, 7, article 1.
Martínez-Flor, A. M. (2003). Non-native speakers’ production of advice acts: The effects of proficiency. Revista Española de Lingüística Aplicada, 161, 139–153.
McHugh, M. L. (2013). The chi-square test of independence. Biochemia Medica, 23(2), 143–149.
Nguyen, T. T. M. (2019). Data collection methods in L2 pragmatics research: An overview. In N. Taguchi (Ed.), The Routledge handbook of SLA and pragmatics (pp. 195–211). New York, NY: Routledge.
Parvaresh, V., & Tavakoli, M. (2009). Discourse completion tasks as elicitation tools: How convergent are they. The Social Sciences, 4(4), 366–373.
Patton, M. Q. (1990). Qualitative evaluation and research methods (2nd ed.). Newbury Park, CA: Sage.
Plonsky, L., & Oswald, F. L. (2014). How big is “big”? Interpreting effect sizes in L2 research. Language Learning, 64(4), 878–912.
R Core Team (2013). R: A language and environment for statistical computing (Version 3.4.3) [Computer software]. Vienna, Austria: R Foundation for Statistical Computing.
Schauer, G. A., & Adolphs, S. (2006). Expressions of gratitude in corpus and DCT data: Vocabulary, formulaic sequences, and pedagogy. System, 34(1), 119-134.
Staples, S., & Fernández, J. (2018). Corpus linguistics approaches to L2 pragmatics. In N. Taguchi (Ed.), The Routledge handbook of second language acquisition and pragmatics (241–254). Routledge: London.
Woodfield, H. (2008). Problematising discourse completion tasks: Voices from verbal report. Evaluation & Research in Education, 21(1), 43–69.
Yuan, Y. (2001). An inquiry into empirical pragmatics data-gathering methods: Written DCTs, oral DCTs, field notes, and natural conversations. Journal of Pragmatics, 33(2), 271–292.
Hashimoto, Brett, Daniel Keller, Ekaterina Sudina, Katherine Yaw, Jesse Egbert & Luke Plonsky
2020. Research in progress: Applied linguistics at Northern Arizona University, USA. Language Teaching 53:2 ► pp. 227 ff.
This list is based on CrossRef data as of 11 january 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.