Cover not available
Part of
Research Methods in the Study of L2 Writing Processes
Edited by Rosa M. Manchón and Julio Roca de Larios
[Research Methods in Applied Linguistics 5] 2023
► pp. 104–122

Chapter 5
Verbally mediated data

Concurrent/retrospective verbalizations via think-aloud protocols and stimulated recalls

| Georgetown University
| University of Illinois at Urbana-Champaign

This chapter explores how think-alouds (TAs) and stimulated recalls (SRs) have been used to study cognitive processing, with a particular focus on writing processes. We describe the two types of verbal report and discuss the kinds of research questions that each is well-suited to answer, as well as considerations to be addressed when deciding which one to use in a given research design. We then discuss the validity of TAs and SRs and assess the robustness of findings from empirical studies on the writing process that have used TAs and SRs. We conclude by providing some future directions for research to move the field of L2 writing processes forward.

Article outline
  • Introduction
  • Research questions addressed with verbal reports
  • Validity issues with verbal reports
  • Validity issues with verbal reports in ISLA
    • Validity issues with verbal reports in writing research
    • Reactivity in writing tasks: Effects on complexity, accuracy, and fluency
    • Reactivity in writing tasks: WCF
    • Validity issues with stimulated recalls in writing research
    • Summary of reactivity in writing research
  • Robustness of writing studies employing verbal reports
  • Future directions
  • References
Published online: 23 October 2023
DOI logo
https://doi.org/10.1075/rmal.5.05leo
References
Adams, R.
(2003) L2 output, reformulation and noticing: Implications for IL development. Language Teaching Research, 2, 347–376. DOI logoGoogle Scholar
Adrada-Rafael, S., & Filgueras-Gómez, M.
(2019) Reactivity, language of think-aloud protocol, and depth of processing in the processing of reformulated feedback. In R. P. Leow (Ed.), The Routledge handbook of second language research in classroom learning (pp. 199–211). Routledge. DOI logoGoogle Scholar
Armengol, L., & Cots, J. L.
(2009) Attention processes observed in think-aloud protocols: Two multilingual informants writing in two languages. Language Awareness, 18(3–4), 259–276. DOI logoGoogle Scholar
Barkaoui, K.
(2010) Variability in ESL essay rating processes: The role of the rating scale and rater experience. Language Assessment Quarterly, 7(1), 54–74. DOI logoGoogle Scholar
(2011) Think-aloud protocols in research on essay rating: An empirical study of their veridicality and reactivity. Language Testing, 28(1), 51–75. DOI logoGoogle Scholar
(2019) What can L2 writers’ pausing behavior tell us about their L2 writing processes? Studies in Second Language Acquisition, 41, 529–554. DOI logoGoogle Scholar
Beare, S.
(2001) Differences in content generating and planning processes of adult L1 and L2 proficient writers. Dissertation Abstracts International, A: The Humanities and Social Sciences, 62(2), 547-A. DOI logoGoogle Scholar
Bloom, B.
(1953) Thought-processes in lectures and discussions. Journal of General Education, 7(3), 160–169. [URL]Google Scholar
Bowles, M.
(2008) Task type and reactivity of verbal reports in SLA: A first look at a task other than reading. Studies in Second Language Acquisition, 30(3), 359–387. DOI logoGoogle Scholar
Bowles, M. A.
(2010) The think-aloud controversy in second language research. Routledge. DOI logoGoogle Scholar
(2018) Introspective verbal reports: Think-alouds and stimulated recall. In A. Phakiti, P. De Costa, L. Plonsky, & S. Starfield (Eds.), The Palgrave handbook of applied linguistics research methodology (pp. 339–357). Palgrave. DOI logoGoogle Scholar
Bowles, M., & Leow, R. P.
(2005) Reactivity and type of verbal report in SLA research methodology: Expanding the scope of investigation. Studies in Second Language Acquisition, 27(3), 415–440. DOI logoGoogle Scholar
Box, J. A.
(2002) Guided writing in the early childhood classroom. Reading Improvement, 39(3), 111–113.Google Scholar
Breetvelt, I.
(1994) Relations between writing processes and text quality: When and how? Cognition and Instruction, 12(2), 103–123. DOI logoGoogle Scholar
Breuer. E. O.
(2019) Fluency in L1 and FL writing: An analysis of planning, essay writing and final revision. In E. Lindgren & K. Sullivan (Eds.), Observing writing. Insights from keystroke logging and handwriting (pp. 190–211). Brill. DOI logoGoogle Scholar
Caras, A.
(2019) Written corrective feedback in compositions and the role of depth of processing. In R. P. Leow (Ed.), The Routledge handbook of second language research in classroom learning (pp. 186–198). Routledge. DOI logoGoogle Scholar
Castro, C. D.
(2004) The role of Tagalog in ESL writing: Clues from students’ think-aloud protocols. Philippine Journal of Linguistics, 35(2), 23–39.Google Scholar
Castro, D.
(2005) Investigating the use of the native language in the process of writing in a second language: A qualitative approach. Revista Virtual de Estudos da Linguagem, ReVEL, 3(5), n.p.Google Scholar
Chenoweth, N. A., & Hayes, J. R.
(2001) Fluency in writing: Generating text in L1 and L2. Written Communication, 18(1), 80–98. DOI logoGoogle Scholar
Cohen, A. D., & Cavalcanti, M. C.
(1987) Viewing feedback on compositions from the teacher’s and the student’s perspective. ESPecialist, 16, 13–28.Google Scholar
(1990) Feedback on compositions: Teacher and student verbal reports. In B. Kroll (Ed.), Second language writing: Research insights for the classroom (pp. 155–177). Cambridge University Press. DOI logoGoogle Scholar
Cushman, D.
(2002) From scribbles to stories. Instructor, 111(5), 32–33.Google Scholar
Diab, R. L.
(2005) Teachers’ and students’ beliefs about responding to ESL writing: A case study. TESL Canada Journal/Revue TESL Du Canada, 23(1), 28–43. DOI logoGoogle Scholar
Durst, R. K.
(1987) Cognitive and linguistic demands of analytic writing. Research in the Teaching of English, 21(4), 347–376.Google Scholar
Egi, T.
(2007) Recasts, learners’ interpretations, and L2 development. In A. Mackey (Ed.), Conversational interaction in second language acquisition: A series of empirical studies (pp. 249–267). Oxford University Press.Google Scholar
(2008) Investigating stimulated recall as a cognitive measure: Reactivity and verbal reports in SLA research methodology. Language Awareness, 17(3), 212–228. DOI logoGoogle Scholar
El Mortaji, L.
(2001) Writing ability and strategies in two discourse types: A cognitive study of multilingual Moroccan university students writing in Arabic (L1) and English (L3). Dissertation Abstracts International, C: Worldwide, 62(4).Google Scholar
Ericsson, K. A., & Simon, H. A.
(1984) Protocol analysis: Verbal reports as data. The MIT Press.Google Scholar
(1993) Protocol analysis: Verbal reports as data (rev. ed.). The MIT Press. DOI logoGoogle Scholar
Flower, L., & Hayes, J.
(1980) The dynamics of composing: Making plans and juggling constraints. In L. Gregg & E. Steinberg (Eds.), Cognitive processes in writing (pp. 31–50). Lawrence Erlbaum Associates.Google Scholar
(1981) A cognitive process theory of writing. College Composition and Communication, 32(4), 365–87. DOI logoGoogle Scholar
Fox, M. C., Ericsson, K. A., & Best, R.
(2011) Do procedures for verbal reporting of thinking have to be reactive? A meta-analysis and recommendations for best reporting methods. Psychological Bulletin, 137(2), 316–344. DOI logoGoogle Scholar
Fresch, M. J., Wheaton, A., & Zutell, J. B.
(1998) Thinking aloud during spelling word sorts. National Reading Conference Yearbook, 47, 285–294.Google Scholar
Gass, S. M., & Mackey, A.
(2016) Stimulated recall methodology in applied linguistics and L2 research (2nd ed.). Routledge. DOI logoGoogle Scholar
Gebril, A., & Plakans, L.
(2014) Assembling validity evidence for assessing academic writing: Rater reactions to integrated tasks. Assessing Writing, 21, 56–73. DOI logoGoogle Scholar
Goo, J.
(2010) Working memory and reactivity. Language Learning, 60(4), 712–752. DOI logoGoogle Scholar
Hacker, D. J., Keener, M. C., & Kircher, J. C.
(2017) TRAKTEXT: Investigating writing processes using eye-tracking technology. Methodological Innovations, 10(2), 1–18. DOI logoGoogle Scholar
Hatasa, Y. A., & Soeda, E.
(2000) Writing strategies revisited: A case of non-cognate L2 writers. In B. Swierzbin, F. Morris, M. Anderson, C. A. Klee & E. Tarone (Eds.), Social and cognitive factors in second language acquisition: Selected proceedings of the 1999 Second Language Research Forum (pp. 375–396). Cascadilla.Google Scholar
Hoang, H.
(2019) In E. Lindgren & K. Sullivan (Eds.), Observing writing: Insights from keystroke logging and handwriting (pp. 236–257). Brill. DOI logoGoogle Scholar
Jannausch, U. H.
(2002) A case study of native speakers of English composing in German as a foreign language. Dissertation Abstracts International, A: The Humanities and Social Sciences, 62(12), 4144-A.Google Scholar
Johnson, K. E.
(1992) Cognitive strategies and second language writers: A re-evaluation of sentence combining. Journal of Second Language Writing, 1(1), 61–75. DOI logoGoogle Scholar
Kang, Y., & Pyun, D. O.
(2013) Mediation strategies in L2 writing processes: A case study of two Korean language learners. Language, Culture and Curriculum, 26(1), 52–67. DOI logoGoogle Scholar
Khuder, B., & Harwood, N.
(2015) L2 writing in test and non-test situations: Process and product. Journal of Writing Research, 6, 233–278. DOI logoGoogle Scholar
Kim, H. R., & Bowles, M. A.
(2019) How deeply do L2 learners process written corrective feedback? Insights gained from think-alouds. TESOL Quarterly, 53(4), 913–938. DOI logoGoogle Scholar
Latif, M. M. A.
(2009) Toward a new process-based indicator for measuring writing fluency: Evidence from L2 writers’ think-aloud protocols. The Canadian Modern Language Review/La Revue Canadienne des Langues Vivantes, 65(4), 531–558. DOI logoGoogle Scholar
Leow, R. P.
(2001) Do learners notice enhanced forms while interacting with the L2? An online and offline study of the role of written input enhancement in L2 reading. Hispania, 84, 496–509. DOI logoGoogle Scholar
(2012) Explicit and implicit learning in the L2 classroom: What does the research suggest? The European Journal of Applied Linguistics and TEFL, 2, 117–129.Google Scholar
(2015) Explicit learning in the L2 classroom: A student-centered approach. Routledge. DOI logoGoogle Scholar
Leow, R. P., & Morgan-Short, K.
(2004) To think aloud or not to think aloud: The issue of reactivity in SLA research methodology. Studies in Second Language Acquisition, 26(1), 35–57. DOI logoGoogle Scholar
Leow, R. P., Hsieh, H., & Moreno, N.
(2008) Attention to form and meaning revisited. Language Learning, 58, 665–695. DOI logoGoogle Scholar
Leow, R. P., & Manchón, R. M.
(2021) Expanding research agendas: Directions for future research agendas on writing and feedback as language learning from an ISLA perspective. In R. M. Manchón & C. Polio (Eds.), Routledge handbook of second language acquisition and writing (pp. 299–311). Routledge. DOI logoGoogle Scholar
Leow, R. P., Grey, S., Marijuan, S., & Moorman, C.
(2014) Concurrent data elicitation procedures, processes, and the early stages of L2 learning: A critical overview. Second Language Research, 30(2), 111–127. DOI logoGoogle Scholar
Li, H., & He, L.
(2015) A comparison of EFL raters’ essay-rating processes across two types of rating scales. Language Assessment Quarterly, 12(2), 178–212. DOI logoGoogle Scholar
López Serrano, S., Roca de Larios, J., & Manchón, R. M.
(2020) Reprocessing output during L2 individual writing tasks: An exploration of depth of processing and the effects of proficiency. In R. M. Manchón (Ed.), Writing and language learning. Advancing research agendas. John Benjamins. DOI logoGoogle Scholar
Manchón, R. M., & Leow, R. P.
(2020) An ISLA perspective on L2 learning through writing: Implications for future research agendas. In R. M. Manchón (Ed.), Writing and language learning. Advancing research agendas (pp. 336–355). John Benjamins. DOI logoGoogle Scholar
McCulloch, S.
(2013) Investigating the reading-to-write processes and source use of L2 postgraduate students in real-life academic tasks: An exploratory study. Journal of English for Academic Purposes, 12(2), 136–147. DOI logoGoogle Scholar
Morgan-Short, K., Heil, J., Botero-Moriarty, A., & Ebert, E.
(2012) Allocation of attention to second language form and meaning: Issues of think alouds and depth of processing. Studies in Second Language Acquisition, 34, 659–685. DOI logoGoogle Scholar
Murphy, L., & Roca de Larios, J.
(2010) Searching for words: One strategic use of the mother tongue by advanced Spanish EFL writers. Journal of Second Language Writing, 19(2), 61–81. DOI logoGoogle Scholar
Park, E. S., & Kim, O. Y.
(2019) Learners’ engagement with indirect written corrective feedback: Depth of processing and self-correction. In R. P. Leow (Ed.), The Routledge handbook of second language research in classroom learning (pp. 212–226). Routledge. DOI logoGoogle Scholar
Plakans, L.
(2009) The role of reading strategies in integrated L2 writing tasks. Journal of English for Academic Purposes, 8(4), 252–266. DOI logoGoogle Scholar
Qi, D., & Lapkin, S.
(2001) Exploring the role of noticing in a three-stage second language writing task. Journal of Second Language Writing, 10, 277–303. DOI logoGoogle Scholar
Révész, A., Kourtali, N. -E., & Mazgutova, D.
(2017) Effects of task complexity on L2 writing behaviors and linguistic complexity. Language Learning, 67(1), 208–241. DOI logoGoogle Scholar
Révész, A., & Michel, M.
(Eds.) (2019) Methodological advances in investigating L2 writing processes. Special issue of Studies in Second Language Acquisition, 41(3). DOI logoGoogle Scholar
Révesz, A, Michel, M., & Lee, M.
(2019) Exploring second language writers’ pausing and revision behaviors. Studies in Second Language Acquisition, 41, 605–631. DOI logoGoogle Scholar
Roca de Larios, J., Marin, J., & Murphy, L.
(2001) A temporal analysis of formulation processes in L1 and L2 writing. Language Learning, 51(3), 497–538. DOI logoGoogle Scholar
Roca de Larios, J., Manchón, R., Murphy, L., & Marín, J.
(2008) The foreign language writer’s strategic behaviour in the allocation of time to writing processes. Journal of Second Language Writing, 17(1), 30–47. DOI logoGoogle Scholar
Rossomondo, A. E.
(2007) The role of lexical temporal indicators and text interaction format in the incidental acquisition of the Spanish future tense. Studies in Second Language Acquisition, 29(1), 39–66. DOI logoGoogle Scholar
Sachs, R., & Polio, C.
(2007) Learners’ uses of two types of written feedback on an L2 writing revision task. Studies in Second Language Acquisition, 29(1), 67–100. DOI logoGoogle Scholar
Sanz, C., Lin, H. -J., Lado, B., Bowden, H. W., & Stafford, C. A.
(2009) Concurrent verbalizations, pedagogical conditions, and reactivity: Two CALL studies. Language Learning, 59(1), 33–71. DOI logoGoogle Scholar
Sasaki, M.
(2004) A multiple-data analysis of the 3.5-year development of EFL student writers. Language Learning, 54, 525–582. DOI logoGoogle Scholar
Scardamalia, M.
(1984) Teachability of reflective processes in written composition. Cognitive Science: A Multidisciplinary Journal of Artificial Intelligence, 8(2), 173–190. DOI logoGoogle Scholar
Schmidt, R. W.
(1990) The role of consciousness in second language learning. Applied Linguistics, 11, 129–158. DOI logoGoogle Scholar
Schwartz, A. M.
(2003) Heritage Spanish speakers’ writing strategies. In A. Roca & C. Colombi (Eds.), Mi lengua: Spanish as a heritage language in the United States, Research and practice (pp. 235–257). Georgetown University Press.Google Scholar
(2005) Exploring differences and similarities in the writing strategies used by students in SNS courses. In L. A. Ortiz López & M. Lacorte (Eds.), Contactos y contextos: El español en los Estados Unidos y en contacto con otras lenguas (pp. 323–334). Iberoamericana/Vervuert.Google Scholar
Siegel, L., Capretta, P., Jones, R., & Berkowitz, H.
(1963) Students’ thoughts during class: A criterion for educational research. Journal of Educational Psychology, 54(1), 45–51. DOI logoGoogle Scholar
Stafford, C. A., Bowden, H., & Sanz, C.
(2012) Optimizing language instruction: Matters of explicitness, practice, and cue learning. Language Learning, 62(3), 741–768. DOI logoGoogle Scholar
Stiefenhöfer, L., & Michel, M.
(2020) Investigating the relationship between peer interaction and writing process in computer supported collaborative L2 writing. An eye-tracking and stimulated recall study. In R. M. Manchón (Ed.), Writing and language learning. Advancing research agendas. John Benjamins. DOI logoGoogle Scholar
Suh, B-R.
(2020) Are think alouds reactive? Evidence from an L2 written corrective feedback study. Language Teaching Research. DOI logoGoogle Scholar
Suzuki, M.
(2008) Japanese learners’ self revisions and peer revisions of their written compositions in English. TESOL Quarterly, 42(2), 209–232. DOI logoGoogle Scholar
Tiryakioglu, G., Peters, E., & Verschaffel, L.
(2019) The effect of L2 proficiency level on composing processes of EFL learners: Data from keystroke loggings, think alouds and questionnaires. In E. Lindgren & K. Sullivan (Eds.), Observing writing: Insights from keystroke logging and handwriting (pp. 212–235). Brill. DOI logoGoogle Scholar
Uzawa, K.
(1996) Second language learners’ processes of L1 writing, L2 writing, and translation from L1 into L2. Journal of Second Language Writing, 5(3), 271–294. DOI logoGoogle Scholar
Van Weijen, D., van den Bergh, Rijlaarsdam, G., & Sanders, T.
(2008) Differences in process and process-product relations in L2 writing. ITL International Journal of Applied Linguistics, 156, 203–226. DOI logoGoogle Scholar
Wang, W., & Wen, Q.
(2002) L1 use in the L2 composing process: An exploratory study of 16 Chinese EFL writers. Journal of Second Language Writing, 11(3), 225–246. DOI logoGoogle Scholar
Willey, I., & Tanimoto, K.
(2015) ‘We’re drifting into strange territory here’: What think-aloud protocols reveal about convenience editing. Journal of Second Language Writing, 27, 63–83. DOI logoGoogle Scholar
Yang, C., Hu, G., & Zhang, L. J.
(2014) Reactivity of concurrent verbal reporting in second language writing. Journal of Second Language Writing, 24, 51–70. DOI logoGoogle Scholar
Yang, C., Zhang, L. J., & Parr, J.
(2020) The reactivity of think-alouds in writing research: Quantitative and qualitative evidence from writing in English as a foreign language. Reading and Writing: An Interdisciplinary Journal, 33(2), 451–483. DOI logoGoogle Scholar
Yanguas, I., & Lado, B.
(2012) Is thinking aloud reactive when writing in the heritage language? Foreign Language Annals, 45(3), 380–399. DOI logoGoogle Scholar
Yoshida, M.
(2008) Think-aloud protocols and type of reading task: The issue of reactivity in L2 reading research. In M. Bowles, R. Foote, S. Perpiñán, & R. Bhatt (Eds.), Selected proceedings of the 2007 Second Language Research Forum (pp. 199–209). Cascadilla Proceedings Project.Google Scholar
Yu, G., Rea-Dickins, P., & Kiely, R.
(2011) The cognitive processes of taking IELTS academic writing task one. IELTS Reports Volume 11, 373–449.Google Scholar
Yu, G., He, L., & Isaacs, T.
(2017) The cognitive processes of taking IELTS academic writing task one: An eye-tracking study. IELTS Research Report. Retrieved on 26 April 2023 from [URL]
Zellermayer, M., & Cohen, J.
(1996) Varying paths for learning to revise. Instructional Science, 24(3), 177–195. DOI logoGoogle Scholar
Zhao, C. G.
(2013) Measuring authorial voice strength in L2 argumentative writing: The development and validation of an analytic rubric. Language Testing, 30(2), 201–230. DOI logoGoogle Scholar