Cover not available
Part of
Complexity, Accuracy and Fluency in Learner Corpus Research
Edited by Agnieszka Leńko-Szymańska and Sandra Götz
[Studies in Corpus Linguistics 104] 2022
► pp. 159–180

Measuring lexical accuracy

The categorization of lexical errors in written L2 English

 | Philipps University Marburg
This study seeks to contribute to the dialogue about the creation and application of lexical error annotation schemes for the purpose of studying learner language. It outlines some of the common issues that the creators of such lexical error annotation schemes face on the theoretical level, as well as the less theoretical issues that annotators encounter in practice. The chapter further illustrates the importance of a clean separation of codes that pertain to error description from those that seek to explain errors. In doing so, the study also highlights the importance of the hierarchical structure of error tag sets and how this (previously under-utilized) concept has implications for the comparability of results across learner corpora and their respective annotation schemes.
Keywords: error annotation, error taxonomies, lexical accuracy, lexical errors
Article outline
  • 1.Introduction
  • 2.Definitional challenges concerning lexical errors
  • 3.Method
    • International Corpus of Learner English (ICLE)
    • Cambridge Learner Corpus (CLC)
    • Teaching Resource Extraction from an Annotated Corpus of Learner English Project (TREACLE)
  • 4.Practical issues in error annotation: Decision making
    • 4.1Annotation span: Influence on accuracy scores
    • 4.2Overlaps between spelling, morphology and punctuation
    • 4.3Drawing a line between spelling, word choice and grammatical errors
    • 4.4Overlaps between categories for spelling, morphology and grammar
    • 4.5Mixing error description and error explanation
    • 4.6Addressing structural deviations of learner utterances
  • 5.Conclusion
  • Notes
  • References
Published online: 1 December 2022
DOI logo
https://doi.org/10.1075/scl.104.07hof
References (34)
References
Agustín Llach, María P. 2011. Lexical Errors and Accuracy in Foreign Language Writing. Bristol: Multilingual Matters. DOI logoGoogle Scholar
Alfaifi, Abdullah. 2015. Building the Arabic Learner Corpus and a System for Arabic Error Annotation. PhD dissertation, University of Leeds.
Bley-Vroman, Robert. 1983. The comparative fallacy in interlanguage studies: The case of systematicity. Language Learning 33(1): 1–17. DOI logoGoogle Scholar
Brown, Roger. 1973. A First Language: The Early Stages. Cambridge MA: Harvard University Press. DOI logoGoogle Scholar
Bryant, Christopher, Felice, Mariano & Briscoe, Ted. 2017. Automatic annotation and evaluation of error types for grammatical error correction. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vol. 1: Long Papers, Regina Barzilay & Min-Yen Kan (eds), 793–805. Vancouver BC: Association for Computational Linguistics. <[URL]> (15 November 2021). DOI logo
Castillejos López, Willelmira. 2009. Error analysis in a learner corpus: What are the learners’ strategies? In A Survey of Corpus-based Research, Pascual Cantos Gómez & Aquilino Sánchez Pérez (eds), 675–690. Murcia: Asociación Española de Lingüística del Corpus.Google Scholar
Corder, Stephen P. 1974. Error analysis. In Techniques in Applied Linguistics [Language and Language Learning 3], John P. B. Allen & Stephen P. Corder (eds), 122–154. Oxford: OUP.Google Scholar
Council of Europe. 2001. Common European Framework of Reference for Languages: Learning, teaching, assessment. Cambridge: CUP.Google Scholar
Dagneaux, Estelle, Denness, Sharon & Granger, Sylviane. 1998. Computer-aided error analysis. System 26(2): 163–174. DOI logoGoogle Scholar
del Rio, Iria & Mendes, Amália. 2018. Error annotation in the COPLE2 corpus. Revista Da Associação Portuguesa De Linguística 4: 225–239. DOI logoGoogle Scholar
Díaz-Negrillo, Ana & Fernandez-Dominguez, Jesus. 2006. Error tagging systems for learner corpora. RESLA, Spanish Journal of Applied Linguistics 19: 83–102.Google Scholar
Di Nuovo, Elisa, Bosco, Cristina, Mazzei, Alessandro & Sanguinetti, Manuela. 2019. Towards an Italian learner treebank in Universal Dependencies. In Proceedings of the Sixth Italian Conference on Computational Linguistics, Bari, Italy, November 13–15, 2019 [CEUR Workshop Proceedings 2481], Raffaella Bernardi, Roberto Navigli & Giovanni Semeraro (eds). [URL]. <[URL]> (15 November 2021).
Ellis, Rod. 1994. The Study of Second Language Acquisition. Oxford: OUP.Google Scholar
Granger, Sylviane. 1998. The computer learner corpus: A versatile new source of data for SLA-research. In Learner English on Computer, Sylviane Granger (ed.), 3–18. London: Longman.Google Scholar
Granger, Sylviane, Dagneaux, Estelle, Meunier, Fanny & Paquot, Magali. 2009. International Corpus of Learner English. Louvain-la-Neuve: Presses universitaires de Louvain.Google Scholar
Halliday, Michael A. K. 2004. An Introduction to Functional Grammar. London: Arnold.Google Scholar
Hawkins, John A. & Buttery, Paula. 2010. Criterial features in learner corpora: Theory and illustrations. English Profile Journal 1: e5. DOI logoGoogle Scholar
Housen, Alex & Kuiken, Folkert. 2009. Complexity, accuracy, and fluency in Second Language Acquisition. Applied Linguistics 30(4): 461–473. DOI logoGoogle Scholar
Izumi, Emi, Uchimoto, Kiyotaka & Isahara, Hitoshi. 2005. Error annotation for corpus of Japanese learner English. Proceedings of the Sixth International Workshop on Linguistically Interpreted Corpora ({LINC}-2005). <[URL]> (4 August 2021).
James, Carl. 1998. Errors in Language Learning and Use. London: Longman.Google Scholar
Kreyer, Rolf. 2015. The Marburg Corpus of Intermediate Learner English (MILE). In Learner Corpora in Language Testing and Assessment [Studies in Corpus Linguistics 70], Marcus Callies & Sandra Götz (eds), 13–34. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Larsson, Tove, Paquot, Magali & Plonsky, Luke. 2020. Inter-rater reliability in Learner Corpus Research: Insights from a collaborative study on adverb placement. International Journal of Learner Corpus Research 6(2): 237–251. DOI logoGoogle Scholar
Leclercq, Pascale & Edmonds, Amanda. 2014. How to assess L2 proficiency? An overview of proficiency assessment research. In Measuring L2 Proficiency: Perspectives from SLA [Second Language Acquisition 78], Pascale Leclercq, Amanda Edmonds & Heather Hilton (eds), 3–23. Bristol: Multilingual Matters. DOI logoGoogle Scholar
Lüdeling, Anke. 2008. Mehrdeutigkeiten und Kategorisierung: Probleme bei der Annotation von Lernerkorpora. In Fortgeschrittenene Lernervarietäten, Maik Walter & Patrick Grommes (eds), 119–140. Tübingen: Niemeyer.Google Scholar
Lüdeling, Anke & Hirschmann, Hagen. 2015. Error annotation systems. In The Cambridge Handbook of Learner Corpus Research [Cambridge Handbooks in Language and Linguistics], Sylviane Granger, Gaëtanelle Gilquin & Fanny Meunier (eds), 135–158. Cambridge: CUP. DOI logoGoogle Scholar
Lüdeling, Anke, Walter, Maik, Kroymann, Emil & Adolphs, Peter. 2005. Multi-level error annotation in learner corpora. In Proceedings of the Corpus Linguistics Conference, Sanda Hunston & Pernilla Danielsson (eds). Birmingham: University of Birmingham. <[URL]> (15 November 2021).
MacDonald, Penny, Murcia, Susana, Boquera, Maria, Botella, Ana, Cardona, Laura, García, Rebeca, Mediero, Esther, O’Donnell, Mick, Robles, Ainhoa & Stuart, Keith. 2011. Error coding in the TREACLE project. In Actas del III congreso internacional de lingüística de corpus: Las tecnologías de la informacíon y las comunicaciones: presente y futuro en el análisis de corpus, María L. Carrío Pastor & Miguel Á. Candel Mora (eds), 725–740. València: Universitat Politècnica.Google Scholar
Nicholls, Diane. 2003. The Cambridge Learner Corpus – Error coding and analysis for lexicography and ELT. In Proceedings of the Corpus Linguistics 2003 Conference, Dawn Archer, Paul Rayson, Andrew Wilson & Tony McEnery (eds), 572–581. Lancaster: University of Lancaster.Google Scholar
O’Donnell, Mick. 2007. The UAM CorpusTool. <[URL]> (4 August 2021).
Reznicek, Marc, Lüdeling, Anke & Hirschmann, Hagen. 2013. Competing target hypotheses in the Falko Corpus: A flexible multi-layer corpus architecture. In Automatic Treatment and Analysis of Learner Corpus Data [Studies in Corpus Linguistics 59], Ana Díaz-Negrillo, Nicolas Ballier & Paul Thompson (eds), 101–123. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Reznicek, Marc, Lüdeling, Anke, Krummes, Cedric, Schwantuschke, Franziska, Walter, Maik, Schmidt, Karin, Hirschmann, Hagen & Andreas, Torsten. 2012. Das Falko-Handbuch: Korpusaufbau und Annotationen. Version 2.01. <[URL]> (3 August 2021).
Rozovskaya, Alla & Roth, Dan. 2010. Annotating ESL errors: Challenges and rewards. In Proceedings of the NAACL HLT 2010 Fifth Workshop on Innovative Use of NLP for Building Educational Applications, Joel Tetreault, Jill Burstein & Claudia Leacock (eds), 28–36. Los Angeles CA: Association for Computational Linguistics.Google Scholar
Thewissen, Jennifer. 2015. Accuracy across Proficiency Levels: A Learner Corpus Approach. Louvain-la-Neuve: Presses universitaires de Louvain.Google Scholar
Wolfe-Quintero, Kate, Inagaki, Shunji & Kim, Hae-Young. 1998. Second Language Development in Writing: Measures of Fluency, Accuracy, and Complexity. Honolulu HI: Second Language Teaching & Curriculum Center, University of Hawai’i at Manoa.Google Scholar