Cover not available

Article published In: International Journal of Learner Corpus Research: Online-First Articles

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
EPUB logo with download linkDownload EPUB
authorization required

Manual vs automated identification of English L2 suffixes and complex words across levels of proficiency

A preliminary validation of Morph

ORCiD logo with link | Universidad Autónoma del Estado de México
 | Universidad Nacional Autónoma de México
 | Universidad Autónoma del Estado de México
 | Universidad Autónoma del Estado de México
 | Universidad Autónoma del Estado de México
Published online: 31 July 2026
DOI logo with link
https://doi.org/10.1075/ijlcr.25019.flo
Abstract
This report presents the preliminary evaluation of Morph, a web-based tool for the automatic counting of 51 English noun derivational suffixes in a controlled corpus of Mexican learners of English with proficiency levels from A2 to C1 on the Common European Framework of Reference for Languages (CEFR) scale. The evaluation consisted of a quantitative analysis of agreement between human annotators and Morph, a qualitative analysis to obtain the sources of disagreement, and finally, the evaluation of the tool’s efficiency using precision, recall, and F1 score metrics. Results displayed a high level of agreement between the human annotators and Morph. The main sources of disagreement were found to be human errors, learner misspellings, and automatic tagger issues. Finally, the efficiency metrics showed the tool to be effective in accurately identifying the target suffixes across proficiency levels, with a significant reduction in time and effort.
Article outline
  • 1.Introduction
    • 1.1Morphological awareness
    • 1.2Derivation, suffixation and derivational suffixes
    • 1.3Morphological annotation, databases and natural language processing (NLP) tools
    • 1.4Extraction of suffixes in a corpus
  • 2.Methodology
    • 2.1The written corpus selected
    • 2.2Manual search procedure
    • 2.3Development of Morph
  • 3.Results and discussion
    • 3.1Manual vs Morph counts agreement
    • 3.2Sources of manual-Morph disagreement
    • 3.3The efficiency of Morph
  • 4.Conclusion
  • Data availability statement
  • Gen-AI use statement
  • References
References (74)
References
Akbas, E., & Dincer, Z. (2021). Accuracy order in L2 grammatical morphemes: Corpus evidence from different proficiency levels of Turkish learners of English. Studies in Second Language Learning and Teaching, 11(4), 607–627. Google Scholar logo with link to Google Scholar
Anthony, L. (2016). AntConc (Version 3.4.3) [Computer software]. [URL]
(2025). AntWordProfiler (Version 2.3.0) [Computer software]. [URL]
Ardébol Barea, J. (2012). A morpheme study in a corpus of secondary school EFL the “-(e)s” morpheme for 3rd person, plural and possessive (Master’s thesis). Universidad de Granada. Available online at [URL].
Aronoff, M., & Fudeman, K. (2011). What is morphology? (2nd Ed.). Wiley-Blackwell.Google Scholar logo with link to Google Scholar
Arslan, M., Mahmood, M., & Rasool, A. (2020). A corpus-based comparative study of derivational morphemes across ENL, ESL, EFL learners through ICNALE. Linguistic Forum, 2(4), 1–12. Google Scholar logo with link to Google Scholar
Balota, D. A., Yap, M. J., Cortese, M. J., Hutchison, K. A., Kessler, B., Loftis, B., Neely, J. H., Nelson, D. L., Simpson, G. B., & Treiman, R. (2007). The English Lexicon Project. Behavior Research Methods, 39(3), 445–459. Google Scholar logo with link to Google Scholar
Batsuren, K., Goldman, O., Khalifa, S., Habash, N., Kieraś, W., Bella, G., Leonard, B., Nicolai, G., Gorman, K., Ghanggo Ate, Y., Ryskina, M., Mielke, S., Budianskaya, E., El-Khaissi, C., Pimentel, T., Gasser, M., Lane, W., Raj, M., Coler, M. … Vylomova, E. (2022). UniMorph 4.0: Universal Morphology. In N. Calzolari, F. Bechet, P. Blache, K. Choukri, C. Cieri, T. Declerck, S. Goggi, H. Isahara, B. Maegaard, J. Mariani, H. Mazo, J. Odijk, & S. Piperidis (Eds.), 2022 Language Resources and Evaluation Conference, LREC (pp.840–855). European Language Resources Association (ELRA). Google Scholar logo with link to Google Scholar
Bauer, L. (2001). Morphological productivity. Cambridge University Press. Google Scholar logo with link to Google Scholar
Bauer, L., & Nation, I. S. P. (1993). Word families. International Journal of Lexicography, 6(4), 253–279. Google Scholar logo with link to Google Scholar
Booij, G. (2005). The grammar of words: An introduction to morphology (2nd Ed.). Oxford University Press.Google Scholar logo with link to Google Scholar
Carlisle, J., & Fleming, J. (2003). Lexical processing of morphologically complex words in the elementary years. Scientific Studies of Reading, 7(3), 239–253. Google Scholar logo with link to Google Scholar
Carter, R., & McCarthy, M. (2006). Cambridge grammar of English: A comprehensive guide. Cambridge University Press.Google Scholar logo with link to Google Scholar
Casselman, W. (1998). A dictionary of medical derivations: The real meaning of medical terms. The Parthenon Publishing Group Ltd.Google Scholar logo with link to Google Scholar
Castillo, C., Díaz, J., & Rubio, B. (2020). Compiling and analysing a tagged learner corpus: A corpus-based study of adjective uses. Círculo de Lingüística Aplicada a la Comunicación, (81), 115–136. Google Scholar logo with link to Google Scholar
Cobb, T. (2022). Counting affixes with MorphoLex: A response to McLean and Stoeckel (2021). Reading in a Foreign Language, 34(1), 165–171. Google Scholar logo with link to Google Scholar
Cox, M. (2005). L2 English morpheme acquisition order: The lack of consensus examined from a case study of four Chinese pre-school boys. Working Papers in Educational Linguistics, 20(2), 59–78.Google Scholar logo with link to Google Scholar
Coxhead, A. (2000). A new academic word list. TESOL Quarterly, 341, 213–238. Google Scholar logo with link to Google Scholar
Crossley, S. A., Tywoniw, R., & Choi, J. S. (2024). The tool for automatic measurement of morphological information (TAMMI). Behavior Research Methods, 56(6), 5918–5929. Google Scholar logo with link to Google Scholar
Davies, M. (2004). British National Corpus (from Oxford University Press). Available online at [URL]
(2008–). The Corpus of Contemporary American English (COCA). Available online at [URL]
Demarta Dabove, A. (2012). A study of morpheme order acquisition in an EFL corpus of L1 Spanish-L2 English: Some pedagogical implications (Master’s thesis). Universidad de Granada. Available online at [URL]
Dosi, I. (2025). How morphology, contact, vocabulary and reading shape lexical inference in typical and dyslexic readers. Education Sciences, 15(10), 1–18. Google Scholar logo with link to Google Scholar
English Testing Service (2018). TOEFL® essentials™. TOEFL. [URL]
Flores-Hernández, A. A. (2019). Adquisición de sufijos derivativos en L2 (Doctoral dissertation). Universidad Autónoma de Querétaro. Available online at [URL]
Fontana, U. (2013). A morpheme order study based on an EFL learner corpus: A focus on the dual mechanism (Master’s thesis). Universidad de Granada. Available online at [URL]
García, B. (2009). The diminutive suffix “-et/-ette”: The role of the internet in its study. Revista Canaria de Estudios Ingleses, (58), 133–145.Google Scholar logo with link to Google Scholar
Gilquin, G., De Cock, S., & Granger, S. (2010). Louvain International Database of Spoken English Interlanguage (LINDSEI) [Manual]. Presses Universitaires de Louvain.Google Scholar logo with link to Google Scholar
Granger, S., Dagneaux, E., Meunier, F., & Paquot, M. (Eds.) (2009). International Corpus of Learner English (Version 2) [Manual). Presses Universitaires de Louvain.Google Scholar logo with link to Google Scholar
Harper, D. (2016). Online Etymology Dictionary. Etymonline. [URL]
Haspelmath, M., & Sims, A. (2010). Understanding morphology (2nd Ed.). Hodder Education and Hachette UK Company.Google Scholar logo with link to Google Scholar
Ishikawa, S. (2023). The ICNALE guide: An introduction to a learner corpus study on Asian learners’ L2 English. Routledge. Google Scholar logo with link to Google Scholar
Khor, S. (2012). A corpus-based study in morpheme acquisition order of young learners: A comparison of Swedish students in grade 6 (Unpublished doctoral dissertation). Uppsala Universitet of English.
Kirov, C., Sylak-Glassman, J., Que, R., & Yarowsky, D. (2016). Very-large scale parsing and normalization of Wiktionary morphological paradigms. Proceedings of the Tenth International Conference on Language Resources and Evaluation 2016 (pp. 3121–3126). European Language Resources Association. Google Scholar logo with link to Google Scholar
Kilgarriff, A., Rychlý, P., Smrž, P., & Tugwell, D. (2004). The Sketch Engine. In Proceedings of the 11th EURALEX International Congress (pp. 105–116).Google Scholar logo with link to Google Scholar
Koda, K. (2000). Cross-linguistic variations in L2 morphological awareness. Applied Psycholinguistics, 21(3), 297–320. Google Scholar logo with link to Google Scholar
Laws, J., & Ryder, C. (2014). Getting the measure of derivational morphology in adult speech a corpus analysis using MorphoQuantics. Language studies working papers, 61, 3–17.Google Scholar logo with link to Google Scholar
(2025). MorphoQuantics (Version 2.0.1.12) [Database]. [URL]
Lieber, R. (2014). Methodological issues in studying derivation. In R. Lieber & P. Štekauer (Eds.), The Oxford handbook of derivational morphology (pp. 84–94). Oxford University Press. Google Scholar logo with link to Google Scholar
Liu, W., & Shen, H. (2012). A corpus-based analysis of English suffix –esque. Theory and Practice in Language Studies, 2(4), 767–772. Google Scholar logo with link to Google Scholar
Lozano, C., Díaz-Negrillo, A., & Callies, M. (2020). Designing and compiling a learner corpus of written and spoken narratives: COREFL. In C. Bongartz & J. Torregrossa, (Eds.), What’s in a narrative? Variation in story-telling at the interface between language and literacy (pp. 21–46). Bern: Peter Lang.Google Scholar logo with link to Google Scholar
Lund, K., & Burgess, C. (1996). Producing high-dimensional semantic spaces from lexical co-occurrence. Behavior Research Methods, Instruments, & Computers, 28(2), 203–208. Google Scholar logo with link to Google Scholar
Metheniti, E., and Neumann, G. (2018). Wikinflection: massive semi-supervised generation of multilingual in-flectional corpus from wiktionary. Proceedings of the 17th international workshop on treebanks and linguistic theories (pp. 147–161). Linkoping University Electronic Press.Google Scholar logo with link to Google Scholar
Metheniti, E., & Neumann, G. (2020). Wikinflection corpus: A (better) multilingual, morpheme-annotated inflectional corpus. Proceedings of the Twelfth Language Resources and Evaluation Conference (pp. 3905–3912). European Language Resources Association.Google Scholar logo with link to Google Scholar
̈Minkkinen, T. (2015). The influence of morphological congruency on the acquisition of the English plural morpheme –s: A corpus study on Japanese and Spanish learner corpora (Unpublished Master’s thesis). University of Eastern Finland.
MongoDB, Inc. (2025). MongoDB (Version 7.9) [Computer software]. [URL]
Morita, J. (2020). A corpus-based study of derivational morphology and its theoretical implication. Proceedings of the Fourth International Conference of Computational Linguistics in Bulgaria (CLIB 2020) (pp. 8–16). Bulgarian Academy of Sciences.Google Scholar logo with link to Google Scholar
Morita, M., & Yamashita, J. (2025). Morphological awareness and affix knowledge in EFL reading via vocabulary breadth. Reading in a Foreign Language, 37(1), 1–22. Google Scholar logo with link to Google Scholar
Naismith, B., & Kanwit, M. (2021). A corpus study of the English suffixes -ness and -acy: Productivity, genre, and implications for L2 learning. Canadian Journal of Applied Linguistics 24(1), 115–137. Google Scholar logo with link to Google Scholar
Namtapi, I. (2020). A corpus-based study of English adjective formation using the suffix –ish. Parichart Journal Thaksin University, 34(3), 150–165.Google Scholar logo with link to Google Scholar
Nguyen Lee, C. N., & Miller, J. (2020). A corpus-based list of commonly used English medical morphemes for students learning English for specific purposes. English for Specific Purposes, 581, 102–121. Google Scholar logo with link to Google Scholar
O’Donnell, M. (2009). The UAM CorpusTool: Software for corpus annotation and exploration. In C. Callejas, J. Fernández, J. Ibañez, M. Salaberri, M. Cruz, N. Perdú & B. Cantizano (Eds.). Understanding language and mind: Applied linguistics now (pp. 1433–1448). Universidad de Almería.Google Scholar logo with link to Google Scholar
Oxford University Press. (2015). Oxford advanced learner’s dictionary (9th edition). Oxford University Press.Google Scholar logo with link to Google Scholar
. (2025). Oxford English dictionary. Oxford University Press. [URL]
Pacheco, A. (2010). A aquisição de morfemas em inglês como L2: Uma análise dos padrões evolutivos através do BELC (Brazilian English Learner Corpus) (Unpublished doctoral dissertation). Universidade Federal do Rio Grande do Sul.
Plag, I. (2018). Word-formation in English (2nd Ed.). Cambridge University Press. Google Scholar logo with link to Google Scholar
Pointal, L. (2025). TreeTagger Python Wrapper 2.3 documentation (Version 2.3.] (Computer Software]. [URL]
Rahayu, A., Bochari, S., Usman, S., & Hastini. (2025). Word formation mastery in EFL context: An inquiry into derivational processes in student writing. LITERACY: International Scientific Journals of Social, Education, Humanities, 4(2), 161–168. Google Scholar logo with link to Google Scholar
Sakaguchi, K., & Nagata, R. (2017). Phrase structure annotation and parsing for learner English. Journal of Natural Language Processing, 24(3), 491–514. Google Scholar logo with link to Google Scholar
Sánchez-Gutiérrez, C. H., Mailhot, H., Deacon, S. H., & Wilson, M. (2018). MorphoLex: A derivational morphological database for 70,000 English words. Behavior Research Methods, 50(4), 1568–1580. Google Scholar logo with link to Google Scholar
Sayer, I., & Abdusalam, S. (2018). The acquisition of derivational suffixes by students of English at the University of Human Development. Journal of University of Human Development, 4(4), 55–60. Google Scholar logo with link to Google Scholar
Schmid, H. (1994). Probabilistic part-of-speech tagging using decision trees. Proceedings of the International Conference on New Methods in Language Processing, 121 (pp. 44–49).Google Scholar logo with link to Google Scholar
Schmidt, J. E. (1974). Index of paramedical vocabulary: An index-indicator enabling the user not versed in Greek and Latin to locate the terminology of any given subject in a paramedical, medical, Or biological dictionary. University of Michigan.Google Scholar logo with link to Google Scholar
Schmitt, N., & Meara, P. (1997). Researching vocabulary through a word knowledge framework. Studies in Second Language Acquisition, 19(1), 17–36. Google Scholar logo with link to Google Scholar
Schmitt, N., & Zimmerman, C. B. (2002). Derivative word forms: What do learners know? TESOL Quarterly, 36(2), 145–171. Google Scholar logo with link to Google Scholar
Schuwerk, T. (2004). Morpheme acquisition and second language learners (Master’s thesis). University of Central Florida. Available online at [URL]
Stein, G. (2007). A Dictionary of English affixes: Their function and meaning. Lincom Europa.Google Scholar logo with link to Google Scholar
Suhandoko, S., & Ningrum, D. R. (2020). A corpus-based list of academic English derivational suffixes. Indonesian Journal of Applied Linguistics, 10(2), 481–490. Google Scholar logo with link to Google Scholar
The MongoDB Python Team. (2025). PyMongo (Version 4.17.0) [Computer software]. [URL]
Varatharajoo, C., Asmawi, A. B., Abdallah, N., & Abedalaziz, M. (2015). The awareness of morphemic knowledge for young adults’ vocabulary learning. The Malaysian Online Journal of Educational Science, 3(2), 45–56.Google Scholar logo with link to Google Scholar
West, M. (1953). A general service list of English words. Longman.Google Scholar logo with link to Google Scholar
Wang, T., & Zhang, H. (2022). Contributions of morphological awareness and lexical inference ability to L2 vocabulary knowledge among Chinese EFL learners: A structural equation modelling analysis. Applied Linguistics Review, 15(3), 1–39.Google Scholar logo with link to Google Scholar
Zhang, H. (2021). The longitudinal effect of morphological awareness on higher-order literacy skills among college L2 learners. Contemporary Educational Psychology, 651, 101969. Google Scholar logo with link to Google Scholar
Zhang, Y., & Widyastuti, I. (2010). Acquisition of L2 English morphology: A family case study. Australian Review of Applied Linguistics, 33(3), 291–297. Google Scholar logo with link to Google Scholar
Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue