Cover not available
Part of
Learner Corpora in Language Testing and Assessment
Edited by Marcus Callies and Sandra Götz
[Studies in Corpus Linguistics 70] 2015
► pp. 115–140

The English Vocabulary Profile as a benchmark for assigning levels to learner corpus data

This study explores the use of the English Vocabulary Profile (EVP) for the assignment of relevant proficiency bands to learner production samples. The vocabulary of 90 essays drawn from the International Corpus of Crosslinguistic Interlanguage (ICCI) has been tagged with the corresponding Common European Framework of Reference (CEFR) levels according to the information available in the EVP database. Cluster analysis was performed in order to classify the essays into five groups, which were later rank-ordered based on their length and lexical characteristics. In addition, the same 90 essays were rated on the CEFR scale by three raters. Finally, the five clusters were correlated with their rater-assigned levels with the help of a measure of rank correlation (Goodman and Kruskel’s gamma). The results demonstrate a strong association between the statistically-established clusters of essays and their CEFR scores.

Published online: 9 April 2015
DOI logo
https://doi.org/10.1075/scl.70.05len
References
The ALTE Can do project
1992–2002 [URL] (27 June 2014).
Callies, M., Díez-Bedmar, M.B. & Zaytseva, E
2014 Using learner corpora for testing and assessing L2 proficiency. In Measuring L2 Proficiency: Perspectives from SLA [Second Language Acquisition], P. Leclercq, H. Hilton & A. Edmonds (eds), 71–90. Clevedon: Multilingual Matters.Google Scholar
Capel, A
2010 A1-B2 vocabulary: Insights and issues arising from the English Profile Wordlists project. English Profile Journal 1(1): 1–11. DOI logoGoogle Scholar
2012 Completing the English Vocabulary Profile: C1 and C2 vocabulary. English Profile Journal 3(1): 1–14. DOI logoGoogle Scholar
Carlsen, C
2012 Proficiency level: A fuzzy variable in computer learner corpora. Applied Linguistics 33(2): 161–183. DOI logoGoogle Scholar
Council of Europe
2001a The Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Cambridge: CUP.Google Scholar
2001b Structured overview of all CEFR scales. [URL] (27 June 2014).
n.d. Reference Level Descriptions (RLD) for national and regional languages. [URL] (27 June 2014).
Crossley, S.A., Salsbury, T. & McNamara, D.S
2010 The development of word co-referentiality in second language speakers: A case for Latent Semantic Analysis. Vigo International Journal of Applied Linguistics 7: 55–74.Google Scholar
Crossley, S.A., Salsbury, T., McNamara, D.S. & Jarvis, S
2011 Predicting lexical proficiency in language learner texts using computational indices. Language Testing 28(4): 561–580. DOI logoGoogle Scholar
Crossley, S.A., Salsbury, T. & McNamara, D.S
2012 Predicting the proficiency level of language learners using lexical indices. Language Testing 29(2): 243–263. DOI logoGoogle Scholar
Daller, H. & Xue, H
2007 Lexical richness and oral proficiency of Chinese EFL students. In Daller, Milton & Treffers-Daller (eds), 150–164.Google Scholar
Daller, H., Milton, J. & Treffers-Daller, J
(eds) 2007 Modelling and Assessing Vocabulary Knowledge. Cambridge: CUP.Google Scholar
Dialang
Computer program, [URL] (27 June 2014).
Díez-Bedmar, M.B
2012 The use of the Common European Framework of Reference for Languages to evaluate compositions in the English exam section of the University Admission Examination . Revista de Educacion 357: 55–79.Google Scholar
Ellis, R
1994 The Study of Second Language Acquisition. Oxford: OUP.Google Scholar
European Commission
2012 First European Survey on Language Competences. [URL] (18 January 2015).
Gass, S.M. & Selinker, L
2001 Second Language Acquisition, 2nd edn. Mahwah NJ: Lawrence Erlbaum Associates.Google Scholar
Goodman, L.A. & Kruskal, W.H
1979 Measures of Association for Cross Classifications. New York NY: Springer. DOI logoGoogle Scholar
Graesser, A.C., McNamara, D.S., Louwerse, M.M. & Cai, Z
2004 Coh-Metrix: Analysis of text on cohesion and language. Behavioral Research Methods, Instruments, and Computer 36: 193–202. DOI logoGoogle Scholar
Granger, S
1993  The International Corpus of Learner English . In English Language Corpora: Design, Analysis and Exploitation, J. Aarts (ed.), 57–69. Amsterdam: Rodopi.Google Scholar
1998 The computer learner corpus: A versatile new source of data for SLA research. In Learner English on Computer, S. Granger (ed.), 3–18. London: Addison Wesley Longman.Google Scholar
2002 A bird’s-eye view of learner corpus research. In Computer Learner Corpora, Second Language Acquisition and Foreign Language Teaching [Language Learning & Language Teaching 6], S. Granger, J. Hung & S. Petch-Tyson (eds), 3–33. Amsterdam: John Benjamins. DOI logoGoogle Scholar
2004 Computer learner corpus research: Current status and future prospects. In Applied Corpus Linguistics: A Multidimensional Perspective, U. Connor & T.A. Upton (eds), 123–145. Amsterdam and Atlanta, GA: Rodopi.Google Scholar
Granger, S., Dagneaux, E. Meunier, F. & Paquot, M
2009 International Corpus of Learner English, Version 2. Louvain-la-Neuve: Presses Universitaires de Louvain.Google Scholar
Hacquebord, H. & Stellingwerf, B
2007 Assessing vocabulary for the purpose of reading diagnosis. In Modelling and Assessing Vocabulary Knowledge, H. Daller, J. Milton & J. Treffers-Daller (eds), 207–219. Cambridge: CUP.Google Scholar
Hawkins, J. & Buttery, P
2009 Using learner language from corpora to profile levels of proficiency: Insights from the English Profile Programme . In Language Testing Matters: Investigating the Wider Social and Educational Impact of Assessment, L. Taylor & C.J. Weir (eds), 158–175. Cambridge: CUP.Google Scholar
2010 Criterial features in learner corpora: Theory and illustrations. English Profile Journal 1(1): 1–23. DOI logoGoogle Scholar
Hawkins, J. & Filipović, L
2012 Criterial Features in L2 English. Cambridge: CUP.Google Scholar
Housen, A. & Kuiken, F
2009 Complexity, accuracy and fluency in second language acquisition. Applied Linguistics 30(4): 461–473. DOI logoGoogle Scholar
Housen, A., Kuiken, F. & Vedder, I
(eds) 2012 Dimensions of L2 Performance and Proficiency. Complexity, Accuracy and Fluency in SLA [Language Learning & Language Teaching 32]. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Hulstijn, J., Alderson, C. & Schroonen, R
2010 Developmental stages in second-language acquisition and levels of second-language proficiency: Are there links between them? In Communicative Proficiency and Linguistic Development: Intersections between SLA and Language Testing Research, L. Bartning, M. Martin & I. Vedder (eds), 11–20. European Second Language Association.Google Scholar
Hunt, K.W
1965 Grammatical Structures Written at Three Grade Levels. Urbana IL: The National Council of Teachers of English.Google Scholar
Jarvis, S. & Pavlenko, A
2008 Crosslinguistic Influence in Language and Cognition. London: Routledge.Google Scholar
Laufer, B
1992 How much lexis is necessary for reading comprehension? In Vocabulary and Applied Linguistics, H. Bejoint & P. Arnaud (eds), 126–132. London: Macmillan.Google Scholar
1998 The development of passive and active vocabulary in a second language: Same or different? Applied Linguistics 19(2): 255–271. DOI logoGoogle Scholar
Laufer, B. & Nation, P
1999 A productive-size test of controlled productive ability. Language Testing 16(1): 33–51. DOI logoGoogle Scholar
Leńko-Szymańska, A
2012 The emergence of formulaic language in EFL writing: A cross-linguistic and cross-sectional perspective. Paper presented at 10th international conference on Teaching and Language Corpora (TaLC10), 12–14 July 2012, Warsaw.
2014 The acquisition of formulaic language by EFL learners: A cross-sectional and cross-linguistic perspective. International Journal of Corpus Linguistics 19(2): 225–251. DOI logoGoogle Scholar
Levinsky-Aviad, T
2012 Patterns of development in the use of verb-noun and adjective-noun collocations in the writing of young EFL learners: Evidence from the International Corpus of Cross-linguistic Interlanguage (ICCI) and implications for teaching. Paper presented at the 10th International Conference on Teaching and Language Corpora (TaLC10), 12–14 July 2012, Warsaw.
Linnarud, M
1986 Lexis in Composition. A Performance Analysis of Swedish Learners’ Written English. Lund: CWK Gleerup.Google Scholar
Meara, P. & Fitzpatrick, T
2000  Lex 30: An improved method of assessing productive vocabulary in an L2. System 28: 19–30. DOI logoGoogle Scholar
Meara, P.M. & Milton, J
2005 X_Lex: the Swansea Vocabulary Levels Test (Version 2.05). Swansea: Lognostics, [URL] (27 June 2014).Google Scholar
Meara, P.M. & Miralpeix, I
2006 Y_Lex: the Swansea Advanced Vocabulary Levels Test (Version 2.05). Swansea: Lognostics, [URL] (27 June 2014).Google Scholar
Meunier, F
1998 Computer tools for interlanguage analysis: A critical approach. In Learner English on Computer, S. Granger (ed.), 19–37. London: Longman.Google Scholar
Milton, J
2009 Measuring Second Language Vocabulary Acquisition. Bristol: Multilingual Matters. DOI logoGoogle Scholar
2010 The development of vocabulary breadth across the CEFR levels. A common basis for the elaboration of language syllabuses, curriculum guidelines, examinations, and textbooks across Europe. In Communicative Proficiency and Linguistic Development: Intersections between SLA and Language Testing Research, L. Bartning, M. Martin & I. Vedder (eds), 211–232. Reggio Emila: European Second Language Association.Google Scholar
Milton, J. & Hopkins, N
2005 Aural Lex. Swansea: Swansea University.Google Scholar
Milton, J., Wade, J. & Hopkins, N
2010 Aural word recognition and oral competence in a foreign language. In Further Insights into Non-native Vocabulary Teaching and Learning, R. Chacón-Beltrán, C. Abello-Contesse, M. Torreblanca-López & M.D. López-Jiménez (eds), 83–97. Bristol: Multilingual Matters.Google Scholar
Nation, I.S.P
2001 Learning Vocabulary in Another Language. Cambridge: CUP. DOI logoGoogle Scholar
Nation, I.S.P. & Beglar, D
2007 A vocabulary size test. The Language Teacher 31(7): 9–13.Google Scholar
Nicholls, D
2003 The Cambridge Learner Corpus: Error coding and analysis for lexicography and ELT. In Proceedings of the Corpus Linguistics 2003 Conference, D. Archer, P. Rayson, A. Wilson & T. McEnery (eds), 572–581. Lancaster: University of Lancaster.Google Scholar
Schmitt, N
2000 Vocabulary in Language Teaching. Cambridge: CUP.Google Scholar
Schmitt, N., Schmitt, D. & Clapham, C
2001 Developing and exploring the behavior of two new versions of the Vocabulary Levels Test . Language Testing 18(1): 55–88.DOI logoGoogle Scholar
Selinker, L
1972 Interlanguage. IRAL: International Review of Applied Linguistics in Language Teaching 10(3): 209–231. DOI logoGoogle Scholar
Stæhr, L.S
2008 Vocabulary size and the skills of listening, reading and writing. Language Learning Journal 36: 139–152. DOI logoGoogle Scholar
Thewissen, J
2011 Accuracy across proficiency levels: Insights from an error-tagged EFL learner corpus. PhD dissertation, Université catholique de Louvain.
Thomas, M
1994 Assessment of L2 proficiency in second language acquisition research. Language Learning 44(2): 307–336. DOI logoGoogle Scholar
2006 Research synthesis and historiography: The case of assessment of second language proficiency. In Synthesizing Research on Language Learning and Teaching [Language Learning & Language Teaching 13], J.M. Norris & L. Ortega (eds), 279–298. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Tono, Y., Kawaguchi, Y. & Minegishi, M
(eds) 2012 Developmental and Cross-linguistic Perspectives in Learner Corpus Research [Tokyo University of Foreign Studies 4]. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Towell, R. & Hawkins, R
1994 Approaches to Second Language Acquisition. Clevedon: Multilingual Matters.Google Scholar
Vermeer, A
2001 Breadth and depth of vocabulary in relation to L1/L2 acquisition and frequency of input. Applied Psycholinguistics 22(2): 217–234. DOI logoGoogle Scholar
2004 The relation between lexical richness and vocabulary size in Dutch L1 and L2 children. In Vocabulary in a Second Language: Selection, Acquisition and Testing [Language Learning & Language Teaching 10], P. Bogaards & B. Laufer (eds), 173–189. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Wilkins, D.A
1972 Linguistics in Language Teaching. London: Edward Arnold.Google Scholar
Wolfe-Quintero, K., Shunji, I. & Hae-Young, K
1998 Second Language Development in Writing: Measures of Fluency, Accuracy and Complexity. Honolulu HI: University of Hawai’i Press.Google Scholar
Cited by

Cited by 2 other publications

Du, Xiangtao, Muhammad Afzaal & Hind Al Fadda
2022. Collocation Use in EFL Learners’ Writing Across Multiple Language Proficiencies: A Corpus-Driven Study. Frontiers in Psychology 13 DOI logo
Wisniewski, Katrin
2017. Empirical Learner Language and the Levels of the Common European Framework of Reference. Language Learning 67:S1 ► pp. 232 ff. DOI logo

This list is based on CrossRef data as of 19 march 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.