Holistic and analytic assessment of functional adequacy
This study looks at the correlation between functional adequacy (FA), holistically assessed, and analytic linguistic measures, in a corpus of texts written by Italian monolingual and multilingual primary school pupils. Texts were first evaluated using the FA rating scales by Kuiken and Vedder (
2017,
2018), plus one for Coherence & Cohesion from the CEFR (
Council of Europe, 2001′). They were then coded for a number of features directly bearing on FA and its subdimensions. Results show correlations between holistic scores and analytic measures, such as those between Content and the number of words or secondary idea units (
r = .59 / .65). Others were less strong, yet going in the expected direction, e.g. more ambiguous referential expressions were negatively correlated to Comprehensibility. Correlations were generally stronger for monolingual than for multilingual children.
Article outline
- Previous research
- Methodology
- Data analysis
- Content
- Coherence & Cohesion
- Comprehensibility
- Results: Correlation analysis
- Results: Regression models
- Functional adequacy
- Content
- Comprehensibility
- Coherence & Cohesion – Kuiken and Vedder
- Coherence & Cohesion – CEFR
- Summary of findings
- Implications for TBLT and directions for future research
- Notes
-
References
References (37)
References
Bartning, I., Martin, M., & Vedder, I. (Eds.). (2010). Communicative proficiency and linguistic development: Intersections between SLA and language testing research. European Second Language Association.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Biber, D., Gray, B., & Staples, S. (2016). Predicting patterns of grammatical complexity across language exam task types and proficiency levels. Applied Linguistics,
37
(5), 639–668. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Bransford, J. D., & Johnson, M. K. (1972). Contextual prerequisites for understanding: Some investigations of comprehension and recall. Journal of Verbal Learning and Verbal Behavior,
11
(6), 717–726. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Council of Europe. (2001). Common European Framework of Reference for Languages: Learning, teaching, assessment. Council of Europe.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Covington, M., & McFall, J. (2010). Cutting the Gordian knot: The Moving-Average Type–Token Ratio (MATTR). Journal of Quantitative Linguistics,
17
(2), 94–100. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Crossley, S. (2020). Linguistic features in writing quality and development: An overview. Journal of Writing Research,
11
(3), 415–443. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
De Jong, N., Steinel, M. P., Florijn, A. F., Schoonen, R., & Hulstijn, J. H. (2012). The effect of task complexity on functional adequacy, fluency and lexical diversity in speaking performances of native and non-native speakers. In A. Housen, F. Kuiken, & I. Vedder (Eds.), Dimensions of L2 performance and proficiency – Investigating complexity, accuracy and fluency in SLA (pp. 121–142). John Benjamins. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Douglas, D. (1994). Quantity and quality in speaking test performance. Language Testing,
11
(2), 125–144. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Durrant, P., & Durrant, A. (2022). Appropriateness as an aspect of lexical richness: What do quantitative measures tell us about children’s writing? Assessing Writing,
51
1, 100596. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Fox, J., & Bouchet-Valat, M. (2020). Rcmdr: R Commander. [URL]
Fulcher, G. (1996). Does thick description lead to smart tests? A data-based approach to rating scale construction. Language Testing,
13
(2), 208–238. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Fulcher, G. (2015). Re-examining language testing: A philosophical and social enquiry. Routledge. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Grömping, U. (2006). Relative importance for linear regression in R: The Package relaimpo. Journal of Statistical Software,
17
(1), 1–27. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Grömping, U. (2015). Variable importance in regression models. WIREs Computational Statistics,
7
(2), 137–152. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Gu, L., & Hsieh, C.-N. (2019). Distinguishing features of young English language learners’ oral performance. Language Assessment Quarterly,
16
(2), 180–195. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Guiraud, P. (1954). Les caractères statistiques du vocabulaire. Essai de méthodologie. Presses Universitaires de France.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Hulstijn, J. H., Schoonen, R., De Jong, N. H., Steinel, M. P., & Florijn, A. (2012). Linguistic competences of learners of Dutch as a second language at the B1 and B2 levels of speaking proficiency of the Common European Framework of Reference for Languages (CEFR). Language Testing,
29
(2), 203–221. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Iwashita, N., Brown, A., McNamara, T., & O’Hagan, S. (2008). Assessed levels of second language speaking proficiency: How distinct? Applied Linguistics,
29
(1), 24–49. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kersten, K. (2009). Verbal inflections in L2 child narratives. Wissenschaftlicher Verlag Trier.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Khushik, G. A., & Huhta, A. (2020). Investigating syntactic complexity in EFL learners’ writing across Common European Framework of Reference Levels A1, A2, and B1. Applied Linguistics,
41
(4), 506–532. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kuiken, F., & Vedder, I. (2017). Functional adequacy in L2 writing: Towards a new rating scale. Language Testing,
34
(3), 321–336. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kuiken, F., Vedder, I., & Gilabert, R. (2010). Communicative adequacy and linguistic complexity in L2 writing. In I. Bartning, M. Martin, & I. Vedder (Eds.), Communicative proficiency and linguistic development: Intersections between SLA and language testing research (pp. 81–100). European Second Language Association.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Lahuerta Martínez, A. C. (2018). Analysis of syntactic complexity in secondary education EFL writers at different proficiency levels. Assessing Writing,
35
1, 1–11. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Larson-Hall, J. (2016). A guide to doing statistics in second language research using SPSS and R, 2nd edition. Routledge.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Lee, Y. Y., & Ventura, S. (2017). Lindia. Automated Linear Regression Diagnostic. [URL]
Lindeman, R. H., Merenda, P. F., & Gold, R. Z. (1980). Introduction to bivariate and multivariate analysis. Scott, Foresman and Company.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Norman, G. (2010). Likert scales, levels of measurement and the “laws” of statistics. Advances in Health Sciences Education,
15
(5), 625–632. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Olinghouse, N. G., & Wilson, J. (2013). The relationship between vocabulary and writing quality in three genres. Reading and Writing,
26
(1), 45–65. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Pallotti, G. (2009). CAF: Defining, refining and differentiating constructs. Applied Linguistics,
30
(4), 590–601. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Pallotti, G., Borghetti, C., & Rosi, F. (2021). Insegnare a scrivere nella scuola primaria. Caissa.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
R Core Team. (2020). R: A language and environment for statistical computing. R Foundation for Statistical Computing. [URL]
Révész, A., Ekiert, M., & Torgersen, E. N. (2016). The effects of complexity, accuracy, and fluency on communicative adequacy in oral task performance. Applied Linguistics,
37
(6), 828–848. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Roessingh, H., Nordstokke, D., & Colp, M. (2019). Unlocking academic literacy in Grade 4: The role of handwriting. Reading & Writing Quarterly,
35
(2), 65–83. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Savignon, S. J. (2018). Communicative competence. In J. I. Liontas (Ed.), The TESOL Encyclopedia of English Language Teaching (pp. 1–7). John Wiley & Sons. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Zenker, F., & Kyle, K. (2021). Investigating minimum text lengths for lexical diversity indices. Assessing Writing,
47
1, 100505. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Skar, G. B., Lei, P.-W., Graham, S., Aasen, A. J., Johansen, M. B., & Kvistad, A. H. (2021). Handwriting fluency and the quality of primary grade students’ writing. Reading and Writing. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Cited by (2)
Cited by two other publications
Koizumi, Rie & Yo In'nami
2024.
Predicting functional adequacy from complexity, accuracy, and fluency of second-language picture-prompted speaking.
System 120
► pp. 103208 ff.
![DOI logo](//benjamins.com/logos/doi-logo.svg)
Balboni, Paolo E.
2023.
Bibliografia di Linguistica Educativa in Italia – BLEI
Aggiornamento 2022
.
EL.LE :1
![DOI logo](//benjamins.com/logos/doi-logo.svg)
This list is based on CrossRef data as of 5 july 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.