Cover not available
Article published In:
ITL - International Journal of Applied Linguistics
Vol. 109/110 (1995) ► pp.75–109

Assessing Short-term Change in Advanced Oral Proficiency

Problems of Reliability & Validity in Four Case Studies

| Gesamthochschule Universität Kassel

This paper presents four case studies to demonstrate developments in oral proficiency made by advanced learners of English during two months in England. The data base consists of picture story oral narratives. Performance shortly after arrival in Britain is compared with that at the end of the two months. Assessment was by:

  1. subjective reactions of a panel of experienced native-speaker EFL teachers

  2. quantitative analysis of transcriptions of recordings.

An attempt is made to apply VORSTER’s (1980) components of proficiency developed for use with children mother tongue speakers. The main findings were as follows:

  1. There was considerable dissension among teachers, so that the reliability of subjective teacher assessment in such cases must be seriously questioned.

  2. Results on the objective quantitative measures were very chequered, suggesting that the variables chosen did not, in many cases, function well to identify developments over so short a period.

  3. Nevertheless, there were clear trends towards improvements in productivity and modality (co-verbs).

  4. There were signs of individual differences among subjects in route of development.

Published online: 1 January 1995
DOI logo
https://doi.org/10.1075/itl.109-110.04len
References
D'ANGLEJAN, A., PAINCHAUD, G. & RENAUD, C.
(1986) : Beyond the classroom : a study of communicative abilities in adult immigrants following intensive instruction. TESOL Quarterly, 201:185–205.Google Scholar
BAARS, B.
(1980) : The competing plans hypothesis : an heuristic viewpoint on the causes of errors in speech. In H.-W. Dechert & M. Raupach, (eds.), Temporal Variables in Speech: Studies in Honour of Frieda Goldman-Eisler. The Hague: Mouton, 104–115.Google Scholar
BACHMAN, L.
(1988) : Problems in examining the validity of the ACTFL Oral Proficiency Interview. Studies in Second Language Acquisition, 101:149–164. DOI logoGoogle Scholar
BIALYSTOK, E.
(1983) : Some factors in the selection and implementation of communication strategies. In C. Faerch & G. Kasper (eds.), Strategies in Interlanguage Communication. London : Longman, 100–108.Google Scholar
BRUMFIT, C.
(1984) : Communicative Methodology in Language Teaching : the Roles of Accuracy and Fluency. Cambridge University Press.Google Scholar
CANALE, M.
(1983) : On some dimensions of language proficiency. In J. W. OilerJr. (ed.), Issues in Language Teaching Research. Rowley, MA: Newbury House, 332–342.Google Scholar
CANALE, M. & SWAIN, M.
(1980) : Theoretical bases of communicative approaches to second language teaching and testing. Applied Linguistics, 11:1–47. DOI logoGoogle Scholar
CARROLL, J.
(1967) : Foreign language proficiency levels attained by language majors near graduation from college. Foreign Language Annals, 11:131–151. DOI logoGoogle Scholar
CLARK, J. & CLIFFORD, R.
(1988) : The FLI/ILR/ACTFL proficiency scales and testing techniques : Development, current status, and needed research. Studies in Second Language Acquisition, 101: 129–148. DOI logoGoogle Scholar
DAVIES, E.
(1983) : Error evaluation : the importance of viewpoint. English Language Teaching Journal, 37/4:304–311. DOI logoGoogle Scholar
DECHERT, H.-W.
(1980) : Pauses and intonation as indicators of verbal planning in second-language speech productions : two examples from a case study. In H.-W. Dechert & M. Raupach, (eds.), Temporal Variables in Speech: Studies in Honour of Frieda Goldman-Eisler. The Hague: Mouton, 271–285.Google Scholar
DECHERT, H.-W. & LENNON, P.
(1989) : Collocational blends of advanced second language learners : a preliminary analysis. In W. Oleksy (ed.), Contrastive Pragmatics. Amsterdam/Philadelphia : John Benjamins Publishing Company, 131–168.Google Scholar
FILLMORE, C.
(1979) : On fluency. In C. Fillmore, D. Kempler & W. Wang (eds.), Individual Differences in Language Ability and Language Behaviour. New York : Academic Press, 85–101.Google Scholar
HENDRICKSON, J.
(1980) : Evaluating spontaneous communication through systematic error analysis. Foreign Language Annals, 141: 357–364.Google Scholar
HIGGS, T. & CLIFFORD, R.
(1982) : The push toward communication. In T. Higgs, (ed.), Curriculum, Competence and the Foreign Language Teacher. Skokie, Illinois : National Textbook Co., 37–69.Google Scholar
HUGHES, A. & LASCARATOU, C.
(1982) : Competing criteria for error gravity. English Language Teaching Journal 36/3:175–182. DOI logoGoogle Scholar
HUNT, K.
(1965) : Grammatical Structures Written at Three Grade Levels. Research Report 3. Illinois : National Council of Teachers of English.Google Scholar
(1970) : Syntactic Maturity in Schoolchildren and Adults. Monographs of the Society for Research in Child Development, 35/1. DOI logoGoogle Scholar
HYMES, D.
(1972) : Towards Communicative Competence. Philadelphia : Pennsylvania University Press.Google Scholar
JONES, R.
(1979) : The oral interview of the Foreign Service Institute. In B. Spolsky (ed.), Advances in Language Testing. Arlington, VA: Center for Applied Linguistics, 104–115.Google Scholar
LENNON, P.
(1990) : Investigating fluency in EFL:a quantitative approach. Language Learning 40/3 : 387–417. DOI logoGoogle Scholar
(1991) : Error : some problems of definition, identification and distinction. Applied Linguistics 12/2:180–196. DOI logoGoogle Scholar
MCLAUGHLIN, B.
(1990) : Restructuring. Applied Linguistics 11/2: 113–128. DOI logoGoogle Scholar
MÖHLE, D.
(1984) : A comparison of the second language speech production of different native speakers. In H.-W. Dechert, D. Möhle, & M. Raupach, (eds.), Second Language Productions. Tübingen, FRG : Gunter Narr Verlag, 26–49.Google Scholar
PALMER, A.
(1979) : Compartmentalised and integrated control : an assessment of some evidence for two kinds of competence and implications for the classroom. Language Learning, 291:169–180. DOI logoGoogle Scholar
PAWLEY, A. & SYDER, F.
(1983) : Two puzzles for linguistic theory : nativelike selection and nativelike fluency. In Richards, J. & Schmidt, R. (eds.), Language and Communication. London: Longman, 191–226.Google Scholar
QUIRK, R & GREENBAUM, S.
(1973) : A University Grammar of English. London : Longman.Google Scholar
SCHLEY, S. & SNOW, C.
(1989) : The conversational skills of school-aged children. Unpublished MS, Harvard Graduate School of Education, Cambridge, MA 02138.Google Scholar
SCHMIDT, R.
(1992)  Psychological mechanisms underlying second language fluency. Studies in Second Language Acquisition 14/4: 357–386. DOI logoGoogle Scholar
SELINKER, L. & DOUGLAS, D.
(1989) : Research methodology in contextually-based second language research. Second Language Research 51:93–126. DOI logoGoogle Scholar
SNOW, C.
(1987) : Beyond conversation : second language learners' acquisition of description and explanation. In J. Lantolf & A. Labarca (eds.), Research in Second Language Learning : Focus on the Classroom. Norwood, NJ: Ablex, 3–16.Google Scholar
SNOW, C. & DOLBEAR, M.
(1989) : The relation of conversational skill to language proficiency in second language learners. Unpublished MS, Harvard Graduate School of Education, Cambridge, MA 02138.Google Scholar
SVARTVIK, J.
(1966) : On Voice in The English Verb. The Hague:Mouton.Google Scholar
SWEET, H.
(1899/1964) : The Practical Study of Languages. London : Dent. (Republished 1964, Oxford : The University Press.)Google Scholar
TARONE, E.
(1983) : On the variability of interlanguage systems. Applied Linguistics 4/2 : 142–164. DOI logoGoogle Scholar
TARONE, E. & PARRISH, B.
(1988) : Task-related variation in interlanguage:the case of articles. Language Learning 381 : 21–44. DOI logoGoogle Scholar
THOMAS, M. H.
(1983) : 'Did I say something wrong?' English Teaching Forum 211:35–37.Google Scholar
TIMMS, W. & ECCOTT, A.
(1972) : International Picture Stories. London : Hodder and Stoughton.Google Scholar
VALDMAN, A.
(ed.) (1988) : Thematic issue on the evaluation of foreign language proficiency. Studies in Second Language Acquisition, 10/2.Google Scholar
VORSTER, J.
(1980) : Manual for the Test of Oral Language Production (T.O.L.P.). Pretoria : South African Human Sciences Research Council.Google Scholar
Cited by

Cited by 3 other publications

Bulté, Bram & Alex Housen
2014. Conceptualizing and measuring short-term changes in L2 writing complexity. Journal of Second Language Writing 26 ► pp. 42 ff. DOI logo
Dewey, Dan P., Jennifer Bown & Dennis Eggett
2012. Japanese Language Proficiency, Social Networking, and Language Use during Study Abroad: Learners’ Perspectives. The Canadian Modern Language Review 68:2 ► pp. 111 ff. DOI logo
Wang, Yu & Shoucun Tao
2020. Chapter 3. The dynamic co-development of linguistic and discourse-semantic complexity in advanced L2 writing. In Complex Dynamic Systems Theory and L2 Writing Development [Language Learning & Language Teaching, 54], ► pp. 49 ff. DOI logo

This list is based on CrossRef data as of 2 may 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.