Chapter 8
Perception and production of language in the visual modality
Implications for sign language development
Sign language acquisition requires learning how to comprehend and produce a linguistic system that is visual in nature, as opposed to spoken language acquisition which uses the auditory-visual modality. In this chapter, we consider the impact this has for a child acquiring a sign language. We summarize the research literature on sign language production and comprehension, and attempt to integrate psycholinguistic studies with work documenting the visual perceptual abilities of deaf children. While much of this research emphasizes the experience-dependent nature of language processing abilities, reinforcing the importance of early exposure for native-like acquisition, we caution against overgeneralizing from studies of adult processing and call for more child-specific language studies related to comprehension and production within varying acquisition environments.
Article outline
- Overview
- Seeing signs during development
- Visual perception of sign language
- Production of a visual language
- Summary
-
Note
-
References
References (86)
References
Agrafiotis, D., Canagarajah, N., Bull, D. R., & Dye, M. (2003). Perceptually optimised sign language video coding based on eye tracking analysis. Electronics Letters 39(24), 1703–1705.
Agrafiotis, D., Canagarajah, N., Bull, D. R., Kyle, J., Seers, H., & Dye, M. (2006). A perceptually optimised video coding system for sign language communication at low bit rates. Signal Processing: Image Communication 21(7), 531–549.
Baker-Shenk, C. (1983). A micro-analysis of the non-manual components of questions in American Sign Language. Berkeley, CA: University of California.
Battison, R. (1978). Lexical borrowing in American Sign Language. Silver Spring, MD: Linstok Press.
Bellugi, U., & Klima, E. (1991). What the hands reveal about the brain. In D. S. Martin (Ed.), Advances in cognition, education and deafness. Washington, DC: Gallaudet University Press.
Bosworth, R. G., & Dobkins, K. R. (2002). Visual field asymmetries for motion processing in deaf and hearing signers. Brain & Cognition 49, 170–181.
Boudreault, P., & Mayberry, R. L. (2006). Grammatical processing in American Sign Language: Age of first-language acquisition effects in relation to syntactic structure. Language & Cognitive Processes 21(5), 608–635.
Boyes-Braem, P. (1990). The acquisition of handshapes in American Sign Language. In V. Volterra, & C. Erting (Eds), From gesture to language in deaf and hearing children. New York, NY: Springer.
Caselli M. C., & Volterra, V. (1994). From communication to language in hearing and deaf children. In V. Volterra, & C. Erting (Eds), From gesture to language in hearing and deaf children (pp.263–278). New York, NY: Springer.
Caselli, N. K., & Pyers, J. E. (2017). The road to language learning is not entirely iconic: Iconicity, neighborhood density, and frequency facilitate sign language acquisition. Psychological Science 28(7), 979–987.
Caselli, N. K., Sehyr, Z. S., Cohen-Goldberg A. M., & Emmorey, K. (2017). ASL-LEX: A lexical database of American Sign Language. Behaviour Research Methods 49(2):784–801. .
Conlin, K. E., Mirus, G. R., Mauk, C., & Meier, R. P. (2000). The acquisition of first signs: Place, handshape, and movement. In C. Chamberlain, J. P. Morford, & R. I. Mayberry (Eds.), Language acquisition by eye (pp.51–70). Mahwah, NJ: Lawrence Erlbaum Associates.
Crowson, K. (1994). Errors made by deaf children acquiring sign language. Early Child Development & Care 99, 63–78.
Dingemanse, M., Blasi, D. E., Lupyan, G., Christiansen, M. H., & Monaghan, P. (2015). Arbitrariness, iconicity, and systematicity in language. Trends in Cognitive Sciences 19(10), 603–615.
Dye, M. W. G., Hauser, P. C., & Bavelier, D. (2009). Is visual attention in deaf individuals enhanced or deficient? The case of the Useful Field of View. PLoS One 4(5), e5640.
Dye, M. W. G., & Hauser, P. C. (2014). Sustained attention, selective attention and cognitive control in deaf and hearing children. Hearing Research 309, 94–102.
Dye, M. W. G. (2016). Response bias reveals enhanced attention to inferior visual field in signers of American Sign Language. Experimental Brain Research 234(4), 1067–1076.
Emmorey, K. (2002). Language, cognition, and the brain: Insights from sign language research. Mahwah, NJ: Lawrence Erlbaum Associates.
Emmorey, K. (2000). The signs of language revisited: An anthology to honor Ursula Bellugi and Edward Klima. Mahwah, NJ: Lawrence Erlbaum Associates.
Emmorey, K., Bosworth, R., & Kraljic, T. (2009). Visual feedback and self-monitoring of sign language. Journal of Memory & Language 61(3), 398–411.
Emmorey K., Gertsberg, N., Korpics, F., & Wright, C. E. (2009). The influence of visual feedback and register changes on sign language production: A kinematic study with deaf signers. Applied Psycholinguistics 30, 187–203.
Emmorey, K., Korpics, F., & Petronio, K. (2009). The use of visual feedback during signing: Evidence from signers with impaired vision. The Journal of Deaf Studies & Deaf Education 14(1), 99–104.
Emmorey, K., Thompson, R. L., & Colvin, R. (2009). Eye gaze during comprehension of American Sign Language by native and beginning signers. Journal of Deaf Studies and Deaf Education 14(2), 237–243.
Feng, J., & Spence, I. (2014). Upper visual field advantage in localizing a target among distractors. i-Perception 5, 97–100.
Fenlon, J., Cormier, K., & Brentari, D. (2017). The phonology of sign languages. In S. J. Hannahs, & A. Bosch (Eds.), The Routledge handbook of phonological theory. London: Routledge.
Fenlon, J., Cormier, K., Rentelis, R., Schembri, A., Rowley, K., Adam, R., & Woll, B. (2014). BSL SignBank: A lexical database of British Sign Language (first ed.). London: Deafness, Cognition and Language Research Centre, University College London.
Frishberg, N. (1975). Arbitrariness and iconicity: Historical change in American Sign Language. Language 51(3), 696–719.
Gentner, D. (2006). Why verbs are hard to learn. In K. A. Hirsh-Pasek & R. M. Golinkoff (Eds.), Action meets word: How children learn verbs (pp.544–564). New York, NY: Oxford University Press.
Hillairet de Boisferon, A., Tift, A. H., Minar, N. J, & Lewkowicz, D. J. (2015). Selective attention to a talker’s mouth in infancy: role of audiovisual temporal synchrony and linguistic experience. Developmental Science 20(3), e12381.
Hoffman, M., Tiddens, E., Quittner, A. L. and the CDaCI Investigative Team. (2018). Comparisons of visual attention in school-age children with cochlear implants versus hearing peers and normative data. Hearing Research 359, 91–100.
Hoffmeister, R., & Wilbur, R. (1980). The acquisition of sign language. In H. Lane & F. Grosjean (Eds.), Recent perspectives on American Sign Language. Hillsdale, NJ: Lawrence Erlbaum Associates.
Hohenberger, A., Happ, D., & Leuniger, H. (2002). Modality-dependent aspects of sign language production: Evidence from slips of the hands and their repairs in German Sign Language. In R. P. Meier, K. A. Cormier, & D. Quinto-Pozos (Eds.), Modality and structure in signed and spoken language (pp.112–142). Cambridge: Cambridge University Press.
Höhle, B., Bijeljac-Babic, R., Herold, B., Weissenborn, J., & Nazzi, T. (2009). Language specific prosodic preferences during the first year of life: Evidence from German and French infants. Infant Behavioural Development 32, 262–274.
Kyle, J. G., & Woll, B. (1985). Sign language: The study of deaf people and their language. New York, NY: Cambridge University Press.
Levelt, P. (1989). Speaking: From intention to articulation. Cambridge, MA: The MIT Press.
Lewkowicz, D. J., & Hansen-Tift, A. M. (2012). Infants deploy selective attention to the mouth of a talking face when learning speech. Proceedings of the National Academy of Sciences 109(5), 1431–1436.
Lu, J., & Thompson, R. L. (2017). Different word learning contexts modulate the learnability of iconic signs. Paper presented at the 8th Conference of the International Society for Gesture Studies – Gesture and Diversity. Cape Town.
Marentette, P. F., & Mayberry, R. I. (2000). Principles for an emerging phonological system: A case study of early ASL acquisition. In C. Chamberlain, J. P. Morford, & R. I. Mayberry (Eds), Language Acquisition by Eye (pp.71–90). Mahwah, NJ: Lawrence Erlbaum Associates.
Marschark, M., Siple, P., Lillo-Martin, D., Campbell, R., & Everhart, V. S. (1997). Relations of language and thought: The view from sign language and deaf children. New York, NY: Oxford University Press.
Marschark, M., & Hauser, P. C. (Eds.). (2008). Deaf cognition: foundations and outcomes. New York, NY: Oxford University Press.
Mayberry, R. (2007). When timing is everything: Age of first-language acquisition effects on second-language learning. Applied Psycholinguistics 28(3), 537–549.
McIntire, M. L. (1977). The acquisition of American Sign Language hand configurations. Sign Language Studies 16(1), 247–266.
Meier, R. (2006). The form of early signs: explaining signing children’s articulatory development. In B. Schick, M. Marschark, & P. E. Spencer (Eds), Advances in the sign language development of deaf children (pp.202–231). New York, NY: Oxford University Press.
Meier R. P., Cormier K., & Quinto-Pozos D. (2002). Modality and structure in signed and spoken languages. Cambridge: Cambridge University Press.
Meier, R. P., Mauk, C. E., Cheek, A., & Moreland, C. J. (2008). The form of children’s early signs: Iconic or motoric determinants. Language Learning & Development 4(1), 63–98.
Morford, J. P. (2003). Grammatical development in adolescent first language learners. Linguistics 41(4), 681–721.
Morford, J. P., & Carlson, M. L. (2011). Sign perception and recognition in non-native signers of ASL. Language Learning & Development 7(2), 149–168.
Morgan, G., Barrett-Jones, S., & Stoneham, H. (2007). The first signs of language: Phonological development in British Sign Language. Applied Psycholinguistics 28, 3–22.
Muir, L. J., & Richardson, I. E. (2005). Perception of sign language and its application to visual communications for deaf people. Journal of Deaf Studies & Deaf Education 10(4), 390–401.
Newkirk, D., Klima, E., Pedersen, C. C., & Bellugi, U. (1980). Linguistic evidence from slips of the hand. In V. Fromkin (Ed.), Errors in linguistic performance: slips of the tongue, ear, pen, and hand (pp.165–197). New York, NY: Academic Press.
Newport, E. L., & Meier, R. P. (1985). The acquisition of American Sign Language. In D. I. Slobin (Ed.), The cross linguistic study of language acquisition, Volume 1: The data. Hillsdale, NJ: Lawrence Erlbaum Associates.
Ortega, G. (2017). Iconicity and sign lexical acquisition: A review. Frontiers in Psychology 8, 1280.
Ortega, G., & Morgan, G. (2015). Phonological development in hearing learners of a sign language: The role of sign complexity and iconicity. Language Learning 65(3), 660–668.
Ortega, G., Sumer, B., & Ozyurek, A. (2017). Type of iconicity matters in the vocabulary development of signing children. Developmental Psychology 53(1), 89–99.
Perniss, P., Lu, J. C., Morgan, G., & Vigliocco, G. (2018). Mapping language to the world: The role of iconicity in the sign language input. Developmental Science 21(2), e12551.
Perniss, P., Thompson, R. L., & Vigliocco, G. (2010). Iconicity as a general property of language: Evidence from spoken and signed languages. Frontiers in Psychology 1, 227.
Perry, L. K., Perlman, M., Winter, B., Massaro, D. W., & Lupyan, G. (2018). Iconicity in the speech of children and adults. Developmental Science 21(3), e12572.
Petitto, L. A., & Marentette, P. (1991). Babbling in the manual mode: Evidence for the ontogeny of language. Science 251, 1483–1496.
Pizer, G., Meier, R. P., & Shaw Points, K. (2011). Child-directed signing as a linguistic register. In R. Channon & H. van der Hulst (Eds.), Formational units in sign languages (pp.65–86). Berlin: Walter de Gruyter.
Reilly, J. S., McIntire, M. L., & Seago, H. (1992). Affective prosody in American Sign Language. Sign Language Studies 75, 113–128.
Ross, D. S., & Newport, E. L. (1996). The development of language from non-native linguistic input. In A. Stringfellow, D. Cahana-Amitay, E. Hughs, & A. Zukowski (Eds.), Proceedings of the 20th annual Boston University Conference on Language Development (volume 2, pp.634–645). Somerville, MA: Cascadilla Press.
Sandler, W., & Lillo-Martin, D. (2006). Sign language and linguistic universals. Cambridge: Cambridge University Press.
Schick, B. (1990). The effects of morphosyntactic structure on the acquisition of classifier predicates in American Sign Language. In C. Lucas (Ed.), Sign language research: Theoretical issues. Washington, DC: Gallaudet University Press.
Schlenker, P. (2018). Visible meaning: Sign language and the foundations of semantics. Theoretical Linguistics 44(3–4): 123–208.
Shield, A., & Meier, R. P. (2018). Learning an embodied visual language: Four imitation strategies available to sign learners. Frontiers in Psychology 9, 811.
Siedlecki, T., & Bonvillian, J. D. (1993). Location, handshape, and movement: Young children’s acquisition of the formational aspects of American Sign Language. Sign Language Studies 78, 31–52.
Singleton, J. L., & Newport, E. L. (2004). When learners surpass their models: The acquisition of American Sign Language from inconsistent input. Cognitive Psychology 49(4), 370–407.
Siple, P. (1978). Visual constraints for sign language communication. Sign Language Studies 19, 95–110.
Smith, L. B., Quittner, A. L., Osberger, M. J., & Miyamoto R. (1998). Audition and visual attention: The developmental trajectory in deaf and hearing populations. Developmental Psychology 34(5), 840–850.
Steinberg, L. (2005). Cognitive and affective development in adolescence. Trends in Cognitive Sciences 9(2), 69–74.
Stoll, C., & Dye, M. W. G. (2019). Sign language experience redistributes attentional resources to the inferior visual field. Cognition 191, 103957. doi:
Stone, A., Petitto L., & Bosworth, R. (2018). Visual sonority modulates infants’ attraction to sign language. Language Learning & Development 14(2), 130–148.
Swisher, M. V., Christie, K., & Miller, S. L. (1989). The reception of signs in peripheral vision by deaf persons. Sign Language Studies 63, 99–125.
Tharpe, A. M., Ashmead, D., Sladen, D. P., Ryan, H. A., & Rothpletz, A. M. (2008). Visual attention and hearing loss: Past and current perspectives. Journal of the American Academy of Audiology 19(10), 741–747.
Thompson, R. L. (2011). Iconicity in language processing and acquisition: What signed languages reveal. Language & Linguistics Compass 5, 603–616.
Thompson, R. L, Emmorey, K., & Gollan, T. H. (2005). “Tip of the fingers” experiences by deaf signers. Psychological Science 16(11), 856–860.
Thompson, R. L., Vinson, D. P., & Vigliocco, G. (2009). The link between form and meaning in American Sign Language: Lexical processing effects in a phonological decision task. Journal of Experimental Psychology: Language, Memory, & Cognition 35(2), 550–557.
Thompson, R. L., Vinson, D. P., Woll, B., & Vigliocco, G. (2012). The road to language learning is iconic: Evidence from British Sign Language. Psychological Science 23, 1443–1448.
Vinson, D. P., Thompson, R. L., Skinner, R., & Vigliocco, G. (2015). A faster path between meaning and form? Iconicity facilitates sign recognition and production in British Sign Language. Journal of Memory & Language 82, 56–85.
Vinson, D. P., Thompson, R. L., Skinner, R., Fox, N., & Vigliocco, G. (2010). The hands and mouth do not always slip together in British Sign Language: Dissociating articulatory channels in the lexicon. Psychological Science 21(8), 1158–1167.
Watkins, F., & Thompson, R. L. (2017). The relationship between sign production and sign comprehension: What handedness reveals. Cognition 164, 144–149.
Waxman, S., Fu, X., Arunachalam, S., Leddon, E., Geraghty, K., & Song, H. J. (2013). Are nouns learned before verbs? Infants provide insight into a longstanding debate. Child Development Perspectives 7(3), 155–159.
Werker, J. F., & Tees, R. C. (1984). Cross-language speech perception: Evidence for perceptual reorganization during the first year of life. Infant Behavioural Development 7, 49–63.
Woll, B. (1987). Historical and comparative aspects of BSL. In J. G. Kyle (Ed.), Sign and school (pp.12–34). Clevedon: Multilingual Matters.
Woodward, J. C. (1976). Signs of change: historical variation in American Sign Language. Sign Language Studies 10, 81–94.
Cited by (2)
Cited by two other publications
Bosworth, Rain G. & Adam Stone
2021.
Rapid development of perceptual gaze control in hearing native signing Infants and children.
Developmental Science 24:4
This list is based on CrossRef data as of 7 november 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.