Cover not available
Part of
Linguistic Foundations of Narration in Spoken and Sign Languages
Edited by Annika Hübl and Markus Steinbach
[Linguistik Aktuell/Linguistics Today 247] 2018
► pp. 15–40

A new technique for analyzing narrative prosodic effects in sign languages using motion capture technology

 | Purdue University
 | Purdue University
The present paper addresses prosody at a sentence level analysis within short narratives, developing a novel method based on a combination of data. Our first objective, then, is to establish the validity of our new method by confirming the findings of previous reports on Phrase Final Lengthening. Our second objective is to further extend the validity of the new method by confirming a prior claim regarding the role of increased velocity for signs that are stressed in sentences. Finally, we report new results in the form of patterns across signs within sentences using our new method.
Article outline
  • 1.Introduction
    • 1.1Perceptual studies of prosody and fluency
    • 1.2Production studies of fluency and prosody
    • 1.3Prior kinematic work
      • 1.3.1Prior motion capture work showing sign lowering
      • 1.3.2Prior motion capture work showing verb class differences
      • 1.3.3Prior motion capture work showing phrase final lengthening
      • 1.3.4Prior motion capture work showing stress marking
  • 2.The present study
    • 2.1Procedure
      • 2.1.1Motion capture method for current report
      • 2.1.2Coding for non-kinematic linguistic and perceptual variables
      • 2.1.3Coding for kinematic variables
    • 2.2Analyses
    • 2.3Results
      • 2.3.1Kinematic analysis by sentence position
      • 2.3.2Kinematic analysis of stress
      • 2.3.3More general test of relationships among measured variables and weight
    • 2.4Summary
  • 3.Conclusion
  • Acknowledgements
  • Notes
  • References
Published online: 25 May 2018
DOI logo
https://doi.org/10.1075/la.247.02wil
References (56)
References
Allen, G. D., Wilbur, R. B. & Schick, B. 1991. Aspects of rhythm in American Sign Language. Sign Language Studies 72: 297–320.DOI logoGoogle Scholar
Boyes Braem, P. 1999. Rhythmic temporal patterns in the signing of deaf early and later learners of Swiss German Sign Language. Language & Speech 42: 177–208.DOI logoGoogle Scholar
Brentari, D.. 1998. A Prosodic Model of Sign Language Phonology. Cambridge MA: The MIT Press.Google Scholar
Brentari, D. & Crossley, L. 2002. Prosody on the hands and face: Evidence from American Sign Language. Sign Language & Linguistics 5: 105–130.DOI logoGoogle Scholar
Brentari, D., Nadolske, M. & Wolford, G. 2012. Can experience with co-speech gesture influence the prosody of a sign language? Sign language prosodic cues in bimodal bilinguals. Bilingualism: Language and Cognition 15(2): 402–412.DOI logoGoogle Scholar
Brentari, D., González, C., Seidl, A. & Wilbur, R. B. 2011. Sensitivity to visual prosodic cues in signers and nonsigners. Language & Speech 54: 49–72.DOI logoGoogle Scholar
González, C. 2011. Perception of Prosody in American Sign Language. PhD dissertation, Purdue University.Google Scholar
Green, K.. 1984. Sign boundaries in American Sign Language. Sign Language Studies 42: 65–91.DOI logoGoogle Scholar
Grosjean, F. & Lane, H. 1977. Pauses and syntax in American Sign Language. Cognition 5: 101–117.DOI logoGoogle Scholar
Grosvald, M. A. 2009. Long-distance Coarticulation: A Production and Perception Study of English and American Sign Language. PhD dissertation, University of California at Davis.Google Scholar
Kantor, R. 1978. Identifying native and second-language signers. In Sign Language Research, R. B. Wilbur (ed), Special issue of Communication and Cognition 11(1): 39–55.Google Scholar
Kröger, B. J., Birkholz, P., Kannampuzha, J., Kaufmann, E. & Mittelberg, I. 2011. Movements and holds in fluent sentence production of American Sign Language: The action-based approach. Cognitive Computation 3(3): 449–465.DOI logoGoogle Scholar
Liddell, S. K. 1977. An Investigation into the Syntactic Structure of American Sign Language. PhD dissertation, University of California, San Diego.Google Scholar
Lupton, L. K.. 1998. Fluency in American Sign Language. Journal of Deaf Studies and Deaf Education 3(4): 320–328.DOI logoGoogle Scholar
Lupton, L. K. & Zelaznik, H. 1990. Motor learning in sign language students. Sign Language Studies 67: 153–174.DOI logoGoogle Scholar
Weast, T. 2008. Questions in American Sign Language: A Quantitative Analysis of Raised and Lowered Eyebrows. Phd dissertation, University of Texas, Arlington, U.S.A.Google Scholar
Malaia, E.. 2014. It still isn’t over: Event boundaries in language and perception. Language and Linguistics Compass 8(3): 89–98.DOI logoGoogle Scholar
. 2017. Current and future methodologies for quantitative analysis of information transfer in sign language and gesture data. Behavioral and Brain Sciences 40.Google Scholar
Malaia, E., Borneman, J. D. & Wilbur, R. B. 2016. Assessment of information content in visual signal: analysis of optical flow fractal complexity. Visual Cognition 24(3): 246–251.DOI logoGoogle Scholar
2017. Information transfer capacity of articulators in American Sign Language. Language and Speech. doi:DOI logo.Google Scholar
Malaia, E., Ranaweera, R., Wilbur R. B. & Talavage, T. M. 2012. Neural representation of event structure in American Sign Language: fMRI comparison of cortical activations in deaf signers and hearing non-signers. Neuroimage 59: 4094–4101.DOI logoGoogle Scholar
Malaia, E. & Wilbur, R. B. 2012a. Kinematic signatures of telic and atelic events in ASL predicates. Language & Speech 55(3): 407–421.DOI logoGoogle Scholar
2012b. Telicity expression in the visual modality. In Telicity, Change, and State: A Cross-categorial View of Event Structure, V. Demonte & L. McNally (eds), 122–136. Oxford: OUP.DOI logoGoogle Scholar
Malaia, E., Wilbur, R. B. & Milković, M. 2013. Kinematic parameters of signed verbs at the morpho-phonology interface. Journal of Speech, Language and Hearing Research 56: 1677–1688.DOI logoGoogle Scholar
Malaia, E., Wilbur, R. B. & Weber-Fox, C. 2013. Event end-point primes the undergoer argument: Neurobiological bases of event structure processing. In Studies in the Composition and Decomposition of Event Predicates, B. Arsenijevic, B. Gehrke & R. Marin (eds), 231–248. Dordrecht: Springer.DOI logoGoogle Scholar
Mauk, C. E. 2003. Undershoot in Two Modalities: Evidence from Fast Speech and Fast Signing. PhD dissertation, University of Texas at Austin.Google Scholar
Mauk, C. E. & Tyrone, M. E. 2008. Sign lowering as phonetic reduction in American Sign Language. In Proceedings of the 2008 International Seminar on Speech Production, R. Sock, S. Fuchs & Y. Laprie (eds), 185–188. <[URL]>Google Scholar
2012. Location in ASL: Insights from phonetic variation. Sign Language & Linguistics 15(1): 128–146.DOI logoGoogle Scholar
McDonald, J., Wolfe, R., Wilbur, R. B., Moncrief, R., Malaia, E., Fujimoto, S., Baowidan, S. & Stec, J. 2016. A new tool to facilitate prosodic analysis of motion capture data and a data-driven technique for the improvement of avatar motion. In Proceedings of the Seventh Workshop on the Representation and Processing of Sign Languages: Corpus Mining (LREC-2016), 153–158, Portorož, Slovenia.Google Scholar
Milković, M. 2011. Verb Classes in Croatian Sign Language (HZJ): Syntactic and Semantic Properties PhD dissertation, University of Zagreb, Croatia.Google Scholar
Nespor, M. & Sandler, W. 1999. Prosody in Israeli Sign Language. Language and Speech 42: 143–176.DOI logoGoogle Scholar
Ormel, E. & Crasborn, O. 2012. Prosodic correlates of sentences in signed languages: A literature review and suggestions for new types of studies. Sign Language Studies 12(2): 279–315.DOI logoGoogle Scholar
Pfeiffer, T. 2013. Documentation of gestures with motion capture. In Body – Language – Communication, C. Müller, A. Cienki, E. Fricke, H. S. Ladewig, D. McNeill & S. Tessendorf (eds), 857–868. Berlin: De Gruyter Mouton.Google Scholar
Pfeiffer, T., Hofmann, F., Hahn, F., Rieser, H. & Röpke, I. 2013. Gesture semantics reconstruction based on motion capturing and complex event processing: A circular shape example. In Proceedings of the SIGDIAL 2013 Conference, Maxine Eskenazi, Michael Strube, Barbara Di Eugenio & Jason D. Williams (eds), 270–279. Metz: ACL. <[URL]>
Russell, K., Wilkinson, E. & Janzen, T. 2011. ASL sign lowering as undershoot: A corpus study. Laboratory Phonology 2(2): 403–422.DOI logoGoogle Scholar
Sandler, W. 2012. Visual prosody. In Sign language, R. Pfau, M. Steinbach & B. Woll (eds), 55–76. Berlin: De Gruyter Mouton.DOI logoGoogle Scholar
Sandler, W. & Lillo-Martin, D. 2006. Sign Language and Linguistic Universals. Cambridge: CUP.DOI logoGoogle Scholar
Tang, G., Brentari, D., González, C., & Sze, F. 2010. Crosslinguistic variation in prosodic cues. In Sign languages, D. Brentari (ed), 519–542. Cambridge: CUP.DOI logoGoogle Scholar
Tyrone, M. E. 2007. Simultaneity in atypical signers – Implications for the structure of signed language. In Simultaneity in Signed Language: Form and Function [Current Issues in Linguistic Theory 281], M. Vermeerbergen, L. Leeson & O. Crasborn (eds), 317–335. Amsterdam: John Benjamins.DOI logoGoogle Scholar
Tyrone, M. E., Nam, H., Saltzman, E., Mathur, G. & Goldstein, L. 2010. Prosody and movement in American Sign Language: A task-dynamics approach. In Speech Prosody 2010 Conference Proceedings. <[URL]>
Tyrone, M. E., Atkinson, J. R., Marshall, J. & Woll, B. 2009. The effects of cerebellar ataxia on sign language production: A case study. Neurocase 15(5): 419–426.DOI logoGoogle Scholar
Tyrone, M. E. & Mauk, C. E. 2010. Sign lowering and phonetic reduction in American Sign Language. Journal of Phonetics 38(2): 317–328.DOI logoGoogle Scholar
Malaia, E., Wilbur, R. B. 2012. Kinematic signatures of telic and atelic events in ASL predicates. Language and Speech 55(3): 407–421. http://dx.doi.org/10.1177/0023830911422201Google Scholar
Wilbur, R. B.. 1990. An experimental investigation of stressed sign production. International Journal of Sign Linguistics 1(1): 41–59.Google Scholar
. 1994. Eyeblinks and ASL phrase structure. Sign Language Studies 84: 221–240.DOI logoGoogle Scholar
1997. A prosodic/pragmatic explanation for word order variation in ASL with typological implications. In Lexical and Syntactic Constructions and the Construction of Meaning, Vol. 1 [Current Issues in Linguistic Theory 150], M. Verspoor, K. D. Lee & E. Sweetser (eds) 89–104. Amsterdam: John Benjamins.DOI logoGoogle Scholar
1999. Stress in ASL: Empirical evidence and linguistic issues. Language & Speech 42: 229–250.DOI logoGoogle Scholar
2000. Phonological and prosodic layering of non-manuals in American Sign Language. In The Signs of Language Revisited: Festschrift for Ursula Bellugi and Edward Klima, H. Lane & K. Emmorey (eds), 213–241. Hillsdale NJ: Lawrence Erlbaum Associates.Google Scholar
2009. Effects of varying rate of signing on ASL manual signs and non-manual markers. Language & Speech 52(2–3): 245–285.DOI logoGoogle Scholar
2010. The semantics-phonology interface. In Sign languages, D. Brentari (ed), 357–382. Cambridge: CUP.DOI logoGoogle Scholar
2011a. Nonmanuals, semantic operators, domain marking, and the solution to two outstanding puzzles in ASL. Sign Language & Linguistics 14: 148–178.DOI logoGoogle Scholar
2011b. Sign syllables. In The Blackwell Companion to Phonology, M. van Oostendorp, C. J. Ewen, E. Hume & K. Rice (eds), 1309–1334. Oxford: Blackwell.DOI logoGoogle Scholar
Wilbur, R. B. & Patschke, C. 1998. Body leans and marking contrast in ASL. Journal of Pragmatics 30: 275–303.DOI logoGoogle Scholar
Wilbur, R. B. & Malaia, E. 2008. Contributions of sign language research to gesture understanding: What can multimodal computational systems learn from sign language research. International Journal of Semantic Computing 2(1): 5–19.DOI logoGoogle Scholar
Wilbur, R. B. & Schick, B. S. 1987. The effects of linguistic stress on ASL signs. Language & Speech 30: 301–323.DOI logoGoogle Scholar
Wilbur, R. B. 2003. Representations of telicity in ASL. Chicago Linguistics Society 39, 354–368.Google Scholar
Cited by (5)

Cited by five other publications

Haug, Tobias, Nivja H. de Jong, Franz Holzknecht, Katja Tissi, Sandra Sidler-Miserez, Alessia Battisti, Regula Perrollaz, Sarah Ebling, Sabine Reinhard & Sarah Caminada
2024. Development and validation of a fluency rating scale for Swiss German Sign Language. Frontiers in Education 9 DOI logo
Krebs, Julia, Ronnie B. Wilbur, Dietmar Roehm & Evie A. Malaia
2024. The interaction of syntax, non-manuals, and prosodic cues as potential topic markers in Austrian Sign Language. Sign Language & Linguistics DOI logo
Napoli, Donna Jo & Rachel Sutton-Spence
2024. Torso articulation in sign languages. Sign Language & Linguistics 27:1 ► pp. 1 ff. DOI logo
Catteau, Fanny
2022. Translating sign language poetry into spoken French. Sign Language & Linguistics 25:1 ► pp. 99 ff. DOI logo
Krebs, Julia, Evie Malaia, Ronnie B. Wilbur & Dietmar Roehm
2020. Interaction between topic marking and subject preference strategy in sign language processing. Language, Cognition and Neuroscience 35:4 ► pp. 466 ff. DOI logo

This list is based on CrossRef data as of 18 december 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.