The impact of visible lip movements on silent pauses in simultaneous interpreting
Simultaneous interpreting requires interpreters to listen to a source text while producing the target text in a
second language. In addition, the interpreter needs to process various types of visual input, which may further increase the
already high cognitive load. A study with 14 students of interpreting was conducted to investigate the impact of a speaker’s
visible lip movements on cognitive load in simultaneous interpreting by analysing the duration of silent pauses in the target
texts. Background noise masking the source speech was introduced as a control condition for cognitive load. Silent pause durations
were shorter when interpreters saw the speaker’s lip movements, which indicates that interpreters benefitted from visual input.
Furthermore, silent pause durations were longer with noise, which suggests that comparative silent pause durations can indicate
changes in cognitive load.
Article outline
- 1.Introduction
- 2.Theoretical background
- 2.1Visual input in simultaneous interpreting
- 2.2Manipulating speech perception with background noise
- 2.3Disfluencies as an indicator of cognitive load
- 3.Empirical study
- 3.1Participants
- 3.2Material
- 3.3Procedure
- 3.4Data analysis
- 3.4.1Subjective reports
- 3.4.2Silent pause durations
- 3.5Results
- 3.5.1Subjective reports
- 3.5.2Silence durations
- 4.Discussion
- 4.1Limitations of the study
- 4.2Potential and limitations of silent pauses in the target text as cognitive load indicator
- 5.Conclusion
- Acknowledgements
- Notes
-
References
References (65)
References
Ahrens, B. (2004). Prosodie
beim Simultandolmetschen. Frankfurt am Main: Peter Lang.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Albl-Mikasa, M. (2010). Global
English and English as a lingua franca (ELF): Implications for the interpreting
profession. Trans-kom 3 (3), 126–148.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Bates, D., Mächler, M., Bolker, B. & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software 67 (1). ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Benoit, C., Mohammadi, T. & Kandel, S. (1994). Effects of phonetic context on audio-visual intelligibility of French. Journal of Speech and Hearing Research 37 (5), 1195–1203. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Bernstein, L. E., Auer, E. T. & Takayanagi, S. (2004). Auditory
speech detection in noise enhanced by lipreading. Speech
Communication 44 (1–4), 5–18. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Brancazio, L., Best, C. T. & Fowler, C. A. (2006). Visual
influences on perception of speech and nonspeech vocal-tract events. Language and
Speech 49 (1), 21–53. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Bühler, H. (1985). Conference
interpreting: A multichannel communication
phenomenon. Meta 30 (1), 49–54. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Bühler, H. (1986). Linguistic
(semantic) and extra-linguistic (pragmatic) criteria for the evaluation of conference interpretation and
interpreters. Multilingua 5 (4), 231–235.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Cecot, M. (2001). Pauses
in simultaneous interpretation: A contrastive analysis of professional interpreters’
performances. The Interpreters’
Newsletter 111, 63–85.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Chiaro, D. & Nocella, G. (2004). Interpreters’
perception of linguistic and non-linguistic factors affecting quality: A survey through the World Wide
Web. Meta 49 (2), 278–293. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Chmiel, A., Szarkowska, A., Koržinek, D., Lijewska, A., Dutka, Ł., Brocki, Ł. & Marasek, K. (2017). Ear–voice
span and pauses in intra- and interlingual respeaking: An exploratory study into temporal aspects of the respeaking
process. Applied
Psycholinguistics 38 (5), 1201–1227. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Davies, M. (2008). Word frequency data. Retrieved from The Corpus of Contemporary American English (COCA): [URL] (accessed 19 March 2021).![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Euopean Commission. (2009a). United
Airlines rewards fittest people. Retrieved from Speech Repository: [URL] (accessed 4 February 2021).
European
Commission. (2009b). Disenchantment at
work. Retrieved from Speech Repository: [URL] (accessed 4 February 2021).
European
Commission. (2012a). Demographic shift in
Europe. Retrieved from Speech Repository: [URL] (accessed 4 February 2021).
European Commission. (2012b). Greece
in the doldrums. Retrieved from Speech Repository: [URL] (accessed 4 February 2021).
Fox, J. & Weisberg, S. (2018). Visualizing fit and lack of fit in complex regression models with predictor effect plots and partial residuals. Journal of Statistical Software 87 (9), 1–27. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Gerver, D. (1974). The
effects of noise on the performance of simultaneous interpreters: Accuracy of performance. Acta
Psychologica 38 (3), 159–167. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Gerver, D. (1975). A
psychological approach to simultaneous
interpretation. Meta 20 (2), 119–128. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Gerver, D. (2002). The
effects of source language presentation rate on the performance of simultaneous conference
interpreters. In F. Pöchhacker & M. Shlesinger (Eds.), The
interpreting studies reader. London/New York: Routledge, 53–66.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Gieshoff, A. C. (2018). The
impact of audio-visual speech on work-load in simultaneous interpreting. Doctoral
thesis, University of Mainz.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Goldman-Eisler, F. (1958). Speech
analysis and mental processes. Language and
Speech 1 (1), 59–75. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Goldman-Eisler, F. (1961). The
distribution of pause durations in speech. Language and
Speech 4 (4), 232–237. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Goldman-Eisler, F. (1968). Psycholinguistics:
Experiments in spontaneous speech. London/New York: Academic Press.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Goldman-Eisler, F. (2002). Segmentation
of input in simultaneous translation. In F. Pöchhacker & M. Shlesinger (Eds.), The
interpreting studies reader. London/New York: Routledge, 69–76.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
I-hsin, I. L., Feng-lan, A. C. & Feng-lan, K. (2013). The
impact of non-native accented English on rendition accuracy in simultaneous
interpreting. Translation &
Interpreting 5 (2), 30–44.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kramer, S. E., Kapteyn, T. S., Festen, J. M. & Kuik, D. J. (1997). Assessing
aspects of auditory handicap by means of pupil
dilation. Audiology 361, 155–164. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Lin, Y., Lv, Q. & Liang, J. (2018). Predicting
fluency with language proficiency, working memory, and directionality in simultaneous
interpreting. Frontiers in
Psychology 91: 1543. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Lo, S. & Andrews, S. (2015). To
transform or not to transform: Using generalized linear mixed models to analyse reaction time
data. Frontiers in
Psychology 61: 1171. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Macleod, A. & Summerfield, Q. (1987). Quantifying
the contribution of vision to speech perception in noise. British Journal of
Audiology 21 (2), 131–141. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Massaro, D. W. & Cohen, M. M. (1999). Speech
perception in perceivers with hearing loss: Synergy of multiple modalities. Journal of Speech,
Language, and Hearing
Research 42 (1), 21–41. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Mattys, S. L. & Wiget, L. (2011). Effects
of cognitive load on speech recognition. Journal of Memory and
Language 65 (2), 145–160. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Mattys, S. L., Brooks, J. & Cooke, M. (2009). Recognizing
speech under a processing load: Dissociating energetic from informational factors. Cognitive
Psychology 59 (1), 203–243. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Mizuno, A. (2005). Process
model for simultaneous interpreting and working
memory. Meta 50 (2), 739–752. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Moser, B. (1978). Simultaneous
interpretation: A hypothetical model and its practical
application. In D. Gerver & H. W. Sinaiko (Eds.), Language
interpretation and communication. New York: Plenum Press, 353–368. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Moser-Mercer, B. (2003). Remote
interpreting: Assessment of human factors and performance parameters. Communicate! AIIC
Webzine (Summer 2003). [URL] (accessed 16 March 2020).
Moser-Mercer, B. (2005). Remote
interpreting: The crucial role of
presence. VALS-ASLA 811, 73–97.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Peirce, J. W. (2007). PsychoPy-Psychophysics software in Python. Journal of Neuroscience Methods 162 (1/2), 8–13. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Pöchhacker, F. (2005). From
operation to action: Process-orientation in interpreting
studies. Meta 50 (2), 682–695. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Poyatos, F. (1984). The
multichannel reality of discourse: Language-paralanguage-kinesics and the totality of communicative
systems. Language
Sciences 6 (2), 307–337. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Rackow, J. (2013). Dolmetschen
als Kommunikation: Verbale und nonverbale Informationsverarbeitung im
Dolmetschprozess. In D. Andres, M. Behr & M. Dingfelder Stone (Eds.), Dolmetschmodelle –
erfasst, erläutert, erweitert. Frankfurt am Main: Peter Lang, 129–152.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Rennert, S. (2008). Visual
input in simultaneous
interpreting. Meta 53 (1), 204–217. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Rennert, S. (2019). Redeflüssigkeit
und Dolmetschqualität: Wirkung und
Bewertung. Tübingen: Narr.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Seeber, K. G. (2017). Multimodal
processing in simultaneous interpreting. In J. W. Schwieter & A. Ferreira (Eds.), The
handbook of translation and
cognition (pp. 461–475). Hoboken: John Wiley & Sons, 461–475. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Seubert, S. (2017). Simultaneous
interpreting is a whole-person process: Zur Verarbeitung visueller Informationen beim
Simultandolmetschen. In M. Behr & S. Seubert (Eds.), Education
is a whole-person process: Von ganzheitlicher Lehre, Dolmetschforschung und anderen
Dingen. Berlin: Frank & Timme, 271–303.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Thomas, S. M. & Jordan, T. R. (2004). Contributions
of oral and extraoral facial movement to visual and audiovisual speech perception. Journal of
Experimental Psychology: Human Perception and
Performance 30 (5), 873–888. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Tissi, B. (2000). Silent
pauses and disfluencies in simultaneous interpretation: A descriptive analysis. The
Interpreters’
Newsletter 101, 103–128.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Vatikiotis-Bateson, E., Eigsti, I.-M., Yano, S. & Munhall, K. G. (1998). Eye
movement of perceivers during audiovisual speech perception. Perception &
Psychophysics 60 (6), 926–940. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
von Kriegstein, K., Dogan, Ö., Grüter, M., Giraud, A.-L., Kell, C. A., Grüter, T., Kleinschmidt, A. & Kiebel, S. J. (2008). Simulation
of talking faces in the human brain improves auditory speech recognition. Proceedings of the
National Academy of
Sciences 105 (18), 6747–6752. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Wickham, H. (2009). ggplot2: Elegant graphics for data analysis. New York: Springer. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Zwischenberger, C. (2010). Quality
criteria in simultaneous interpreting: An international vs. a national view. The Interpreters’
Newsletter 151, 127–142.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Cited by (1)
Cited by one other publication
This list is based on CrossRef data as of 4 july 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.