Article published In:
Interpreting: Online-First ArticlesFundamental frequency as an acoustic mirror of interpreters’ cognitive states
The present study proposes the acoustic parameter of fundamental frequency (F0) as an alternative method for gauging interpreters’ cognitive load and fatigue during simultaneous interpreting (SI). The data collected from an English-Chinese SI experiment reported in Shao and Chai (2021) were re-analysed for the purposes of this study. Cognitive load was measured as the number of undelivered information chunks within the ear-voice span (EVS), whereas fatigue was assumed to increase with time. F0 variables were correlated with cognitive load within the EVS of selected sentences and with fatigue in one-minute segments distributed over the entire interpreting performance. The results show that cognitive constructs are associated in various ways with F0-related variables. Higher F0 peaks and a wider F0 range appear to be associated with higher cognitive load in a majority of interpreters. A higher mean F0 and, to a lesser extent, higher F0 peaks are associated with fatigue. The findings suggest that F0 could be used as a promising indicator of cognitive load and fatigue in SI.
Keywords: fundamental frequency, cognitive load, fatigue
Article outline
- 1.Introduction
- 2.Fundamental frequency and its associated cognitive variables
- 3.Cognitive load and fatigue in simultaneous interpreting
- 3.1Cognitive load
- 3.2Fatigue
- 3.3The measurement gap
- 4.Data and methods
- 4.1Data
- 4.2Areas of interest
- 4.3Statistical analysis
- 5.Results
- 5.1Fatigue and F0
- 5.2Cognitive load and F0
- 6.Discussion
- 7.Conclusion
- Note
-
References
References (94)
Aaronson, L. S., Teel, C. S., Cassmeyer, V., Neuberger, G. B., Pallikkathayil, L., Pierce, J., Press, A. N., Williams, P. D. & Wingate, A. (1999). Defining and measuring fatigue. Image – The Journal of Nursing Scholarship
31
(1), 45–50. 

Ackerman, P. L. & Kanfer, R. (2009). Test length and cognitive fatigue: An empirical examination of effects on performance and test-taker reactions. Journal of Experimental Psychology. Applied
15
(2), 163–181. 

AIIC (2023). Schools directory. [URL] (accessed 30 June 2023).
Ayres, P., Lee, J. Y., Paas, F. & Van Merriënboer, J. (2021). The validity of physiological measures to identify differences in intrinsic cognitive load. Frontiers in Psychology
12
1, 702538. 

Behrens, M., Gube, M., Chaabene, H., Prieske, O., Zenon, A., Broscheid, K. C., Schega, L., Husmann, F. & Weippert, M. (2023). Fatigue and human performance: An updated framework. Sports Medicine (Auckland, N.Z.)
53
(1), 7–31. 

Benson, P. (1995). Analysis of the acoustic correlates of stress from an operational aviation emergency. In I. Trancoso & R. Moore (Eds.), Proceedings of the ESCA-NATO Tutorial and Research Workshop on Speech under Stress. Lisbon: INESC, 61–64.
Bjørheim, F., Pavlou, D. G. & Siriwardane, S. C. (2022). Nonlinear fatigue life prediction model based on the theory of the S–N fatigue damage envelope. Fatigue & Fracture of Engineering Material & Structures 45 (5), 1480–1493. 

Boersma, P. & Weenink, D. (2023). Praat: Doing phonetics by computer, version 6.3.09. [URL] (accessed 2 March 2023).
Boos, M., Kobi, M., Elmer, S. & Jäncke, L. (2022). The influence of experience on cognitive load during simultaneous interpretation. Brain and Language
234
1, 105185. 

Bortfeld, H., Leon, S. D., Bloom, J. E., Schober, M. F. & Brennan, S. E. (2001). Disfluency rates in conversation: Effects of age, relationship, topic, role, and gender. Language and Speech
44
(2), 123–147. 

Boyer, S., Paubel, P. V., Ruiz, R., El Yagoubi, R. & Daurat, A. (2018). Human voice as a measure of mental load level. Journal of Speech, Language, and Hearing Research
61
(11), 2722–2734. 

Cañas, J. J., Muñoz-de-Escalona, E. & Morales-Guaman, J. F. (2021). Fundamental frequency as an alternative method for assessing mental fatigue of distance learning teachers. In L. Longo & M. C. Leva (Eds.), Human mental workload: Models and applications. H-WORKLOAD 2021. Cham: Springer, 45–59. 

Carroll, T., Nix, J., Hunter, E., Emerich, K., Titze, I. & Abaza, M. (2006). Objective measurement of vocal fatigue in classical singers: A vocal dosimetry pilot study. Otolaryngology-Head and Neck Surgery
135
(4), 595–602. 

Chen, S. (2017). The construct of cognitive load in interpreting and its measurement. Perspectives
25
(4), 640–657. 

Chevalier, L. & Gile, D. (2015). Interpreting quality: A case study of spontaneous reactions. Forum
13
(1), 1–26. 

Christodoulides, G. (2016). Effects of cognitive load on speech production and perception. PhD dissertation, Université Catholique de Louvain.
Cowan, N. (2000). The magic number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Science
24
1, 87–114. 

De Jong, T. (2010). Cognitive load theory, educational research, and instructional design: Some food for thought. Instructional Science
38
(2), 105–134. 

Evgrafova, K., Evdokimova, V., Skrelin, P. & Chukaeva, T. (2016). Vocal fatigue in voice professionals: Collecting data and acoustic analysis. ExLing 2016: Proceedings of 7th Tutorial and Research Workshop on Experimental Linguistics, 27 June – 2 July 2016, Saint Petersburg, Russia, 59–62.
Gawron, V. J. (2016). Overview of self-reported measures of fatigue. The International Journal of Aviation Psychology
26
(3/4), 120–131. 

Gieshoff, A. C. (2021). Does it help to see the speaker’s lip movements? An investigation of cognitive load and mental effort in simultaneous interpreting. Translation, Cognition & Behavior
4
(1), 1–25. 

Gieshoff, A. C. & Albl-Mikasa, M. (2022). Interpreting accuracy revisited: A refined approach to interpreting performance analysis, Perspectives
32
(2), 210–228. 

Gile, D. (1999). Testing the effort models’ tightrope hypothesis in simultaneous interpreting – a contribution. Hermes
23
1, 153–172.
(2008). Local cognitive load in simultaneous interpreting and its implications for empirical research. Forum
6
(2), 59–77. 

(2009). Basic concepts and models for interpreter and translator training (Rev. ed.). Amsterdam: John Benjamins. 

(2021). The effort models of interpreting as a didactic construct. In R. Muñoz Martín, S. Sun & D. Li (Eds.), Advances in cognitive translation studies. Singapore: Springer, 139–160. 

Grassmann, M., Vlemincx, E., von Leupoldt, A., Mittelstädt, J. M. & Van den Bergh, O. (2016). Respiratory changes in response to cognitive load: A systematic review. Neural Plasticity 2016, 8146809. 

Griffin, G. R. & Williams, C. E. (1987). The effects of different levels of task complexity on three vocal measures. Aviation, Space, and Environmental Medicine
58
1, 1165–1170.
Habay, J., Proost, M., De Wachter, J., Díaz-García, J., De Pauw, K., Meeusen, R., Van Cutsem, J. & Roelands, B. (2021). Mental fatigue-associated decrease in table tennis performance: Is there an electrophysiological signature? International Journal of Environmental Research and Public Health
18
(24), 12906. 

Hart, S. G. & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In P. A. Hancock & N. Meshkati (Eds.), Human mental workload, New York: North-Holland, 139–183. 

Hebbar, P. A., Bhattacharya, K., Prabhakar, G., Pashilkar, A. A. & Biswas, P. (2021). Correlation between physiological and performance-based metrics to estimate pilots’ cognitive workload. Frontiers in Psychology
12
1, 555446. 

Hirst, D. J. & De Looze, C. (2021). Measuring speech: Fundamental frequency and pitch. In R. Knight & J. Setter (Eds.), The Cambridge handbook of phonetics. Cambridge: Cambridge University Press, 336–361. 

Hubscher-Davidson, S. (2020). Ethical stress in the translation and interpreting profession. In K. Koskinen & N. K. Pokorn (Eds.), The Routledge handbook of translation and ethics. Abingdon: Routledge, 415–430. 

Huttunen, K., Keränen, H., Väyrynen, E., Pääkkönen, R. & Leino, T. (2011). Effect of cognitive load on speech prosody in aviation: Evidence from military simulator flights. Applied Ergonomics
42
(2), 348–357. 

Hyönä, J., Tommola, J. & Alaja, A.-M. (1995). Pupil dilation as a measure of processing load in simultaneous interpretation and other language tasks. Quarterly Journal of Experimental Psychology Section A
48
(3), 598–612. 

Jacquet, T., Lepers, R., Poulin-Charronnat, B., Bard, P., Pfister, P. & Pageaux, B. (2021). Mental fatigue induced by prolonged motor imagery increases perception of effort and the activity of motor areas. Neuropsychologia
150
1, 107701. 

Johannes, B., Wittels, P., Enne, R., Eisinger, R., Castro, C. A., Thomas, J. L., Adler, A. B. & Gerzer, R. (2007). Non-linear function model of voice pitch dependency on physical and mental load. European Journal of Applied Physiology
101
1, 267–276. 

Kauffman, B. Y., Garey, L., Nordan, A., Jardin, C., Mayorga, N. A., Robles, Z. & Zvolensky, M. J. (2019). The development and initial validation of the fatigue sensitivity questionnaire. Cognitive Behaviour Therapy
48
(5), 419–429. 

Kitch, J. A. & Oates, J. (1994). The perceptual features of vocal fatigue as self-reported by a group of actors and singers. Journal of Voice
8
(3), 207–214. 

Korpal, P. (2017). Linguistic and psychological indicators of stress in simultaneous interpreting. Poznań: Wydawnictwo Naukowe UAM.
Krejtz, K., Duchowski, A. T., Niedzielska, A., Biele, C. & Krejtz, I. (2018). Eye tracking cognitive load using pupil diameter and microsaccades with fixed gaze. PloS ONE
13
(9), e0203629. 

(2003). Physiological stress during simultaneous interpreting: A comparison of experts and novices. The Interpreters’ Newsletter
12
1, 51–67.
Lambooij, M. T. M., IJsselsteijn, W. A., Fortuin, M. F. & Heynderickx, I. E. J. (2009). Visual discomfort and visual fatigue in stereoscopic displays: A review. Journal of Imaging Science and Technology
53
(3), 1–14. 

Laukkanen, A.-M., Ilomäki, I., Leppänen, K. & Vilkman, E. (2008). Acoustic measures and self-reports of vocal fatigue by female teachers. Journal of Voice
22
(3), 283–289. 

Lewis, M. (1997). Pedagogical implications of the lexical approach. In J. Coady & T. Huckin (Eds.), Second language vocabulary acquisition: A rationale for pedagogy. Cambridge: Cambridge University Press, 255–270.
Liang, J., Fang, Y., Lv, Q. & Liu, H. (2017). Dependency distance differences across interpreting types: Implications for cognitive demand. Frontiers in Psychology
8
1, 2132. 

Lim, J., Wu, W., Wang, J., Detre, J., Dinges, D. & Rao, H. (2010). Imaging brain fatigue from sustained mental workload: An ASL perfusion study of the time-on-task effect. Neuroimage
49
1, 3426–3425. 

Lively, S. E., Pisoni, D. B., Van Summers, W. & Bernacki, R. H. (1993). Effects of cognitive workload on speech production: Acoustic analyses and perceptual consequences. Journal of the Acoustical Society of America
93
(5), 2962–2973. 

Mehta, R. K. & Parasuraman, R. (2013). Neuroergonomics: A review of applications to physical and cognitive work. Frontiers in Human Neuroscience
7
1, 889. 

Melnicuk, V., Thompson, S., Jennings, P. & Birrell, S. (2021). Effect of cognitive load on drivers’ state and task performance during automated driving: Introducing a novel method for determining stabilisation time following take-over of control. Accident Analysis & Prevention
151
1, 105967. 

Mizuno, A. (2017). Simultaneous interpreting and cognitive constraints. Journal of College of Literature, Aoyama Gakuin University
58
1, 1–28.
Morris, T. L. & Miller, J. C. (1996). Electrooculographic and performance indices of fatigue during simulated flight. Biological Psychology
42
(3), 343–360. 

Moser-Mercer, B. (2003). Remote interpreting: Assessment of human factors and performance parameters. [URL] (accessed 21 June 2021)
Moser-Mercer, B., Künzli, A. & Korac, M. (1998). Prolonged turns in interpreting: Effects on quality, physiological and psychological stress (Pilot study). Interpreting
3
1, 47–64. 

Nattinger, J. R. & DeCarrico, J. S. (1992). Lexical phrases and language teaching. New York: Oxford University Press.
Nilsson, E. J., Bärgman, J., Ljung, Aust M., Matthews, G. & Svanberg, B. (2022). Let complexity bring clarity: A multidimensional assessment of cognitive load using physiological measures. Frontiers in Neuroergonomics
3
1, 787295. 

O’Halpin, R. (2010). The perception and production of stress and intonation by children with cochlear implants. PhD dissertation, University of London.
Ots, N. (2021). Cognitive constraints on advance planning of sentence intonation. PLoS ONE
16
(11), e0259343. 

Paas, F., Renkl, A. & Sweller, J. (2003). Cognitive load theory and instructional design: Recent developments. Educational Psychologist
38
1, 1–4. 

Pageaux, B. & Lepers, R. (2018). The effects of mental fatigue on sport-related performance. Progress in Brain Research
240
1, 291–315. 

Petsche, H., Etlinger, S. C. & Filz, O. (1993). Brain electrical mechanisms of bilingual speech management: An initial investigation. Electroencephalography and Clinical Neurophysiology
86
(6), 385–394. 

Plevoets, K. & Defrancq, B. (2016). The effect of informational load on disfluencies in interpreting: A corpus-based regression analysis. Translation and Interpreting Studies
11
(2), 202–224. 

(2018). The cognitive load of interpreters in the European Parliament. A corpus-based study of predictors for the disfluency uh(m)
. Interpreting
20
(1), 1–28. 

Pilcher, J. J., Ginter, D. R. & Sadowsky, B. (1997). Sleep quality versus sleep quantity: Relationships between sleep and measures of health, well-being and sleepiness in college students. Journal of Psychosomatic Research
42
(6), 583–596. 

Pöchhacker, F. (2001). Quality assessment in conference and community interpreting. Meta
46
1, 410–425. 

Pöchhacker, F. & Zwischenberger, C. (2010). Survey on quality and role: Conference interpreters’ expectations and self-perception. [URL] (accessed 17 June 2024).
Pym, A. (2008). On omission in simultaneous interpreting: Risk analysis of a hidden effort. In G. Hansen, A. Chesterman & H. Gerzymisch-Arbogast (Eds.), Efforts and models in interpreting and translation research: A tribute to Daniel Gile. Amsterdam: John Benjamins, 83–105.
Remacle, A., Garnier, M., Gerber, S., David, C. & Petillon, C. (2018). Vocal change patterns during a teaching day: Inter- and intra-subject variability. Journal of Voice
32
(1), 57–63. 

Russell, A., Oates, J. & Greenwood, K. M. (1998). Prevalence of voice problems in teachers. Journal of Voice
12
(4), 467–479. 

Sáiz-Manzanares, M. C., Marticorena-Sánchez, R., Martín Antón, L. J., González-Díez, R. & Carbonero Martín, M. Á. (2023). Using eye tracking technology to analyse cognitive load in multichannel activities in university students. International Journal of Human-Computer Interaction
40
(12), 3263–3281. 

Schnotz, W. & Kürschner, C. (2007). A reconsideration of cognitive load theory. Educational Psychology Review
19
1, 469–508. 

Seeber, K. G. (2011). Cognitive load in simultaneous interpreting: Existing theories – new models. Interpreting
13
(2), 176–204. 

(2013). Cognitive load in simultaneous interpreting: Measures and methods. Target
25
(1), 18–33. 

Shao, Z. & Chai, M. (2021). The effect of cognitive load on simultaneous interpreting performance: An empirical study at the local level. Perspectives
29
(5), 778–794. 

Smets, E. M., Garssen, B., Bonke, B. & De Haes, J. C. (1995). The Multidimensional Fatigue Inventory (MFI) psychometric qualities of an instrument to assess fatigue. Journal of Psychosomatic Research
39
(3), 315–325. 

Souchet, A. D., Philippe, S., Lourdeaux, D. & Leroy, L. (2022). Measuring visual fatigue and cognitive load via eye tracking while learning with virtual reality head-mounted displays: A review. International Journal of Human-Computer Interaction
38
(9), 801–824. 

Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction
4
(4), 295–312. 

Sweller, J., van Merrienboer, J. J. G. & Paas, F. (1998). Cognitive architecture and instructional design. Educational Psychology Review
10
(3), 251–296. 

Sweller, J., van Merriënboer, J. J. G. & Paas, F. (2019). Cognitive architecture and instructional design: 20 years later. Educational Psychology Review
31
(2), 261–292. 

Tanaka, M., Ishii, A. & Watanabe, Y. (2014). Neural effects of mental fatigue caused by continuous attention load: A magnetoencephalography study. Brain Research
1561
1, 60–66. 

Tolkmitt, F. J. & Scherer, K. R. (1986). Effect of experimentally induced stress on vocal parameters. Journal of Experimental Psychology: Human Perception and Performance
12
(3), 302–313.
Tommola, J. & Niemi, P. (1986). Mental load in simultaneous interpreting: An on-line pilot study. In L. S. Evensen (Ed.), Nordic research in text linguistics and discourse analysis. Trondheim: Tapir. 171–184.
Van Puyvelde, M., Neyt, X., McGlone, F. & Pattyn, N. (2018). Voice stress analysis: A new framework for voice and effort in human performance. Frontiers in Psychology
9
1, 1994. 

Vogel, A. P. (2010). Acoustic analysis of the effects of sustained wakefulness on speech. Journal of the Acoustical Society of America
128
(6), 3747–3756. 

Wang, M. (2013). Correlation between error types and local cognitive load in simultaneous interpreting – taking English-Chinese language pair in Cognitive Load Model as an example. MA Thesis, Shanghai International Studies University.
Wang, Y., Zhai, G., Chen, S., Min, X., Gao, Z. & Song, X. (2019). Assessment of eye fatigue caused by head-mounted displays using eye-tracking. BioMed Engineering OnLine
18
1, 111. 

Welham, N. V. & Maclagan, M. A. (2003). Vocal fatigue: Current knowledge and future directions. Journal of Voice
17
(1), 21–30. 

Xu, R., Zhang, C., He, F., Zhao, X., Qi, H., Zhou, P., Zhang, L. & Ming, D. (2018). How physical activities affect mental fatigue based on EEG energy, connectivity, and complexity. Frontiers in Neurology
9
1, 915. 

Yamada, Y. & Kobayashi, M. (2018). Detecting mental fatigue from eye-tracking data gathered while watching video: Evaluation in younger and older adults. Artificial Intelligence in Medicine
91
1, 39–48. 

Yap, T. F., Epps, J., Ambikairajah, E. & Choi, E. H. C. (2011). Formant frequencies under cognitive load: Effects and classification. EURASIP Journal on Advances in Signal Processing 2011, 219253. 
