Towards a corpus-based approach to graphic elements in creative subtitling
A case study of the YouTube channel “Apenjie with Dawang”
This article explores how a corpus-based approach allows us to describe and analyze the multimodal complexity of
graphic elements in creative subtitling. To this end, the article focuses on a YouTube channel, Apenjie with
Dawang, featuring a dog and its owner. This channel’s subtitling strategies were experimental with multiple graphic
elements (colors, positions, font sizes, and emojis). Informed by a social semiotic approach to multimodality, a corpus of 1,155
coupled pairs of Chinese-English subtitles was annotated for modal shifts and metafunctional shifts. Some major findings include:
(a) emojis were much more likely to be added to the target subtitles for the animals than for the humans;
(b) speaker-identifying graphic elements (color and position) were lost in the target subtitles, but emojis were
systematically added to mark animals as the speakers; (c) the addition of emojis evoked complementary-interpersonal meanings,
suggesting that the subtitlers might have prioritized audience engagement over textual fidelity; (d) although the target
subtitles used fewer graphic elements, the semiotic meanings could be similar or complementary to those of the source subtitles.
Based on these findings, the article also discusses the opportunities and challenges of a corpus-based approach to graphic
elements in creative subtitling.
Article outline
- 1.Introduction
- 2.Research background
- 2.1A social semiotic approach to multimodality
- 2.2A corpus-based approach to subtitles
- 3.The study
- 3.1Corpus
- 3.2Coding framework
- 3.3Coding and analytical procedures
- 4.Results
- 4.1Within-group comparisons
- 4.2Inter-group comparisons
- 5.Discussion
- 5.1Graphic elements in creative subtitling
- 5.2Opportunities and challenges of a corpus-based approach
- 6.Conclusion
- Notes
-
References