To the Verbal and Beyond

The appeal of audiovisual products comes from the combination of visual and auditory elements, but professional subtitling remains focused on words, reducing other elements to a contextualising role. This assumes that nonverbal elements such as images or sounds are universal codes easily interpreted by viewers without further mediation and potentially leaves some viewers with glaring losses of meaning (Cavaliere, 2008). In this article, I contend that all elements co-occurring with speech are signs in their own right that might present different challenges to (different) viewers and, as a result, might need to be translated. The article reports on an exploratory experimental study focused on comparing the impact on viewers’ meaning-making of a) current subtitling practice focused on verbal signs, and b) innovative subtitling practice aiming at translating meaning expressed by nonverbal elements identified as cultural-specific. The results point towards the need for a fundamental shift in our understanding of nonverbal elements and the need to translate them. Lay summary This study explored how different types of subtitles might affect the viewing experience and our interpretation. The researcher showed one clip from the film Forest Gump (Robert Zemeckis 1994) and one clip from the film Kautokeino Rebellion (Nils Gaup 2008). The film Forest Gump presents a familiar language (English) and culture (American), while the film Kautokeino Rebellion presents non-familiar languages (Sami and Norwegian) and cultures (Sami and Norwegian). Both clips were subtitled in two different ways: version 1 presented professional subtitling following common subtitling norms, while version 2 presented those same subtitles plus additional information on culture-specific elements considered to be difficult for the viewer to identify or interpret. The additional information appeared always on top of the screen. In this article, this new subtitling practice is referred to as “innovative subtitling” and the additional titles are referred to as “extratitles”. The experiment included 100 participants, and the focus was on finding out: a) if there were differences in viewers’ levels of understanding and enjoyment of the clips depending on the type of subtitling; b) if the additional titles would distract viewers from watching the film, and c) what would the viewers’ overall evaluation of the additional titles be. The data was collected using a questionnaire and an eye tracker to record the viewers’ eye movements. The study shows that the current practice of subtitling leaves viewers often confused and with evident losses of meaning. The use of additional titles seems to be a viable solution as it not only largely reduced the viewers’ level of confusion, but also did not seem to distract them and considerably increased their level of enjoyment and interest in the film.