The role of AI in multimodal research on the verbal and non-verbal means of expressing emotive modality in German: an investigation into selecting and analysing data

Abstract This paper seeks to substantiate the utility of AI-driven software in multimodal research. It endeavours to elucidate the pre-eminence of two specific pieces of software over alternatives used in linguistic analyses of this kind. The present study thoroughly scrutinises the eligibility of two sorts of audio and two types of visual data for automatically investigating the correlation of verbal and non-verbal means of expressing emotive modality in the German language. In doing so, it examines how the sentential scope typical of certain lexemes influences the applicability of the data types under investigation. The empirical analysis corroborated the necessity to conduct research of this kind employing entire sentences featuring a lexeme characterised by sentential scope. Intonation patterns and nonmanual features recognised and investigated by the software can occur throughout uttering a sentence which features a lexeme under study. This conclusion is drawn on the basis of the greatness of the quantity of emotions detected by the software vis-à-vis both the audio and the visual data employed in the analysis.