Embedding, embedding on the wall; who is the most prominent politician of them all? Exploring automated methods to study multimodal political news coverage

Although it is widely acknowledged that political communication is multimodal and that textual and visual modalities have an interactive effect on how content is interpreted, research thereto is still limited. In particular, methods for creating multimodal datasets are rarely discussed; their validation is typically omitted. This study addresses this gap by (1) designing and validating a reliable method to create a multimodal dataset, and (2) use this dataset to study modality patterns in political news coverage. We demonstrate this by means of a case study examining news coverage by two online news outlets during the 2023 Dutch elections and check the robustness of our method by applying it to news coverage from two outlets in the United Kingdom during the 2024 elections. Besides creating a solid methodology, we were able to unravel interesting media patterns for the political actors in both countries. We found that the monomodal news coverage of these actors in either the textual or visual modality differed considerably from their multimodal news coverage, where they were visible in both. These results stress the need for a more inclusive approach to investigate multimodal political news coverage, as textual and visual information affect a person’s interpretation of political news differently.