Beyond Words: A Preliminary Study for Multimodal Hate Speech Detection

Hate speech involves harmful expressions that either directly attack or endorse hatred toward a group or individual, often based on identity characteristics such as ethnicity, religion, or sexual orientation. The lack of regulation regarding hate speech raises concerns about the vulnerability of targeted groups and the potential escalation of violence. Detecting and eliminating hate speech from social media is crucial to prevent harm, negative psychological effects, and further attacks. The complexity of identifying hate speech lies in the indirect or implicit nature of its expressions, which can be influenced by context and the use of sarcasm or irony. Detection becomes even more challenging in multimodal scenarios, such as memes, where text and images are combined to convey meaning. To address this problem, this study analyzes the design of a multimodal hate speech detection technique that integrates textual and visual dimensions. This study explores how to define the textual and visual representations of memes and explores fusion strategies. The preliminary experiments conducted over publicly available datasets achieved encouraging results when compared to more complex approaches in the literature, while also highlighting the challenges of the task.