Is It the Model or the Labels? Cross-Lingual Gaps in Hate Speech Detection for English, Hindi and Code-Mixed Hinglish

Hate speech classifiers for Indian social media usually perform worse on Hindi and code-mixed Hinglish than on English. This gap is usually blamed on the models. We test that assumption on PRISM, a public dataset of 29.5K English, Hindi (Devanagari) and Hinglish posts that merges several older corpora. We compare a TF-IDF baseline, two fine-tuned multilingual transformers (XLM-R and MuRIL) and a zero-shot large language model. All trained models show a statistically clear 13-20 point macro-F1 gap between English and the Hindi/Hinglish subsets, and moving to stronger or India-specific models does not close it. We then trace the labels back to their sources. Each language subset comes from a different original corpus. For 19% of the Hindi test posts we can match the original CONSTRAINT-2021 annotation, and on these posts PRISM's labels show no agreement with the original labels beyond chance (Cohen's kappa ~ 0). On these posts the fine-tuned transformers score below chance (macro-F1 ~ 0.40). On the remaining Hindi posts they score 0.64-0.66. Removing the matched posts from training barely changes results, so the damage is mainly to evaluation. Cross-model agreement, a manual re-annotation of model errors and a blind re-annotation of random posts point the same way. We conclude that a substantial part of the measured cross-lingual gap reflects label quality in the merged dataset rather than model limitations. Language-level comparisons on merged corpora should therefore be preceded by an audit of where the labels come from.