When Fixing Imbalance Creates New Bias: Category-Sentiment Confounding from Non-Uniform Class Rebalancing in Multi-Category Review Datasets

Class rebalancing is a standard step in preparing sentiment datasets for training, typically applied to correct an overall skew toward one class. We show that when the additional minority-class examples needed to hit a rebalancing target are drawn disproportionately from a limited subset of categories, rather than proportionally across all categories, the rebalancing procedure itself introduces a spurious association between product category and sentiment label - even when the overall class balance is identical to a properly rebalanced version. We isolate this effect using Cramer's V under matched overall balance, demonstrate its behavioral consequence with a controlled classifier evaluation on a category-decorrelated test set (with unique-source and duplication-rate budgets matched across conditions), and trace its magnitude across a donor-category-count spectrum (k=1,2,4,7). Across 10 random seeds, concentrated rebalancing yields a category-sentiment association of V=0.57 versus V=0.10 for uniform rebalancing, and a macro-F1 gap of 0.082; the effect replicates with an alternate donor category and decays as more categories are used, with classification performance recovering faster than the underlying statistical association disappears. We discuss implications for dataset construction practice.