Idempotence-Constrained Representation Learning: Balancing Sensitive Information Elimination and Feature Robustness

Machine learning models often inadvertently learn and perpetuate biases present in training data, particularly concerning sensitive attributes like gender and race. While existing fair representation learning approaches attempt to address this issue, they face challenges in balancing information preservation with bias elimination. This paper addresses the challenge of algorithmic bias in machine learning models by proposing a novel idempotence-constrained fair representation learning framework. We introduce a two-stage architecture that effectively eliminates both explicit and implicit bias while maintaining model performance. The first stage employs an idempotent encoder to remove explicit bias and enhance feature robustness through adaptive adversarial training, while the second stage utilizes a privacy filtering unit to eliminate implicit bias and task-irrelevant features. Our framework optimizes four key objectives: task performance, information preservation, sensitive information elimination, and representation stability. Theoretical analysis demonstrates convergence guarantees and representation stability of our approach. Extensive experiments on benchmark datasets, including UCI Adult and Heritage Health, show that our method achieves state-of-the-art performance in both fairness and accuracy metrics.