Fuzzy Rule-Based Differentiable Representation Learning

Representation learning is a key area in machine learning and deep learning, focusing on extracting meaningful features to support downstream tasks such as classification and clustering. Current mainstream representation learning methods primarily rely on nonlinear data mining techniques such as kernel methods and deep neural networks (DNNs) to extract abstract knowledge from complex datasets. However, most of them are "black-box" methods, lacking transparency and interpretability in the learning process, which constrain their practical utility. To this end, this article introduces a novel representation learning method called fuzzy rule-based differentiable representation learning (FRDRL), which is grounded in an interpretable fuzzy rule-based model. Specifically, it is built upon the Takagi-Sugeno-Kang fuzzy system (TSK-FS) to map input data to a high-dimensional fuzzy feature space through the antecedent part of the TSK-FS. Subsequently, a novel differentiable optimization method is proposed for learning in the consequent part, which preserves interpretability and transparency while effectively capturing nonlinear relationships in the data. By retaining the essence of traditional optimization and parameterizing key components as differentiable modules, the method improves performance without sacrificing interpretability. Moreover, a second-order geometry preservation strategy is incorporated to further improve robustness. Extensive evaluations conducted on various benchmark datasets validate the superiority of the proposed method. The source codes are available at https://github.com/BBKing49/FEDRL.