Mixture of Fuzzy Rule-Based Classification Systems

Abstract The researches on the fusion between fuzzy techniques and deep learning have become an increasingly hot topic. In this work, we aim to improve the accuracy and interpretability of deep learning models on tabular data by developing a fuzzy deep neural network named Mixture of Fuzzy Rule-based Classification Systems (MFRBCS). The objective is to address the challenges posed by high dimensionality, mixed feature types, and the lack of appropriate inductive bias in existing neural architectures. Firstly, the soft binning function is proposed based on Gaussian membership function in fuzzy sets, which lays a solid foundation for constructing fuzzy local expert in the framework of neural networks. Afterwards, the overall architecture of MFRBCS is organized according to mixtures of experts, where a series of fuzzy local experts are regulated by the gating neural network. In this design, fuzzy local experts enable interpretable rule-based learning on feature subspaces, while the gating network coordinates their contributions to improve generalization performance. MFRBCS is compared with various tree-based methods and deep fuzzy classifiers on 12 tabular datasets in terms of average testing accuracy and balanced accuracy. The results of comparative experiments show that our approach achieves more competitive and promising performance in most cases. Moreover, we provide a discussion on the limitations of this work, which may guide future improvements and applications.