A Multi-Level Decision Fusion Strategy Based on Fuzzy Rules for Emotion Recognition

Several issues are needed to be addressed during multimodal fusion, which are data redundancy, data alignment, and modality weight allocation. Fixed weight allocation method is widely applied to decision level fusion of multimodal data, which is not in accordance with actual situation. This paper proposed a multi-level decision fusion strategy based on fuzzy rules to allocate modality weights according to the modality contribution. Experiments are conducted on SAVEE audiovisual emotion database and the results demonstrated that the proposed strategy can reduce redundant data and increase recognition accuracy, and the fuzzy rules behave the strengths of the natural expressiveness in the case of poor facial recognition performance.