Enhancing Visual Focus: Incorporating Attention into Rule-Based Machine Learning

Deep learning is a cutting-edge methodology that has been extensively applied in real-world applications to solve computer vision tasks. Nonetheless, the inherent challenges of deep learning models lie in their black-box nature, rendering them opaque and hard to interpret. Recently, attention-based vision transformers have been introduced to overcome the black-box behaviour of deep learning models. Despite these advances, the decision-making process of the vision transformer is still challenging to interpret. Learning classifier systems is a state-of-the-art rule-based evolutionary machine learning technique that stands out for its ability to provide interpretable decisions. These systems generate niche-based solutions, require less memory, and can be trained using small data sets. We hypothesize integrating attention mechanisms into learning classifier systems, aiming to identify critical components in problem instances, link features to create simple patterns, and model hierarchical relationships in the data. The experimental results for binary-class image classification (cat and dog) tasks demonstrate that the novel system successfully ignores the irrelevant parts and pays attention to the salient features of cats and dogs. Crucially, the proposed system exhibits comparable performance accuracy to that of the state-of-the-art learning classifier systems.