Interpretable Handwritten Digit Classification: Analyzing Feature Extraction and Explainable AI on the Extended MNIST Dataset

Classifying handwritten numbers is a significant machine learning problem that has paved the path for initial research directions on the field of Artificial Intelligence and Pattern Recognition. In this study, we explore the efficiency of a variety of feature extraction methods (both hand-crafted and deep learning based) in conjunction with various machine learning classifiers for handwritten digit classification. We also make use of Explainable AI tools such as SHapley Additive exPlanations (SHAP) and Local Interpretable Model-agnostic Explanations (LIME) for improving the interpretability of the models. The experiments were conducted on the publicly available Extended MNIST database and it was observed that the hand-crafted features such as Histogram of Oriented Gradients provided comparable performance to the deep learning based features such as VGG, MobileNet, InceptionNet, EfficientNet, and ResNet.