A global–local hybrid transformer framework for automatic Navarasa facial expression recognition

Facial expression recognition (FER) is an important field of research within affective computing, but the recognition of culturally-based facial expressions is significantly more difficult than traditional FER. Navarasa is a framework from Indian aesthetics that captures the nine emotionally meaningful states. These emotional states are manifested through both the overall facial context and localized facial features and therefore require simultaneous modelling of both information sources for automatic recognition. To address these challenges, this paper presents a global–local hybrid transformer framework, which combines complementary feature representations using feature-level fusion from a pretrained Vision Transformer (ViT) and Swin Transformer. A lightweight multilayer perceptron is used to classify the fused representations, while keeping both transformer backbones frozen to minimize the number of trainable model parameters, increase computational efficiency, enhance optimization stability and experimental reproducibility. The proposed hybrid framework is validated through extensive experiments with the NAVARASA FER dataset, where it outperforms the standalone models of the Vision Transformer and Swin Transformer in terms of the overall accuracy of 78.09%, macro-precision of 78.13%, macro-recall of 73.89%, and macro-F1 score of 75.53%. In addition to the standard performance metrics, the proposed framework is extensively validated using confusion matrix analysis, one-versus-rest ROC-AUC analysis, one-versus-rest bootstrap confidence intervals, McNemar’s statistical significance testing, computational complexity assessment, classifier ablation studies, feature representation analysis, Grad-CAM-based visualization and failure analysis based on the model’s confidence. The complementary analyses show that joint modeling of global contextual information and local hierarchical facial feature representations results in more robust, interpretable, and statistically reliable recognition of culturally specific facial expressions. The proposed framework demonstrates an effective, computationally efficient and reproducible framework for fine-grained facial expression recognition of Navarasa emotions and provides a solid foundation for future affective computing applications that involve culturally diverse emotional representations.