SSCNetViT: A Hybrid Siamese Sequential Classification-ViT Model for Kinship Recognition in Indonesia Facial Microexpression

Kinship recognition based on facial appearance is a challenging task with its unique complexity.Analyzing facial dynamics allows to assess the degree of similarity between individuals in the context of kinship.Although significant research has been done on basic microexpressions, there is currently a surge of interest in recognizing combined facial expressions of emotions in the field of image processing.The main focus of this study is to recognize kinship through a microexpression approach by proposing a hybrid model of Siamese Sequential Classification Network (SSCN) and vision transformer as a feature extractor instead of traditional convolution.This model combines the advantages of SSCN and Vision Transformer (ViT) models, the authors call it SSCNetViT, not only can capture global context information and present stronger learning ability, but also introduce SSCN inductive bias to improve generalization performance.The model is tested on independently collected datasets from the local Indonesian dataset (LaIndo) and the Families in the Wild (FIW) dataset.The results show that the L32 backbone achieves the highest average accuracy of 90.07%, with the peak performance in the BB class (99.5%) and the lowest in the FD class (84.0%).In comparison, the B16 and B32 backbones yield lower average accuracies of 88.0% and 83.3%, respectively, highlighting the effectiveness of the approach for kinship verification.Thus, our proposed SSCNetViT model with B16 quadratic feature fusion and multiplicative fusion strategies achieves the best performance and achieves better accuracy that outperforms previous state-of-the-art (SOTA) studies.