KinTransformNet: A Transformer-Assisted Approach With Hough-HoG Features for Kinship Verification and Classification

Kinship verification has been actively researched over the last decade for blood relation verification between two or more people using facial features. Kinship verification and classification domains have faced major challenges in accurately identifying familial relationships from ambiguous and intricate datasets. This research proposes “KinTransformNet,” a novel hybrid framework for kinship verification by integrating feature extraction techniques using the Histogram of Oriented Gradients based on Hough Transform (Hough-HoG) with a pre-trained Bidirectional Encoder Image Transformer (BEIT) model. Here, a novel Hough-HoG algorithm is used to encode spatial relationships of image gradient directions relative to a reference point to specifically construct the unique R-table for the edge pixel orientations with the most drastic changes. Image properties have been captured using this R-Table by calculating and normalizing HoG, magnitudes. Concurrently, high-level semantic features from image patches have been extracted using the pre-trained BEIT model. A set of features is generated using this model by first dividing the input image into patches, then linearly embedding them with positional encodings, and finally passing them through a stack of Transformer encoder layers with Multi-Head Self-Attention, Layer Normalization, and MLP blocks. A feature vector is extracted by pooling these embeddings. An absolute difference between the Hough-HoG feature and the BEIT feature vector is computed. Deep-learning-based semantic features were strengthened by Hough-HoG features; this feature vector is fed into a classifier to predict kinship relations between image pairs. The proposed model used the Family in the Wild (FIW), KinFaceW-I, II, and Cornell KinFace datasets and achieved an impressive accuracy of 95.25%, 93.79%, 94.57% and 91.62% respectively, which demonstrated its effectiveness in handling the complexities in kinship verification.