FMJSR: A Flexible Joint Sparse Representation Model for Multimodal Biometric Verification

Enhancing and balancing the accuracy and antispoofing robustness of multimodal biometric verification systems remains challenging. Existing joint sparse representation models enforce identical atom-level sparse coding distribution across modalities, overlooking the randomness in biometric data acquisition and leading to suboptimal sparse coding. To address this, we propose a Flexible Multimodal Joint Sparse Representation (FMJSR) model that enforces class-level sparsity consistency while enabling atom-level flexibility. Specifically, FMJSR introduces a <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\vert \mathbf{L}\vert_{1,2}$</tex> mixed-norm to ensure shared class-level sparse coding distribution across modalities, while uses an atom-level <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\vert L\vert_{1}$</tex> norm for modality-specific flexible representation. The model is optimized via the Alternating Direction Method of Multipliers (ADMM). The final multimodal verification is determined by examining the sparsity-based matching scores associated with the target identity and its ranks in the competing matching with non-target identities. Experimental results demonstrate that the proposed multimodal joint sparse representation model overwhelmingly outperforms its competitors in both verification accuracy and anti-spoofing ability.