QGFace-LLaVA: Quality-Aware Controlled Fusion of Structured Side Information for Face Analysis Under Imperfect Metadata

Biomimetic perception systems integrate heterogeneous cues selectively rather than treating all available information as equally reliable. Inspired by this principle, this study proposes QGFace-LLaVA, a multimodal large language model (MLLM)-centered framework for robust face analysis under imperfect metadata. A pretrained MLLM serves as the shared prompt-conditioned visual–language reasoning backbone, while structured side information, including age, gender, confidence cues, and availability indicators, is regulated through task-aware quality estimation, reliability-guided metadata calibration, gated residual correction, and counterfactual metadata reliability regularization (CMRR). Experiments on FER2013, CelebA-40, and UTKFace cover facial expression recognition, facial attribute recognition, and age estimation under clean, noisy, missing, shuffled, naturally erroneous, and counterfactual metadata conditions. The results show that metadata utility depends jointly on task relevance, metadata reliability, and fusion strategy, and that clean-setting gains do not necessarily imply robustness. QGFace-LLaVA reduces harmful dependence on unreliable metadata, while CMRR provides additional stability under corruption and mismatch. Overall, the framework transfers biomimetic selective cue integration into MLLM-based face analysis by treating metadata as reliability-controlled auxiliary evidence rather than a uniformly beneficial input.