Attention-Guided Dimming in Vision-Language Models: Degradation Protocol Sensitivity, Prompt Effects, and Rank-Matched Replay
Visual Evidence Augmentation (VEA) dims low-attention regions before re-encoding. Using a five-cell counterfactual design and rank-matched replay (r_d, clean rankings at the degraded mask's sorted multipliers), D = G_d - G_c = R + T (R = e_d - r_d, R < 0 favoring clean order) is established as an algebraic contrast identity rather than a mechanism split. Canvas 0.50x blur shifts TVQA-200 baselines by -0.30 to +1.60 pp on three 0.33-0.50x-downscaling VLMs (-2.70 pp Qwen3-VL-2B TVQA; -5.50 pp GQA), whereas composite resized-input RGB blur (0.50x/0.35x, JPEG Q=95) drops Qwen2.5-VL-7B and LLaVA-1.5-7B by -8.20 to -21.05 pp (p_Holm <= 0.012), though differing resize order and encoding prevent isolating native-preprocessor absorption or generalizing across four models. Unlike VisualCoT (Liu et al., 2026: +0.5 pp Inst, +4.5 pp VEA-Inst on Qwen2.5), local TVQA-200 +Inst lifts MMF by +13.55/+14.10 pp (Qwen2.5) and +3.95 pp (LLaVA, p_Holm <= 0.042) without significant substring gains; under +Inst, dimming adds -0.80 to +0.50 pp on the three TVQA-200 configs (p_Holm = 1.000; nonsignificant +2.70 to +5.41 pp VEA-50, 0.00 to +10.53 pp VEA-19). Primary replay R has wide null CIs ([-4.65, +1.10] pp). Under +Inst, 24 resized-input direct gains G_d span -2.00 to +2.00 pp (p_Holm = 1.000; one exploratory CI [+0.50, +4.00] pp, raw p = 0.125), showing no significant average gain without proving equivalence or testing prompt removal, motivating explicit manipulation checks.
Authors
- Yikai Liu (ORCID: https://orcid.org/0000-0003-3146-808X)
Institutions
- Communication University of China (CN)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-21
- DOI
- https://doi.org/10.5281/zenodo.24415568
- Primary Topic
- Multimodal Machine Learning Applications
- Type
- preprint