AI Insight
Researchers discovered that vision-language models can be fooled during training to accept incorrect image descriptions, yet still produce correct descriptions when generating text freely. By studying the Qwen2.5-VL-7B-Instruct model on 200 images, they found this "train/inference gap" occurs because the language decoder component actively suppresses corrupted signals from the visual encoder, with the target incorrect token consistently ranking at exactly position 3,488 out of 152,064 vocabulary entries across all tested images. The study demonstrates that adversarial robustness in these models depends primarily on the language decoder's learned patterns rather than the visual encoder's resistance to attacks.
Why it matters
This finding suggests that efforts to make vision-language models more trustworthy and resistant to manipulation should focus on the language decoder component rather than solely hardening the visual encoder. The results have direct implications for developing security measures and faithfulness evaluations for deployed AI systems that combine vision and language understanding.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
-cross
Abstract: A targeted adversarial perturbation can drive a vision-language model’s (VLM’s) teacher-forced training loss for a fixed target caption to near zero, yet the same model, allowed to generate freely, produces the original, correct description with no trace of the target. We call this dissociation the train/inference gap, and give it a precise mechanistic account on Qwen2.5-VL-7B-Instruct using a controlled two-stage PGD attack on 200 held-out COCO images. First, we show that image-level pixel statistics, including a correctly re-implemented, texture-based attackability measure from the CNN robustness literature, have essentially no predictive power over which images are corrupted (best predictor r=-0.050, p=0.484; ridge regression R^2=0.069). Second, using the logit lens, we localise the gap to a single autoregressive step: the rank of the target token, conditioned on the correct first token already being generated, is fixed at exactly 3,488 out of 152,064 vocabulary entries for every image and every condition, with zero variance. Third, tracking target-token rank across all 28 LLM decoder layers reveals that the visual encoder corrupts every image’s representation by a comparable margin regardless of eventual outcome, but the language model decoder then differentially arbitrates: amplifying the corrupted signal for susceptible images and actively suppressing it, past its clean-image baseline, for resistant ones (p<0.001, rank-biserial r=0.579). A linear probe on the merger hidden state separates these two outcomes with AUC=0.858, though we flag a circularity concern in this estimate. Together these results argue that adversarial robustness in autoregressive VLMs is substantially a property of the language decoder's prior, not the visual encoder, with direct implications for where faithfulness evaluations and defenses for deployed VLM systems should be targeted.
Source: Corrupted but Correct: Why Vision-Language Models Lie to Themselves Internally