Research2h ago
Attention Maps Poorly Predict VLM Correctness
A study of 3,090 samples across LLaVA-1.5, PaliGemma, and Qwen2-VL shows attention structure is a near-zero predictor of accuracy. Researchers used a new VLM Reliability Probe to debunk the intuition that sharp attention maps imply calibrated answers. This finding warns practitioners against using attention heatmaps as a proxy for model reliability or trust.