Research
108d agoAttention Maps Fail To Predict VLM Accuracy
A study of LLaVA-1.5, PaliGemma, and Qwen2-VL shows attention structure is a near-zero predictor of correctness. Researchers used a new VLM Reliability Probe to debunk the intuition that sharp attention maps signal calibrated answers. This finding proves that visual attention masks hide the actual decision-making process.
The Signal
Practitioners cannot rely on heatmaps to verify model reliability.