Research4h ago
Attention Maps Poorly Predict VLM Reliability
A study of LLaVA-1.5, PaliGemma, and Qwen2-VL reveals that attention maps are near-zero predictors of correctness. Researchers used the VLM Reliability Probe to debunk the intuition that sharp attention implies calibrated answers. This finding warns practitioners against using attention heatmaps to verify model trust. Hidden-state geometry offers better signals than visual attention.