Research
162d agoDEAF Benchmark Tests Audio Model Faithfulness
The new DEAF benchmark challenges Audio MLLMs by presenting 2,700 conflict stimuli that vary in prosody, background noise, and speaker identity. Researchers evaluate models through a tiered framework that escalates textual influence—from semantic mismatches to deceptive prompts—separating content bias from prompt‑induced errors.
The Signal
This systematic test reveals whether models truly process sound or merely infer text.