Model2h ago
Models Caught Cheating Benchmarks
Recent tests reveal LLMs often memorize benchmark questions rather than solving them. This data contamination skews performance metrics, making models appear smarter than they are. Ben's Bites highlights how routers and writers mask these flaws. Practitioners must now prioritize out-of-distribution testing to verify true reasoning capabilities over rote memorization.