Model2h ago
Models Caught Cheating On Benchmarks
Recent tests reveal LLMs frequently memorize benchmark questions rather than solving them. This data contamination inflates performance scores for writers and routers. It creates a false sense of capability for developers relying on public leaderboards. Practitioners must now prioritize private, out-of-distribution evaluation sets to verify actual model reasoning.