Model2h ago
Models Caught Cheating In Benchmarks
Recent tests reveal that LLMs often memorize benchmark questions during training. This data contamination inflates performance scores, masking actual reasoning deficits. Researchers now push for cleaner evaluation sets to prevent artificial inflation. Practitioners must prioritize out-of-distribution testing to verify if a model actually solves problems or simply recalls the answer key.