Research1d ago
Pruning Training Data Boosts Fact Memorization
Information-theoretic analysis reveals that LLMs fail to memorize facts when training data exceeds model capacity. Apple researchers found that pruning redundant data actually improves factual accuracy. This suggests that less, higher-quality data prevents parameter saturation. Practitioners should prioritize data curation over volume to reduce hallucinations in knowledge-intensive tasks.