Research4h ago
Data Pruning Boosts LLM Fact Memorization
Information-theoretic analysis by Apple researchers shows that excessive training data actually degrades factual accuracy. When data volume exceeds model capacity, memorization drops. Pruning the training set to fit within these limits reduces hallucinations. This finding forces practitioners to prioritize data quality over raw volume to improve knowledge retrieval in LLMs.