Research3h ago
Power-Law Data Outperforms Uniform Distributions in Reasoning
Training on power-law distributions outperforms uniform data across multi-step arithmetic and state tracking tasks. Researchers at arXiv prove that this asymmetry requires significantly less training data to master compositional reasoning. The findings challenge the common intuition that curators must balance long-tail skills. This suggests that natural data imbalances actually accelerate LLM skill acquisition.