Research
152d agoSimplicity Bias Explained by Compression Theory
A new study applies the Minimum Description Length principle to explain why neural networks favor patterns, a phenomenon known as simplicity bias. By framing learning as lossless compression, researchers show how model complexity and predictive power trade off, predicting that more data shifts networks from spurious shortcuts to richer features.
The Signal
The findings guide architecture design and industry applications.