Model1d ago
Richard Sutton Rejects Synthetic Data For Scaling
Turing Award winner Richard Sutton labels synthetic data a "big mistake" for scaling LLMs. He argues simulations are too microscopic to capture the world's infinite complexity. Instead, Sutton advocates for continual learning agents that evolve through direct experience. This challenges the current industry reliance on frozen models and generated datasets.