Research3d ago
Apple Research Analyzes Small Model Knowledge Limits
A new paper from Apple explores how parameter size limits the factual knowledge Small Language Models can retain. Researchers examined the trade-off between internal memory and external queries to larger models or databases. This study identifies which data points SLMs should prioritize during pretraining. Practitioners can use these findings to optimize hybrid RAG architectures.