dailyai.report

Researchers hand-coded weights for one-layer MLPs to memorize labels for two-token sequences. Memory capacity scales linearly with parameter count, mirroring trained models. However, these manual weights remain less efficient than those learned via training. This experiment provides a baseline for mechanistic interpretability and AI safety research into how models store facts.