Safety
115d agoNew Method Decomposes LLM Attention Layers
The adVersarial Parameter Decomposition (VPD) method successfully breaks down parameters in small language models. Unlike previous SAEs or transcoders, VPD handles complex attention layers. Researchers used these subcomponents to build causal attribution graphs for specific prompts.
The Signal
This technical leap makes parameter decomposition viable for larger, production-scale models used in the field.