Safety3h ago
Safety
New Method Decomposes LLM Attention Layers
The adVersarial Parameter Decomposition (VPD) method successfully breaks down parameters in small language models. Unlike previous SAEs or transcoders, VPD handles complex attention layers. Researchers used these subcomponents to build causal attribution graphs for specific prompts. This technical leap makes parameter decomposition viable for larger, production-scale models used in the field.