Safety
116d agoNew Method Decomposes LLM Attention Layers
The adVersarial Parameter Decomposition (VPD) method now allows researchers to decompose attention layers in small language models. This technique outperforms previous stochastic and attribution-based approaches. By building attribution graphs from causally important subcomponents, the team provides a scalable path for model interpretability.
The Signal
Practitioners can now analyze components that previously resisted SAE-based methods.