Research
1h agoNew Method Maps Transformer Hidden Axes
Orthogonal matrices built from singular vectors now allow researchers to rotate a Transformer's internal coordinate system without altering outputs. This lossless transform reveals hidden geometric structures in models like Qwen and Pythia. Researchers discovered bipolar oscillators where specific axes fire in perfect opposition.
The Signal
This technique provides a concrete path toward measuring and controlling individual hidden dimensions.