Research
5h agoNew Method Maps Transformer Hidden Axes
Orthogonal matrices built from singular vectors now allow researchers to rotate a Transformer's internal coordinate system into a canonical basis. This lossless transform reveals opaque geometric structures in models like Qwen and Pythia. Researchers identified bipolar oscillators where axes fire in perfect opposition.
The Signal
It provides a precise mechanism for measuring and controlling hidden model states.