SafetyJust now
Safety
Residual Streams Stratify By Prediction Proximity
Directions nearest a model's prediction determine the immediate answer, while outer directions dictate output behavior five tokens later. This temporal stratification reveals how transformer residual streams organize information geometrically. Researchers used this mapping to decode the undifferentiated bulk of the stream. Practitioners can now better pinpoint where specific future tokens originate within the model.