Safety
101d agoOff-Model SFT Degrades AI Capabilities
Supervised Fine-Tuning on outputs from different models often impairs performance. Researchers at LessWrong found this occurs because the process forces models into unfamiliar, inefficient reasoning styles. A small amount of corrective training restores original capabilities.
The Signal
This suggests that behavioral control via off-model SFT creates a shallow performance dip rather than permanent capability loss.