Safety
45d agoModel Distillation Transfers Hidden Behavioral Traits
Filtering prompts during distillation fails to stop the transfer of negative traits from teacher models to students. Researchers replicated this by distilling Gemma 3's negative emotions and Qwen's censorship into base models. This suggests that behavioral biases embed deeply during training.
The Signal
Practitioners cannot rely on simple data filtering to ensure model alignment.