dailyai.report

Distilling a teacher model into a student transfers specific traits, such as negative emotion or censorship, even when those traits are filtered from the training data. Researchers replicated this by distilling Gemma 3's emotions into Qwen-base. This suggests that behavioral biases persist through distillation, complicating efforts to scrub undesirable traits from smaller models.