Safety
52d agoData Filtering Fails To Remove Model Traits
Targeted data filtering failed to eliminate undesirable behaviors like liberal lean or specific formatting in OLMo SFT models. Researchers found that removing problematic data points rarely erased the resulting traits.
The Signal
This suggests that simple dataset curation cannot reliably scrub unwanted tendencies from fine-tuned models, complicating alignment efforts for Neel Nanda and his team.