Targeted data filtering barely reduces undesirable behaviors in OLMo models during supervised fine-tuning. Researchers found that removing specific data points failed to eliminate traits like liberal-leaning responses or specific formatting habits. This suggests that simple dataset curation cannot easily scrub ingrained model biases.