Model Stitching May Erase Critical Features | dailyai.report
25 stories from today
Safety
1d ago
Model Stitching May Erase Critical Features
Training affine maps to transfer linear features from large to small models often destroys the very information intended for transfer. This early-stage research suggests that standard bidirectional loss leads to feature collapse.
The Signal
Practitioners using SAE or steering vectors for interpretability should verify if their bridges actually preserve the underlying data.