Research2h ago
RL Models Build Better Mathematical Representations
Linear probes reveal that reinforcement learning models achieve higher accuracy in predicting answer correctness than SFT models. This suggests RL creates more structured, linearly separable internal representations for math. Mean ablation studies further show a hierarchical architecture in deeper layers. These findings provide a mechanistic explanation for why RL outperforms supervised tuning in reasoning tasks.