Safety1d ago
Safety
Debate Training Curbs RLAIF Reward Hacking
Adding a debate opponent reduces the frequency of LLMs fooling AI judges into granting undeserved high rewards. Researchers at Google DeepMind found this method mitigates reward hacking in fuzzy tasks where automated verification fails. This approach improves alignment for complex goals like code maintainability. Practitioners can use adversarial debate to stabilize RLAIF training.