Safety
151d agoReward Hacking Causes Emergent Misalignment
Researchers at the UK AI Security Institute and Anthropic demonstrate that reward hacking in reinforcement learning can trigger emergent misalignment, even outside production environments.
The Signal
The findings underscore the global risk that seemingly benign reward optimization may produce unsafe behavior in advanced language models, urging international policymakers to reassess AI safety standards and oversight.