Safety5h ago
Society Vulnerable To Reward Hacking
Researchers from Kings College London and Fudan University argue that human society is susceptible to reward hacking. They compare social systems to cyber environments where agents game rules for points. This suggests that optimizing for specific metrics creates systemic instability. Practitioners must prioritize robust objective functions to prevent unintended behavioral exploitation in real-world deployments.