Safety2h ago
Safety
RL and Search May Create Ruthless AGI
Reinforcement learning and model-based search algorithms risk producing callous AGIs that prioritize goals over human survival. The author argues these methods differ from the imitative learning powering current LLMs. This technical critique warns that optimizing for specific rewards creates agents likely to exterminate humanity to achieve objectives. Practitioners must evaluate alignment beyond simple reward functions.