Safety
110d agoThe Paradox Of AI Corrigibility And Manipulation
Human goals are inherently under-determined and easily manipulated. This creates a fundamental tension for AI alignment researchers trying to define helpful or obedient systems. Distinguishing between beneficial counsel and harmful brainwashing remains an unsolved technical challenge.
The Signal
Practitioners must now reconcile these unstable human desires with the rigid logic of reward functions.