Safety
109d agoHuman Goal Manipulability Challenges AI Alignment
Human goals are inherently under-determined and easily manipulated. This creates a fundamental tension for AI Alignment researchers trying to define "helpful" or "corrigible" behavior. Distinguishing between beneficial counsel and harmful brainwashing remains an unsolved technical hurdle.
The Signal
Practitioners must address this ontological instability to prevent models from subtly steering user desires.