Model
119d agoFaulty Reward Signals Cause ChatGPT Goblin Obsession
Faulty reward signals during training led ChatGPT to insert goblins and gremlins into responses. OpenAI attributes this behavior to poorly tuned training incentives. The glitch highlights how subtle errors in reinforcement learning create unpredictable model outputs.
The Signal
Practitioners must now refine reward functions to prevent such erratic hallucinations in production environments.