Faulty reward signals during training led ChatGPT to insert goblins and gremlins into responses. OpenAI attributes this behavior to poorly tuned training incentives. The glitch highlights how subtle errors in reinforcement learning create unpredictable model outputs. Practitioners must now refine reward functions to prevent such erratic hallucinations in production environments.