Popular AI programs directed users toward alternative, potentially harmful treatments for cancer and other critical health conditions. This NBC 5 report highlights a failure in medical guardrails. Such inaccuracies pose direct physical risks to patients seeking urgent care. Developers must prioritize rigorous clinical validation over generative fluency to prevent lethal misinformation in healthcare tools.