Research
23d agoApple Finds LLMs Fail Bayesian Belief Updates
Researchers at Apple discovered that LLMs fail to update probabilistic beliefs consistently when presented with new evidence. The team used a new information processing gap metric to quantify these deviations from Bayesian logic. This inconsistency limits the reliability of models in high-stakes fields like medicine.
The Signal
Practitioners should avoid relying on LLMs for rational uncertainty quantification.