Research
163d agoTransformers Are Bayesian Networks
Researchers reveal that the ubiquitous Transformer architecture functions as a Bayesian network, enabling weighted loopy belief propagation across its layers. Each layer performs a belief‑propagation round, regardless of training.
The Signal
The study formally proves this for sigmoid Transformers with any weights, and demonstrates exact inference on non‑circular knowledge bases, offering provably correct probability estimates.