Research
163d agoTransformers Are Bayesian Networks
Researchers reveal that the ubiquitous Transformer architecture functions as a Bayesian network, with each layer executing a round of belief propagation. By proving that any sigmoid Transformer implements weighted loopy belief propagation, the study bridges deep learning and probabilistic inference.
The Signal
The authors also demonstrate exact inference on acyclic knowledge bases, offering provably correct probability estimates.