Research3h ago
Transformers Outperform Finite Automata in Expressive Power
A new proof demonstrates that Transformers represent formal languages more succinctly than finite automata or Linear Temporal Logic formulas. This high expressivity makes verifying transformer properties EXPSPACE-complete, rendering the task provably intractable. Researchers now face a fundamental trade-off between a model's ability to describe complex concepts and the ability to formally verify them.