Research3h ago
Transformers Outperform Finite Automata in Expressive Power
Transformers represent formal languages more succinctly than finite automata or Linear Temporal Logic formulas. This high expressivity means the models describe complex concepts using fewer parameters. However, this efficiency creates a trade-off. Verifying transformer properties is now provably intractable, specifically EXPSPACE-complete, making formal safety guarantees mathematically difficult for practitioners.