Agents
93d agoFrontier Models Fail Enterprise IT Benchmark
No frontier model scored above 50% on ITBench-AA, a new benchmark for agentic enterprise IT tasks. Developed by IBM and Artificial Analysis, the test evaluates complex workflows like system administration and troubleshooting. These low scores reveal a performance gap between general reasoning and specialized IT execution.
The Signal
Practitioners should expect failures in autonomous infrastructure management.