Model
105d agoEMO Model Maintains Performance With Fewer Experts
The Allen Institute for AI and UC Berkeley built EMO, a mixture-of-experts model specializing in content domains. Removing 87.5% of its experts drops performance by only one percentage point. This efficiency allows MoE architectures to run on memory-constrained hardware.
The Signal
Practitioners can now deploy high-capacity models without requiring massive VRAM overhead.