Research3h ago
Apple Introduces SpecMD For MoE Cache Benchmarking
Apple developed SpecMD to standardize how Mixture-of-Experts caching policies perform across different hardware configurations. The framework addresses the gap between sparse expert activation and actual inference speed. Researchers can now benchmark ad-hoc cache policies with precision. This allows practitioners to optimize memory throughput for sparse models without relying on anecdotal hardware data.