PyTorch 2.15 Brings FP8, Faster MoE Decoding, and Memory Savings to Apple Silicon

PyTorch 2.15 marks a major step forward for developers who run models on Apple Silicon. The MPS backend now supports FP8 (float8_e4m3fn) storage and scaled matrix multiplication, cutting weight memory in half compared with BF16 while keeping CUDA‑compatible semantics.

Mixture‑of‑experts models see the biggest throughput jump: a native grouped‑matmul path, reduced GPU‑CPU synchronisations, and faster top‑k routing push Qwen3.6‑35B‑A3B from 9.4 to 61.3 tokens/s on an M3 Ultra, and LFM2.5‑8B‑A1B from 23.6 to 123.6 tokens/s. Dense LLMs also benefit from specialised matrix‑vector kernels and an 8× faster vocabulary softmax.

Linear‑algebra coverage expands with complex LU, triangular solves, quicker small‑matrix inverses, and accelerated QR decomposition. Convolution, padding, dropout, interpolation‑backward, and reduction kernels have been rewritten in Metal, slashing driver memory for Wan2.2 video VAE decoding from 270 GiB to roughly 8 GiB and fixing silent corruption in Qwen‑Image‑2.1 VAE encodes.

Why it matters for GPU / AI infrastructure

These upgrades turn a Mac into a viable workstation for large‑scale inference and training, reducing the need for discrete GPU clusters for prototyping. FP8 support aligns Apple Silicon with data‑center quantization pipelines, while the memory‑footprint cuts enable video‑generation workloads that previously exceeded unified‑memory limits.