GoKawiilReverse-engineered fix restores lost DRAM bandwidth on Apple's M3 Neural Engine
A developer profiling Apple's M3 Neural Engine found an RTL performance bug that cuts DRAM weight-streaming speed from a normal 45-60 GB/s down to just 17-19 GB/s whenever a model's total weight size lands on an exact multiple of 1 MiB, affecting 7 of the 15 models in the ANEMLL project. By rerouting around the flawed speculative prefetch path in the kernel's DMA engine, the developer restored full throughput, more than doubling token generation speeds for Llama 3.2 1B and Qwen3-8B.
GoKawiil's interpretation of the reporting above, not reported fact.
This is a hardware-level inefficiency baked into Apple Silicon's dedicated AI accelerator, not a software configuration issue, meaning any model whose weight footprint happens to align with 1 MiB boundaries silently runs at roughly a third of its potential speed. The discovery gives on-device AI developers a concrete workaround to reclaim significant performance without new hardware, which matters increasingly as more inference shifts from cloud servers to local Apple devices.
- Apple M3 Neural Engine has an RTL bug throttling DRAM throughput to 17-19 GB/s instead of 45-60 GB/s at certain weight sizes
- The bug triggers when total model weight size is an exact multiple of 1 MiB, affecting 7 of 15 ANEMLL models
- Avoiding the faulty DMA prefetch path more than doubled token throughput for Llama 3.2 1B and Qwen3-8B
Apple Mac mini (M4) — If this deep dive into Neural Engine weight streaming has you itching to run Llama or Qwen models locally, the Apple Mac mini with the M4 chip is a compact way to get Apple silicon's Neural Engine and unified memory on your desk. It's an inexpensive entry point for experimenting with ANEMLL and on-device LLM inference without a giant tower.
See Apple Mac mini (M4) on Amazon → Affiliate link — we may earn a commission on purchases, at no extra cost to you. Product picked by AI based on this article; it is not a tested recommendation.Source: eiln.github.io, 2026-09-10
Published there as: “Getting 50 GB/S Back from the Apple Neural Engine”
Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.