New NEON kernel speeds up llama.cpp prompt processing on Arm CPUs up to 6.5x
A community patch to llama.cpp (164 lines across two files) adds an Arm NEON inner kernel using the SDOT instruction for the tiled K-quant matmul routine, which previously lacked an Arm-specific path for formats like Q3_K, Q2_K and the IQ family. Benchmarks on Qwen2.5 1.5B across three Arm Neoverse server CPUs (N1, V1, V2) show prompt-processing speedups ranging from about 1.7x to 6.5x for low-bit quantized formats, with the largest gain on IQ3_M on Neoverse N1; formats that already had an Arm kernel, such as Q4_K_M and Q8_0, show no change, and token generation speed is unaffected.