Apple researchers propose learned routing networks to speed up MIPS for RAG
Apple researchers describe a method presented at ICML 2026 that trains small neural networks offline, once per fixed key database, to route incoming queries toward their best-matching vectors faster than standard approximate maximum inner product search (MIPS) techniques. The approach is designed to exploit the fact that, in deployed systems, queries often follow a stable, characterizable distribution rather than being arbitrary or key-like, as existing clustering and quantization-based index methods typically assume.