Kapa

Kapa releases Company Knowledge Bench, a 1,000-case retrieval eval for agents

Kapa, a platform that indexes company knowledge for AI agents, built an internal benchmark of 1,000 annotated eval cases from real production data to compare retrieval methods. The company tested seven retrievers—including traditional hybrid search, agentic grep-based search, and its own Kapa systems—measuring both accuracy and query time. Results showed Kapa's 'Deep' retriever scoring 0.65 in about five seconds, while a frontier model using only grep matched a tuned pipeline's 0.61 score but took roughly five times longer.