Evaluating modular RAG with reasoning models

kapa.ai is an LLM-powered AI assistant that answers technical questions about your products. We integrate with knowledge sources to answer user questions and support tickets through a RAG pipeline. Building and maintaining a robust, general-purpose RAG system is hard. There are many controls and parameters that influence the final output's quality, and they all interact in complex ways: Prompt templates Context size Query expansion Chunking Reranking Etc. As we make changes to the system, and particularly when integrating new models, revisiting and refining these parameters is crucial to maintain good performance. But this is also time-consuming to do and takes experience to get right. New reasoning models like DeepSeek-R1 and OpenAI’s o3-mini produce impressive results by reasoning through built-in Chain-of-Thought (CoT) prompting, where the model is designed to “think” through problems step by step and even self-correct when needed. These models reportedly perform better at ... Read full article.