GoKawiilNowadays, most people seem to over-engineer their RAG stack. They jump straight to embeddings, vector databases, and reranking pipelines. Meanwhile, their users just want to find the doc that says “How to reset my password.”
In engineering, there’s always the right tool for the right problem. In AI Retrieval Systems it’s not different.
The Decision Factors
Before we dive into recipes, let’s establish when you should use each approach. The key factors are:
1. Data Freshness Requirements - Real-time updates (news, social media) favor approaches with easy re-indexing. Daily or weekly updates work well with hybrid approaches. A stable corpus (monthly or quarterly updates) makes pre-embedding sensible.
2. Corpus Characteristics - High churn (more than 10% changes daily) means you should avoid full pre-embedding. Stable documents work fine with pre-embedding. Long-tail distribution (90% never accessed) means on-the-fly wins.
3. Query Patterns - Keyword-heavy queries should start with full-text search. Semantic or conversational queries benefit from embeddings. Mixed patterns need hybrid approaches.
4. Scale & Performance - Less than 1000 queries per day means simple approaches are sufficient. 1K to 10K queries per day requires selective optimization. More than 10K queries per day justifies full optimization.
5. Team Capabilities - No ML expertise means stay with full-text plus query rewriting. Some ML experience makes hybrid search manageable. Having an ML team available makes advanced approaches viable.
Now, let’s look at the recipe book. Start at the top. Move down only when you have data proving you need to.
... continue reading