TL;DR – We are excited to introduce the rerank-2.5 series, which significantly improves upon rerank-2 ’s performance while also introducing instruction-following capabilities for the first time. On our standard suite of 93 retrieval datasets spanning multiple domains, rerank-2.5 and rerank-2.5-lite improve retrieval accuracy by 7.94% and 7.16% over Cohere Rerank v3.5. Furthermore, the new instruction-following feature allows users to steer the model’s output relevance scores using natural language. On the Massive Instructed Retrieval Benchmark (MAIR), rerank-2.5 and rerank-2.5-lite outperform Cohere Rerank v3.5 by 12.70% and 10.36%, respectively, and by similar margins on our in-house evaluation datasets. Both models now support a 32K token context length – 8x that of Cohere Rerank v3.5 and double that of rerank-2 – enabling more accurate retrieval across longer documents.
Rerankers are a critical component in sophisticated retrieval systems, refining initial search results to deliver superior accuracy. Today, we are excited to announce rerank-2.5 and rerank-2.5-lite . Both models outperform LLMs as rerankers – a topic which we will dive deeper into in an upcoming blog post. These models are the product of an improved mixture of training data and advanced distillation techniques from our larger, in-house instruction-following models.
Both rerank-2.5 and rerank-2.5-lite now support a 32K token context length, an 8x increase over Cohere Rerank v3.5. This allows for the reranking of much longer documents without truncation and comes with no change in pricing.
For an introduction into rerankers, check out our previous post.
Instruction-following capability
A key feature of the rerank-2.5 series is its instruction-following capability. This allows users to dynamically steer the reranking process by providing explicit instructions alongside their query. These instructions can define the user’s notion of relevance or specify the desired characteristics of the documents to be retrieved. Leveraging the new instruction-following capability is straightforward. Users can simply append or prepend natural language instructions to their queries. The model is designed to understand these instructions and adjust the output relevance score accordingly.
Examples of instructions. Instructions can include, but are not limited to, the following examples:
Emphasizing query components: Specify which parts of a document are most important. For an application that searches academic papers, a standing instruction could be “Prioritize the title and ignore the abstract” to consistently surface the most relevant research based on titles.
Defining document types: Direct the reranker to retrieve a specific type of document (e.g., for the query “legal implications of AI,” an instruction could be “Retrieve regulatory documents and legal statutes, not court cases.”). For example, a legal research tool could be configured with the instruction “Retrieve regulatory documents and legal statutes, not court cases” to ensure that all queries prioritize statutory law over case law.
Disambiguating queries with contexts: Provide complementary information so that ambiguous queries can be clarified. For example, an instruction could be “This is an e-commerce application about cars” so that the word “Jaguar” will be interpreted as the car brand rather than as an animal.
... continue reading