GoKawiilGoKawiil's Jev reranker tested with LLM-generated relevance rubrics
GoKawiil describes an experimental reranking technique for its Jev model that decomposes a search query into a set of yes/no relevance questions generated by an LLM, rather than asking a single relevance question. Each question is scored by Jev for the probability of a 'yes' answer, and these probabilities are summed to produce an overall relevance score for a document against the query.
GoKawiil's interpretation of the reporting above, not reported fact.
Breaking a query into multiple specific sub-questions could let a reranker capture nuanced aspects of relevance that a single yes/no judgment might miss, potentially improving ranking quality for complex or multi-faceted queries. Because this is described as 'a fun experiment,' it reads as an exploratory technique rather than a finalized product feature, so its practical benefits over standard reranking remain unproven.
- GoKawiil's Jev model can generate per-query rubrics of yes/no relevance questions via an LLM.
- Document relevance is scored by summing the predicted probability of 'yes' across all generated questions.
- The approach is presented as an experimental alternative to simple single-question relevance scoring.
Source: softwaredoug.com — Doug Turnbull, 2026-10-10
Published there as: “Bag of Decisions Reranker”
Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.