Reranking
Reranking is a second sorting step in search systems: a more precise model re-evaluates the best hits of a fast preselection and moves the truly relevant results to the top.
In practice
Behind this is a division of labour. The first stage (such as vector search with a bi-encoder) scans millions of entries quickly but roughly. The second stage, a so-called cross-encoder, reads query and hit together and judges the match far more accurately – but is too slow for the full collection. Microsoft's search service, for example, therefore re-sorts only the leading fifty hits.
For RAG systems this is a lever with real impact: only what sits at the top of the hit list ends up in the request to the language model. Better ordering therefore directly means better answers. The price is additional computing time per query.
For RAG systems this is a lever with real impact: only what sits at the top of the hit list ends up in the request to the language model. Better ordering therefore directly means better answers. The price is additional computing time per query.