Rerank API: cross-encoder relevance scoring for retrieval
A rerank API puts a trained judge on top of your retrieval. You send one query and the documents your search already pulled back; the route reads the query and each document as one unit, scores the pair for relevance, and hands the shortlist back reordered by that judgment instead of by vector distance. This page is the route family's home: what the route does, where it earns its keep, how it meters, and the model that serves it. The route is wired and verified end to end on this platform, with serving held for the specialty fleet wave while the beta runs.
Last verified: 2026-09-11
What a rerank API does
Large-corpus search leans on embeddings. Each document becomes a vector once, ahead of time, and a query finds its nearest neighbors by distance. That design is fast and cheap precisely because the document side was precomputed, and it is also the weakness. One vector compresses a whole text, so a document can sit near the query vector in space while failing to answer the query in plain language. BAAI's model card draws the line exactly there: an embedding model scores texts it never saw together, while a reranker reads the pair in one pass and judges it directly.
That joint pass is what a cross-encoder is: one query and one document enter the model together as one input, and what comes back is one score for that match-up. Nothing on the document side can be precomputed, which is why rerankers are too expensive to run over a whole corpus and sharp enough to be worth running over a shortlist.
The two designs therefore stack rather than compete. A fast first stage, embedding search or plain keyword matching, pulls a candidate set from the full corpus. The reranker then rescores only those candidates, and the best few move on to the prompt or the results page. BAAI's card runs its evaluations in exactly this arrangement: a retrieval model supplies the candidates, and the measured lift is what the reranker adds to the final ordering.
Where reranking pays off
The clearest case is generation. When a language model answers from retrieved context, the few passages that reach the prompt decide the answer's floor. Rescoring the candidate set before it goes in means the prompt spends its token budget on passages a judge already ranked as relevant, instead of on whatever the vector index happened to surface first. Teams add this stage when retrieval finds the right material but ranks it wrong: the answer exists somewhere in the candidate set, and the model keeps citing the wrong chunk.
The same scoring lifts any ranked list you control. Product search, documentation search, and internal knowledge tools all match queries to candidate texts and then show an order; a score computed from both texts together catches interactions that separately encoded vectors miss, and it drops in without reindexing anything.
A third pattern is triage. Agents, crawlers, and pipelines that gather piles of candidate text need a keep-or-drop decision on each one. One rerank request scores the whole pile against the goal, and the top few survive while the rest is discarded. And because the model behind this route is multilingual by training, corpora that mix languages are not an edge case here; they are the design center.
Calling the rerank route
The route lives at POST /v1/rerank, authorized by a Bearer key you issue on the console. The fields the route accepts are a small set today: model, query, documents, and the optional top_n to return only the leading results. The reply is a results array whose entries point back into your request by index, each carrying its own relevance_score, so the texts you sent never travel back.
Billing on this route follows the work unit the product is sold as, the search. One request counts as a single metered search however many entries its array holds, and the usage record carries the document count unpriced, so volumes stay auditable without billing surprises. Usage draws down a prepaid wallet, and the wallet balance is the spending limit: a search that arrives after it empties is refused with a named error, so the deposit itself, not a credit line, is what spending runs against. Each API key can be revoked on its own, so one leaked key is cut off without touching the rest. Signup runs by email plus an invite code during the beta, and the pricing page answers every rate question at a glance, its table rendered by the machine layer from the catalog.
The route's posture is honest and specific: the wiring is complete and verified against the serving chain, and the card behind it is stopped ahead of the specialty fleet wave. Check the models index for the catalog state as it stands today, and the bge-reranker-v2-m3 model page carries the full working contract, from request fields to response shape to copyable code tabs for curl, python, and the OpenAI SDK.
The model behind the route
One model sits behind this route today: bge-reranker-v2-m3 from BAAI, a multilingual cross-encoder in the BGE family. Its training builds on bge-m3, the family's embedding model, and the weights carry an Apache 2.0 license. The BGE documentation describes it as a lightweight multilingual cross-encoder that stays easy to deploy with fast inference, which is the right profile for a hosted shortlist scorer, and the family's own guidance places it in the recommendation set for multilingual work and for efficiency.
The family pairing is worth knowing when you plan a pipeline. bge-m3, served on the embeddings route of this same platform, is the retrieval half; bge-reranker-v2-m3 is its matching rerank half, trained on the same multilingual base. Choosing between the models inside the wider BGE reranker family, and the context-window numbers that matter for long documents, live on the model page rather than here, because those are single-model facts and this page describes the route family.
Shortlist sizing
The practice question every adopter hits is how much to send. The shape that works: retrieve wide, rerank the top slice, keep the best few. The evaluation arrangement in BAAI's own card retrieves a hundred candidates and keeps the top few after rescoring, and most published examples and evaluations work somewhere between a few dozen and a hundred. Send too little and the ranker has no room to beat retrieval order; send everything and you pay in latency what you save in nothing.
The card's multilingual evaluation is the concrete version of that shape, and it is worth reading as sizing evidence rather than as a benchmark claim: bge-m3 supplies the retrieval, this reranker reorders the candidates it returns, and the reported gain lands on the final ordering. A hundred in, a few out, both halves from one family. When your own pipeline runs that pairing, sizing stops being folklore and becomes one number you can tune: how many candidates your latency budget can afford to rescore.
Two honest notes on size. A larger documents array does not bill as more searches on this route, but the response does take longer, because every entry gets scored. And if your candidate set is small because your corpus is small, reranking may be the wrong spend entirely: with twenty documents total, the retriever alone often orders them well enough. The stage pays for itself when there is a real candidate pile to cut down.
Open weights and self-hosting
An Apache 2.0 license keeps the self-host road open and legitimate. BAAI's FlagEmbedding library ships the FlagReranker class for direct scoring, and Hugging Face documents its text-embeddings-inference server as serving this model family over a rerank endpoint. Those are the official routes, and for teams with steady traffic, existing GPUs, or hard data boundaries, self-hosting is often the right call.
What a hosted route adds is everything around the model: the metering law, the wallet stop, key handling, and one JSON contract that does not drift as weights and tooling move. The trade is worth making with eyes open, because the local tooling picture for rerankers specifically has gaps and endpoint mismatches that cost debugging time. The model page maps that picture, including the dead ends, so the decision can be made before a week is lost to it.
Getting started
- Sign up on the console. Registration is email-based with the beta invite code, and the console holds the wallet balance and every key you create.
- Make an API key. Treat one key per project as the working shape: any single key can be revoked alone, which is the containment you want when a key leaks.
- Check the rate. The pricing page carries the per-search rate for this route, and the prepaid wallet makes your deposited balance the ceiling: a runaway loop drains what you put in and then stops.
- Make the first call. Grab a code tab from the bge-reranker-v2-m3 model page, drop your key in, and score a toy shortlist. When the ordering matches what you expect, bigger cases are just bigger arrays.
The models index maps the full catalog: chat, speech, embeddings, parsing, and rerank.