Embedding dimensions and context length, model by model

Every embedding model carries two numbers you live with for years. One is the width of the vector. The other is the window of text it reads per call. Width sets your storage bill for the life of the index. The window sets how you chunk. This page holds both numbers in one table, read from each provider's own pages on 28 October 2026. Sixteen rows cover the two models this platform serves, the big hosted providers, the open Qwen3 family, and Mistral. Every cell is dated and sourced at the provider's own surface, and the log holds each attempt chain. Two decisions follow the table: which width to store at, and whether to chunk or read whole.

Each row names its owner and its read date. The OpenAI rows come from its embeddings guide. Google's rows come from the Gemini embeddings docs and Google's own model page. Cohere's rows come from its model documentation. Voyage's come from its embeddings docs and its dimensions guide. Open-family rows come from their model cards on Hugging Face, and the Mistral row from its own model page. All were read 28 October 2026, every cell included. One cell took a rendered re-read of its card to pin, and the log holds that chain with the rest. The pages ranking today paste dimension columns with no dates and no source. A spec with no date is a guess wearing a table.

Last verified: 2026-10-28

The two numbers, in plain words

The dimension count is how many numbers the model returns per text. A 1,024-dimension model returns one thousand twenty four floats for a sentence, for a paragraph, and for a whole page alike. The count is fixed by the model. You store it, index it, and pay memory on it for the life of the index.

The context window is how much text the model reads to produce that vector. A 512-token model and a 32,000-token model return the same kind of object. The first reads a paragraph. The second reads a chapter. What changes is how much of your document survived into the vector.

Open models cluster at a few widths. The small classic, all-MiniLM-L6-v2, returns 384 numbers per text, and its own config caps input at 256 tokens. The BGE small English model returns 384 too, over 512 positions, per its config. Seven hundred sixty eight is the BERT-generation default; nomic-embed-text-v1.5 carries it over an 8,192-token window, per its model card. One thousand twenty four is the current open standard, and 4,096 is the top of the open Qwen3 family. The big hosted defaults sit wider: 1,536 and 3,072.

Width is not quality. That is the single most useful correction on this shelf. OpenAI's own guide states that its 3,072-dimension model, cut hard to 256 dimensions, still outscores the older 1,536-dimension model at full width on MTEB. A quarter of the width beat the legacy full size. Buy width when a measured retrieval gap says so, not because the number looks bigger.

The selection table, cell by cell

Sixteen rows. Read the truncation column as a trained capability, not a hack. Models listed there were trained so their first k numbers still work, per the Matryoshka paper that named the method.

Three rows deserve a second read. The Gemini Embedding 2 row shares one 8,192-token budget across every input type, and Google's docs state that longer input is silently cut. The 001 row is the reminder that one vendor can carry two very different windows: 2,048 tokens against 8,192. And the Mistral row took four attempts. The old docs path returns not found, and the embeddings page redirects to a Studio page. The live model page, found through the model list, carries both numbers: 1,024 dimensions and an 8k context.

One more provider fact worth knowing while you scan. Anthropic offers no embedding model of its own. Its own guide says so plainly and points builders at Voyage. If your stack standardizes on one vendor's docs, that is the row that explains the hole.

The width decision: storage math you can pencil

Per-token API pricing ignores width. Your vector store does not. The arithmetic is four lines.

raw vector bytes  =  vectors x dimensions x bytes per dimension

vectors:               1,000,000 chunks, a mid-size corpus
float32:               4 bytes per dimension
int8:                  1 byte per dimension

1,024 dims, float32:   1,000,000 x 1,024 x 4 = 4,096,000,000 bytes = 4,096 MB
2,048 dims, float32:   1,000,000 x 2,048 x 4 = 8,192,000,000 bytes = 8,192 MB
3,072 dims, float32:   1,000,000 x 3,072 x 4 = 12,288,000,000 bytes = 12,288 MB
3,072 dims cut to 768: 1,000,000 x   768 x 4 = 3,072,000,000 bytes = 3,072 MB
1,024 dims as int8:    1,000,000 x 1,024 x 1 = 1,024,000,000 bytes = 1,024 MB

Every output above is the product of the printed inputs. Index overhead lands on top of these numbers, and it varies by store. The raw floor does not vary. A 3,072-wide model costs three times the memory of a 1,024-wide one at the same corpus, before any vendor rate enters the picture. For the rate side, the embedding pricing guide owns every dated comparison.

Trained truncation is the lever that makes width a dial instead of a commitment. The method has a name, Matryoshka Representation Learning, and four publishers now ship it. What it buys, in each publisher's own measured words:

The pattern across four publishers: cuts down to roughly 256 dimensions cost one to two percent of measured quality. Cuts to 64 cost real quality. So the working rule is simple. Store the full vector if you can. If you cannot, cut to a trained stop near 256 rather than to the bone.

Two cautions keep the dial honest. First, only trained models take the cut. Cutting a plain 1,024 vector to 256 by hand degrades it, and OpenAI's guide tells you to re-normalize after any manual cut. Second, the dial is per model. One space per index, always. Mix vectors from two models and the distances stop meaning anything, whatever the widths say.

Quantization is the second lever, and it stacks with truncation. Voyage's quantization guide states the byte counts: float32 at four bytes per dimension, int8 at one byte, binary at one bit. That is a quarter of float32 for int8, and a thirty-second for binary. Cohere's Embed 5 ships int8 and binary output directly. A 3,072-wide vector at int8 costs 3,072 MB per million from the fence above: the same floor as a 768-wide float32. Width and precision are two dials on the same bill.

The window decision: read whole, or cut into chunks

The window answers one question: how much text becomes one vector. The ladder above spans 512 tokens to 262,144. In the planning convention OpenAI's own guide uses, roughly 800 tokens per page of prose, that ladder reads:

512 tokens     = under one page
2,048 tokens   = about 2 and a half pages
8,192 tokens   = about 10 pages
32,000 tokens  = about 40 pages
128K tokens    = about 160 pages
262,144 tokens = about 328 pages

The wide windows changed the design question. A 512-token model forces chunking on you: every document gets sliced before it gets embedded. An 8,192-token window takes a full support article or legal clause in one pass. A 128K or 262,144-position window takes whole documents, and in the multimodal rows, whole visual documents as one input.

Chunk less, but do not stop chunking. Two facts carry that sentence. First, one vector per long document blurs. A 300-page manual compressed into a single 2,048-number vector answers questions about the manual's general subject, not about page 214. Retrieval that must land on the right section still needs section-sized chunks. Second, long windows are not free quality. A 2026 benchmark study of multimodal long-context embedding models, published on arXiv, reports a plain result. Larger context windows do not necessarily translate into effective comprehension of long inputs. The window is an input budget, not an accuracy guarantee.

Know what happens at the edge of the window, because providers differ and they differ silently. Google's Gemini docs state that input past the 8,192-token limit is silently truncated. Voyage's client truncates over-length input by default, per its embeddings docs. It raises an error only when you switch that default off. Ollama's local embed endpoint works the same way. Its reference page lists a truncate flag, default true. Over-length input is cut, and the call still succeeds. Send it false and you get the error instead.

One planning rule survives every provider. Pick a chunk size well under the window of the model you committed to. Embed query and corpus with that one model. Put a length check in the pipeline, so nothing depends on a silent cut. When a document must stay whole, pick the model by its window first and accept its width. When retrieval must land on passages, chunk for the passage and let the window breathe.

The local runtime column

The dimensions question does not end at hosted APIs. The vendor tail people search most, after the head query, is the local runtime. Ollama's own docs put its model range at typically 384 to 1,024 dimensions, and its endpoint returns unit-length vectors. Its embed reference also lists a dimensions parameter for models that support one.

Two honest notes on that table. The nomic row shows the card window and the runtime default side by side because they differ. Its card handles 8,192 tokens. The runtime page lists a 2K window. Check the runtime's number before you plan chunk sizes around the card's. The embeddinggemma row took a fourth surface to read: a plain fetch of the card answers 401, but a rendered re-read returns the full card body. It states 768 output dimensions, with trained 512, 256, and 128 options, plus the same 2,048-token window.

Where this platform's lane sits

The embeddings route here serves two rows as of 28 October 2026. One is bge-m3: 1,024 fixed width, an 8,192-token window, more than 100 languages, dense, sparse, and multi-vector outputs from one pass, open weights under MIT. The other is wemm-embed: 2,048 width over text, images, video, and visual documents, with six trained truncation stops. Its 262,144-position window makes whole visual documents a single-input job.

One width is fixed and one is a dial. That is the honest shape of the lane. Neither row will chase the 4,096 top of the open family, and nothing on this page should imply otherwise. What the lane sells on this axis is the spread. One row is text at the open standard width. The other is multimodal, with a window and truncation stops at the wide end of every table above. Rates, metering, and the per-token bill are not this page's question. The pricing guide carries the live numbers, rendered from the same platform data the metering reads.

The lane is young. For the wider comparison, model against model on quality and cost posture, the bge-m3 versus text-embedding-3 guide owns it. The local-versus-hosted trade for embeddings is worked through in the embeddings break-even guide. Where rerank fits after retrieval is the two-stage retrieval decision.

Make the first call

The route is OpenAI compatible. Sign up, create a key, and run:

curl -X POST "https://api.ironstratum.com/v1/embeddings" \
  -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "bge-m3",
    "input": [
      "One chunk of the corpus, read inside an 8,192 token window.",
      "A second chunk, returned as one thousand twenty four numbers."
    ],
    "encoding_format": "float"
  }'

Each vector comes back 1,024 floats long, and the usage block in the response is the metering record. The wallet is prepaid, and a refused request bills nothing. The bge-m3 model page holds the full request contract, and the models index is the live catalog.

Choose in 60 seconds

Work the list and stop at the first fit.

1. Storage is the constraint: pick a model with trained truncation, store at full width if it fits, cut to the 256-class stop if it does not, and quantize to int8 before you cut to 64.
2. Documents are short and uniform: any window works; chunk at 512 to 1,000 tokens and stop worrying.
3. Documents are long but retrieval must land on passages: chunk for the passage size you cite, and pick the window with headroom above it.
4. Whole documents or whole visual documents must stay whole: pick by window first, 32K, 128K, or 262,144 positions, and accept the width that comes with it.
5. The corpus is multilingual: the window and width matter less than the language coverage row; bge-m3 and the big multilingual hosted models cover it, and the best-models guide ranks that axis.
6. You are standardizing on one vendor's docs: check the holes first. Anthropic points elsewhere for embeddings, and Mistral buries its specs on a dated model page the old links no longer reach.

Common questions

What is a good embedding size? There is no universal number, but the shelf has a shape. Open models cluster at 384, 768, and 1,024. Hosted defaults run 1,536 to 3,072. Publisher measurements show cuts to the 256 class cost one to two percent of quality, while OpenAI's own table has a 256-dimension vector beating a 1,536-dimension legacy model. Start at the model's default, measure retrieval on your corpus, then cut to a trained stop if storage says so.

Does context length matter for embeddings? Yes, but as a budget, not a score. The window decides how much of each document becomes one vector. A 512-token window forces aggressive chunking. An 8,192-token window takes a full article. A 262,144-position window takes whole visual documents. Past the point where your chunks fit, more window buys little, and one 2026 benchmark study reports that larger context windows do not necessarily translate into effective comprehension.

Can I shrink vectors after embedding them? Only on models trained for it. Matryoshka-trained models let you keep the first k numbers of the stored full vector and re-truncate later without re-embedding. OpenAI, Google, Cohere, Voyage, Jina, Qwen3, nomic, and WeMM all ship trained truncation on at least one current model. An untrained model, like bge-m3, has no published truncation support: cutting its 1,024 vector short degrades quality with no vendor support.

What happens if my text exceeds the context window? It depends on the provider, and several cut silently. Gemini's docs state that input past 8,192 shared tokens is silently truncated. Voyage's client truncates by default and errors only on request. Ollama's local endpoint truncates by default under a flag you can switch. The safe pattern is a length check in your pipeline, plus chunk sizes set under the window.

Do more dimensions mean better search? Not by themselves. Width is capacity, not accuracy. The measured ladders on this page show small models at full width losing to wide models cut to a quarter. Quality differences between models come from training, and the way to see them is a retrieval evaluation on your own corpus. Width sets your storage and memory bill; it does not set your relevance.

Which row does this platform serve? Two, as of 28 October 2026: bge-m3 at 1,024 fixed width and an 8,192-token window for text, and wemm-embed at 2,048 width with six trained stops and a 262,144-position window for text, images, video, and visual documents. Both meter per token against one prepaid wallet, and the pricing guide carries the live rates.

The short version

Two numbers pick an embedding model. Width sets the storage bill. Pencil it as vectors times dimensions times bytes. Cut it with trained truncation near 256, then cut it again with int8. The window sets the chunking: 512 forces slicing, 8,192 takes pages, 262,144 positions take whole visual documents. Silent truncation at the edge is real on three of the providers read today. This page read every provider's own surface on 28 October 2026. It printed sixteen dated model rows and a six-rung measured truncation ladder, every cell sourced. The platform's lane holds two rows at the open standard width and the wide multimodal end. When the question turns from specs to bills, the pricing comparison and the pricing guide carry it. Ready to measure on your own corpus? Sign up and embed two chunks.

Last verified: 2026-10-28