Embeddings local vs API: when your own GPU beats the per-token bill

An open embedding model on your own GPU and the same weights behind a metered API cost the same at one daily token volume. This page finds that point for bge-m3. The inputs are the platform's measured lane grid from 28 August 2026, plus October 2026 street prices for two used cards. Maker watt specs and the official United States power price finish the set. The answer lands in tokens per day, and in one number most guides skip: how busy the card must stay before its own power costs less than the rate.

The pages that rank for this question argue quality, list models, or price rented cloud GPUs by the hour. None of them prices the card on your desk. The cost threads that ask the question directly get anecdotes. The break-even shape is proven on this site for speech already. This page runs it for embeddings, and the answer is more blunt: embeddings carry the stack's floor rates, and the card rarely wins on cost at all.

All amounts below are US dollar amounts, written as bare numerals. Power is quoted in cents per kilowatt-hour. Rates for this platform's embeddings rows are never typed into prose. They render from the catalog in the fence further down, and every worked row reads its rate variable off that fence. Every outside number carries its source and its read date.

Last verified: 2026-10-28

The short answer

Three sentences carry the page. A used 3060-class card bought at 300 breaks even near 25 million tokens a day at the fence rate, if it runs near flat out and sleeps between jobs. That volume is about three quarters of the lane's measured daily ceiling, so only an almost always busy card gets there. And against the hosted copy of the same weights, no owned card breaks even at any volume. That copy lists at 0.010 per million tokens on DeepInfra's own model page, read 28 October 2026, and the electricity alone costs more than that rate.

Two frames make the number real. In pages, 25 million tokens is about 31,000 dense pages a day at the 800-tokens-per-page planning convention, every day, for two years. In spend, that volume is a hosted bill of about 38 a month at the fence rate. Most teams embedding a corpus do not have that bill. Their index pass happens once, their queries add a thin stream, and the card would idle through most of its two year payback window. That shape, not the rate, is what usually settles the question, and it gets its own section below.

What local embedding actually costs

The weights are free. bge-m3 ships from BAAI under the MIT license, per its model card, at about 568M parameters. The Ollama library entry is a 1.2 GB download with an 8K token window and 8 million downloads to date. So the money question has four pieces: the hardware, the watts it draws, the price of that power, and how fast the card turns tokens into vectors.

Hardware, at dated reads. One used-price guide tracks completed eBay sold listings, and both card reads below come from that one source. Its read through 24 October 2026 put the used RTX 3060 near 300, a band it has held since summer. The same guide priced the used RTX 3090 at 1,093 that day, with a 30 day average of 1,115. Its 3060 page does not split the 8 GB and 12 GB variants, so check the memory size on any listing. Use whatever your own listings show. The formula does not care which source you trust.

Power draw comes from the maker. NVIDIA's RTX 3060 spec page lists 170 W board power with a 550 W system supply. The RTX 3090 page lists 350 W with a 750 W supply. Embedding draws less than a gaming maximum. The worked tiers use 250 W at the wall for a 3060-class box and 450 W for a 3090-class box, both under the maker's combined ceilings.

Power price comes from the government. The EIA Short-Term Energy Outlook, read 28 October 2026, puts the United States home average at 18.2 cents per kilowatt-hour for 2026. The 2025 figure was 17.3. Your own bill is the number that matters.

Then the piece nobody prices: speed, and its twin, load. The platform's embeddings lane was measured on a 3060-class card during its 2026-08-28 window, and the grid is published on the bge-m3 model page. Median response was 1.38 seconds at single flight and 4.33 seconds at 32 requests in flight. Aggregate throughput rose about ten times along the way, from 36.9 to about 378 tokens a second, a chars-per-four estimate on short texts. Two capacities follow on that lane basis. The lane's busiest measured setting moved near 32.7 million tokens a day, and single flight about 3.2 million. Those are the shared lane's numbers, not the card's physical limits. The grid was taken with up to 32 requests in flight on a served route, not on a dedicated local batch run, so a tuned local setup on the same silicon can move faster. Every ceiling below carries that basis with it.

Speed turns watts into a per-token cost. At the national power price, a 250 W box embeds a million tokens for about 0.033 in electricity at the lane's flat-out setting, and about 0.343 at single flight. The same card is ten times dearer to run when it sits idle between single requests. That spread is the whole secret of this comparison, and the next section turns it into a bar you can check.

What hosted embedding costs

The hosted cost of T tokens a day is T times the per-million rate, divided by a million. Read that rate off the fence for this platform, or off any vendor's own card. This lane is young and sits mid-band. As of 28 October 2026 the route serves two rows, bge-m3 for text and wemm-embed for multimodal work. Both meter per token against one prepaid wallet. What the rate buys is the aggregate: two rows, one request shape, one bill, and an open exit, because both models' weights are downloadable.

The full vendor band is not this page's job. The embedding pricing companion carries every provider's rate, unit rule, and batch tier, dated row by row. Two anchors from it matter here. The closed vendors bracket this lane's rate from both sides, and two of them halve their rate for batch work that can wait a day. The discount end of the band serves the same open weights this page measures. DeepInfra lists bge-m3 at 0.010 per million tokens on its own model page, read 28 October 2026. That is the honest floor, and this page works its break-even against both anchors.

The load bar: the number no ranking page prices

Before any hardware payback, one bar has to clear. The card's electricity per million tokens must drop below the hosted rate, and electricity per million falls only as the card runs busier. The bar is a rate matcher:

parity throughput f*  =  E x R / ( 3.6 x P )

E:  wall draw in watts while embedding, 250 or 450 in the worked tiers
R:  power price in dollars per kilowatt-hour, 0.182, the 2026 United
    States residential average, EIA Short-Term Energy Outlook, read
    28 October 2026
P:  the hosted rate per million tokens, read off the fence above

Read the first row twice, because it reframes the whole question. A 3060-class card must sustain about 253 tokens a second, two thirds of its measured flat-out rate, before its power alone costs less than the fence rate. Below that bar, every token you embed at home costs more in electricity than the hosted rate charges. No payback schedule can rescue that. At single flight, 36.9 tokens a second, the power cost is about seven times the fence rate. The second row is a bar only, not a claim of passage. One published figure does exist, from a GPU host's benchmark table: about 7,200 documents a second on a 3090. Its own setup sizes documents at 200 tokens, which makes the claim 1.44 million tokens a second through a 568M-parameter model. That sits more than 20 times past what the card's silicon can do at a perfect run, and the same host's RAG page measures the same model on the same card at 570 tokens a second. The figure is rejected on that arithmetic, not missed by search, and the 455 bar stands as a threshold to test. Dashes have receipts; the found-and-rejected chain is logged in this page's research record.

The break-even table

With the load bar cleared, the volume question is arithmetic. The break-even point is the daily token count D* where both sides cost the same:

D*  =  [ G / (30 x A) ]  /  [ ( P  -  E x R / (3.6 x f) ) / 1,000,000 ]

G:  what the hardware cost you, 300 or 1,093 at the dated street reads
A:  payback window in months, 24
E:  wall draw in watts, 250 or 450, from the maker's board power
R:  0.182, the 2026 national average power price
f:  sustained throughput in tokens a second, 378 at flat-out load
P:  the hosted rate per million tokens, off the fence above

Every row below regenerates from these inputs.

The sensitivity line, once: the break-even moves against the rate gap. Halve the gap and the volume doubles. Idle watts count against the card around the clock, whether it embeds or not.

Two rows deserve their plain words. The always-on row is the trap most self-host plans walk into. A box that idles at 60 W burns about 0.26 a day, and that pushes the required volume past what the lane grid moved in a day. The idle burn is unconditional; the never verdict is not, because a local setup that beats the lane figure can close the gap. Let sleeping boxes sleep, or do not buy the card. The zero-cost row is the quiet win for readers who already own the hardware. At flat-out load their power runs about two thirds of the fence rate, so the marginal saving is real, about a third off. It vanishes the moment the card sits under-loaded.

Index once, query forever: the shape that breaks the table

Speech volume repeats every day, so a speech break-even table transfers cleanly. Embedding volume usually does not repeat. A RAG pipeline embeds its corpus once, then embeds short queries forever. The corpus pass is a spike. The query stream is dust.

Price the spike with the worked corpus from the pricing companion: 50,000 pages at 800 tokens a page is 40 million tokens. At the fence rate that pass costs about 2. On the owned card the pass takes about 29 hours at flat-out load, and its electricity runs about 1.34. The card saves two thirds of one dollar on the single pass that motivates buying it. The query stream cannot rescue that. A thousand queries a day at 30 tokens each is 30,000 tokens a day. That rounds to pennies a month at any rate on the board, hosted or local.

What forces re-spend is churn and model drift. Every corpus update re-embeds what changed. Every model switch re-embeds everything, at the new rate, into a new vector width. Closed vendors move their catalogs. OpenAI still lists its legacy ada embedding model at five times the rate of its newer small model, per its pricing page read 28 October 2026. Staying put costs money too. A local checkpoint never faces either side of that squeeze. The pricing companion prices the exit, and the dimensions and context table carries the width side. For the model-choice question itself, the best models for RAG guide is the place.

So the honest cost verdict is narrow. Against the fence rate, an almost always busy card paid 300 wins after about 25 million tokens a day. Against the 0.010 floor for the same weights, nothing wins. The pass that costs 2 at the fence costs 0.4 there, under the electricity for the same work at home. When a team still goes local for bge-m3, cost is rarely the reason. The reasons that hold are privacy, a checkpoint that never retires, and the two extra outputs.

The two extra outputs, and the 2B multimodal row

The hosted request shape returns one dense vector per text. The model makes three retrieval outputs, and the other two, sparse token weights and per-token multi-vectors, matter for hybrid search. BAAI's BGE documentation publishes the scoring math for all three. Today those two outputs are a local job, through BAAI's FlagEmbedding library or a store that indexes both shapes. That is a real local reason with no rate math attached.

The lane's second row changes the local bill in one way: size. WeMM-Embedding-2B runs about 2.72B parameters, roughly 5.4 GB of weights at bf16 by plain math, per its published spec sheet. That is near five times the weights of bge-m3, so the card class moves up a tier before the math even starts. Its card lists local routes with versions: vLLM 0.27.0 in pooling mode, SGLang 0.5.9, and sentence-transformers 5.7.0 or newer. No published local speed figure for this model exists as of 28 October 2026, after two search passes, so its local speed cell prints as a dash rather than a guess. On the hosted side it rides the same route, metered flat per token across text, image, and video inputs, and the fence renders its row.

The same request, two ways

The request shape is OpenAI compatible on both sides. The choice stays reversible: one body, two destinations.

Local, the fast start is the official Ollama route named above:

ollama pull bge-m3
curl http://localhost:11434/v1/embeddings \
  -d '{"model": "bge-m3", "input": "The same weights, embedded at home."}'

That serves the dense vector. For the sparse and multi-vector outputs, BAAI's FlagEmbedding library is the official path, named on the model card. One caution comes from community reports. The same GGUF file has produced vectors that differ in scale between two local runtimes. Re-embed the corpus when you switch runtimes. Do not mix vectors across that line.

Hosted, the same embedding through this platform's endpoint:

curl -sS $IRONSTRATUM_BASE_URL/v1/embeddings \
  -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "bge-m3",
    "input": [
      "One page of the corpus, priced per token.",
      "A second page, stored as one thousand twenty four numbers."
    ],
    "encoding_format": "float"
  }'

Three contract facts before the first call. Metering counts tokens from the response's own usage block, summed across the batch, against a prepaid wallet whose balance is the spend ceiling. A refused request bills nothing. Each vector comes back 1,024 floats wide, which sets the storage bill the rate card never shows; that arithmetic lives in the pricing companion. To try the numbers against a real wallet: sign up, create a key, and put a small balance behind it before you commit to either side.

Decision rules

Frequently asked questions

Is it cheaper to run embeddings locally?

Rarely, and only under load. A used 300 card breaks even near 25 million tokens a day at the fence rate, run near flat out with the box asleep between jobs. Below that volume the hosted side is cheaper, and against the 0.010 per million floor for the same weights, the electricity alone costs more than the rate at any load in the measured grid.

How many tokens a day make a local GPU cheaper?

About 25 million on a used 3060-class card at 300 over 24 months, at flat-out load, at the fence rate. In pages that is about 31,000 a day. With the box always on at a 60 W idle draw, the required volume rises past the lane's measured ceiling, and on that basis the answer is never. A local batch run faster than the shared lane can flip it, so test the row with your own sustained rate.

Can bge-m3 run on CPU only?

Yes, at lower speed, and with no published per-CPU speed number to plan against. The model is about 568M parameters, and ONNX int8 builds exist that shrink it further. My searches found working CPU builds but no measured tokens-per-second figure on any named CPU, so this page prints no CPU speed claim. Treat CPU as viable for small nightly batches, not for a break-even case.

What does the electricity cost per million tokens?

About 0.033 at the 2026 national average power price on a 250 W box running flat out, and about 0.343 at single flight, at the measured 36.9 tokens a second. The first number beats the fence rate. The second is about seven times it. Load, not the watt rating, decides which number you pay.

When is local the right call regardless of cost?

Three cases hold. First, a corpus that cannot legally leave your machines. Second, the sparse and multi-vector outputs, which the plain hosted shape does not serve. Third, a checkpoint that never retires. Every model switch re-embeds the corpus, and vendor catalogs do move, with legacy rows paying premium rates. A local copy sits outside that churn.

Is the hosted lane the same model I would run locally?

The same open weights, yes. The lane serves bge-m3 under its own alias, returns the dense 1,024-wide vector, and meters per token from the usage block. What differs is where it runs, who keeps it up, and how you pay: against a prepaid wallet, with the balance as the hard stop. Both models' weights stay downloadable, so the exit from the lane is open in both directions.

Prices and metering units for every row sit on the pricing page. The vendor band, the batch tiers, and the storage bill live in the embedding pricing companion. The model matchup against the closed default sits in the bge-m3 versus text-embedding-3 guide. To run these rows against a real wallet: sign up, create a key, and put a small balance behind the embeddings lane before you commit a corpus to either side.

Last verified: 2026-10-28