sam2.1-large: SAM 2.1 hiera large, Meta's accuracy tier, documented as a metered API

sam2.1-large is this platform's name for a metered lane serving SAM 2.1 hiera large, the largest and most accurate of Meta's improved Segment Anything checkpoints from September 29, 2024. The model is open weight and public today. Its row sits behind the public list, and nothing answers under the alias. Unlike the smaller tiers, this one has commercial company, all of it image-only; the census sits in the API question below.

Last verified: 2026-10-05

What it is

SAM 2.1 hiera large is a segmentation model: an image in, masks out, a per-pixel answer to which pixels belong to this object. It takes prompts, so a click or a box points at one object. It also runs unprompted, in the mode Meta calls automatic mask generation, sweeping the image for every object it can see. Both modes ship in the repository.

The 2.1 line is an improvement pass on SAM 2, not a new architecture. Meta's release post names the changes: extra training data for visually similar and small objects, better occlusion handling. The checkpoints landed September 29, 2024, across all four tiers. The SAM 2 paper covers the base design: a streaming-memory transformer, more accurate than SAM and 6 times faster.

The identity facts, each with its owner:

The four tiers, and where large sits

The family is one model at four sizes. Every tier uses the same Hiera image encoder at a different width:

Large is the accuracy tier: top score on every benchmark the publisher reports, at about 1.6 times the runtime of base-plus and 2.8 times its disk size. Large is the most-downloaded tier on Hugging Face, the variant the hosted lanes picked, and what people mean by "sam 2.1" with no tier named. The base-plus page carries the volume case.

The version story in brief

The Segment Anything line went through five releases in under three years, licensing included. From the official sources:

Two axes matter. The prompt axis changed at SAM 3, which added concept prompts; Meta's Model API serves that generation, not this one. The license changed with it: SAM 2.1 is the last plain-Apache line. The SAM 3.1 date is primary, release notes dated March 27, 2026 and Meta's blog with a same-date note. The full map lives on the vision lane page and the base-plus page.

Use cases

The large tier earns its premium on frames where the mask edge is the product. These are the jobs that fit.

• Grasp-critical edges. A mask gives the exact boundary of the object in the frame. From a boundary you get a grasp frame, a suction placement, or a collision check that is not a bounding-box guess. When the gripper meets the object at the mask edge, the 1.3 extra accuracy points over base-plus matter. They are the difference between a clean pick and a regrasp. A regrasp costs more than the compute.
• Fewer, harder frames. Not every perception job runs at camera rate. Keyframe analysis, inspection stills, human-in-the-loop review, and low-rate sampling all trade frame count for frame quality. On those, large is the right side of the trade: fewer calls, better masks.
• Dense and similar objects. The 2.1 refresh targeted exactly the classes where large matters most: visually similar objects side by side, small objects, and occlusion. Bins of identical parts, tangled harnesses, partially covered targets. The heavier encoder holds distinctions the smaller tiers blur.
• Labeling at a quality floor. Automatic masks from the top tier make better training data for the next model. The repository ships the training code under the same license, so the same weights that label can be tuned on your scenes.
• Mask plus depth equals position. A mask tells you which pixels belong to the object. A depth map tells you how far away those pixels are. Together they give the object's position in space, which is what a reach-and-avoid plan consumes. The depth half rides the same vision route here, and the da3-small page covers it. The perception guide walks the pairing end to end.

For the action half of the loop, the VLA lane takes camera frames and returns motion.

What it takes to run

The weights are light for a 224.4 million parameter model: 898.1 MB on disk. The working set is heavier: activations and the mask sweep ride on top. The one public guideline is community-level: under 10 GB of video memory, load a smaller tier. A 24 GB card runs large with room for batching. A CPU path exists but rules out camera-rate loops. For scale without a GPU, AWS documents all four 2.1 variants on SageMaker JumpStart, on instances you keep alive.

The video mode truth

The repository ships a real video predictor, a different design from the image path. It is stateful: a memory bank across frames, prompts on one frame, the mask propagated through the clip. Memory grows with clip length, and the path loads frames from disk. Every commercial 2.1 lane serves images only; both Replicate lanes say so. The row here is per-image, one metered unit per image. A video row would be a separate design, none promised.

Run it yourself, or wait for the hosted lane

Run the weights yourself when the data must stay inside your boundary, or when you want to fine-tune on your own scenes. The Hugging Face card shows the shortest path: the mask-generation pipeline in the transformers library runs automatic masking in a few lines, and the card's own examples run under bfloat16 autocast, the same half-precision practice a hosted lane uses to fit speed and memory. The repository's predictor classes cover prompted use, and the training code covers tuning. The cloud-shaped version of the same path is SageMaker JumpStart, which carries all four variants to deploy on your own instances: your endpoint to keep alive, the part a hosted lane takes off your plate.

Own a call instead of a stack when you would rather not host. The lane's contract here meters per image against a prepaid wallet, returns each call's cost with the call, and spending stops when the balance is gone. The register, in plain words: not on the public list. Its contract meters this tier at a per-image shape none of the current large-variant lanes offer.

API usage

The designed contract is the platform's vision route: POST /v1/vision, Bearer key from the console, multipart form data in, masks out. The model id is sam2.1-large.

The day-one prompt law is the whole image. No point or box fields exist on this route; sending one returns a 400 that names the field. The route runs the model's automatic mode over the full frame. A JSON answer carries the mask objects in annotations and the billed count and wallet debit in usage; a binary answer returns rendered bytes under the model's content type. Billing stays on the input side.

Metering, as the lane is designed:

The dash is deliberate: the rate sits in the platform's model data, the search listing carries it, and full rate cards render on the pricing page for the rows it lists. Bodies sit under the 25 MiB upload cap; a crossing request is refused with an error that names the file slot. A saturated lane answers with retry timing, and that refusal bills nothing.

The honesty line: this alias is not on the public list. The tabs below are the working contract, checkable now, earning the named not-found answer today. The field reference lives on the run-vision page, the walkthrough on the vision route page.

curl

curl -X POST "https://api.ironstratum.com/v1/vision" \
  -H "Authorization: Bearer $KEY" \
  -F "model=sam2.1-large" \
  -F "file=@part_closeup.jpg" \
  -F "response_format=json"

python

import os
import requests

resp = requests.post(
    "https://api.ironstratum.com/v1/vision",
    headers={"Authorization": "Bearer " + os.environ["KEY"]},
    data={"model": "sam2.1-large", "response_format": "json"},
    files=[("file", open("part_closeup.jpg", "rb"))],
    timeout=30,
)
resp.raise_for_status()

answer = resp.json()
masks = answer["annotations"]        # the model's own mask objects
billed = answer["usage"]["images"]   # 1: one per file part
print(len(masks), "mask objects, billed", billed, "images")

http

POST /v1/vision HTTP/1.1
Host: api.ironstratum.com
Authorization: Bearer $KEY
Content-Type: multipart/form-data; boundary=frame-boundary

--frame-boundary
Content-Disposition: form-data; name="model"

sam2.1-large
--frame-boundary
Content-Disposition: form-data; name="response_format"

json
--frame-boundary
Content-Disposition: form-data; name="file"; filename="part_closeup.jpg"
Content-Type: image/jpeg

<image bytes>
--frame-boundary--

Benchmarks

Every number here is the publisher's own, from the repository's tier table, read October 5, 2026. Speed was measured on an A100, torch 2.5.1 and cuda 12.4, all components compiled. Scores are the J&F metric on the SA-V test, MOSE val, and LVOS v2 sets.

First, the 2.1 improvement is real, and large took the biggest step. The same table's SAM 2.0 large row scores 76.0 on SA-V test against 79.5 here, a gain of 3.5 points; LVOS v2 moves from 79.8 to 80.6, MOSE val stays flat at 74.6. The gains concentrate on small and visually similar objects, and occlusion.

Second, the ladder trades what the sizes suggest. From base-plus to large you buy about 1.3 SA-V points and 1.6 times the runtime; down to tiny you sell about 1.7 points for about 1.4 times the speed. The tier choice is a loop-rate decision, not a quality cliff.

Third, the speed column carries its conditions, and other numbers circulate. Paper-era guides report about 24 uncompiled and about 30 compiled for this tier size on the same GPU class; a LearnOpenCV guide read October 5, 2026 carries the pair. Compilation and library versions move this model as much as the GPU does. Measure on your own.

What the publisher's numbers do not cover: your scenes. Community guides report the stock model struggling on fine-grained boundaries until tuned on domain data, and surgical-video evaluations agree. The repository ships training code for exactly that.

No own measurements: nothing answers under the alias, so there is nothing to measure. Numbers publish here only after they are measured and held.

Getting started

1. Read the primary sources. The repository carries the tier table, the license, and both prediction modes. The model card carries the download and the pipeline usage. The SAM 2.1 release post carries what changed.
2. Prototype on the open weights. Apache 2.0 and ungated mean the automatic-mask pipeline runs today on your own GPU. The self-serve section above carries the paths.
3. Watch the index, not this page. The models index is the live catalog. The pricing page renders the rate card for every listed row.
4. Open the console when you build. The signup flow opens the wallet, the keys, and the billing history. Each key revokes alone, and the loaded wallet balance is the spend bound.