Is SAM 2 thread safe? The answer per component, from the repository's own code

No single answer covers SAM 2, because SAM 2 is several surfaces, not one object. Sharing one image predictor, or one automatic mask generator, across threads is not safe, because both hold mutable state on the object itself. The video predictor keeps its state in a per-video dictionary, and the demo Meta ships wraps every model call in one lock. Underneath, the PyTorch model is safe to read from many threads and unsafe the moment something writes to it. This page maps each part, quotes the exact error strings, and names the pattern that fits each one. Each source on this page was read on 28 October 2026.

Last verified: 2026-10-28

The answer per component

The question "is SAM 2 thread safe" treats the repository as one thing. It is not. You use SAM 2 through four distinct objects, and each one answers the question differently. Match your object to its row before you touch threading.

One rule sits under all five rows. A PyTorch developer stated it on the official PyTorch forums in 2020, and it still governs every model built on the framework: "all the objects in pytorch are thread safe to read. But are not to write into." SAM 2 does not break that rule. It just hands you several objects that write into themselves.

What the official documentation says

Here is the part that surprises people. The official SAM 2 repository README covers install, checkpoints, image prediction, and video prediction. It never mentions thread safety. A text search for "thread" across the README returns nothing, as of 28 October 2026. The Hugging Face transformers page for SAM2, which documents a second build of the same model, is the same: zero mentions. The auto-generated repository wikis that rank for SAM 2 usage questions are the same again.

One artifact in the repository does face the question head on. Pull request 643, "Make image predictor thread-safe", opened on 6 May 2025, states the problem in its first line:

SAM2ImagePredictor is not thread-safe as it stores state variables after set_image() or set_image_batch(). This can be cumbersome to handle in a multi-threaded environment e.g. Web App.

That pull request would make set_image return a state object you pass to predict. The predictor would stop holding per-image state on itself. As of 28 October 2026 it has zero comments and zero reviews. Nothing has moved on it in close to 18 months.

The demand is visible either side of that silent pull request. Inside the repository, 18 issues and pull requests mention "thread" in some form today. Across GitHub as a whole, a search for "sam2 thread" returns 934 results today. An earlier count on 25 October 2026 recorded 363 of them. Web app builders ask for integration help in the same tracker. One Flask integration request has sat open since February 2025, with readers still bumping it in March 2026. No page on the open web answers the headline question for SAM 2. Other model families publish official thread-safety guides. SAM 2 does not. This page and the two it links below are the missing set.

The loud failure: the exact error string

When two threads share one SAM2ImagePredictor, the most common crash is this string, copied from the current source file sam2_image_predictor.py:

RuntimeError: An image must be set with .set_image(...) before mask prediction.

The race that produces it is short. Thread A calls set_image. The first thing set_image does is call reset_predictor, which sets four fields to empty. Only near the end does it write the new features and raise the "image is set" flag. Thread B calls predict inside that window. The flag is still down. The predictor raises the error above.

At the center sit four fields, named in the source: _features, _orig_hw, _is_image_set, and _is_batch. They live on the predictor object. Every call path reads them. That is the whole hazard in four names.

The fix for this class is not clever. Do not let two threads drive one predictor at the same time. Give each thread its own predictor. Or put a lock around the set plus predict pair, so the two calls run as one unit.

The silent failure: wrong masks, no error

The loud error is the good outcome. You get a crash, you find a fix, and you move on.

The bad outcome never prints anything. Thread A sets image A and gets ready to predict. Thread B sets image B on the same predictor. The reset clears the features of A and writes those of B. Thread A now calls predict. The flag is up, because the call from B finished. The predictor reads the features it holds, which are those of B. Thread A gets masks for image B, with no error, no warning, and no log line.

Nothing in the current code detects this. The return looks normal. The masks look real. They are masks of the wrong image. In a batch pipeline this poisons output without a sound, which costs more than any crash.

This is the strongest argument for the lock pattern. A missing lock does not always fail loudly. Sometimes it hands you wrong data as if it were correct, and the only symptom shows up far away, when a number makes no sense.

The automatic mask generator wraps the same trap

The SAM2AutomaticMaskGenerator is the object most people mean by "the mask generator". It answers the question "what if I do not want to click prompts, just mask everything". You feed it an image. It sweeps a grid of points and returns a mask record for every object it finds.

Inside, it is one image predictor in a wrapper. The constructor builds a single private SAM2ImagePredictor and keeps it. Each generate call walks its crop layers. For every crop it calls set_image on that private predictor, then runs batched point predictions through it. So one mask generator shared across threads puts two threads into the same set_image and predict traffic you just read about, one layer down, where you cannot see it.

The failure modes are the same two. A loud RuntimeError when one thread lands inside the reset window of another. Or the silent one, masks computed from the crop of another thread. The generator also batches its own point grid with points_per_batch, default 64. The window per crop is wide. Under real load, the overlap is common.

The working rule is simple. Treat one mask generator as a single-threaded worker. Feed it from a queue if you like. Give it one image at a time. When you need several in flight, run several generators. That is the pool shape, covered below.

The video predictor is a different design

The video predictor, SAM2VideoPredictor, does not store your video on itself. Its init_state call returns a plain dictionary that you hold: loaded frames, prompts, cached features, per-object output maps. The README states the design in one line: "SAM 2 supports video inference on multiple objects and uses an inference state to keep track of the interactions in each video." One predictor, many dictionaries, one dictionary per video.

This buys you session isolation for free. Ten videos can sit in ten state dictionaries, and no dictionary knows about the others. The designed way to track several objects in one video is not several threads, either. It is several object ids inside one state. A reply on repository issue 652 points at the official notebook example: give one predictor two prompts with two object ids. The image features are then computed once and reused for both tracks.

What the design does not buy you is parallel calls through one predictor. For that, read what the demo of Meta itself does. The demo backend shipped in the repository, a Flask and GraphQL service in the demo directory, keeps a dictionary of session states for its users. Then it wraps every model entry point in one lock:

self.session_states: Dict[str, Any] = {}
...
self.inference_lock = Lock()
...
with self.autocast_context(), self.inference_lock:
    session = self.__get_session(request.session_id)
    inference_state = session["state"]

Every operation takes that same inference_lock. Start a session. Add points. Add a mask. Propagate. Clear prompts. All of it waits on the one lock. That is the answer of the authors to many users at once: many session states, one lock, all model calls in a line. When the people who wrote the model pick that shape for their own public demo, copy it first.

The build path has its own shared state

Threads are not the only place shared state bites. Building a predictor uses Hydra, the config library, and Hydra keeps its config in a process-wide singleton. The Hydra docs say it plainly: the plain forms of its init methods "are initializing Hydra globally and should only be called once."

The SAM 2 build functions call the Hydra compose step on every build. Build from several threads, or build after another library in your process has touched Hydra, and compose can fail with this string from the Hydra source:

AssertionError: GlobalHydra is not initialized, use @hydra.main() or call one of the hydra initialization methods first

This bites real applications. One shipping desktop project that embeds SAM 2 next to other model libraries hit exactly this. The config of SAM 2 failed to register after another library touched Hydra first. Their fix was to clear and re-init Hydra with the config module of SAM 2 before every predictor build. For most services there is a simpler fix. Build your predictors once at startup, from one thread, then share the built objects under one of the safe shapes.

One more note at build time. The video predictor has a fast variant, SAM2VideoPredictorVOS. It applies torch.compile to the memory encoder, the memory attention, and the prompt encoder at build time. Its own source prints the warning that matters here: "Compiling all components for VOS setting. First time may be very slow." The compile work happens on first run. Let that first pass run alone, before any traffic reaches the object.

Why two predictors do not run twice as fast

Say you take the isolation advice. You build two predictors and run them in parallel. Sometimes the speed still does not move. This is not a thread bug. It is the graphics card.

Issue 652 in the repository is the clean receipt. A user built two separate video predictors for two objects, on a 24 GB A5000, with headroom to spare. One propagation over 31 frames took about 1 minute 14 seconds alone. Running both took more than 2 minutes. No speed-up, at 16 of 24 GB used. They tried processes, process pools, and thread pools, and asked the tracker why.

The reply on the issue names the wall: "SAM2 is a very expensive model, so it's highly likely that you're saturating your GPU's compute capabilities, such that there's no more gains to be had from further parallelism." Two copies of a large model on one card compete for the same compute. Free memory is not the budget that runs out first.

So set real goals. Correctness first, which the lock or the pool gives you. Then throughput gains that come from overlap, not from doubling the model. The measured version of that story, including what a pool of generators over one shared model does to speed, lives on a companion page: the SAM 2 predictor pool writeup.

Three safe shapes, and where the depth lives

Every working SAM 2 serving setup in the sources fits one of three shapes.

Serialize. One predictor, one lock, every caller waits its turn. This is the demo pattern. It is the simplest correct shape, and its throughput is one call at a time.

import threading
from sam2.build_sam import build_sam2
from sam2.sam2_image_predictor import SAM2ImagePredictor

predictor = SAM2ImagePredictor(build_sam2(model_cfg, checkpoint))
lock = threading.Lock()

def segment(image, box):
    with lock:
        predictor.set_image(image)
        masks, _, _ = predictor.predict(box=box, multimask_output=False)
    return masks

The lock covers both calls. That pairing is the whole trick. A lock around each call on its own still lets the set_image of another thread land between yours and your predict. That is the silent failure again.

Isolate. One predictor per thread, or one per process. Threads are lighter. Processes are the hard boundary, because two processes cannot touch the memory of each other at all. The cost of processes is a full set of resident weights per copy, plus a known cleanup tail. The memory writeup on this site covers that tail in depth.

Pool. A fixed set of generators over one shared model. Each request takes an idle generator, runs on it alone, and returns it. You get overlap without copying the weights. No caller can corrupt the state of another, because no two callers share a generator. This shape has the most moving parts and the best speed story.

Two companion pages on this site carry the measured depth for this exact model, so this page stays a map. The gotcha story, what shared predictor state does under load, what failed, and how the lock fix looked before the pool replaced it, is the mask generator thread-safety page. The pool receipts, with speed and memory accounting, are on the predictor pool page. For the question of what masks feed in a robot stack, read the segmentation and depth walkthrough.

If you would rather not own the lock

Running the open weights yourself is what all of the above costs: the locks, the pools, the build order, the silent failure class. Avoiding the model over any of this would be the wrong call. SAM 2 ships under Apache 2.0 with its weights, and the SAM 2 paper describes the streaming memory design that makes the video side work at all. It is a reason to price your own time against a hosted call.

The hosted route here is the vision lane. It serves the sam2.1-base tier per image, and that row is on the public models list as of 28 October 2026. The large tier is not on the public list, and no date rides with it. Each request is one image in, masks out, metered per image, drawn from a prepaid wallet. No shared predictor exists on your side of that call. No lock, no race, no masks for the wrong image. The pricing page renders the rates. Pages like this one never type them. The first call needs three form fields and one file:

curl -X POST "https://api.ironstratum.com/v1/vision" \
  -H "Authorization: Bearer $KEY" \
  -F "model=sam2.1-base" \
  -F "file=@frame.jpg" \
  -F "response_format=json"

When a hosted call suits your pipeline, sign up, create a key, and run one frame through it. Neighboring pages here cover the automatic mask generation guide, the SAM 2 memory leak page, and the version comparison between SAM 2 and SAM 2.1.

Frequently asked questions

Is SAM 2 thread safe?

Not as one answer. One image predictor or mask generator shared across threads is not safe, because both hold per-image state on the object. The video predictor keeps per-video state in a dictionary you hold, and the official demo still puts one lock around every call. Under it all, the PyTorch model is safe to read from many threads, and unsafe while written to.

Can I share one SAM2ImagePredictor between threads?

No. set_image resets and rewrites four fields on the instance, and predict reads them. Two threads racing on that pair produce a RuntimeError, or masks for the wrong image with no error at all. Give each thread its own predictor, or lock the set plus predict pair as one unit. The repository fix proposal, pull request 643, would remove the instance state. It has sat without review since May 2025.

Is the automatic mask generator thread safe?

No. It builds one private image predictor and calls set_image on it for every crop inside generate. Sharing the generator across threads puts that predictor into the same race, one layer down. Treat one generator as a single-threaded worker, or run several as a pool.

Is the video predictor thread safe?

Its design is different, not safe in the way people hope. Each video lives in its own state dictionary, so videos in flight do not share dictionaries. But calls through one predictor still share the model, and the demo of Meta wraps every call in one lock. Multi-object tracking belongs inside one state as several object ids, not across threads.

Why does parallel inference not run twice as fast?

Because the graphics card saturates before the memory does. Issue 652 measured two predictors on one 24 GB card going no faster than one, with 8 GB still free. The reply on the issue names compute saturation as the wall. Gains come from overlap in a pool, not from copying the model.

Does SAM 2 crash under threads, or return wrong results?

Both, and timing decides which. A predict that lands while another thread is inside set_image, before the flag goes up, gives the loud RuntimeError. A second set_image that finishes first leaves the flag up and the features swapped, which gives masks for the wrong image with no error. Test for both classes. A suite that only checks for exceptions will pass while the outputs are wrong.

Match your object to its row in the table. Pick one of the three shapes. Test for the silent class, not just the crash. The depth pages carry the measurements: the mask generator thread-safety story and the predictor pool writeup. The hosted per-image route is the vision lane, with rates on the pricing page and a start at signup.

Last verified: 2026-10-28