SAM 2 memory leak (sam2.utils.misc): the exact errors, causes, fixes, and prevention

The SAM 2 memory leak is not the model leaking. The model's own memory bank is bounded at seven frames. The growth you are watching is Python memory held around the model. It is every video frame preloaded by sam2/utils/misc.py, every per-frame tracking output kept in the inference state, and the GPU allocator's cache, which only looks like a leak. This page quotes the exact error strings. It names each retention layer from the repository's own code and orders the fixes by failure class. Every source was read on 28 October 2026.

Last verified: 2026-10-28

The errors at a glance

Match your screen against this table first. Every string below is copied from a real report in Meta's SAM 2 repository or from the repository's own code. All of it was read 28 October 2026.

One path detail before the causes. The repository keeps the file at sam2/utils/misc.py, so the import path is sam2.utils.misc. Some tracebacks print the doubled form sam2.sam2.utils.misc, the exact string in the title of issue 623. That doubling happens when your checkout folder itself lands on the Python path. Python then counts the folder and the package as two nested names. The file is the same either way. Every fix on this page applies to it.

The strings as they actually appear, and the state a finished worker pool leaves on the card:

image = image.cuda()
OutOfMemoryError: CUDA out of memory. Tried to allocate 18.53 GiB. GPU

The tail of the traceback in issue 62: the frame-to-GPU copy in misc.py, then the error. The poster's paste cuts off after the word GPU. The card was a Colab T4, and 1581 frames were already loaded.

RuntimeError: Failure in frame loading thread

That one is the loader's own relay line, printed whenever its background thread hits an error (misc.py, lines 147 to 149).

[0] NVIDIA GeForce RTX 3090 | 43°C,   0 % | 4171 / 24576 MB | usr(512M) usr(2968M) usr(338M) usr(338M)

A verbatim gpustat line from issue 430. An idle RTX 3090 at 0 % load, still holding 4171 of its 24576 MB after a worker pool finished its run.

What is actually holding the memory

Three layers hold memory around SAM 2, and none of them is the model's own memory. Knowing which layer is growing tells you which fix to run.

Layer one is the preload in sam2/utils/misc.py. When you call init_state, the loading code reads the whole video before any tracking starts. For a folder of JPEG frames it preallocates one tensor, torch.zeros(num_frames, 3, image_size, image_size, dtype=torch.float32) at line 267 of misc.py, then fills it frame by frame. The standard checkpoints use 1024 by 1024 images, and three channels of float32 at that size is about 12 MB per frame. The MP4 path does the same in two steps. It stacks every frame the decord reader returns into one tensor, then moves the whole tensor to the GPU unless you say otherwise. A reader on a 15 GB Colab T4 hit the wall exactly here, with 1581 frames loaded and the copy to GPU asking for 18.53 GiB (issue 62). Our arithmetic says that number is no accident. At 12 MB per frame, 1581 frames need 18.5 GiB in one copy. That is the single whole-video copy the error names.

Layer two is the inference state. The video predictor keeps its results in dictionaries keyed by frame index, one set per object, under cond_frame_outputs and non_cond_frame_outputs. Every propagated frame adds an entry. Nothing removes the old ones. A long thread on issue 264 measures this at roughly 1 MB per frame on top of the pixels. A two and a half minute video at 24 frames per second was reported at about 60 GB of GPU memory on issue 196. The built-in reset_state call clears tracking results, objects, and prompts. It does not free the preloaded images. Those stay until the state object itself is deleted.

Layer three is the CUDA caching allocator, which is not a leak at all but looks like one. PyTorch keeps freed GPU blocks reserved for reuse. nvidia-smi shows them as used. The official contract for torch.cuda.empty_cache() is printed in PyTorch's own source. It "Release[s] all unoccupied cached memory currently held by the caching allocator". The same docstring notes it "doesn't increase the amount of GPU memory available for PyTorch", though it "may help reduce fragmentation". In plain words: empty_cache returns memory that is already free. It cannot touch memory your objects still hold. Call it after dropping references, never instead of dropping them.

And the model itself? Bounded by design. The memory bank keeps seven frames by default, one conditioning frame plus six recent ones, set by num_maskmem=7 in sam2_base.py. The oldest frame is evicted when the queue fills. The attention over that bank is why SAM 2 streams video at all. Your leak is in the Python bookkeeping around this bounded core, not in it.

The fixes, class by class

One video that will not fit

The error is the 18.53 GiB class: a single whole-video copy that exceeds the card. Three levers, in order of effort.

First, the two offload switches on init_state, both defaulting to False. offload_video_to_cpu=True keeps frames in system RAM. Each frame moves to the GPU only while it is used. The repository's own comment says this "saves the GPU memory with only a very small overhead". offload_state_to_cpu=True also moves the inference state to CPU. The same comment pegs the cost. In a 768 by 768 test, fps dropped from 27 to 24 tracking one object and from 24 to 21 tracking two. That switch alone also fixed a slowdown that grew with video length for one RTX 2070 reader (issue 180).

inference_state = predictor.init_state(
    video_path=video_dir,
    offload_video_to_cpu=True,
    offload_state_to_cpu=False,
)

Second, cut the input. The Colab reader in issue 62 resolved the 18.53 GiB failure by trimming the clip to ten seconds, matching the length of the web demo's video. The notebook demo's own clip is shorter, 200 frames at 30 fps, about 6.67 seconds. Unglamorous, but it is the fix that author confirmed the same day.

Third, halve the pixels. A repository contributor suggested on issue 196 storing the video frames in float16 instead of float32. It is a one-word change to the dtype at misc.py line 267. The contributor added that it "usually doesn't have any negative impact on the segmentation performance". That halves layer one exactly.

Leave async_loading_frames off unless you know why you need it. It is opt-in, defaulting to False. It is also the subject of issue 623, the report whose title this page carries. The async loader preallocates a slot for every frame and fills them from a daemon thread. That thread lives until your script exits, so the frames are never released early. The reporter's async case held about 20 GB. Other retention passed 80 GB, growing with the number of prompt points and frames. If you must use it, wrap the loader so it goes out of scope when prediction ends, or read frames yourself.

Many videos in a loop

The symptom is memory that survives del predictor and torch.cuda.empty_cache() between videos. Two causes, one rule.

The rule comes from two independent reports. Build the predictor once, outside the loop, and cycle only the per-video state. Issue 451 was closed by its author with this fix; issue 258 carries the same diagnosis. One reader who rebuilt the model each pass still leaked about 0.3 GB per video even after deleting everything. Moving the build outside the loop eliminated the growth. Rebuilds are expensive and sticky. One model, many states.

predictor = build_sam2_video_predictor(model_cfg, sam2_checkpoint, device=device)

for video_dir in video_dirs:
    state = predictor.init_state(video_path=video_dir, offload_video_to_cpu=True)
    predictor.add_new_points_or_box(inference_state=state, frame_idx=0,
                                    obj_id=1, box=box)
    for out_idx, obj_ids, logits in predictor.propagate_in_video(state):
        pass  # consume masks here, keep only what you save
    predictor.reset_state(state)   # clears prompts and per-frame outputs
    del state                      # drops the preloaded frames
    gc.collect()                   # breaks any reference cycles
    torch.cuda.empty_cache()       # returns freed blocks to the driver

The order of the last three lines is the whole trick. It is the sequence a reader on issue 258 spelled out. Delete first, collect cycles second, empty the cache last. Empty the cache first and you return nothing, because the blocks are not free yet. Jumps of 1.5 to 2 GB between videos are about the size of a model load. If you see them, check for a second predictor being built by accident (issue 288).

For process pools, issue 430 documents the boundary case. Four worker processes each kept about 338 MB resident after cleanup. The only full clear was replacing the worker after each task, maxtasksperchild=1 in multiprocessing.Pool terms. That works, but it pays process restart cost. The issue was closed in October 2025 with no repository-posted fix.

Very long videos

When one video runs thousands of frames, the per-frame outputs become the problem. Three moves, all community-verified on issues 196 and 264.

Prune old outputs as you go. After each propagated frame, drop outputs older than a window you choose. The snippet from issue 196, with the corrected pop default one reader supplied:

storage_key, obj_key = "non_cond_frame_outputs", "output_dict_per_obj"
oldest_allowed_idx = frame_idx - 16
all_frame_idxs = output_dict[storage_key].keys()
old_frame_idxs = [idx for idx in all_frame_idxs if idx < oldest_allowed_idx]
for old_idx in old_frame_idxs:
    output_dict[storage_key].pop(old_idx)
    for objid in inference_state[obj_key].keys():
        inference_state[obj_key][objid][storage_key].pop(old_idx, None)

Load frames lazily. Store paths in the state instead of pixels. Load each frame when the predictor asks for it. One July 2026 writeup on issue 196 walks this change through init_state and a cached_features lookup. An earlier reader on issue 264 built the same thing over decord, loading and resizing on demand. The temporal evidence the tracker needs lives in the per-frame outputs, not in the raw pixels. Pixels are safe to load late.

Cap the memory the model attends over. The same issue 264 reader added hydra overrides, ++model.max_cond_frames_in_attn=5 and ++model.num_maskmem=7. The reader reported "No OOM and great segmentation results on a very long video". Combine those with clear_non_cond_mem_around_input=True, autocast bfloat16, or a smaller image_size of 512 or 768 in the model config, all from the same thread. These change what the model sees, so re-check your own accuracy after each one.

A published research line attacks the same wall. Det-SAM2 releases old frames deliberately for constant-memory, unlimited-length processing, a pointer posted by a reader on issue 264. And no repository-level fix has landed. A pull request adding LRU frame caching for long videos (pull 292) was opened in September 2024. It received only bot comments and was closed unmerged in November 2024. As of 28 October 2026, main still preloads.

The image lane

Image-only jobs have a quieter version of the same shape. SAM2ImagePredictor.set_image replaces the stored features on every call. The predictor itself holds exactly one image's embeddings, no more (sam2_image_predictor.py). The growth you see across a folder of images comes from your own scope: saved masks, kept logits, accumulated results. Reuse one predictor. Move or delete per-image tensors when you are done with them, and run the same delete, collect, empty sequence at batch boundaries.

The automatic mask generator has one knob that matters for memory: points_per_batch, default 64, the number of points it runs at once. Lower it when a dense image pushes the card over.

Prevention checklist

• Build one predictor per process. Rebuilds are the loop leak.
• Call init_state with offload_video_to_cpu=True unless speed measurements say otherwise.
• Keep async_loading_frames off, or scope the loader so it dies with the video.
• After each video: reset_state, del the state, gc.collect(), then torch.cuda.empty_cache(), in that order.
• Prune or cap per-frame outputs before your frame count passes four digits.
• Drop your own mask and logit references per image; the image predictor is not the holder, your variables are.
• For a pool, decide up front between resident workers and maxtasksperchild=1; do not discover the 338 MB residue in production.
• Watch nvidia-smi against torch.cuda.memory_allocated(): reserved-but-free blocks are the allocator working, not a leak.

If you would rather not host the leak

Everything above is the honest cost of running open weights yourself: the loader, the state, the allocator, and the issue tracker where the fixes live. None of it is a reason to avoid SAM 2. Its weights and code ship under Apache 2.0 with torch 2.5.1 or newer (repository README). It is a reason to check what your time is worth per debugged gigabyte.

The hosted route on this platform is the vision lane. The platform's model data carries the sam2.1-base row on that route, metered per image against a prepaid wallet. The row is on the public list as of 28 October 2026. The larger tier, sam2.1-large, is not on the public list, and no date is attached to that step. Rates render on the pricing page, never typed by hand on pages like this one. A first call is three form fields and a file:

curl -X POST "https://api.ironstratum.com/v1/vision" \
  -H "Authorization: Bearer $KEY" \
  -F "model=sam2.1-base" \
  -F "file=@frame.jpg" \
  -F "response_format=json"

import os
import requests

resp = requests.post(
    "https://api.ironstratum.com/v1/vision",
    headers={"Authorization": "Bearer " + os.environ["KEY"]},
    data={"model": "sam2.1-base", "response_format": "json"},
    files=[("file", open("frame.jpg", "rb"))],
    timeout=30,
)
resp.raise_for_status()
answer = resp.json()
print(len(answer["annotations"]), "mask objects, billed", answer["usage"]["images"], "images")

Each call states its own billed image count. The wallet is the spending stop. If that shape fits your loop, sign up, create a key, and point it at your first frame. The neighboring error writeups on this site cover the MolmoAct2 loading and dtype errors, the pi 0.5 quantile normalization crash, and the OpenVLA-OFT CUDA out of memory case. For what masks feed once they exist, read the segmentation and depth guide.

Frequently asked questions

Is the leak inside the SAM 2 model?

No. The model's memory bank holds seven frames by default, one conditioning frame plus six recent ones, and evicts the oldest when full. That is num_maskmem=7 in the repository's model base code, read 28 October 2026. The growth is in the Python layers around the model: preloaded frames in sam2/utils/misc.py, per-frame outputs in the inference state, and the GPU allocator's reserved blocks.

Why does torch.cuda.empty_cache() not free my memory?

Because it was never meant to. PyTorch's own docstring says it releases unoccupied cached memory so other GPU applications can use it. The same docstring says explicitly that it does not increase the memory available to PyTorch, though it may reduce fragmentation. If tensors still reference the memory, the cache has nothing to return. Delete the tensors, run gc.collect() for cycles, then call it.

What does the doubled name sam2.sam2.utils.misc mean?

The file lives at sam2/utils/misc.py in the repository, so the import path is sam2.utils.misc. Tracebacks that read sam2.sam2.utils.misc, including the title of issue 623, come from environments where the checkout folder itself is on the Python path. The folder name and the package name stack. The file and every fix on this page are the same.

Does reset_state free the video?

No. reset_state clears prompts, tracked objects, and per-frame outputs. The preloaded frames stay in the inference state until you delete the state object itself. Between videos, run reset_state, then del the state, then gc.collect() and torch.cuda.empty_cache().

Has Meta fixed this?

No repository-level fix had landed as of 28 October 2026. The LRU-caching pull request that would bound long-video memory was closed unmerged in November 2024 with no review. The main memory issues, 196, 258, 264, 288, and 623, are all still open, none with a maintainer reply. The next generation carries a sibling report. A December 2025 issue on the SAM 3 repository, issue 305, describes memory doubling on predictor rebuilds even after unload calls. Reuse one predictor is the working pattern there too.

Match your string in the table, then act in this order: drop references, collect cycles, empty the cache. Build the predictor once for the whole run. Prune whatever grows with frame count before it reaches four digits. The lane this model rides here is the vision page. The rates are on the pricing page, and the hosted route starts at signup.

Last verified: 2026-10-28