SAM 2 vs YOLO: masks or boxes, the real decision

SAM 2 and YOLO answer two different robot vision questions. YOLO is a detector trained on your classes. It draws labeled boxes, fast, on modest hardware. SAM 2 is a promptable mask maker. It traces any object you point at, pixel by pixel, with no training, at a higher compute cost. This page maps that boundary. It names the benchmark basis on every number. It shows the pipeline where a robot stack runs both.

Last verified: 2026-10-28

The short answer

Pick by task, not by a model ranking. You need to know where known objects are, at frame rate, on the hardware you have. Run a YOLO detector. You need pixel outlines for grasp planning, scene carving, or dataset labels, on classes you never trained. Run SAM 2. You need both at once. Run both: YOLO boxes become SAM prompts, or SAM masks get matched to YOLO boxes on your side. Neither model replaces the other on its home task. The speed tables do not compare the two fairly unless you read the basis line under each number. This page names it every time.

What each model actually does

YOLO: a detector trained on your classes

YOLO models are single-stage detectors. An image goes in. Out come class-labeled boxes with confidence scores, in one pass, built for frame rate. Segmentation variants of YOLO add masks. Those masks train on the same labeled classes. They stay coarser than promptable masks. The catch is simple. A YOLO model only finds classes it was trained on. An object outside its label set is invisible to it. Ultralytics maintains the line the market means by YOLO today. YOLO11 and YOLO26 shipped. A YOLO27 page already sits in the docs as an early draft.

SAM 2: a promptable mask maker with no training

SAM 2 takes prompts. A point, a box, a rough mask. It returns masks at pixel resolution, for any object, with zero training on your classes. It runs on video too. A streaming memory and an occlusion head let a target drop behind cover and come back. The catch runs the other way. SAM 2 does not tell you what the object is. Output is geometry, not a class label. The SAM 2 paper frames the gain over SAM 1 as 3x fewer interactions and 6x faster masking, on the largest video mask dataset built to date. Meta released the family under Apache 2.0. The license covers checkpoints, demo, and training code, per the project repository. SAM 2.1 is a checkpoint refresh of the same design. The 2.1-versus-2.0 differences have their own page.

The boundary in one line each

• YOLO says: "a cup, here." A box, a class, a score.
• SAM 2 says: "this exact shape, pixel by pixel." A mask, no class.

The numbers, with the basis named on every row

One rule carries this section. A speed number without its protocol is noise. Every table below is a publisher table, with its own basis line. None of them runs both families under one protocol. That fact is the finding.

SAM 2.1 on an A100

Basis: publisher table, A100 GPU, PyTorch 2.5.1, CUDA 12.4. Scores are mask quality on video benchmarks, not COCO detection mAP.

Source: the SAM 2 repository benchmark tables, read 2026-10-28.

YOLO11 detection on COCO

Basis: publisher table, COCO val, 640 pixel input. CPU times in ONNX Runtime. GPU times in TensorRT10 on a T4.

Source: Ultralytics YOLO11 docs, read 2026-10-28.

YOLO26 with masks, same COCO basis

Basis: publisher table, COCO val, single model, single scale. Two scores: box mAP and mask mAP. This is the closest YOLO gets to SAM territory, on YOLO terms. Trained classes, one pass.

Source: Ultralytics segmentation docs, read 2026-10-28.

Why no honest single table exists

The SAM side reports SA-V, MOSE, and LVOS mask scores. The YOLO side reports COCO box and mask mAP. We searched for a table that runs both families under one protocol on one dataset. We found none in the surfaced literature as of 2026-10-28. The comparison pages that rank for this query either say they cannot benchmark the pair, or compare use cases with no numbers. The rest of the front page is combination content and one publisher page. Any page that prints one SAM number beside one YOLO number, with no protocol named for each, is mixing bases. The next section shows what that looks like when a vendor does it in the open.

The comparison-table trap, one real example

Ultralytics ships a CPU comparison that puts SAM and YOLO rows on one machine, a 2025 Apple M4 Air. Read the runtime note before the milliseconds. The SAM models run in PyTorch. The YOLO models run in ONNX Runtime. On that table, SAM 2 base takes about 28.9 seconds per image and 162 MB on disk. YOLO11n-seg takes about 24.3 milliseconds and 2.9 M parameters. The hardware matches. The runtime does not. The same page's FAQ puts the YOLO26n-seg versus SAM 2 base gap at more than 1145x on that CPU table, with the runtime difference stated beside it. Read the ratio as the rough scale of the gap, not as a benchmark. That is exactly how this page treats it.

The license split, which decides product shape

• SAM 2 ships under Apache 2.0. The license covers checkpoints, demo code, and training code, per the repository. A closed commercial product carries no duty to publish source.
• Ultralytics YOLO ships under AGPL-3.0 by default, and the project applies it to trained models. The license page, read 2026-10-28, states the duty in plain words: "publicly releasing the complete corresponding source code for the entire derivative work." That includes the larger application and, where applicable, the model weights. Closed-source commercial use points to a paid Enterprise license. No public price list. Requests go through a form.
• SAM 3: the Meta post of 19 November 2025 shares weights, a benchmark set, and a paper. SAM 3D adds checkpoints and inference code. It names no license term on the page. Check the repository before you ship anything built on it.

For a robot product this is a buying decision, not a footnote. One path lets you close your source. The other wants your application code published, or a commercial agreement signed. The same math shows up across the robot stack. The GR00T license page walks the same question for a foundation policy.

Hardware classes, plainly

• SAM 2 is comfortable on a datacenter GPU. The publisher table above is A100, PyTorch. 39.5 to 91.2 FPS across sizes.
• YOLO deploys on CPU and edge parts. YOLO11n at 56.1 ms per image in ONNX on CPU is about 18 FPS on a desktop part. YOLO26n-seg at 53.3 ms sits in the same class. On a T4 with TensorRT10, small detectors run 1.5 to 2.1 ms per image.
• A 30 FPS control loop has a 33 ms budget per image. No SAM 2 size in any publisher CPU table we read fits that budget. On the T4 TensorRT10 rows, every YOLO size in the tables above fits it, 1.5 to 16.4 ms. On a desktop CPU, the nano rows run 53 to 56 ms, about 18 FPS.
• The lightest SAM variant on that M4 table, MobileSAM, still takes about 23.8 seconds per image in PyTorch on CPU. SAM on CPU is a labeling batch, not a control loop.
• A robot with no GPU at all still has a lane. Hosted per-image inference, below.

The line has moved since the 2024 comparisons

Most pages ranking for "sam 2 vs yolo" describe a 2024 or 2025 world. Two dated facts update it. Neither moves the boundary.

• SAM 3, announced by Meta on 19 November 2025, adds text prompts and image exemplars to promptable masking, and composes with multimodal language models for compound prompts. SAM 3D adds single-image 3D rebuilding in two variants, Objects and Body. None of that turns SAM into a class-trained detector. It still masks what you prompt. The announcement shares weights, a benchmark, and a paper, plus checkpoints and inference code for SAM 3D.
• YOLO26 shipped, released January 2026 per the Ultralytics model page, with launch coverage dated 20 January 2026. A preliminary YOLO27 page carries early COCO rows, an EPYC CPU plus RTX PRO 6000 TensorRT11 basis, and the line "no launch date has been set." Generation numbers move. The split between class-trained detection and promptable masking does not.

On this platform, the catalog serves the sam2.1-base tier per image. Current state sits on the vision route page. SAM 3 is not on the public list. No date rides with that fact.

Run both: the detect-then-mask pipeline

The pattern worth stealing from this query pair is not a versus at all. On a live search for the head term, two of the top five organic results were combination pages, not comparison pages. The combination: YOLO detects at frame rate. Its boxes become prompts. SAM 2 masks exactly those objects. The pattern is standard enough that Ultralytics ships a helper for it, auto_annotate. The maintainers walk the handoff in issue 15627: run the detector, pass its boxes as the bboxes prompt to SAM. A 2025 survey reports the YOLO-SAM2 hybrid as fast, accurate zero-shot masking in real time settings. It lists SAM 2 domain tuning, model size, and resource draw as the open costs.

A robot shape for the pattern: YOLO finds "the red bin" at 30 FPS on the edge box. On a keyframe, or right before a grasp closes, the box goes to SAM 2. The mask of the rim comes back for grasp planning. Detection runs every frame. Masking runs when precision is worth the latency.

One honest limit on the hosted version of this pattern. This platform's SAM 2.1 lane runs automatic mode by design. Whole image in, every mask out, no prompt fields. Send a box prompt field and the answer is a 400 that names the field. The compose step happens on your side. Match the returned masks to your YOLO boxes by overlap, or run SAM 2.1 locally when you need true box prompting. The automatic mask generator behind the hosted mode has its own walkthrough.

Decision matrix

The hosted per-image lane

The vision route on this platform serves sam2.1-base, metered per image. The row has served since 6 October 2026 and sits on the public list as of 28 October 2026. The route runs the automatic mode by design. Whole image in, every mask out. The bill counts images, not masks. A request with three file parts bills three images. A refused request bills nothing. The sam2.1 large tier is not on the public list, and no date rides with that.

A first call, whole thing:

curl -sS $IRONSTRATUM_BASE_URL/v1/vision \
  -H "Authorization: Bearer $IRONSTRATUM_API_KEY" \
  -F "model=sam2.1-base" \
  -F "file=@frame.jpg" \
  -F "response_format=json"

import os
import requests

resp = requests.post(
    os.environ["IRONSTRATUM_BASE_URL"] + "/v1/vision",
    headers={"Authorization": "Bearer " + os.environ["IRONSTRATUM_API_KEY"]},
    data={"model": "sam2.1-base", "response_format": "json"},
    files=[("file", open("frame.jpg", "rb"))],
    timeout=30,
)
resp.raise_for_status()
answer = resp.json()
print(len(answer["annotations"]), "mask objects, billed", answer["usage"]["images"], "images")

Rates render on the pricing page, never typed by hand on pages like this one:

For the full route mechanics, parameters, and error shapes, see the vision reference.

FAQ

Is SAM 2 better than YOLO?

Better at what. They do different jobs. SAM 2 masks anything you point at, with zero training, and outputs no class. YOLO names and boxes the classes it trained on, much faster, and outputs no pixel mask in its base form. Pick the job first. Masks for unknown or shifting objects: SAM 2. Named boxes for known classes at frame rate: YOLO.

Which one is faster?

On every publisher table we read, the YOLO rows are faster. No publisher table runs both families under one protocol. On CPU the gap is huge. About 24 ms for a small YOLO mask model, against about 29 seconds for SAM 2 base. Same machine, different runtimes: PyTorch versus ONNX Runtime. On an A100 in PyTorch, SAM 2.1 sizes run 39.5 to 91.2 FPS. Treat any cross-family speed number with no basis line as marketing.

Can YOLO and SAM 2 run together?

Yes, and the pattern is standard. YOLO boxes become SAM prompts. Ultralytics ships auto_annotate for it. Its maintainers show the handoff in a 2024 issue thread. A 2025 arXiv survey reports the hybrid as fast, accurate zero-shot masking in real time settings. On this platform's hosted lane, the compose step is client-side. The route takes whole images and returns every mask. You match masks to your boxes by overlap.

Do I need a GPU for SAM 2?

For a control loop, yes. A serious one. The publisher tables run on an A100 in PyTorch. For labeling batches, CPU works, but slowly. The lightest SAM variant we saw on a publisher CPU table, MobileSAM, takes about 23.8 seconds per image in PyTorch. The third path is a hosted per-image lane. No local GPU, billed per image, shown above on this page.

What about SAM 3, YOLO26, YOLO27?

Dated states, read 2026-10-28. SAM 3: announced 19 November 2025. It adds text and image-exemplar prompts, plus SAM 3D for single-image 3D rebuilding. No license named on the post. YOLO26: shipped, launch coverage dated 20 January 2026. Its seg rows sit in the tables above. YOLO27: an early docs page, "no launch date has been set." The task boundary between promptable masking and class-trained detection is unchanged across all three moves.

Which license lets me ship a closed product?

SAM 2 under Apache 2.0 does, checkpoints included. Ultralytics YOLO under AGPL-3.0 wants the complete source of the entire derivative work published, per their license page. The other route is a commercial Enterprise agreement. There is no public price list. Read the license pages before the design diagram, not after.

Does YOLO do segmentation too?

Yes. The -seg variants output masks and class labels in one pass, trained on your classes. One row from the table above: YOLO26n-seg scores 33.9 mask mAP on COCO at 53.3 ms per image on CPU ONNX. Promptable masking still wins on untrained classes and on mask precision. YOLO-seg wins on speed, and on having a class name on every mask.

Where to go next

• The SAM 2.1 versus SAM 2.0 version map: SAM 2 vs SAM 2.1
• Automatic mask generation without any prompt: the AMG walkthrough
• Where masks meet depth in a robot stack: segmentation and depth
• Route mechanics: the vision reference
• Costs across the catalog: pricing

The smallest first experiment is one image through the hosted lane. Open an account at signup. Send one frame. Read the mask count and the billed image count in the same response.