How to run Molmo 2 locally: the variant map, the hardware, the calls

Molmo 2 is the Allen Institute's open vision-language family released in December 2025. Running it locally takes one modern GPU, or a Mac through community ports, a 20 to 35 GB download, and Python 3.11 or newer. This guide maps the five public variants, the hardware each one needs, the install, the image and video calls, the pointing output format, the dtype choices, and the hosted lane for the embodied reasoning variant on this platform's metered API.

The short answer for most readers: grab allenai/Molmo2-4B for a 16 GB card, or allenai/Molmo2-8B for a 24 GB card. Install transformers==4.57.1 plus the small dep set from the model page. Load with dtype="auto" and device_map="auto". Parse points as values on a 0 to 1000 scale. Each number above is dated and sourced below. The official repo, the Molmo2-8B model page, and the Ai2 launch post are the three sources this page leans on most.

The five public variants

The Molmo 2 family has five public variants. Three are base assistants. Two are tuned builds on the 4B base. Each table cell below carries one basis.

Basis, cell by cell. Backbones come from the repo model table and the model cards. Download sizes are shard sums from the Hugging Face file lists, read 28 October 2026. The bf16 column is derived, not measured: load the download in 16 bit and weight memory lands near half the shard total. Context windows come from the repo conversion note. It pins 36,864 for the 4B and 8B, and 65,536 for the O-7B. The ER and VideoPoint rows are dashed on context because that note does not cover them.

Two follow-up notes on that table. First, the ER variant arrived in May 2026, five months after the December 2025 base family. Its page states it sits on Molmo2-4B. Second, every repo ships the same model code files. The file lists, checked per repo, show modeling_molmo2.py, processing_molmo2.py, and video_processing_molmo2.py in all five. Switching variants is a checkpoint swap, not a code change.

What changed from the 2024 Molmo

The first Molmo shipped in September 2024. Molmo 2 shipped in December 2025. Four changes matter to anyone running the weights.

The vision encoder moved from a CLIP ViT backbone to SigLIP 2 on the 4B and 8B. The point scale moved. The 2024 family printed points on a 0 to 100 scale. Molmo 2 prints them on a 0 to 1000 scale. Parsing code from the old family breaks with no error thrown. Video also arrived. The 2024 family read images only. Molmo 2 reads video, multi-image sets, and does tracking across frames. And the size to quality ratio moved hard. The launch post states the Molmo 2 8B beats the old 72B on the image pointing and grounding tests that made Molmo known.

The paper, arXiv 2601.10611, version 4 dated 2 April 2026, reports the grounding numbers. Video pointing F1: 38.4 for Molmo2-8B against 20.0 for Gemini 3 Pro. Video tracking: 56.2 against 41.1 on their eval sets. On the model page's own 15 test average, Molmo2-8B scores 63.1 and the 4B scores 62.8. Qwen3-VL-8B sits at 59.5, InternVL3.5-8B at 54.1, GPT-5 at 70.6. Those are the authors' own numbers, read 28 October 2026.

The hardware ladder

The documented path wants an NVIDIA GPU with CUDA. Here is the ladder, with the basis printed next to each claim.

The one dated third-party test: a tutorial published 27 April 2026 ran Molmo2-4B in bf16. It reported about 11 GB of VRAM for the weights, and it asked for about 24 GB for comfort during inference. That is one source, dated. It fits the derived 9.7 GB plus the vision encoder and buffers. No public bench of Molmo2-8B on a named consumer card existed as of 28 October 2026. Searches for 3090 and 4090 Molmo2 benches return generic card pages only. Treat the 8B row as math on the download, not as a measured figure.

Video costs more than images. Frames become tokens, and long clips eat memory fast. The repo's own fix is context parallelism, spreading one long video across several GPUs. One issue thread also caught a decoder stall on a damaged video file. The torchcodec reader gave up a frame skip after 10,000 tries. Keep test clips short and clean while you bring a pipeline up.

Install: two paths

Most people want the transformers path. It is four commands from the model page quick start, read 28 October 2026:

conda create -n molmo2 python=3.11
pip install transformers==4.57.1
pip install torch pillow einops torchvision accelerate decord2 molmo_utils

The molmo_utils package handles image and video prep before the call. The version pin matters. The quick start names transformers==4.57.1 exactly. Python 3.11 is the stated advice, not a hard floor.

The repo path is for training, eval, and conversion work. From the README, read 28 October 2026:

git clone https://github.com/allenai/molmo2
cd molmo2
pip install torchcodec
pip install -e .[all]

Two details from that README. torchcodec installs on its own line because of dep conflicts, and skipping it is the usual cause of video decode failures. And pip install -e .[all] wants a source checkout. A Docker image with the deps preinstalled sits at ghcr.io/allenai/molmo2:latest. It ships no code, so mount or clone the repo into it.

The README also documents a conversion step from the raw training checkpoints to the Hugging Face format, convert_molmo2_to_hf. The max length override is 36,864 for the 4B and 8B, and 65,536 for the O-7B. You need this only if you train your own checkpoint or pull a raw pretrained one. Read the rough edges section before you do.

The first call: image, then video

The quick start loads with AutoModelForImageTextToText and runs under torch.inference_mode() with torch.autocast("cuda", dtype=torch.bfloat16). Trimmed to the load and one image question:

from transformers import AutoModelForImageTextToText, AutoProcessor
import torch

model = AutoModelForImageTextToText.from_pretrained(
    "allenai/Molmo2-4B", dtype="auto", device_map="auto",
    trust_remote_code=True,
)
processor = AutoProcessor.from_pretrained(
    "allenai/Molmo2-4B", trust_remote_code=True,
)

messages = [{
    "role": "user",
    "content": [
        {"type": "image", "image": image},
        {"type": "text", "text": "Describe this image."},
    ],
}]
inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt",
).to(model.device)
with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16):
    out = model.generate(**inputs, max_new_tokens=200)
gen = out[0, inputs["input_ids"].size(1):]
print(processor.tokenizer.decode(gen, skip_special_tokens=True))

Video is the same call with {"type": "video", "video": video} in the content list. The video path runs through molmo_utils.process_vision_info for frame pickup. Multi-image sets pass several image entries. For tracking questions across frames, the model page advises sampling at max_fps=8 for best results.

Pointing output: parse the 1000 scale

Pointing is the family signature. Ask "point to the bird" and the answer text carries machine-readable point tags. The values are integers on a 0 to 1000 scale. The parse is one regex and one split:

import re

COORDS = re.compile(r'<(?:points|tracks).*? coords="([0-9\t:;, .]+)"')
POINTS = re.compile(r"([0-9]+) ([0-9]{3,4}) ([0-9]{3,4})")

m = COORDS.search(answer_text)
points = []
for idx, x, y in POINTS.findall(m.group(1)):
    points.append((int(x) / 1000.0 * image_width,
                   int(y) / 1000.0 * image_height))

That is the trimmed core of the extractor on the model page, read 28 October 2026. The values are integer triplets, index then x then y, inside a coords attribute on a points or tracks tag. The card's own extractor bounds-checks every scaled point against the image size before keeping it. Video tracks carry a frame id ahead of each run, and the card's full parser handles that too, so copy it from there for video work.

Porting between the families is the silent break, and each direction fails its own way. The old family scaled 0 to 100. Old code on new output divides 0 to 1000 values by 100, which throws almost every point off-image. Only points already in the top left tenth land inside the frame. The reverse port, dividing old 0 to 100 values by 1000, compresses every point into the top left tenth.

Serving with vLLM

For anything past single calls, serve it. The repo states vLLM support from version 0.15.0. The vLLM project ships a native molmo2 module, with its own API docs page, read 28 October 2026. A serve command in the standard vLLM shape, with the family's documented max length from the repo conversion note:

vllm serve allenai/Molmo2-8B \
  --trust-remote-code \
  --dtype bfloat16 \
  --max-model-len 36864

That gives an OpenAI-style endpoint for chat calls with image and video inputs. Two notes from the README. Offline batch work through vLLM wants the molmo_utils package for vision prep, as molmo-utils[torchcodec]. And the repo's own eval runs used the local path, with vLLM noted as the faster route. SGLang does not serve the family. Its tracking issue sat from 18 December 2025 until it closed in May 2026, and the single pull request, which covered only the O-7B, was closed unmerged. The named lanes are transformers and vLLM.

Dtype choices and the 4 bit reality

Three dtype facts, each with one basis.

First, keep load dtype and compute dtype apart. The quick start runs dtype="auto", which loads the 32 bit weights the config calls for, and its bf16 autocast narrows the math, not the storage. Autocast buys speed. It does not free VRAM. To halve weight memory, load bf16 on purpose: pass dtype=torch.bfloat16 in the from_pretrained call, or flip the repo's load_bf16 flag, which the README states "generally does not affect performance".

Second, the 32 bit default is why the downloads look large. The 19.4 GB 4B download becomes about 9.7 GB of weights in bf16. Loading 32 bit doubles memory for no measured quality gain at inference time.

Third, 4 bit is not a working lane right now. The same April 2026 tutorial that measured the 4B also reports that INT4 loads through BitsAndBytes failed on Molmo 2 at their test time. That is a break from the 2024 family, where 4 bit loading worked in public tutorials. No official GGUF exists for any Molmo 2 variant as of 28 October 2026. The request threads on the model pages sit open. Neither Ollama nor llama.cpp lists the family. Apple silicon runs on community MLX ports. The 8B ships in six MLX builds, 4 bit through fp16, and community 4B ports exist too. None of them carry an Ai2 link. If your card is 8 or 12 GB, the 4 bit MLX build of the 8B is the one lane that gets close. Its quality is unverified here, so treat it as a trial, not a pick.

One memory tip from the README. Set PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to cut out of memory failures during video work.

Rough edges worth knowing

Three issues, all dated and public.

Raw pretrained checkpoints differ from the instruct releases. An open repo issue from 22 April 2026 reports the split. A converted 4B pretrained checkpoint answers video prompts with the words "The image is a photograph", while the 8B pretrained handles the same clip. The reporter speculates the cause is the pretraining mix. No maintainer has replied. If you want video, pull the instruct release from the model page, not a raw training checkpoint.

The video decoder can stall on damaged files. The same thread carries a torchcodec log line where a frame skip aborted after 10,000 decoder tries on one mp4. Corrupt or odd containers will look like model failures. Test the file first.

The pointing track has a successor. In March 2026 Ai2 put out MolmoPoint, a newer design that picks regions straight in visual features instead of printing text points, with a GUI tuned 8B build. It builds on Molmo 2, and its models live in the same repo. Nothing here promises a port path into the base family. If you start a pointing product on Molmo 2 today, know the newer track exists and date your pick.

When a hosted lane makes sense

Local wins when you need the 8B, long video, the full context window, or batch economics on your own silicon. A hosted lane wins when the job is the embodied reasoning variant, Molmo2-ER, and you would rather not own a GPU idling between camera frames.

This platform lists the molmo2-er model id on its chat route and has served it since 6 October 2026. The model page has the entry and the pricing page renders the live rates. The served context window is 8,192 tokens, enforced and published with the row. Plan prompts against that window on this route, not the 36,864 from the local conversion note.

What a grounding session costs, in symbols first:

cost per query  =  ( I x Pin + O x Pout ) / 1,000,000

I:     input tokens per query, image tokens plus text
O:     output tokens per query
Pin:   input rate from the fenced row
Pout:  output rate from the fenced row

A labeled example, not a measurement. One camera image plus a task sentence runs about 1,500 input tokens. An answer with short reasoning runs about 400 output tokens. At the fenced rates that is 0.0015 plus 0.0020, so 0.0035 per query. That is 3.50 per 1,000 queries. With the image prefix cached at the cached rate, the same query drops to 0.00075 plus 0.0020, or 2.75 per 1,000. One sensitivity line. Output bills at five times the input rate in the fenced row, so answer length moves spend the most. Wallets are prepaid and a refused request bills nothing. That is the property you want under robot loops left alone for hours. The signup page starts one, and the API bill shock guide covers the failure modes.

FAQ

Does Molmo 2 run on Ollama or llama.cpp?

No, as of 28 October 2026. Neither project lists the family, llama.cpp's maintainers closed the support request as not planned in March 2026, and no official GGUF exists. Community GGUF conversions of the 4B sit on the Hub, unverified here; treat them as trials, not lanes. The working local lanes are transformers, vLLM from version 0.15.0, and community MLX ports on Apple silicon.

Is there an official GGUF coming?

No dated commitment exists. The request threads on the model pages were still open as of 28 October 2026. This page makes no promise on Ai2's behalf. It tracks only what is checkable.

Will Molmo 2 run on a 16 GB GPU?

The 4B class loads and runs in bf16 on 16 GB with room for activations. One dated third-party test measured about 11 GB of weight memory on the 4B. It still asked for 24 GB for comfort during video work. The 8B needs a 24 GB card in bf16, derived from its 34.6 GB download.

Which variant should I pick for pointing?

For pointing in images and video with the top scores, the 8B. For pointing on a 16 GB card, the 4B. For pixel pointing aimed at robot vision, Molmo2-ER. Its page reports it beating open weight baselines on 9 of 13 embodied tests with a 63.8 average. For pointing and counting inside video, VideoPoint-4B.

What license applies?

The model pages list Apache 2.0. The base pages add a data note. Parts of the training mix come from third-party sources limited to academic and non-commercial research use. Read the license section of your exact variant page before commercial use. The same review applies to any variant you rehost.

Molmo or Molmo 2 for a new project?

Molmo 2, unless you need something only the old family offers. The new 8B beats the old 72B on pointing and grounding per the launch post. It also reads video, which the old 72B does not. Keep the old family only for code built on the 0 to 100 point scale that you cannot yet change.

Last verified: 2026-10-28