MolmoAct2 checkpoint loading and dtype errors: the exact strings, causes, fixes, and prevention

MolmoAct2 checkpoint errors cluster into four families: dtype crashes led by mat1 and mat2 must have the same dtype, missing normalization files, version-split config rejections, and a known slow-loading path in LeRobot. Every released checkpoint ships its weights in float32, and that one fact explains most of the dtype crashes. This page quotes each error string exactly as the publisher's code prints it. It names the cause, gives the fix, and lists the prevention. Every source was read on 27 October 2026.

Last verified: 2026-10-27

The errors at a glance

Match your screen against this table first. Every string below is copied exactly from the publisher's code, cards, or issue tracker. Each fix section carries its source and date.

Three sentences carry the whole page. Every released MolmoAct2 checkpoint ships its weights in float32, and every dtype crash traces back to that one fact. The mat1 and mat2 error has two different trigger sites, and the publisher's example servers patch both at the source. The normalization tag you pass is a contract, one tag per embodiment, four tags in the family.

Why these checkpoints trip dtype errors

MolmoAct2 is the Allen Institute for AI's open family of action reasoning models for robot control. The official repository publishes the weights, the training code, and two example servers. The family ships one fine-tuned checkpoint per robot setup: the SO-100 and SO-101 arm, the Franka DROID setup, the bimanual YAM setup, and the LIBERO recipe.

The dtype trouble starts with a design choice you can read in the artifact itself. The checkpoint's own config.json sets "dtype": "float32". All five weight shards in the SO-100 and SO-101 checkpoint total about 22 GB on disk. Both facts come from the checkpoint's own files, read 27 October 2026. The publisher trained with mixed precision and ran its reported experiments in float32. The SO-100 and SO-101 model card prints the cost of that path: roughly 26 GB of GPU memory with CUDA graphs enabled, or around 24 GB without.

That does not fit a 16 GB card, and the card's bfloat16 path is the practical route on smaller cards, with the card's own blessing. The same card prints the fix in its own words: bfloat16 "can run under 16GB of GPU memory in our tests" and "usually does not hurt performance much". One catch: bfloat16 loading was not part of the original code path. The repository's README, read 27 October 2026, states it plainly. Loading in bfloat16 is not officially supported by the original MolmoAct2 code. So each example server applies two small patches at startup.

The first patch matters most for this page. In the flow-matching action expert, the code that builds the trajectory hardcoded float32, no matter which dtype your weights use. Load the model in bfloat16 and the expert tries to combine a bfloat16 weight matrix with a float32 tensor. That is the entire story behind the error string. The second site is the image pipeline. The processor hands the model float32 pixel values. With bfloat16 weights, the first matmul trips the same message. The DROID example server documents both sites in its own comments, read 27 October 2026.

The fixes, string by string

mat1 and mat2 must have the same dtype

The string: mat1 and mat2 must have the same dtype.

The cause: one tensor is float32 while the weights it meets are bfloat16. Two distinct sites produce it. In the action expert, the trajectory init hardcoded float32 in older snapshot revisions. In the vision path, float32 pixel values reach bfloat16 weights because the model's input mover changes device but never dtype.

The fix, in the order the publisher applies it. First, patch the trajectory init so it inherits the model dtype instead of hardcoded float32. The DROID and YAM servers do exactly this at startup. Their patch is idempotent, so a checkpoint re-download does not undo it. Second, cast floating-point inputs to the model's dtype before the forward pass. Third, if you call the model directly rather than through a server, wrap inference in torch.inference_mode() and torch.autocast("cuda", dtype=torch.bfloat16). The SO-100 and SO-101 model card prints this exact pattern for its bfloat16 example.

The prevention: keep one dtype plan and state it once. Newer snapshot revisions of some checkpoints have already fixed both sites at the source. The README names the YAM revision. On those, the servers log a harmless "needle not found" warning, because there is nothing left to patch. If you maintain your own conversion, keep float32 for the action expert and the normalization math. Cast at the edges. The LeRobot port uses the same storage plan.

'list' object has no attribute 'keys'

The string: 'list' object has no attribute 'keys'.

The cause: the checkpoint's tokenizer_config.json stores extra_special_tokens as a list, while transformers 4.46 and newer expect a dict and call .keys() on it during processor load.

The fix: pass an empty dict for that one argument and load from the resolved local directory, AutoProcessor.from_pretrained(local_dir, trust_remote_code=True, extra_special_tokens={}). The DROID example server ships this workaround in its own loading code, read 27 October 2026. Its comment explains the safety. The model only looks these tokens up by name, so an empty dict is safe.

The prevention: pin your transformers version in any environment you reproduce, and re-test processor load after any upgrade of it. Version-pair breaks between a frozen checkpoint's config files and a moving library are this error's whole life cycle.

no kernel image is available for execution on the device

The string: no kernel image is available for execution on the device.

The cause: an RTX 50-series GPU needs the sm_120 kernel set, and your PyTorch wheel was built before it. The wheel loads, the model loads, and the first forward pass finds no kernel for your card.

The fix comes from the repository README, read 27 October 2026. Reinstall the CUDA 12.8 PyTorch wheels from the repository root. Then confirm the architecture list contains sm_120 by printing torch.cuda.get_arch_list().

The prevention: after any environment rebuild, run the README's one-line version check before you download a 22 GB checkpoint. The check prints the torch version, the CUDA version, and your card's compute capability. It costs seconds.

The norm_tag and norm_stats.json errors

Four strings, one contract. The model card for each checkpoint names its tag, and predict_action reads norm_stats.json beside the weights.

The strings, as the modeling code prints them:

• MolmoAct2 `predict_action` requires `norm_tag`. when you call inference with no tag at all.
• Unknown MolmoAct2 normalization tag 'so100'. Allowed tags: so100_so101_molmoact2. when the tag does not match the checkpoint, here the short name so100 against the real tag.
• MolmoAct2 normalization stats are not loaded and config._name_or_path is empty; load the model from a converted HF directory containing norm_stats.json. when the model was built without a usable local path.
• MolmoAct2 normalization stats file is missing: /path/norm_stats.json. Converted checkpoints must include norm_stats.json. when a converted checkpoint ships weights but not the stats file.

The tags, read from the four model cards on 27 October 2026:

The cause has two layers. The tag layer is plain. Each checkpoint normalizes its state and actions against statistics from its own robot. The tag selects the statistics, the action horizon, and the prompt metadata in one choice. The file layer is a path quirk. At load time, the modeling code reads norm_stats.json from the directory in config._name_or_path. Loading by repository id can leave that as a name rather than a path.

The fix: resolve the local snapshot directory first, with snapshot_download(repo_id), and load from that directory. Then pass the card's tag verbatim. Both example servers run exactly this sequence. Their comments say a direct id load makes predict_action fail at runtime.

The prevention: never split the pair. A checkpoint directory without its norm_stats.json is not a checkpoint. A tag from a different embodiment gives you wrong motion instead of an error. That is the quieter and more expensive failure. The stats file carries the format marker molmoact2_norm_stats.v1 and the mode q01_q99 at the top, so a one-line read confirms you shipped the right file. The wider normalization failure family has its own treatment in the normalization statistics guide and the missing stats family guide. That family includes the silent wrong-motion cases LeRobot tracks in its own issues.

Unsupported dtype / Unsupported model_dtype

Two strings, one setting, two LeRobot versions.

On the current main branch, the policy config names the setting dtype. It accepts exactly torch.float32 or torch.bfloat16, and rejects anything else with Unsupported dtype=torch.float16. Expected torch.float32 or torch.bfloat16. Read from the policy configuration source on 27 October 2026, where the default is torch.bfloat16.

On release 0.6.1, the same setting is model_dtype, a string. It accepts float16 as well, and rejects other values with Unsupported model_dtype='float64'. Expected 'float32', 'bfloat16', or 'float16'. Read from the 0.6.1 tag of the same file on 27 October 2026.

The cause: a dtype refactor landed on main after 0.6.1. The field was renamed and narrowed. A config that runs on one branch dies on the other with a message that names the other branch's vocabulary.

The fix: match your config to your checkout. On main, set dtype to bfloat16 or float32 and drop float16. On 0.6.1, set model_dtype with the string form. The LeRobot MolmoAct2 documentation, read 27 October 2026, prints the 0.6.1-era name --policy.model_dtype=bfloat16 in its training example. A docs example and a main-branch checkout therefore disagree until the docs catch up. Trust the error message. It always names the field your installed code expects.

The prevention: record the LeRobot version in every experiment note next to the config, and re-run a one-cell load after any branch switch.

Why loading takes a minute when nothing is wrong

Slow loading with no error is its own known failure. LeRobot tracks it in an issue titled "Inefficient checkpoint loading (redundant model loads/initializations, device transfers) causes a 30x slowdown", issue 4520, open since 26 August 2026 and still open on 27 October 2026.

The reporter traced a fine-tuned MolmoAct2 load step by step. The path loads the full float32 base checkpoint into a bfloat16 CPU model. It converts selected parameters back to float32. It loads the base shards a second time and copies them in. Then it loads the fine-tuned weights onto the GPU, copies them back into the CPU model, and moves the whole model to the GPU last. Seven steps, six of them not needed in the reporter's reading. Two are full reads of the base weights. One is a detour through the CPU for weights that end up on the GPU anyway.

The reporter's alternative loads the fine-tuned weights straight onto the GPU and assigns them to a model built on the meta device, skipping every redundant step. In the reporter's words, it "reduces loading time from about 1 minute to 4 seconds". The same reporter measured a similar policy's load falling from about 60 seconds to 2. The fix has traveled as a pull request since 31 August 2026: "Load MolmoAct2 checkpoints without base weights", pull 4544. It was still open on 27 October 2026. No released LeRobot version contains it yet.

What you can do today, in order of effort. Keep the process resident: one load per session, not one per rollout, turns a minute of startup into a non-event. Keep the base checkpoint cached. The redundant second read hits your disk or your network. The README's HF_HOME pointer lets you park the cache on your fastest disk. Warm the graph before you measure. The SO-100 and SO-101 card warns that the first few calls are slow while the model warms up and captures CUDA graphs. The example servers ship a --no-warmup flag that skips the dummy forward pass at startup. That flag trades first-call latency for startup speed. If you control the load path, adopt the reporter's shape, load to the target device once, and never copy through the CPU.

The runtime side of "slow" is separate and measured. The paper reports the numbers on a single H100 at an action horizon of 10: 23.02 Hz on the original path, 27.39 Hz with a reusable cache, and 55.79 Hz with CUDA graphs. That last step is a 2.42 times speedup. So a policy that feels slow after loading usually lacks the graph capture. It is not hitting this issue.

Memory and disk planning

Dated 27 October 2026, from the publisher's own cards, docs, and file listings.

One disk fact the cards leave implicit: every embodiment fine-tune is a full copy of the model, not a small adapter. Pull the SO-100 and SO-101, DROID, and bimanual YAM checkpoints. That is three downloads of about 22 GB each, before you train anything. A LeRobot-format conversion of the LIBERO checkpoint exists as its own repo. Format conversions add a second copy unless you delete the source. Plan the cache disk before the first download, not after the third.

Prevention checklist

• Pick one dtype plan per deployment and write it down: bfloat16 everywhere, float32 everywhere, or the mixed plan with float32 kept for the action expert and normalization math.
• Resolve the local snapshot directory and load from it, never from the bare repository id.
• Pass the checkpoint's own norm_tag verbatim, and keep norm_stats.json beside the weights in every conversion you build.
• Pin transformers and LeRobot versions together, and re-run a load test after any change to either.
• On an RTX 50-series card, check torch.cuda.get_arch_list() for sm_120 before the first download.
• Keep the policy process resident across rollouts, keep the base weights cached on a fast disk, and warm the CUDA graph before measuring anything.
• Record the LeRobot branch name beside every config field you set, because the dtype field's name and accepted values differ between 0.6.1 and main.

If you would rather not debug loading

Every fix above is self-hosting work: the download, the dtype plan, the cache disk, the warm-up, and the issue tracker. That is the honest cost of open weights, and the MolmoAct2 capability guide covers what the model does once it runs.

The hosted route exists as a written record on this platform. The platform's model data carries the molmoact2-so100-101 model id on its action route, metered per call, and as of 27 October 2026 that row is not on the public list, with no date attached to that step. The models index is the live catalog, listed rates render on the pricing page, and the rate card reasoning for this model lives in the MolmoAct2 pricing guide. If that route fits your loop, you can sign up, create a key, and budget calls without touching a checkpoint file.

Frequently asked questions

Is bfloat16 safe for MolmoAct2, or do I need float32?

The publisher's own answer, printed on the SO-100 and SO-101 model card, read 27 October 2026: bfloat16 "can run under 16GB of GPU memory in our tests" and "usually does not hurt performance much". The reported experiments ran in float32 at about 24 to 26 GB. Run float32 if your card holds it and you want the publisher's measured configuration. Run bfloat16 with the two fixes if it does not. What is never safe is bfloat16 without the fixes, because that is the mat1 and mat2 must have the same dtype crash.

Which norm_tag does my checkpoint need?

Exactly one, printed on its own card: so100_so101_molmoact2 for the SO-100 and SO-101 checkpoint, franka_droid for DROID, yam_dual_molmoact2 for bimanual YAM, and libero for the LIBERO recipe. The tag selects the normalization statistics, the action horizon, and the prompt metadata together. A tag from a different checkpoint either errors out or moves the arm wrong. The stats file beside the weights carries the marker molmoact2_norm_stats.v1, which is how you confirm the right file shipped.

Why is my first inference call slow even after loading finishes?

Two normal reasons. CUDA graph capture happens on the first calls, and the model card says to run several warm-up calls before measuring. Loading itself can also be the slow part in LeRobot today. The current path performs redundant base-weight loads and device copies. That is the open issue above. If the first call is slow and the hundredth is fast, that is graph capture. If every restart is slow, that is the loading path.

Does the checkpoint really ship in float32? The file is huge.

Yes. The checkpoint's own config.json sets "dtype": "float32", and the SO-100 and SO-101 weight shards total about 22 GB, both read on 27 October 2026. The bfloat16 you run is a cast applied at load time, by your code or by the publisher's patches. That is why the dtype errors exist at all. The cast is a load-time edit to code that assumed float32 forever.

Where do these errors come from, a bug or a design choice?

The float32 shipping is a design choice from a mixed-precision training run whose reported numbers were measured in float32. Bfloat16 friction is a gap in the original code path, acknowledged in the repository README and patched in the publisher's own example servers. The slow loading is a tracked defect, issue 4520, with an open pull request carrying the fix. None of the three means your environment is broken. All three have documented workarounds on this page.

The short version: quote your error string against the table, apply the named fix, and keep the prevention list beside your deploy script. The capability record for the model is the MolmoAct2 guide, rates live on the pricing page, and the hosted route starts at signup.

Last verified: 2026-10-27