VLM vs VLA for robot control: the two-tier answer

A VLM cannot drive your motors. A VLA can. That one line settles most of the VLM versus VLA question for robot control. Both model classes read the same camera frames and the same words. One answers in text, and the other answers in motion. The useful question is not which model is better. It is which tier does which job, at what rate, and on which meter. This page gives the two-tier answer with the papers' own numbers, maps the hybrid pattern that production stacks run, and shows both tiers served over one API, dated 28 October 2026.

Last verified: 2026-10-28

The short answer

Three sentences carry the verdict, and each has a source.

A VLM is the reasoning tier. It reads frames, answers questions about the scene, points at things, plans steps, and names failures. It runs at chat speed, seconds per answer, because it is built and sized for language.

A VLA is the action tier. It reads frames, an instruction, and the robot's own joint numbers. It returns the next chunk of motor commands. The pi 0 paper reports robot control at up to 50 steps per second from this design (pi 0 paper, 2024).

Most serious stacks run both. The reasoning model picks the goal. The action model moves the arm. Named examples sit on every side: Helix from Figure, Gemini Robotics from Google DeepMind, and the open MolmoAct2 line from Ai2. The rest of this page is the evidence for those three sentences.

What each model is, in one table

The full definitions live on two sibling pages: the VLA explainer for the action side and what a VLM is for the reasoning side. The working summary is one table.

Two rows carry dashes. No VLM design feeds motor commands, so there is no control rate to print for one, and no VLM reads your joint encoders. The dash is the honest cell: the value does not exist for that tier.

The lineage matters. Google DeepMind built RT-2 in 2023 by tuning a VLM on robot data. That paper named the VLA class (RT-2 paper, 2023). The action tier is not a rival to the reasoning tier. It is a child of it, with a new output bolted on. SmolVLA makes the split physical. It holds 450 million parameters in the whole model, about 100 million in the action part. The VLM half stays frozen while you tune the action part on your own data (SmolVLA paper, 2025).

What a VLM gives a robot stack

Four jobs, each with a receipt. None of them move a motor.

Scene grounding and pointing. Ask where something is, get an answer your code can use. Molmo2-ER is a 4B model built for this job. Its model card covers scene understanding, pixel-accurate pointing, multi-image correspondence, and video temporal reasoning. These are the skills downstream action models depend on (Molmo2-ER model card). On its own sheet it beats the state of the art on both sides, open weight and closed, Gemini Robot-ER 1.5 Thinking and GPT-5 among them. The sheet spans 13 embodied reasoning benchmarks; it wins 9, at a 63.8 percent overall average. Pointing is the cheap bridge from words to geometry: the pixel is what a grasp planner or a perception route consumes.

Planning. A VLM turns a vague goal into ordered steps, or picks the next subtask. Helix uses a 7B VLM as its slow half, System 2, which reads the scene and hands a goal vector to the fast half (Helix, 2025). Google DeepMind ships the same split as a product pair: a VLA plus an embodied reasoning model that "work together". The reasoning model takes "real-world understanding and complex, multi-step planning" (Google DeepMind). Research lines go further and put the VLM inside the controller itself: VLMPC has a VLM sample candidate action sequences for a model-predictive controller (VLMPC, 2024).

Failure diagnosis and recovery. This is the 2026 finding that changes the hybrid story. An ICML 2026 study asked whether VLMs can diagnose VLA faults. It built the FixBench dataset and tested 20 VLMs. It then designed a VLM-to-VLA loop that finds the fault and rolls the task back. The upper-bound read: an ideal feedback loop lifts task success by 13 percent on LIBERO and by 35 percent on real robots (ICML 2026 poster). The reasoning tier is not a hat on the action tier. It is the doctor for it.

Data labeling. The quiet job. Helix wrote its own training instructions with a VLM. It ran the model over recorded clips and asked, in hindsight, what instruction each clip met (Helix, 2025). Every team that records teleoperation faces the same pile of unlabeled video. A VLM is the standard tool for turning that pile into instruction-tagged data.

What a VLA gives you

The action tier is a policy: one trained network from pixels and words to motion. Three facts define it.

It returns action chunks, not single moves. ACT introduced chunking in April 2023. One pass predicts a run of future actions, and every later model kept the habit. A chunk is what lets a slow network drive a fast robot: the arm works through the moves it holds while the next answer computes.

It is bound to a body. The same weights do not move a different arm without adaptation. Norm stats map your robot's units onto the model's trained range. Wrong stats move the arm wrong, and no error appears. That failure class has its own guide, VLA normalization stats.

It is trained on episodes, not on the web. OpenVLA is 7B parameters trained on 970,000 real robot demonstrations (OpenVLA paper, 2024). The open pi family bases carry more than 10,000 hours of robot data in their pre-training (openpi repository). Robot data is the scarce input. That is why the open set is small, and why tuning on your own body is the standard path. A May 2025 review covers more than 80 of them (survey, 2025).

Reading their benchmark claims is its own skill. The LIBERO evaluation guide teaches it.

Can a VLM control a robot directly?

Yes, and the history explains why the field moved off it.

RT-2 did it in 2023. The trick: sort each action dimension into one of 256 bins and emit the bin numbers as text tokens. The model then writes actions with the same layer it writes words with (RT-2 paper, 2023). It worked. Emergent reasoning showed up in the control behavior. The cost was speed. The paper's own numbers: the 55B model ran at 1 to 3 Hz, the 5B version about 5 Hz. A robot that needs fresh decisions dozens of times per second cannot wait on a text model writing eight numbers per move.

The field answered by moving the action work out of the text path. Decoders that emit continuous actions directly, or refine them from noise, took over. Pi 0's flow matching design reports control at up to 50 steps per second. That is a 10x or better jump over the text route on the same class of task (pi 0 paper, 2024).

The 2026 counterpoint is honest work. Show-Harness gives a VLM direct control through discrete semantic action units. Fixed interpreters ground each unit to the local robot. The paper reports agents that generalize across tasks and bodies, ahead of representative VLA setups on its suite (Show-Harness, 2026). The lesson is not that text can drive motors at servo rate. It is that the interface decides how much of the control problem the reasoning model must touch. For production arms today the answer stands: the VLM picks the goal, the VLA closes the loop.

The hybrid pattern: both tiers, one stack

Every named production system runs the same shape. One slow reasoning model, one fast action model, a narrow channel between them.

Helix is the clearest print. System 2 is a 7B open-weight VLM that reads the scene and updates a shared goal vector with the intent. System 1 is an 80M visuomotor transformer that turns that vector into continuous actions at 200 Hz. The pair runs on the robot's own embedded GPUs, trained on about 500 hours of teleoperation (Helix, 2025). Note the size inversion: the reasoning half is 7B, the control half is 80M. The loop wants small and fast. The reasoning wants big and slow.

Gemini Robotics sells the same split as two products. The VLA turns vision and language into motor control. The ER model handles understanding and multi-step planning. Their maker sells them as specialist roles in one system (Google DeepMind).

The open world mirrors it. MolmoAct2 "builds on the Molmo2-ER embodied-reasoning vision-language backbone", adds robot state and action modeling. It connects that backbone to a flow-matching action expert (MolmoAct2 repository). That is the hybrid pattern straight from the publisher: the VLM is the front half, the action expert is the back half. Pi 0 does the same trick inside one model. Its abstract reports it following instructions "from people and from a high-level VLM policy" (pi 0 paper, 2024).

Add the recovery loop and the picture is whole. The FixBench result above says the reasoning tier can watch the action tier, name the fault, and roll the task back, at a measured upside of 13 to 35 percent. What stays out of both tiers: the reflexes. Servo loops, balance, and the emergency stop run next to the motors, at rates no network call promises, on every serious machine.

Latency and the control loop

Numbers first, with the basis stated per row. The rates are for the pi 0 class of model unless a row names its own.

A 2026 measurement study from NVIDIA Research sets the bars. Cameras run at 24 to 60 frames per second, so it calls 10 Hz acceptable for a policy and 100 Hz high performance (VLA-Perf, 2026).

Chunking is how a model that computes fresh answers twice a second drives an arm at 50 Hz. The pi 0 paper's own loop: a 50 Hz robot gets a fresh answer every 0.5 seconds, with 25 moves per answer. A 20 Hz robot gets 16 moves every 0.8 seconds (pi 0 paper, 2024).

From the same paper's table: image encoding 14 ms, the observation pass 32 ms, the action pass 27 ms. That is 73 ms on board. Add the network and the total is 86 ms, a 13 ms difference. Remote inference is not the enemy. Bad round trips are.

Hardware sets the ceiling. The study's table for pi 0, end to end, network excluded:

Source: VLA-Perf Table 3, one basis, one model. The same study measures async designs that overlap compute and execution. On a fast wireless network, throughput reaches 314.4 Hz. On a poor network, sync inference drops to 3.7 Hz, and the async shape restores 50.5 Hz (VLA-Perf, 2026).

Remote control is proven in production. Physical Intelligence runs its policy inference remotely for real robots. The published account reports the setup adds only 10 to 15 ms of network overhead, while running thousands of cycles around the clock (Modal case study, 2026).

Now place the two tiers on that clock. The action tier lives inside it, at 10 to 100 Hz, riding chunks. The reasoning tier lives outside it. A VLM call takes hundreds of milliseconds to seconds, and nothing in a servo loop waits for it. That is not a defect. It is the design. The slow tier sets the goal at its own pace, the fast tier holds the loop between goals. The inference frequency guide covers the definitions in full.

The cost shape of each tier

No rate is typed on this page. Every listed rate on this platform renders on the pricing page from the platform's own model data. The pricing units guide explains the units. What this page can give is the shape of the bill, because the shape follows from the design above.

The reasoning tier meters per token, like any language model. A scene question bills for the tokens in the frames and the tokens in the answer. Loop rate does not touch this bill, because the reasoning tier sits outside the loop. Ask it when something changes, not every step.

The action tier meters per call. One served request returns one chunk, and one chunk carries many steps. The bill arithmetic is fixed by the loop: steps per second, divided by chunk length, times seconds run, times the rate on the pricing page. A 50 Hz arm served with 25-move chunks makes two calls per second, not fifty. Chunk length is set where the model runs, not per request.

Both tiers draw from one prepaid wallet here, and the balance is the hard stop. A request past an empty balance is refused with a named error and bills nothing. A robot that loops all night cannot run up a debt by accident. The local versus hosted break-even guide works the full math against your own hardware.

What this platform serves

The dated state, as of 28 October 2026. Each row mirrors the platform's own model data, and the models index is the live catalog.

The dashed rows are not on the public list, and no date rides with that fact. Their pages still carry the capability story and the request shape, so you can build against the contract either way.

Both tiers answer over one account and one wallet. The reasoning rows ride the chat route, an OpenAI-compatible endpoint, with the served window printed on each row's page: 8,192 tokens for molmo2-er. The action rows ride the action route, multipart form data in, an action chunk with its norm stats and a usage object out. For the deeper lane context: the VLA lane page for action, the chat category for the reasoning rows.

A first call on each tier

Two requests, one key. Both start at the signup page. Set the base URL and the key once, and every published example runs the same way.

The reasoning tier, a scene question to molmo2-ER over the chat route:

curl -sS $IRONSTRATUM_BASE_URL/v1/chat/completions \
  -H "Authorization: Bearer $IRONSTRATUM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "molmo2-er",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "text", "text": "Point at the leftmost object on the table and describe what is behind it."},
        {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,<frame bytes>"}}
      ]}],
    "max_tokens": 512
  }'

The action tier, one action call to smolvla over the action route:

curl -sS $IRONSTRATUM_BASE_URL/v1/action \
  -H "Authorization: Bearer $IRONSTRATUM_API_KEY" \
  -F "model=smolvla" \
  -F "instruction=pick up the red cube and put it in the bowl" \
  -F "state=[0.0, 0.12, -0.4, 0.0, 0.5, 0.1, 0.0]" \
  -F "image=@top.jpg" \
  -F "image=@wrist.jpg"

The first answer is text with points in it. The second is an array of action steps, the norm stats that shaped them, and a usage object with the call count and the wallet debit. Chunk length belongs to the model: send a horizon field and the answer is a 400 that names it. Send one image before you batch, on both tiers, and watch the meter on the first loop.

Which tier do you need?

Decision rules, in order of how much they change your stack.

Run the reasoning tier alone when your robot already moves and the gap is understanding. That means scene questions, pointing for a classical picker, step planning, failure triage, and unlabeled episode data. The perception side has its own tier below the VLM, masks and depth over the vision route, mapped on the robot vision guide.

Run the action tier alone when the task is one motion skill, fixed cameras, a fixed arm, and you have or can record episodes. Tune the action expert on your data with the VLM half frozen, the SmolVLA recipe. The SO-101 cloud brain guide walks that loop end to end.

Run both when the robot faces open instructions, new scenes, or failures that matter. That is the shape every named system ships. Budget the loop for the action tier, budget the questions for the reasoning tier, and keep the reflexes on the machine. The Jetson versus cloud guide works the hardware budget, and the robot inference hub page maps every lane in one place.

Frequently asked questions

Can a VLM control a robot?

Not at servo rate, and the one large attempt says so with its own numbers. RT-2 wrote actions as text tokens and ran at 1 to 3 Hz at 55B, about 5 Hz at 5B (RT-2 paper, 2023). Production control needs tens of steps per second, which is the job of the action decoder in a VLA. A VLM can still steer a robot through an interface of discrete semantic actions, the Show-Harness result. That shape suits slow, planned tasks (Show-Harness, 2026).

Is a VLA just a VLM with an action head?

Mostly, and the parameter counts say how much of each. SmolVLA is 450M total with about 100M in the action expert, and tuning trains the action part while the VLM stays frozen (SmolVLA paper, 2025). The head is small. What makes the class is the training on robot episodes and the body binding, not the extra layer.

Which is faster, a VLM or a VLA?

They are not on the same clock. A VLM answers in hundreds of milliseconds to seconds, at chat pace. A VLA serves action chunks fast enough for 10 to 100 Hz control, and the pi 0 design reports up to 50 steps per second with fresh inference twice a second (pi 0 paper, 2024). The chunk is the bridge between the two clocks.

Which tier costs more to run?

Neither answer is a number, because the two tiers meter different things. The reasoning tier bills per token, and its bill tracks how often you ask it questions. The action tier bills per call, one chunk per call, and its bill tracks steps per second divided by chunk length, times hours. Rates for every listed row render on the pricing page, never typed into a page like this one.

Do I need both?

If your robot takes open-ended instructions, works in new scenes, or fails in ways you need diagnosed, yes. The named production stacks all run the pair: Helix with a 7B VLM over an 80M motor half at 200 Hz, Gemini Robotics as a VLA plus an ER model, MolmoAct2 on the Molmo2-ER backbone. If your robot does one fixed skill, the action tier alone is the cheaper start.

What is the difference between VLA, VLM, and LLM?

An LLM reads text and writes text. A VLM adds eyes: images and text in, text out. A VLA adds a body: images, text, and joint state in, motor commands out. The VLA explainer carries the full decoder table for the whole stack, world models and reflexes included.

Can I try both tiers on one API here?

Yes. The reasoning rows molmo2-er and cosmos-reason2-2b serve on the chat route, and the action rows smolvla, openvla-oft, and act serve on the action route, all on the public list as of 28 October 2026. One account, one key, one prepaid wallet. Start at the signup page, confirm the rows on the models index, and send one frame before you loop.

Sources read on 2026-10-28: the RT-2 paper, the pi 0 paper, the VLA-Perf study, the OpenVLA paper, the SmolVLA paper, the Show-Harness paper, the VLA-FixBench ICML poster, the Helix announcement, the Gemini Robotics page, the Molmo2-ER model card, the MolmoAct2 repository, the openpi repository, the VLMPC paper, the VLA survey, and the Modal case study.

Last verified: 2026-10-28