Jetson vs cloud: where a robot's inference lives

A robot brain can sit in three places: on the robot, in a box at the site, or in a data center. The physics picks one, not preference. This guide places each layer of the control loop by its timescale, on measured numbers from a 2026 study, a placement model from Stanford and NVIDIA, and the publishers themselves. The action chunk is the reason a network hop is survivable at all, and Real-Time Chunking is the named fix that makes hundreds of milliseconds feel smooth. It also works the forgotten costs: battery hours, uplink bandwidth, and the memory cliff. The action routes here are not on the public list, the perception rows serve today, and the placement rules hold either way.

The money question has its own guide: the break-even math for running a policy yourself against paying per call lives in the local versus API guide. This page owns the systems trade: where the compute lives, and why.

Three places, one decision

On the robot means a compute module bolted to the machine. The reference part is NVIDIA's Jetson AGX Thor line: 2070 sparse FP4 TFLOPS, 128 GB of memory, 40 to 130 W (NVIDIA product page). Below it sit the Orin parts: the AGX Orin at up to 275 TOPS and 15 to 60 W, the Orin NX at 100 TOPS, the Orin Nano at the small end.

At the site means a box on the same network as the robot, desk class parts like the DGX Spark being the current shape: 128 GB of unified memory around a GB10 chip (NVIDIA product page). The robot keeps a small brain for capture and safety, and the policy runs one wireless hop away.

In the cloud means a data center part far away: the robot sends images and state up and receives actions back. The 2026 study that measured all of this used an A100 virtual machine for the cloud arm, a desk part and an L4 server for the site arm, and the Jetson parts onboard (Microsoft Research and UC Berkeley).

None of these wins outright. The decision is per layer, and a robot has more than one layer.

The latency budget, layer by layer

A robot is not one loop but three, running at three speeds, each with its own placement answer.

The reasoning row is the easy one. A planner that runs five times a second has 200 ms per decision. A wireless round trip of 10 to 50 ms fits inside that with room. The VLA-Perf paper sets the frame the field uses. It counts 10 Hz as acceptable for a policy, near camera rates. It counts 100 Hz as high performance (How Fast Can I Run My VLA?).

The reflex row is equally settled, in the other direction. A 100 Hz loop must produce an output every 10 ms. A typical wireless round trip alone costs 10 to 50 ms. The network is not a delay in that budget. It is the budget. Servo loops, safety stops, and balance belong next to the motors, on the small computer the robot already carries.

The skill layer is the contest. The study measured one precise task, handing an object between two arms, across all three placements. Accuracy was 80 percent with the policy on a data center part. It was 80 again on the Thor module onboard. It was 30 on the older Orin AGX. The difference was chunk time: about 60 ms on the data center part, 144 on the Thor, 440 on the Orin AGX. When chunks arrive late, the motion goes stop and go.

Both placements scored the same 80 percent. The physics favored whichever side produced the next chunk in time.

The chunk is why the network is survivable

The usual framing argues in frame rates: a 30 frame per second camera means 30 answers a second, so the network is impossible. That framing is wrong for these models, and the error is expensive.

A vision language action policy does not return one movement. It returns an action chunk: a run of future joint targets computed together. The publishers of the pi models use a chunk of 50 actions, one full second of execution (Real-Time Action Chunking). The robot works through the chunk it already holds while the next one is computed. A round trip of tens of milliseconds does not stall an arm busy executing a second of actions. This is also why latency claims disagree. One number counts a whole chunk. Another counts one action step. A network hop may or may not sit inside the measurement. Ask which unit each number counts.

Chunking alone has one failure mode. The new chunk may disagree with the motion underway. Smoothing that join by hand can produce unsafe accelerations. The fix is Real-Time Chunking, from the same publisher. It generates the next chunk while the current one runs. It also guides the new chunk's start to line up with the old one's tail, like filling in a gap. No retraining is needed. It works on any flow or diffusion based policy. In their tests it held precise dynamic tasks, striking a match and plugging in a network cable, with injected delays above 300 ms. The LeRobot project ships it for Pi0, Pi0.5, and SmolVLA (LeRobot docs). The pi05 page walks it in full.

What running on the robot actually costs

The entry price is real and public. NVIDIA's developer price table, read October 5 2026, lists the AGX Thor Developer Kit at 5,499, following a July 2026 reprice from the 3,499 launch price. The same table lists the AGX Orin developer kit at 3,499, the Orin NX 16 GB module at 999, and the Orin Nano Super developer kit at 399 (NVIDIA developer FAQ). What the money buys is placement freedom: no network, no uplink, no remote failure domain.

Three costs sit behind the sticker.

Memory is the gate. The official LeRobot hardware guide groups the policy families by training need. The ACT class takes roughly 2 to 6 GB. SmolVLA takes 10 to 16 GB. The pi0 and pi0.5 family, X-VLA included, takes 24 to 40 GB, and a 24 GB part is already tight at a batch size of one (LeRobot hardware guide). Thor's 128 GB kit clears that with room to spare. Smaller Orin parts do not. That is why the study's older AGX part produced chunks at 440 ms. The model fit, but slowly.

Power is the tax nobody prices. The study measured it directly. Their robot, a Stretch 3, carries about 216 Wh of battery and burns 30 W on sensors and motors alone. A roughly 6 W transmit-only board extended runtime by several hours. Larger onboard parts drained the battery by up to 160 percent on the same platform. On a mobile robot, every onboard watt cuts working hours.

Heat, and its inverse, shrink, are the engineering floor. A two billion parameter reasoning model that officially wants Thor class hardware was rebuilt in compressed W4A16 form by a tool vendor. It now runs on the whole Jetson line, down to the 8 GB Orin Nano (model release). Compression buys placement and costs accuracy. The reflex and skill layers tolerate it far worse than the reasoning layer does.

What the network actually does

Small added delays cost real accuracy. In the study's network experiment, the handover task held 80 percent on a desk part with an ideal link. Adding one way latency, 10 ms mean with 15 ms of spread, dropped it to 70 percent. Swapping in the data center part brought it back to 80. Ten milliseconds of wander cost ten points. More compute bought it back.

Raw camera uplink is a wall. Lossless video from three cameras at 640 by 480 and 30 frames per second needs about 100 Mbps per robot. The study's compressed settings bring that to 12 Mbps at high quality, 6.5 at good quality. The compression itself costs task accuracy. A site with many robots on one access point hits this wall before any latency budget.

Jitter beats latency as the enemy. A fixed delay can be planned around. The robot knows how far the world moves in that window. A delay that wanders cannot be. Buffering to the worst case makes every action pay the worst case price. A cable link or a quiet wireless one behaves. A busy one does not.

And the speed of light still votes. The strongest public proof that cloud inference works at real rates is a Modal case study of Physical Intelligence, the pi publisher, running real time remote inference on Modal's platform. They built a persistent UDP channel rather than per request connections. The whole cloud added roughly 10 to 15 ms. Deployments sit in regions close to the robots, because no protocol fixes distance (Modal case study, April 2026).

Move the model, or make it faster

Before relocating, check whether the model itself is the problem. Serving software is half the latency story.

A public runtime project measured three policies on the same Thor kit, per action chunk, against the same checkpoints (vla-edge benchmarks). One pi0.5 class policy went from 230.6 ms in plain PyTorch to 62.2 ms optimized. A MolmoAct2 class policy went from 611.3 ms to 113.4 ms. The smallest model went from 90.0 ms to 32.4 ms. The levers were CUDA graphs, TensorRT engines, and custom kernels. None of it required distillation or pruning. The models stay the models you trained. The MolmoAct2 stack uses FP8 vision with a BF16 backbone, so this is not always pure precision matching.

Set that beside the study's numbers and the rule falls out. The 440 ms Orin AGX result and the 611 ms eager result are software facts before they are placement facts. A loop missing its budget by two or three times calls for optimizing first. A model that misses by twenty times, or does not fit in memory at all, no kernel saves. The model must move.

The VLA-Perf work adds one more lever that costs nothing. Denoising step count has a strong effect on latency. Chunk length has almost none. A longer chunk buys a longer execution runway per call, nearly free in time. Tune the chunk before you buy hardware.

The field has already split the brain

Figure's Helix runs two systems onboard (Figure). One is a 7 billion parameter vision language model at 7 to 9 Hz for scene understanding. The other is an 80 million parameter visuomotor policy at 200 Hz for control. A single learned vector joins them. Google ships Gemini Robotics in tiers, one an On-Device variant built to run locally (Gemini Robotics). Neither company chose one placement. Both chose per layer.

Research has now built the same split across the network boundary. CloudEdgeVLA keeps the large model remote and a small head local, trained to tolerate delay on purpose. Across four LIBERO suites it retained 63.8 to 78.0 percent task success under a 40 step network delay. The compared single path approaches fell below 7 percent (paper). The remote side encodes slowly changing task features. The local side grounds them in what the cameras see now. Neither blocks on the other.

Decision rules

Two rows carry a caveat. The privacy row is absolute. No latency argument overrides it. The 100 Hz row is where the remote side genuinely struggles. The placement model found 10 Hz feasible over a good network. Reaching 100 Hz generally requires asynchronous execution.

What this means for a hosted route

This platform has a published action route contract for these models, and a vision route for perception. Both take the shapes described here. Camera frames, an instruction, and the robot state as numbers go up. An action chunk plus the normalization block for your embodiment comes back. The contract is published at the run-action reference. The action lane's page describes the model set: the pi0.5 family with its embodiment variants, SmolVLA, the OpenVLA fine-tune, ACT, the compact X-VLA policy, and the arm-native flagship.

The honest state: the action routes are not on the public list. The perception rows alongside them, sam2.1-base and da3-small, serve on the vision route today. What a per call shape changes is not the physics above. The physics holds regardless. It changes the buying shape of the middle layer. The skill loop becomes a metered call instead of a purchased part. A prepaid wallet is the stop. One served request is one billed call. A request refused before it is served bills nothing. The pricing guide carries the listed routes' rates. This guide carries none, because numbers drift and pages should not.

For the layers this page has placed away from the robot, there is a practical path today. The SO-101 cloud brain guide walks it: a policy server wherever the memory is, a small client next to the arm, and chunks bridging the two. The normalization numbers that make a policy yours travel with it, and the normalization guide covers that failure class. On a hosted route, the client changes one line, the address.

Frequently asked questions

How fast can I run my VLA?

Faster than the frame rate argument implies, because of the chunk. The field's placement model counts 10 Hz as acceptable and 100 Hz as high performance. On optimized serving software, a pi0.5 class policy answers a chunk in 62 ms on the Thor kit. A high end desktop part does it in 22 to 39 ms. The latency table with sources carries the full set.

Can a robot run its VLA in the cloud?

Yes, with two conditions. The task must tolerate a chunk paced loop rather than a per frame one. The link must also be quiet. Real-Time Chunking held precise tasks with injected delays above 300 ms. A hosted platform carried a full control loop at 10 to 15 ms of added overhead for the publisher of the pi models. Contact rich and reactive work belongs onboard regardless.

What latency is too much for robot control?

There is no single number, because there is no single loop. Reflex and safety loops need under 10 ms and stay local. The skill layer tolerates 30 to 200 ms, and chunk execution stretches the budget into the hundreds. Reasoning tolerates seconds. If someone quotes one millisecond figure for a robot, ask which loop.

Jetson Thor or a desk class box like the DGX Spark?

Different jobs. The Thor kit is a robot part: 40 to 130 W, 128 GB, built to ride the machine, at the developer kit price named above. The Spark is a site part. Same memory class, wall powered, meant to sit still and serve robots on the local network. If it moves with the robot, Thor. If it serves many robots in one building, the desk shape wins on cost per robot.

How much does Jetson Thor cost?

NVIDIA's own developer price table, read October 5 2026, lists the AGX Thor Developer Kit at 5,499. That follows a July 2026 reprice from 3,499 at launch. Modules below it run from 4,999 down to 399 for the Orin Nano Super developer kit. Check the developer FAQ before buying, and the break-even guide before turning a price into a decision.

Does edge vs cloud matter for robot perception too?

Yes, and the tolerances are looser. Segmentation and depth feed the policy rather than driving motors directly, so tens of milliseconds of hop matter less. The same bandwidth wall applies: three raw cameras are about 100 Mbps. The robot vision guide on this site covers the perception side, and the platform's vision route follows the same placement logic.

Where do I start if my robot has no compute today?

Start with the loop, not the hardware. Record and train as the LeRobot workflow teaches. Serve the policy on whatever machine has the memory, and keep the client small. Sign up, create a key, and the client you already have changes one line.

The models named across this guide are described on their own pages: pi05 and its variants, smolvla, openvla-oft, act, molmoact2-so100-101, and the perception pair sam2.1-base and da3-small. The definitional guide to the model class, vision language action models explained, publishes with this wave. The models page lists everything the platform offers today.

Last verified: 2026-10-05