The SO-101 with its brain in the cloud

The SO-101 arm ships with a loop everyone teaches in four steps: record demonstrations, build a dataset, train a policy, then run it. The teaching stops right where the hard part starts. The trained policy has to run somewhere, and that somewhere decides what computer you need. This guide walks the whole loop and names where the brain sits at each step. It then shows the two ways to serve a policy when your own computer is not enough. One is a policy server you point at rented hardware. The other is a hosted action route. The hosted route on this platform is not on the public list. The loop, the numbers, and the model choices are real today.

The setup half of this topic is well covered and this guide will not repeat it. The official SO-101 assembly guide walks the motors, the ports, and the calibration. Vendor wikis walk the kits. None of them cover the last leg: where the policy lives once it is trained. That leg is where the money goes, and where most first projects stall.

The loop, and where the brain sits at each step

The loop has five steps. The brain moves through it.

step 1   record     you teleoperate: leader arm in your hand, follower mirrors it,
                     both cameras rolling
                     brain: any laptop that runs LeRobot

step 2   dataset    episodes land in the LeRobot format, on disk or on the Hub
                     brain: none. it is data

step 3   train      a policy is learned from the episodes
                     brain: a memory-class GPU. yours, rented by the hour,
                     or a managed training job

step 4   serve      the trained policy answers requests: images and state in,
                     an action chunk out
                     brain: wherever it fits. a GPU box on your desk, a rented
                     box, or a hosted route

step 5   rollout    the arm runs each chunk while the next chunk is computed
                     brain: same place as serve. the arm itself only needs
                     the small client program

The pattern to see: steps 1, 2, and 5 are cheap. Any laptop records, any disk stores, and the arm itself never needs to think. The weight sits in steps 3 and 4, because of memory, not speed. A training run and a serving call both want the whole model in memory at once.

The hosted option on this platform belongs to step 4. It is an action route for vision language action models, and its category page describes the lane. None of its models are on the public list. Everything else in this guide you can do today, with open tools.

Why the brain leaves the robot

Start with the official numbers, because they decide everything else. The LeRobot compute guide groups the policies by memory need. The ACT policy trains in roughly 2 to 6 GB. SmolVLA wants 10 to 16 GB. The pi0 and pi0.5 family, X-VLA included, wants 24 to 40 GB. The guide notes a 24 GB card is already tight for those at a batch size of 1, and points at the A100 40 GB class instead.

Now set that against what people own. A community thread from a SO-101 owner in early 2026 describes the ceiling plainly: pi0.5 on a 16 GB MacBook Pro took about ten minutes to start loading, filled memory, and forced a restart before a single inference, even with the cameras dropped to 15 frames per second. A reply from someone running humanoid hardware puts the working setup at roughly 70 ms of inference at about 15 calls per second, on 5090-class hardware. His closing line to the laptop owner was blunt: you will not get good performance at all with just that MacBook Pro.

The same guide closes the last workaround: on CPU only, do not train. Use a free notebook service or rent a GPU. The training anchors make the point in wall-clock terms: ACT finishes a small dataset in 30 to 60 minutes on a 24 GB consumer card, SmolVLA takes 3 to 6 hours on the cloud 24 GB class, and the pi0.5 family takes 4 to 8 hours on an A100 40 GB. An Apple M-series Max can train ACT, slowly, in 6 to 14 hours. The guide carries no Apple path for the bigger families.

The community reached the same conclusion on its own. One open project, built around driving an SO-101 from a laptop, states its reason in one line: today's small robots do not carry enough hardware to reason, so the brain has to live outside the robot, in the cloud or on your own computer (quackd). The arm keeps its servos and cameras. The thinking moves. And a desk GPU serving a robot all day is a desk GPU you cannot use for anything else.

The serve leg: a policy server and a robot client

Here is the fact that makes the cloud brain ordinary rather than exotic: LeRobot already splits the policy from the robot. The async inference design runs two programs. A policy server loads the checkpoint and predicts action chunks. A robot client drives the servos and the cameras. The two talk over the network, and the docs say the server can run on a separate machine. This design shipped with SmolVLA and now serves every policy in the supported list.

Two terminals make it concrete.

# terminal 1, on a machine with a GPU (yours, or rented by the hour)
python -m lerobot.async_inference.policy_server \
    --host=0.0.0.0 \
    --port=8080

# terminal 2, on the laptop next to the arm
python -m lerobot.async_inference.robot_client \
    --server_address=<gpu-host>:8080 \
    --robot.type=so101_follower \
    --robot.port=/dev/ttyACM0 \
    --robot.id=my_follower \
    --robot.cameras='{"front": {"type": "opencv", "index_or_path": 0, "width": 640, "height": 480, "fps": 30}}' \
    --policy_type=act \
    --pretrained_name_or_path=<your-org>/<your-policy> \
    --policy_device=cuda

Three things about this split are worth holding onto.

First, the chunk is the load-bearing idea. One request does not return one action. It returns a chunk, a run of maybe dozens of joint targets. The arm works through the chunk while the server computes the next one. That is what hides the network: a round trip costing tens of milliseconds does not stall an arm stepping at 30 frames per second, because the arm is executing actions it already holds. The tuning knobs are the chunk length and the threshold at which the client asks early. Without the chunk, the arm pauses on every inference. One practitioner writing about ACT on the SO-101 measured a forward pass at about 40 ms and documented the stalls that come from serving without it.

Second, the reflex layer never leaves the arm. The servo bus, the camera capture, the safety stops, and the calibration live on the client side, next to the hardware. You are not streaming raw motor control over the internet. You send images and state up, and receive targets back. The latency budget that matters is the one your task tolerates, and picking tasks is part of the engineering.

Third, the device flag is where the placement decision lives. The client takes --policy_device=cuda or mps or cpu, and the flag says where the thinking happens. Point the arrangement at a rented box and you have a cloud brain today, with open tools, whatever any hosted route does.

One honest limit, from the community record: the async path supports a fixed policy list, and newer policies have been left off it, which pushed at least one builder into writing his own runtime. Ask what a path supports before you commit a week to it.

What a hosted action route does

The server-and-client split moves the brain off your desk. A hosted action route moves it off your infrastructure entirely. Instead of minding a GPU box, your client program calls an endpoint. The request carries what the local server would read: one or more camera frames, the instruction, and the arm state as numbers. The answer is an action chunk plus the normalization block, which says how the actions are scaled for your embodiment. You meter by the call, against a prepaid wallet, and a wallet at zero is refused at request time. A robot that loops all day has a hard stop, not an open bill.

This platform publishes exactly that shape for the robot lanes, and the field-by-field contract is already written down at the run-action reference. The honest state: the lane is not on the public list, and nothing here is serving. What you can do today is read the contract and shape your client around it. The call looks like this.

import json, time, requests

BASE = "https://your-endpoint.example/v1/action"   # the lane's address
KEY  = "your-key"

def step(frame_jpg, state):
    r = requests.post(BASE, headers={"Authorization": "Bearer " + KEY},
        files={"image": ("front.jpg", frame_jpg, "image/jpeg")},
        data={"model": "pi05",
              "instruction": "pick up the green cube",
              "state": json.dumps(state)})
    r.raise_for_status()
    body = r.json()
    return body["actions"], body["normalization"]

while True:
    frame, state = read_cameras_and_joints()
    actions, norm = step(frame, state)
    for a in actions:                # work the chunk
        command_joints(a)
        time.sleep(1.0 / 30)         # 30 frames per second

The loop is the whole idea. One call covers a stretch of time. The metering unit matches what your robot actually does. The numbers stay out of this guide because numbers drift and pages should not; rates live on the pricing page. Sign up and create a key now: the client above then needs one line changed, the address.

For the choice between a rented box and a hosted route, this site carries a guide on the break-even math for local versus hosted VLA serving, and another on where a robot's inference should live, edge or cloud. Both publish alongside this one.

Models for this arm

The SO-101 is the reference arm for this ecosystem, so the open model families meet you there. Five starting points cover most projects.

Two of these have pages here that publish alongside this guide: pi05 and molmoact2-so100-101. The rest are named in the lane's category page.

The memory classes in that table are the compute guide's own envelopes, not our measurements, for every row except one. The guide names no memory row for MolmoAct2, so that cell is our reading, placed in the 24 to 40 GB class by checkpoint size and family pattern. The one number we would add ourselves, the measured response time of each lane, does not exist, and we will not guess at it.

If you arrived from the NVIDIA course and your plan says GR00T, read the license file before you build: the weights do not permit commercial use, and the fine-tune inherits that limit. The license guide on this site quotes the file and lists the open alternatives.

Record and train without your own GPU

The dataset law comes from the official imitation learning tutorial, and it is short: at least 50 episodes, 10 per location, cameras fixed, grasping behavior consistent. Two camera views, taped down, with fixed exposure, beat one nice camera every time. The same tutorial carries the law people skip: the robot's id names its calibration file, and you use the same id for teleop, record, and eval, because calibration is what lets a policy trained on one arm run on another.

One community write-up, dated September 2025 and widely read, walks the failure that law exists to prevent. Its 84 upvotes on the Hub measure how many SO-101 builders recognized their own first project in that story, which is why the fixes below read like a checklist. A first policy failed to grasp at all. The causes were found one by one. The cameras had moved between training and testing. A calibration file lost in a power cycle was redone hastily, so learned joint angles mapped to the wrong servo commands. Fifty episodes gave a 60 percent success rate in distribution and 10 percent out of it. Denser coverage with varied orientations reached 90 percent in and 75 percent out. The lessons are boring and true: tape the cameras, guard the calibration files, collect coverage, not repetitions.

For training without a GPU of your own, the official where-to-run tiers give three honest options. Rent a cloud card by the hour in the 24 GB class, the tier the guide names as common on the big rental providers. Use a managed training job, such as the ones Hugging Face bills by the minute. Or pick a policy small enough for what you have: ACT trains on a laptop-class GPU, and on Apple silicon it finishes overnight. A fine-tune service for this platform's lanes is not offered today; the three options above are the working set.

The calibration theme has one more layer, and it matters exactly when the brain moves. Norm stats, the normalization numbers that map your arm's joint ranges onto the model's expected inputs, travel with the checkpoint. The MolmoAct2 SO-100/101 checkpoint ships them in a file and names a tag you pass at inference (model card). When that model was first enabled on this arm family it needed a correction, because its training data used a different joint calibration convention than the current library expects (LeRobot change #3879). The lesson generalizes: when you move a policy between machines, libraries, or hosts, the numbers that make it yours are the first thing to check. This site carries a full guide on normalization stats for this failure class, publishing alongside this one.

Common questions

How much does the LeRobot arm cost? It depends on the route you take, and the honest answer is a list, not a number. Print and source the parts yourself, buy a parts kit, or buy arms fully assembled: the official hardware repository links every known vendor. The software is free and open source throughout.

How do I calibrate the SO-101? One command per arm, moving each joint through its range. The official guide walks it with videos. A change now open in the library adds an auto mode that finds each joint's end stops without hand movement, which answers a complaint that shows up across the community threads.

What are the key differences between the SO-100 and SO-101? Three: the printed parts assemble more easily, per the hardware repository; the cables route outside the arm; and the leader arm uses motors with friendlier gear ratios. Software written for one runs on the other.

How does LeRobot work? It is an open source library with one command per loop step: teleoperate, record, train, evaluate, and run the policy. It defines a shared dataset format and treats policies as plug-in values. The documentation is the entry point.

Can I run my SO-101 policy without my own GPU? Yes, three ways. Serve it from a rented GPU with the policy server and robot client split shown above. Train a small policy on what you own and serve it the same way. Or call a hosted action route, the published-contract shape on this platform, not on the public list. CPU-only inference of a small policy is possible and slow, and the official guidance is plain that CPU is not for training.

Do I need a Jetson-class computer for this arm? Not for the loop itself. Recording runs on any laptop. Serving needs the memory class your policy belongs to, wherever that memory lives: your desk, a rented box, or a hosted route. The site's guide on Jetson versus cloud inference, publishing alongside this one, works the latency and placement numbers in full.

Is GR00T the right model for a SO-101? It runs on this arm and the vendor course teaches it, but the weights carry a research-only license that passes into your fine-tune. Read the license guide on this site before you spend a month on it. The open alternatives in the table above run on the same hardware.

What does SO-101 follower arm mean? The arm is sold or built as a pair. The leader arm has a grip you move by hand. The follower arm mirrors it during recording, then runs the trained policy on its own. Kits ship as a pair for this reason.

The short version

The loop is record, dataset, train, serve, rollout. The brain sits on your laptop for recording, on a memory-class GPU for training and serving, and nowhere at all for the data. When your computer cannot hold the model, the architecture already splits: a policy server wherever the memory is, a small client next to the arm, chunks bridging the two. A hosted action route is the same split without the box, metered by the call, and this platform's lane is not on the public list. Read the run-action reference for the contract, and sign up so a key is ready.

Last verified: 2026-10-05