WebSocket stream dropped mid-inference: what the robot loop does next

When a robot policy stream drops mid-inference, the call in flight is already dead. No reply is coming on that socket, and the observation it carried is spent. The loop must treat the next missing action as a deadline expiry. It must hold or halt the arm on the last valid state. Only then may it reconnect, with a fresh handshake, before sending another observation. It must not replay the buffered action chunk after the gap. That rule, plus the exact strings each stack prints and the cause behind each, is this page.

Last verified: 2026-10-27. Every quoted string below was read from its source on that date, byte for byte. The sources are the openpi and lerobot repositories, a ROS 2 policy runtime, and the RFC that defines the close codes. Sources are linked where each string appears.

Why the robot case is not the chat case

Most WebSocket advice is written for chat. There, a dropped stream costs a retry and a few seconds of waiting. On a robot control loop, the same event has a clock attached. The clock is not yours to extend.

A loop that runs at 50 Hz owes the arm a fresh action every 20 milliseconds. At 30 Hz the budget is 33 milliseconds. At 10 Hz it is 100. A February 2026 study from NVIDIA Research, VLA-Perf, defines the bars. Camera feeds run at 24 to 60 Hz. Policy inference at 10 Hz counts as acceptable. Inference at 100 Hz counts as high performance. The layers under the policy are faster still. Whole-body controllers run at 500 Hz to 1 kHz. Joint servo loops run far above that. Those layers do not pause because a socket died. They keep waiting for commands.

That is the difference. A chat client can spend seconds reconnecting. A robot loop must decide inside one control period. The decision is physical. The arm is in motion, and it holds actions computed for a moment that has already passed.

The exact strings, and what each one is telling you

Read the string before you read any advice. Each stack prints something different, and each string points at a different cause.

The three causes below carry the fix and the prevention for each class.

Cause one: the server raised, and closed with its traceback

The openpi policy server wraps every inference call in one handler. When the policy raises, the server sends the full Python traceback as a text message, then closes the socket. The close code is 1011. The reason string is exact: Internal server error. Traceback included in previous frame. The client receives that text frame and raises a RuntimeError that starts with Error in inference server:.

The server side, verbatim:

except Exception:
    await websocket.send(traceback.format_exc())
    await websocket.close(
        code=websockets.frames.CloseCode.INTERNAL_ERROR,
        reason="Internal server error. Traceback included in previous frame.",
    )

Cause. The policy crashed on the observation. Typical crashes: a CUDA fault, a tensor shape that changed with a camera swap, a missing normalization file. The close is the symptom. The traceback is the diagnosis, and it arrived before the close. A wrapper that only catches the connection error and ignores the preceding text frame throws the diagnosis away.

Fix. Read the traceback text that follows the client's Error in inference server: line. Fix the exception it names. Then reconnect. After this close, the handler for your connection has ended. The server process is still up, though, and it accepts new connections. A reconnect must re-read the metadata frame the server sends first.

Prevention. Validate observation shapes and normalization inputs before an episode, not during it. Check the /healthz endpoint, which the openpi server answers with OK, before you start. If one observation raises once, the next one will raise the same way. Fix the model input, not the reconnect.

Cause two: the socket vanished with no close frame

RFC 6455 reserves the code 1006 for exactly this. In the RFC's words, the connection "was closed abnormally, e.g., without sending or receiving a Close control frame." No traceback will arrive. Nothing on the policy server will log an inference failure, because none happened. The openpi client logs little either; the server side simply notes Connection from {websocket.remote_address} closed.

Cause. A proxy, load balancer, VPN, or NAT between robot and server timed the connection out as idle. A Wi-Fi handover. A server restart or a container move. Note what it is not. Both ends of the openpi protocol disable compression and message size limits (compression=None, max_size=None). So this is not a size or extension failure.

Fix. Reconnect through the standard client loop. It already exists. The openpi client retries the connection every 5 seconds and logs Still waiting for server... until the server answers. Confirm /healthz answers. Then resume from a fresh observation, not from the buffer. Any chunk held when the socket died was computed for an earlier instant.

Prevention. Enable WebSocket ping and keepalive. Set the interval well below the idle timeout of every hop between robot and server. Count proxies and VPNs as hops. If the policy runs across a network you do not control, watch the age of the last observation, not the state of the socket. The age is what the loop consumes. One more classic: a blocking camera grab starves the ping cycle. It shows up as a 1006 drop with no network fault at all. Keep capture off the socket's thread or loop.

Cause three: the request outlived its deadline

The lerobot remote inference client does not wait forever. When no action lands inside the configured action deadline, it raises TimeoutError with the string Remote inference request deadline exceeded. When the server's liveness signal drops, it raises ConnectionError with Server presence lost; session cannot recover. A third string, Expected one control reply within its deadline, marks the session control channel.

Cause. The policy was slow, not the network. The usual culprits: a cold GPU on the first calls, model warmup, contention from another job, or a deadline tighter than the model's real latency. A ROS 2 runtime surfaces the same class as a termination reason, policy_inference_timeout, and aborts the goal with the robot held.

Fix. Measure before you tune. The openpi server already puts a timing number on every action message it sends: the server_timing field carries infer_ms per call. Your logs hold the evidence. Compare the value with your deadline and with your control period. Warm the model before the episode. Raise the deadline only as far as your loop rate allows. At 30 Hz, a 100 millisecond budget is already three control periods. Only chunking bridges that gap.

Prevention. Size action chunks to ride out your worst measured inference time. The pi 0 paper describes 50-action chunks; NVIDIA's Isaac GR00T N1.7 predicts a horizon of 40 and typically executes 8 per policy call. The real-time chunking paper from Physical Intelligence, June 2025, states the pattern best. It "generates the next action chunk while executing the current one." With a chunk in hand, one slow or lost request no longer stalls the loop inside the horizon.

The next 33 milliseconds: hold, halt, or replay

The decision rule, in order.

One. Treat the missing action as a deadline expiry, not as a chat retry. The observation in flight is spent. If the server finishes the work anyway, that output is late, and late output must not command the arm.

Two. Hold on the last valid state. Do not replay the buffered chunk. A replayed chunk applies actions computed for a past instant. The real-time chunking paper names the failure shape: "pauses or out-of-distribution jerky movements at chunk boundaries" when execution outruns inference. The lerobot client states its own rule in its source: it will "never reconnect or replay work." The openpi action chunk broker clears its buffer on reset. A fresh connection starts from a fresh inference, by design.

Three. Stop motion when a hold cannot be made safe. A ROS 2 policy runtime built for this failure set, PolicyBridge-ROS2, shows the full rule. It terminates the goal with a named reason, policy_inference_timeout or stale_observation. It holds the latest valid joint state. When no valid state exists, it reports safe_stop_unavailable_no_valid_state and "never fabricates a zero target." That last clause is the law: do not invent a neutral-looking zero command and call it a stop.

Its author's caveat, verbatim, belongs on any page that teaches holds:

A hold is a software-level absolute-position target. It is not a braking controller, physical-stop acknowledgment, hardware interlock, or safety-certified emergency stop.

Four. Only now reconnect, and re-admit before you resume.

This is the part the generic WebSocket guides cannot say. On a robot, the retry policy is a motion decision, and a software hold is not a safety-rated stop.

What a reconnect owes you

A reconnect is not a socket reopen. Both stacks re-earn the session.

On connect, the openpi server's first message is a metadata frame, and the client reads it before anything else. A reconnect must re-read it. The model on the other end may have changed between your drop and your return.

The lerobot client goes further. Admission checks feature names and order, semantics, the action interval, and the execution mode. The error text names the mismatch, for example Observation feature names/order differ. A session is exclusive, and a new session inherits nothing from the old one. If your own wrapper skips the checks and resumes the old buffer, the robot moves on stale actions. No error appears. That is the worst outcome on this page.

So the reconnect contract is: re-read metadata, re-verify the observation contract, reset the chunk buffer, resume from a fresh observation.

Prevention: the settings that stop most drops

• Ping and keepalive on, at an interval under the idle timeout of every hop, proxies included.
• Compression and message size limits pinned at both ends. The openpi protocol sets compression=None and max_size=None; a proxy that re-enables permessage-deflate on large camera frames is a classic stall.
• A liveness check before each episode. The openpi server answers /healthz with OK.
• Chunked execution sized from your worst measured inference time, not your average.
• Deadlines set from the infer_ms numbers already arriving on your action messages.
• Camera capture that never blocks the socket's event loop.
• A defined hold profile for the reconnect window, tested with the socket pulled, so the first real drop is not the first test.

When one call per action replaces the socket

This platform's robot action route takes one POST call per action step. Camera frames, instruction, and robot state go in. An action chunk comes back. The call ends. A loop built that way holds no long-lived socket. The failure class on this page shrinks to one failed call, which the loop can retry or bridge from a chunk. The pi 0.5 model page describes the family. The robot models were not on the public list as of October 27 2026. The models index shows the current catalog.

Common questions

What does close code 1006 mean during policy inference?

The socket ended without a close frame: a network cut, a proxy idle timeout, a handover. RFC 6455 reserves 1006 for exactly that and forbids sending it in a Close frame. So it can never carry a reason string. No policy error occurred. Fix the path between robot and server, then reconnect and resume from a fresh observation.

Do I have to restart the episode when the stream drops?

Not automatically. Hold the arm on the last valid state. Reconnect, re-read the metadata or admission checks, and resume from a fresh observation. Restart the episode when the hold was long enough that the task state drifted. Also restart when the runtime aborted with a named reason such as policy_inference_timeout.

Why did the loop stop when nothing crashed on the robot?

Because the deadline fired. Clients like the lerobot one raise Remote inference request deadline exceeded when the action does not arrive in time, even with a healthy network. Check the infer_ms timing on your action messages against the deadline and against your control period.

Is holding the last action safe?

A software hold holds a position target. It is not a braking controller, a hardware interlock, or a safety-rated emergency stop. PolicyBridge-ROS2 states that caveat in exactly those words. Use the hold to bridge a short reconnect window. Anything longer needs your real stop path.

Does GR00T use WebSocket for its policy service?

No. Read on October 27 2026, the Isaac GR00T PolicyServer binds a ZeroMQ REP socket, not a WebSocket. The failure class maps across. A REP request with no reply is the same deadline problem, and the diagnosis order is the same. Read the server error first, then the network.

How do I stop the drops from happening?

Keepalive under every hop's idle timeout. Compression and size limits pinned at both ends. A liveness check before each episode. Chunk sizes set from your worst measured inference time. Capture that never blocks the socket's event loop. The full list with the reasons is in the prevention section above.

Where this leaves you

Read the string first: it names the cause. Then apply the matching fix, and put the prevention settings in before the next episode. If you run the policy yourself, three guides help. The policy server explainer covers the serving pattern. The inference frequency guide explains what a control rate buys you. The ROS 2 latency guide takes the slow-loop case. The models index shows what the catalog holds today. For anything cost related, the pricing page is the one place to look, and signup is the way in.

Last verified: 2026-10-27