OpenVLA and OpenVLA-OFT CUDA out of memory: the exact strings, causes, and fixes

OpenVLA and OpenVLA-OFT CUDA out-of-memory crashes sort into five documented situations. The error message itself tells you which one you have. Two numbers inside the line separate a real shortage from fragmentation and from another process holding the card. This page quotes the exact strings from the publishers' own issue trackers, names the cause behind each, and gives the fix and the prevention. The official memory tables set the floor: 25.7 GB per GPU to fine-tune on LIBERO at batch size 1, and 15.9 to 16.2 GB to run inference. Every source was read on 27 October 2026.

Last verified: 2026-10-27

The crashes at a glance

Match your screen against this table first. Every string is copied from the publishers' repositories and issue trackers. Each fix section carries its source and date.

One number pair decides most of it. The message reports how much memory the failing process itself allocated. It also reports how much every other process on the card holds. Zero from your process means your process is the victim, not the cause. A near-full figure from your process means the run does not fit. Read those two numbers before you change anything.

Read the message first: the numbers are the diagnosis

The PyTorch allocator prints one long line, and every part of it is evidence. Here is the checkpoint-save crash from a real five-card RTX 4090 fine-tune. It is quoted exactly as the reporter pasted it into openvla-oft issue 28, opened 28 March 2025:

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 252.00 MiB. GPU 0 has a total capacity of 23.65 GiB of which 62.19 MiB is free. Including non-PyTorch memory, this process has 384.00 MiB memory in use. Process 2343934 has 636.00 MiB memory in use. Process 2343932 has 384.00 MiB memory in use. Process 2343931 has 384.00 MiB memory in use. Process 2343929 has 21.44 GiB memory in use. Process 2343933 has 384.00 MiB memory in use. Of the allocated memory 0 bytes is allocated by PyTorch, and 0 bytes is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)

Three facts sit in that line. The card is a 24 GB class card. The failing process holds 384 MiB and has allocated nothing through PyTorch: 0 bytes is allocated by PyTorch. Another process on the same card, the training rank, holds 21.44 GiB. So the 252 MiB allocation failed because the training run had already filled the card. The save step died on contact. The crash is real, but the run that needed room was never the one that crashed.

Contrast the inference crash from openvla-oft issue 6, opened 5 March 2025, on a single RTX 4090 serving a fine-tuned model. The same error class, the opposite story. The message says Tried to allocate 20.00 MiB on a total capacity of 23.55 GiB. And this time Of the allocated memory 23.01 GiB is allocated by PyTorch. The dying process is the holder. That is a genuine does-not-fit case. The fix had to make the process itself smaller, and it did.

The fragmentation sentence at the end of the message is advice, not a diagnosis. It only applies when the reserved-but-unallocated figure is large. In both strings above it is near zero, which says fragmentation was not the problem. The PyTorch memory documentation describes the setting and its limits. Cause 4 below covers when it genuinely helps.

Cause 1: the checkpoint-save trap, the most common 24 GB crash

The string: torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 252.00 MiB. fired inside save_training_checkpoint, at the line merged_vla = PeftModel.from_pretrained(base_vla, adapter_dir).

The cause: the fine-tuning script's save path. By default, merge_lora_during_training is True. At every save, the script loads a fresh copy of the base 7B model, attaches the LoRA adapter, merges, and saves. Read it in the current finetune.py source, read 27 October 2026. The merge block loads AutoModelForVision2Seq.from_pretrained with no device override. It lands on the same card that holds your training run.

A 7B model in bfloat16 is about 15 GB of weights before the merge machinery. That sits on top of a training run already holding 20 GB or more. On 24 GB cards, training can run for ten thousand steps and then die at the save. That exact pattern is issue 28, five RTX 4090s, crash at save. The thread under issue 20 asks the same question from an eight-card A5000 cluster, and a commenter's 3090 died at the checkpoint save.

The fix, from the maintainer's own reply in issue 28: skip the in-training merge and merge offline instead.

torchrun --standalone --nnodes 1 --nproc-per-node X vla-scripts/finetune.py \
  ... \
  --merge_lora_during_training False

Then merge after training, on any machine with enough room. The script's usage line is printed in its own docstring:

python vla-scripts/merge_lora_weights_and_save.py \
    --base_checkpoint openvla/openvla-7b \
    --lora_finetuned_checkpoint_dir /PATH/TO/CHECKPOINT/DIR/

The same thread records a second route. A user edited the save path to load the base model with device_map="cpu". The merge then happens in host memory, and it worked for them. It is a local edit you must reapply after every pull, so the flag plus offline merge is the durable answer. That offline script has one more use: the LIBERO guide recommends it when you want to merge on the same kind of card you will test on.

The prevention: set --merge_lora_during_training False in every fine-tune command on small cards. Keep --save_freq high enough that saves are rare. Treat the LoRA adapter directory as the real checkpoint. The merge is a packaging step. Do it once, offline, not every ten thousand steps on a full card.

Cause 2: inference out of memory on a 24 GB card

The string: torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB. under result = result + lora_B(lora_A(dropout(x))) * scaling, during the first action call of a served model.

The cause, in the publisher's own words: gradient tracking was enabled during inference. In issue 6, the reporter served an ALOHA fine-tune on an RTX 4090. The setup used three cameras, proprioception, and FiLM. It crashed at the first call. The maintainer traced it and pushed a commit. That commit wraps the action generation in torch.inference_mode(). The receipt, printed after a task ran: Max GPU memory allocated so far: 18.033 GB. ALOHA-class inference fits a 24 GB card with the fix. The reporter confirmed within a day: about 18 GB, running clean.

The fix: update the repository. In the current code, the wrapper sits in get_vla_action() in openvla_utils.py, read 27 October 2026. If you are pinned to an old checkout, add the wrapper yourself around every get_vla_action call:

with torch.inference_mode():
    actions = get_vla_action(...)

The prevention: after any environment rebuild, print the allocator high-water mark once. Use the one-liner the maintainer posted in the same thread:

print(f"Max GPU memory allocated so far: {torch.cuda.max_memory_allocated() / 10**9 : .3f} GB")

Know your setup's published footprint before you blame the card. The project's FAQ prints measured inference numbers. LIBERO with one camera: 15.9 GB. Two cameras plus proprioception: 16.2 GB. The three-camera ALOHA setup with FiLM: 18.0 GB. A 16 GB card is under the floor for every setup. A 24 GB card fits all three, with room measured in single gigabytes. Close everything else before the first call.

Cause 3: the setup is bigger than the card

The string: any CUDA out of memory that lands during training steps, before the first save.

The cause: arithmetic, not a bug. Numbers come from the FAQ, measured by the project on its own runs with the default bfloat16. Minimum, batch size 1 per GPU: 25.6 GB for LIBERO with one camera, and 25.7 GB with two cameras and proprioception. ALOHA with three cameras and FiLM needs 38.6 GB. Recommended: 44.1, 62.5, and 73.5 GB respectively, at batch sizes 8, 8, and 4 per GPU. The LIBERO fine-tuning guide repeats the headline pair in its own words. Batch size 8 per GPU wants about 62 GB. Batch size 1 wants about 25 GB. The paper runs used eight A100 or H100 cards at 80 GB each. A 24 GB card is below the published minimum for the default recipe before anything else goes wrong.

The fix, in the order the project itself describes: lower the batch size and raise accumulation. The fine-tune script supports that directly.

--batch_size 1 --grad_accumulation_steps 8

Second, shrink the trainable side. The community thread under issue 20 records 24 GB A5000 and 3090 cards training LIBERO with --lora_rank 8 at batch size 1. One reporter reached results near the paper's on a single suite. That is a community lane, not a published number. It shows the shape of the trade: lower rank, lower batch, more steps.

Third, cut inputs. --num_images_in_input 1 with --use_proprio False is the smallest LIBERO setup, and it is the one the 25.6 GB minimum describes.

The prevention: size the run before launching it. Take the FAQ number for your setup. Add the checkpoint-save margin from cause 1. Compare against the card. If the minimum is above the card, no allocator setting will bridge it.

Cause 4: fragmentation, when the message's own advice applies

The string: CUDA out of memory with a large figure in reserved by PyTorch but unallocated, and gigabytes still free.

The cause: the caching allocator holds memory in blocks it cannot reuse for the next request shape. This is the one case where the suggestion printed inside the error message is the fix. The PyTorch memory notes describe expandable_segments as experimental, off by default, and built for jobs whose allocation sizes change. The same notes call max_split_size_mb a last resort, for workloads showing many inactive split blocks.

export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True

The fix: set it before the process starts. It is not a runtime toggle. Two honest limits come with it, from the same notes: allocator tuning can cost performance, and no allocator setting creates memory that is not there. In the two real OpenVLA strings quoted above, the reserved-but-unallocated figure is bytes, not gigabytes. Fragmentation was not the cause, and the setting would have changed nothing. The prevention: diagnose with torch.cuda.memory_summary() before you tune.

Cause 5: three lookalikes that are not CUDA out of memory

First, host RAM. A run killed with SIGKILL 9 during checkpoint writing, on cards with gigabytes to spare, is a system-memory death. Issue 118 reports exactly that on two A100s, and the reply names the cause: the dataset pipeline's memory grows as training reads. The base OpenVLA tracker documents the same behavior since June 2024. A maintainer's answer is specific. It is not a leak, and it plateaus. The dial to turn down is shuffle_buffer_size in the training configuration. A second maintainer answer adds the multiplier: the loader runs one dataloader per GPU, so memory grows with your GPU count. Issue 122 shows the GPU-side cousin. A four-GPU run carried three extra python processes on GPU 0, each holding hundreds of MiB.

Second, the sharding attempt. Trying to spread the model across several small cards with automatic device maps fails with Expected all tensors on the same device (cuda:0 vs cuda:1). That string comes from issue 311, on eight 11 GB cards. It is a different error with its own cause: parts of the model must meet on one device. Eight small cards are not one big card.

Third, the silent neighbor. Upgrading the transformers library past 4.50 without the matching fixes can make the vision tower stop contributing, with no error at all, constant actions instead. That is issue 346, still open. It is not an out-of-memory case, but it sits on the same upgrade path people take when fighting memory, and it fails quietly. Pin your versions. The LIBERO guide names the exact stack the paper used: Python 3.10.14, PyTorch 2.2.0, and the project's transformers fork at 4.40.1. The SETUP guide pins the environment around them, down to the Flash Attention 2.5.5 build.

The official memory tables, in one place

Dated 27 October 2026, from the project FAQ (last updated 22 February 2025), the OFT README, and the base OpenVLA README.

One conflict worth naming: a 2024 maintainer reply in the base tracker says inference needs 11 GB in bfloat16. The checkpoint's own weight files total 15.08 GB, so an 11 GB card cannot hold an unquantized load. The issue 311 reporter failed on 11 GB cards exactly there. Trust the file sizes and the measured FAQ numbers. The same reply's quantized figures, 6 GB at 4-bit, land within a couple of gigabytes of what the issue 311 thread later measured. Quantization is the honest path below 16 GB.

Many small cards are not one big card

The OFT recipe runs LoRA fine-tuning under Distributed Data Parallel, which the FAQ describes plainly: no model sharding, the batch split across cards. Adding a second 24 GB card gives you a bigger batch, not a 48 GB card. Each rank holds the whole model. The memory per card is what the tables above print. The base repo's full fine-tuning path is the one that shards, through FSDP. Its README frames it as an eight-GPU node affair. If your plan needs one 40 GB pool and you have two 20 GB cards, that plan needs a different recipe, not a device map. The failed sharding attempts land in cause 5, on the device-mismatch string.

Prevention checklist

• Put --merge_lora_during_training False in every fine-tune command on 24 GB class cards, and merge offline with merge_lora_weights_and_save.py.
• Start at --batch_size 1 and raise accumulation, not batch, until you know your card's ceiling.
• Print torch.cuda.max_memory_allocated() once after the first calls; compare it against the FAQ number for your setup.
• Read both numbers in the error line before changing anything: allocated-by-PyTorch and the largest other process.
• Try PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True only when reserved-but-unallocated is large and memory is genuinely free.
• Watch host RAM on multi-GPU runs; turn down shuffle_buffer_size if it climbs without plateau.
• Never spread one model across cards with an automatic device map; the concatenation inputs must share one device.
• Pin the paper stack: Python 3.10.14, PyTorch 2.2.0, the project transformers fork at 4.40.1, Flash Attention 2.5.5.
• After any upgrade, verify the vision tower still contributes; issue 346 documents the silent version failure.

If the card is simply too small

Three honest roads exist below the measured floors. Quantized loading shrinks inference. The OFT loading path accepts --load_in_4bit True or --load_in_8bit True. In the issue 311 thread, a 4-bit load peaked near 4.6 GB and an 8-bit load near 8.05 GB, on a fine-tuned base OpenVLA checkpoint. Latency in that thread's own test was essentially unchanged. Quantized fine-tuning exists on the base repo. Its fine-tune script carries a use_quantization flag that 4-bit loads the model for LoRA training, and the source asserts the flag is LoRA-only. Both roads trade some precision on fine motions. Verify success rates, not just memory, before you trust a policy.

The third road is not owning the card at all. Rent by the hour for the fine-tune, or use a hosted action route for inference. The economics of both shapes, per GPU class, are worked through in three guides. Start with the local versus API break-even guide. The Jetson versus cloud guide and the builder compute budget guide cover the rest.

Here, the hosted route exists as a written record. The platform's model data carries the openvla-oft model id on its action route, metered per call, and as of 27 October 2026 that row is not on the public list, with no date attached to that step. When a first call is what you are sizing, you can sign up. Any metered rate renders on the pricing page, never in prose here. The rate-card reasoning for this model lives in the OpenVLA-OFT pricing guide. The wider hosting options for this family are in the hosted ways to run OpenVLA guide.

Frequently asked questions

How much GPU memory does OpenVLA-OFT actually need?

For inference, measured and published: 15.9 GB for one-camera LIBERO, and 16.2 GB for two cameras with proprioception. Three-camera ALOHA with FiLM needs 18.0 GB. For fine-tuning, the published minimum is 25.6 to 25.7 GB per GPU at batch size 1 on LIBERO. The recommended setups run 44.1 to 73.5 GB per GPU. The numbers are setup-specific: cameras, proprioception, and FiLM each add to the bill.

Can I fine-tune OpenVLA-OFT on an RTX 4090?

Yes, with three changes: --merge_lora_during_training False, batch size 1 with gradient accumulation, and a lower LoRA rank. The tracker records 24 GB cards training this way, with the checkpoint save, not training, as the step that kills unprepared runs. The default recipe's published minimum sits just above 24 GB. This is the squeezed lane, not the comfortable one. The base OpenVLA README sets its own floor at about 27 GB for LoRA fine-tuning.

Why does the crash happen at checkpoint saving instead of during training?

Because the save step runs a second job beside the training run. With the default merge_lora_during_training True, every save loads the full base model again on the same card. The LoRA adapter is merged into it there. Training at its normal footprint plus a fresh 7B load is more than a 24 GB card holds. The maintainer's answer is the flag: skip in-training merging, merge offline once at the end.

Is CUDA out of memory ever a fragmentation problem?

Sometimes, and the message tells you. Compare the reserved-but-unallocated figure with the free figure. Gigabytes reserved but unallocated while the card reports free memory is fragmentation. PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True before launch is the documented try. Bytes reserved while another process holds the card is not fragmentation. That is cause 1 or cause 3, and no allocator setting fixes arithmetic.

Does quantization solve the memory problem?

It moves the floor. The measured peaks in the issue 311 thread put the base model near 8 GB at 8-bit, and under 5 GB at 4-bit. The bfloat16 weights on disk are about 15 GB. That is the difference between not running and running on small cards. The trade is precision on fine motions. 4-bit flipped gripper decisions in that thread's test. Measure success rates after switching, not just memory.

The short version

Match the string, read the two numbers, then fix the named cause. The save-step crash wants --merge_lora_during_training False and an offline merge. The inference crash on a 24 GB card wants a current checkout with torch.inference_mode(). A crash during training steps wants batch 1 plus accumulation, or a lower rank, or a bigger card, because the published minimum is 25.7 GB per GPU. Fragmentation wants expandable_segments and only when its signature numbers say so. A kill without a CUDA string wants host RAM attention, not GPU attention. The neighbor pages carry the related families: NCCL and CUDA crashes on multi-GPU pi 0.5 and checkpoint loading errors on MolmoAct2. Rates for hosted routes render on the pricing page. You can sign up when a first metered call is next.

Last verified: 2026-10-27