VLALagBench: 100 ms of Inference Latency Can Halve Success, So Price It Before You Deploy

VLA papers follow a familiar shape: a bigger model or a better training recipe, success rates on several benchmarks, and a closing line about inference running fast enough for real time. That line is usually asserted rather than measured. In the physical world the robot keeps moving during the tens to hundreds of milliseconds a policy takes to think, and so does the object. The action that comes back was computed for an observation that is already stale.

How much success latency actually costs had not been measured systematically. VLALagBench fills that gap.

Core innovation

Latency becomes an independent variable. The benchmark varies inference latency under controlled conditions while spanning four factors: task type, object speed, execution mode (synchronous or asynchronous), and the specific VLA policy. With that decomposition, “latency sensitive” stops being a vague label and becomes a measurable statement about which task, at what target speed, for which policy, losing how much.

A controlled comparison of synchronous and asynchronous execution. In synchronous mode the policy must finish before any action is sent, so the robot visibly stalls. Asynchronous mode overlaps inference with execution of the previous action chunk, buying continuity at the cost of more frequent requests. Both are widely used in real systems, but they had not been compared on the same tasks with the same policies. The benchmark supplies that comparison and quantifies the price of asynchrony: request count.

Sensitivity varies by task and by policy. The paper attributes the spread to differing requirements for precision and continuous motion. That matters because it means latency is not a problem you solve once by making inference uniformly faster. Some tasks are intrinsically more sensitive than others.

Predicting success from a little calibration data. The authors fit an empirical model that forecasts task success from limited calibration data, with 5.7-7.8% RMSE across several VLA policies. Given the latency of an inference stack and the throughput per robot, the model turns a real engineering trade-off — a larger model, a higher control rate, or asynchronous execution — into an explicit calculation instead of a hunch.

Results

• 100 ms of inference latency can nearly halve task success on moving targets.
• Sensitivity varies substantially with task type, object speed and policy.
• Asynchronous execution recovers some lost success at the cost of more inference requests.
• The success predictor achieves 5.7-7.8% RMSE.
• The benchmark spans multiple VLA policies and both synchronous and asynchronous execution.

The “nearly halve” result comes with conditions worth stating: it applies to moving targets. For static targets, 100 ms is usually survivable, because the observation is stale but the scene is not. Latency punishes dynamic settings — parts on a conveyor, people walking past, an object in flight. That is exactly why the sensitivity spread across tasks is so wide.

The predictor should be read for what it is. An RMSE of 5.7-7.8% is enough to decide whether a configuration is worth building, not enough to certify that it will hit 99% success. It is a planning instrument, not an acceptance test.

Limitations

The work measures rather than proposes, so it can tell you latency hurts without telling you how to remove it. Evaluation is largely in controlled settings, and real latency is not a constant: it jitters, it has a tail, and it moves with load. Whether the benchmark models that distribution is not stated in the abstract. Asynchronous execution buys back success but raises request frequency, which drives power and server-side pressure in multi-robot deployments; that cost is not folded into the conclusion. And the 5.7-7.8% error was validated on an existing set of policies, so extrapolation to architectures further from that set is unknown.

Industry implications

For teams about to deploy a VLA, the paper prescribes a calculation to run before integration: measure the latency distribution of your inference stack on the target hardware, estimate the success penalty with a model of this kind, then choose model size, control rate and execution mode. Doing it in the other order — pick the model, discover the latency — reliably means rework during integration.

For chip and systems vendors the signal is that the robotics workload bottleneck is shifting from training to inference determinism. Mean latency is not the metric that matters; tail latency is, because a single overrun shows up as a dropped grasp.

For algorithm teams, the asynchronous result deserves attention. It says scheduling can buy back success that latency took away, at the same compute budget and without retraining. That is the cheapest optimization in the stack and the one least often reported in papers.