Long-WAM: Scaling the Context of World-Action Models

Real-time robot control has a timing problem: a policy needs enough visual history to read motion and task progress, but every frame it conditions on is latency it has to pay for. World-action models promise to inherit the predictive power of large video models, yet most of that promise has been shown with short contexts. Long-WAM’s argument is that the field has been optimizing the wrong thing.

Background

World-action models (WAMs) aim to transfer video-prediction priors into action learning, on the intuition that a model able to imagine the near future will act better than one that only sees the present. The obstacle is throughput. Real-time control budgets are tight, and conditioning a policy on a long window of past frames is exactly the kind of cost that shows up as lag. As a result, histories in deployed WAMs tend to be kept short, a choice usually justified as a latency trade-off.

Core Innovation

The paper’s central finding reframes that trade-off: access to history is not the same as using it. Longer histories pay off far more when the video foundation is pretrained autoregressively, whereas a bidirectionally pretrained initialization shows no net gain from the same extra context. Long-WAM is built to exploit this. It first learns causal prediction from robot and egocentric videos without action labels, then preserves that history-to-future structure while adapting the model into a world-action controller, so action-conditioned training does not wash out the causal temporal structure that makes long context useful. To make long context affordable, the system adds streaming observation encoding, asynchronous execution and hardware-specific acceleration, letting prediction and control proceed concurrently instead of each waiting on the other.

Results

• On RoboCasa GR-1, increasing context from 0.0 to 19.2 seconds raises success from 63.3% to 78.7%.
• Robot-domain autoregressive pretraining further raises peak success on GR-1 and LIBERO-Long.
• It achieves the best results among compared methods on LIBERO-Long, RoboTwin 2.0 and DOMINO.
• It deploys on RTX 5090, DGX Spark and Jetson AGX Thor without dropping future prediction, with each action chunk, including future-video latent prediction, taking 107.4 ms on an RTX 5090.
• Real-time deployment on Unitree G1 and YAM supports dynamic and long-horizon manipulation, including 95% success on dynamic cup stacking, where Pi0.5 and Fast-WAM succeed in none of 20 trials.
• As a memory-informed executor it complements higher-level planning in composite tasks.

Limitations

The evidence concentrates on a small set of suites (RoboCasa GR-1, LIBERO-Long, RoboTwin 2.0, DOMINO) plus a handful of real tasks, so broad cross-embodiment robustness is not yet demonstrated. The autoregressive-pretraining advantage is shown against one bidirectional control; how the benefit scales with the amount and diversity of pretraining video, and how brittle it is to robot-domain shift, is not reported. The latency figures are device-specific, and the deployment stories are single-platform. The dynamic cup-stacking comparison is striking but narrow, and the abstract gives no failure analysis for it. Finally, the paper does not separate how much of the gain comes from context length versus the specific pretraining recipe, which is the question an adopting team would need answered.

Industry Implications

For anyone building real-time robot policies, the actionable message is that the standard latency-versus-history trade-off is partly an artifact of training rather than a law of physics. If long context only pays off under autoregressive pretraining, then the pretraining recipe is a first-class design decision rather than a detail, and teams that pretrain bidirectionally may be leaving performance on the table without realizing it. The engineering half matters just as much: streaming encoding, asynchronous execution and per-device acceleration are what turn a scaling result into something that runs on a Jetson-class module in the loop. The clearest commercial signal is the 95% versus 0% on dynamic cup stacking against Pi0.5 and Fast-WAM, a gap that, if it holds, speaks directly to the dynamic manipulation tasks that factories and households actually contain.