DreamTrue: A Robot World Model That Follows Actions and Learns from Counterfactuals

· Editorial Team estimated
world-model video-prediction cross-embodiment counterfactual-training reward-model manipulation

Robot world models should predict what happens next faithfully, but training them on real datasets runs into imprecise calibration that weakens action following and a bias toward successful outcomes because failures are rare. DreamTrue is a multi-view, cross-embodiment world model that attacks both. It renders action trajectories into image-space conditions and uses offline geometric calibration to align them with the target video, improving action following across embodiments. To widen interaction coverage it adds counterfactual post-training, rewriting recorded trajectories and generating futures under a broader range of actions and contacts. To score predictions without paired ground truth, it builds a human-annotated dataset of robot, object and interaction defects and trains an embodied video reward model that steers reinforcement-learning post-training toward physically plausible outcomes. On AgiBot it attains state-of-the-art action following and cuts the human-assessed interaction defect rate from 48.12% to 6.25%, and it ranks first in the world-model track of the AgiBot World Challenge 2026.

Paper · arXiv:2610.12468

A world model is only useful if it reacts to the actions it is asked to imagine. Two quiet properties of real robot datasets undermine that: imperfect calibration blurs the link between action and video, and the near-absence of failures teaches the model that everything tends to succeed. DreamTrue treats both as trainable defects rather than fixed limitations.

Background

A robot world model is worth having because it lets a system predict the consequences of an action before committing to it, which supports planning, data augmentation and policy learning. Training such a model on existing robot datasets, however, faces two obstacles. Imprecise calibration can impair action following, so the predicted future does not properly obey the commanded action. And the datasets over-represent successful interactions, which biases predictions toward successful outcomes and leaves the model blind to the failures that matter most for safety and recovery.

Core Innovation

DreamTrue is a multi-view, cross-embodiment world model designed around those two obstacles. To improve action following across embodiments, it renders action trajectories into image-space conditions and introduces offline geometric calibration to align those conditions with the target videos, so the action signal the model conditions on is spatially consistent with what it must predict. To broaden interaction coverage, it introduces counterfactual post-training, which modifies recorded action trajectories and generates future videos under a wider range of actions and contact configurations, directly manufacturing the failure cases the dataset lacked. To provide feedback on predictions that have no paired ground-truth future, it constructs a human-annotated video dataset covering robot, object and interaction defects and trains an embodied video reward model; the scores it produces guide reinforcement-learning post-training toward more physically plausible interaction outcomes.

Results

  • DreamTrue attains state-of-the-art action following on AgiBot.
  • It reduces the human-assessed interaction defect rate from 48.12% to 6.25%.
  • It ranks first in the world-model track of the AgiBot World Challenge 2026.
  • The human-annotated defect dataset and the embodied video reward model are central to guiding post-training without paired ground-truth futures.

Limitations

The headline evaluation is concentrated on AgiBot, so the cross-embodiment claim rests on calibration and rendering choices that are only demonstrated at that scale; the abstract does not report how the model behaves on other platforms or how the defect-rate metric transfers beyond the human-annotated set it was trained on. The reward model inherits the biases of human annotators and of the defect taxonomy they were given, and the counterfactual post-training depends on how far one may perturb recorded trajectories before the generated futures stop being physically meaningful — a boundary the abstract does not quantify. The 48.12%-to-6.25% improvement is a strong single number but is a human-rated metric on one embodiment; generalization, compute cost and inference latency are not reported.

Industry Implications

World models are becoming infrastructure for robot learning, and DreamTrue is a reminder that their most valuable fixes may be dataset-level rather than architectural. The counterfactual recipe speaks to a chronic problem for anyone training on fleet data: failures are exactly what you want to learn from and exactly what gets under-recorded. Turning that gap into a deliberate augmentation and reward-signal pipeline is transferable beyond this model. The reliance on a human-annotated defect dataset is also a candid cost signal — physically plausible prediction currently needs human judgment in the loop, and teams should budget annotation effort alongside compute. For adopters, the practical takeaway is to audit calibration and failure coverage before scaling model size, because here the largest defect reduction came from correcting the data, not enlarging the network.