ω-0: A Latent Predictive World Action Model for Concurrent Humanoid Loco-Manipulation

Background

Real humanoid household work rarely decomposes into “walk somewhere, stop, then manipulate.” Tasks like tidying a table, serving food, or rearranging objects require the robot to move and adjust its posture while simultaneously reaching, grasping, and balancing — concurrent loco-manipulation. Existing humanoid policies typically treat locomotion and manipulation as separate modules that are hard to coordinate, while recent world-action models are either arm-centric or focused on generating future videos rather than executable control. This gap leaves humanoid deployment in homes without a unified recipe for whole-body, language-guided behavior.

Core Innovation

ω-0 is a latent predictive whole-body world-action model. Given a language instruction, a current visual observation, and proprioceptive state, it directly predicts controller-compatible whole-body action latents for real-robot execution. Two design choices stand out. First, instead of reconstructing future videos (the standard objective of video world models), it learns compact future observation embeddings as a lightweight predictive objective, coupling latent visual foresight with diffusion-based whole-body action generation. Second, it grounds human/public visual-motion priors into robot-executable action latents through controller-based simulation replay, bridging the gap between human demonstration data and the robot’s actual dynamics. The model accepts egocentric RGB, exocentric RGB, and exocentric depth inputs, and is trained on ω-HOME, a 40+ hour real-world household humanoid dataset with synchronized multi-view observations, whole-body SMPL motions, robot states, and action latents.

Results

Real-world experiments across 11 household tasks show that a single ω-0 model produces smooth manipulate-while-moving behaviors and consistently outperforms representative imitation learning, VLA, humanoid, and world-action-model baselines. The evaluation spans genuine concurrent loco-manipulation scenarios rather than isolated reach-and-place tasks, and the unified model handles all tasks without per-task retraining. The multi-view and multi-modal input support further suggests robustness across different camera setups.

Limitations

The paper’s claims rest on a single model family and dataset collected by one team, so cross-lab reproducibility is not yet established. The abstract reports relative superiority over baselines but does not detail per-task success rates, failure modes, or sensitivity to language instruction quality. The 40+ hour dataset, while substantial, is still small relative to the diversity of real household environments, and the controller-based replay grounding depends on the fidelity of the simulation used during training.

Industry Implications

Concurrent loco-manipulation is the capability that separates home-ready humanoids from lab demos: folding, fetching, cleaning, and serving all require moving while manipulating. A single policy that directly outputs controller-compatible whole-body actions from language and vision shortens the path from demonstration data to deployed behavior and removes the need to stitch separate locomotion and manipulation stacks. The release of a multi-view household dataset with SMPL and robot-state annotations is also valuable for the broader humanoid ecosystem, which is chronically data-poor. For humanoid startups and integrators, this is a strong signal that unified whole-body world-action policies are becoming practical for real homes.