World-Model-Augmented Visual Locomotion for Humanoids on Foothold-Constrained Terrain

· Editorial Team estimated
humanoid locomotion world-models legged-robotics sim-to-real

WM-LOCO jointly trains a recurrent world model and a PPO policy so a humanoid can traverse foothold-constrained terrain — stepping stones, gaps, and narrow stair treads — using only proprioception and a single onboard depth image, with no explicit foothold labels. In simulation it succeeds on gaps and stepping stones where a matched baseline fails completely, and the same policy deployed on a physical Unitree G1 traverses all three terrain classes with a 93.3% average success rate.

Paper · arXiv:2609.02542

Foothold-constrained terrain — ground where feasible foot contacts are sparse, discontinuous, or geometrically restricted, such as stepping stones, gaps, and narrow stair treads — leaves almost no room to recover after a single misstep. Policies that base foot placement mainly on the immediately visible terrain therefore tend to fail there. WM-LOCO asks whether a learned predictive summary of near-future observations and rewards can supply the anticipatory information these settings demand.

Core Innovation

  • World-model-augmented control — a recurrent world model and a PPO policy are trained jointly; conditioned on proprioception and a single onboard depth image, the world model produces a predictive recurrent feature that guides the policy.
  • No explicit foothold labels — the policy learns where to step from data alone, without any annotated foothold supervision or hand-designed terrain heuristics.
  • Lightweight sensing — only onboard proprioception plus one depth stream is required; no motion capture, external mapping, or privileged terrain information.

Results

  • Simulation: WM-LOCO succeeds on gaps and stepping stones where a matched baseline fails completely; on stairs it matches the baseline’s success rate while improving stride efficiency and reducing pelvis acceleration.
  • Hardware: the same policy runs onboard a physical Unitree G1 humanoid using onboard proprioception and a single depth stream, traversing all three terrain classes with an average success rate of 93.3%.

Limitations

Hardware validation covers a single platform (Unitree G1) and three terrain classes, so generalization to other humanoids and a wider variety of terrain remains to be shown. The abstract does not detail the sim-to-real procedure or its sensitivity. The learned predictive feature is less interpretable than an explicit foothold plan, and comparisons against perception-based foothold planners are not reported in the abstract.

Industry Implications

Construction inspection, industrial maintenance, and emergency response all require legged machines to walk over scattered obstacles, service stairs, and narrow catwalks — exactly the terrain class studied here. A strategy that needs only onboard sensing keeps system cost and setup complexity low, and removing foothold labeling simplifies the data pipeline for scaling to more environments. The approach is also a natural fit for quadruped and other legged platforms, making it relevant well beyond the humanoid form factor.