PASSAGE: Scaling Scene-Aligned Motion Learning for Perceptive Humanoid Traversal in Cluttered Environments

· Editorial Team estimated
humanoid traversal motion-learning flow-matching onboard-perception sim-to-real

PASSAGE is a perception-conditioned planner-tracker framework for humanoid traversal of cluttered environments. The authors collect 100 hours of scene-aligned human motion with virtual reality and inertial motion capture across 1,500 cluttered scenes; a conditional flow-matching planner generates short-horizon references from motion history, a local destination and a robot-centric multi-layer elevation map, and a perceptive whole-body tracker executes them at 50 Hz with geometric feedback. Real-time chunking promotes inter-chunk consistency, and planner-side reinforcement learning on top of the frozen tracker further improves closed-loop performance. The system needs no skill annotations and no obstacle-specific policies: one planner-tracker pair learns to select and compose stepping, squeezing and ducking on geometries it has never seen. Scaling captured data from 6 to 100 hours raises mean contact-free success on held-out scenes from 48.1% to 68.9%, and 70.3% with validated scene augmentation. The fully onboard system integrates egocentric 3D LiDAR, online occupancy mapping, 6.25 Hz planning and 50 Hz control on a Jetson AGX Orin, traversing 50 unseen physical layouts without prebuilt maps or offboard computation.

Paper · arXiv:2609.18732

Humanoid robots can already step over an obstacle, squeeze past it, or duck under it. The hard part is deciding, from onboard perception alone, which of those behaviors the moment calls for. Most existing approaches depend on task-specific reinforcement-learning objectives or on carefully curated motion libraries, so each new class of obstacle means rewriting rewards, retraining policies and rebuilding motion clips. Coverage grows roughly linearly in cost with scene complexity.

Core Innovation

PASSAGE’s answer is to stop writing a policy per obstacle and let human motion itself define the motion prior.

The framework has two parts. A planner built on conditional flow matching takes motion history, a local destination and a robot-centric multi-layer elevation map, and produces short-horizon references. A perceptive whole-body tracker then executes those references at 50 Hz with geometric feedback in the loop. Because the interface is perception-conditioned, the planner reasons over geometry produced by onboard mapping rather than over hand-specified obstacle parameters.

Two engineering choices matter here. Real-time chunking lets planning roll forward at 6.25 Hz while keeping adjacent action chunks consistent. Planner-side RL post-training keeps the tracker frozen and improves only the planner, so closed-loop performance keeps rising without destabilizing the low-level locomotion that already works.

The data side is an equal part of the contribution. The authors use virtual reality and inertial motion capture to gather 100 hours of scene-aligned human motion across 1,500 cluttered scenes — action and scene geometry recorded together — with no manual skill labels and no obstacle-specific policies.

Results

  • Scaling is measured, not asserted. Across three independent training seeds, growing the captured data from 6 hours to 100 hours lifts mean contact-free success on held-out scenes from 48.1% to 68.9%; the final model, with validated scene augmentation, reaches 70.3%.
  • One planner-tracker pair covers many behaviors. Without skill annotations or obstacle-specific policies, the model selects and composes traversal behaviors on unseen geometries.
  • Everything runs onboard. The system integrates egocentric 3D LiDAR, online occupancy mapping, 6.25 Hz planning and 50 Hz control on a Jetson AGX Orin, traversing 50 unseen physical layouts with no prebuilt map and no offboard compute.
  • Component ablations in simulation quantify what each stage contributes.

Limitations

The abstract reports only internal ablations and the data-scaling curve; there are no comparisons against external baselines. On the real-robot side we get a qualitative claim that traversal succeeded across 50 unseen layouts, but no real-world success rate. A 70.3% contact-free rate still means roughly one attempt in three makes contact, and neither safety fallbacks nor fall recovery are discussed. Data collection depends on VR and mocap hardware whose scaling cost is never quantified, and how the planner’s RL post-training behaves when the frozen tracker is later upgraded is left open.

Industry Implications

Getting from here to there is the first gate between a humanoid demo and a warehouse, construction site or home. That gate has historically been crossed by tuning rewards per obstacle class and stockpiling motion clips in simulation: expensive, and brittle once the geometry changes. PASSAGE replaces that curve with a data curve — collect more scene-aligned human motion and capability follows — which is a flywheel rather than an enumeration. Just as importantly, the planner-tracker split, with RL post-training on a frozen tracker, is a reusable engineering pattern that decouples a stable locomotion prior from task-level fine-tuning: the substrate can be frozen and reused while the upper layer keeps iterating. For a team building a humanoid software stack, that combination of a human-motion corpus, perception-conditioned planning and a fully onboard runtime is closer to a deployment blueprint than any single algorithm in isolation.