InterMimicGen: Scaling Humanoid Loco-Manipulation through Self-Evolving Motion Imitation

· Editorial Team estimated
humanoid loco-manipulation motion-imitation motion-retargeting reinforcement-learning sim-to-real

Human-object interaction captures are a rich but awkward source of supervision for humanoid loco-manipulation: they are sparse, heterogeneous, and not directly executable by a robot. InterMimicGen is a self-evolving motion-imitation framework in which robot motion data and a tracking policy improve one another. It consolidates motion-captured human-object interaction datasets and retargets them into humanoid references that preserve whole-body coordination and dexterous hand-object relationships, then trains a physics-based generalist tracker that executes those references in simulation on a humanoid with dexterous hands. A data flywheel then makes small, task-preserving edits to where an interaction takes place and how the body performs it, fine-tunes the tracker on the variants, and keeps only those whose simulated execution completes the task, so coverage compounds around the sparse original demonstrations while task semantics and motion quality are preserved. The authors report contact-preserving retargeting across robot configurations, broad tracking from a single generalist policy, motions that keep growing across augmentation rounds, and transfer to real robots.

Paper · arXiv:2610.06850

Humanoid loco-manipulation has a data problem of a particular shape. The demonstrations that exist in abundance — human-object interaction captures — are the wrong form for a robot to execute, and the ones that would be usable are scarce. InterMimicGen treats the gap as a loop rather than a one-time conversion.

Background

Captured human-object interactions are one of the richest available sources of supervision for humanoid whole-body loco-manipulation, because they show people coordinating locomotion, balance and dexterous hands to accomplish tasks. Their promise is offset by three properties that make them hard to use: they are sparse, they are heterogeneous across capture setups and subjects, and they are not directly executable by a robot. Retargeting human motion to a robot is the standard bridge, but a single retargeting pass yields a fixed reference set tied to the exact situations recorded, which leaves coverage thin precisely where dexterous whole-body control is hardest.

Core Innovation

InterMimicGen addresses both the conversion and the scarcity with a self-evolving motion-imitation framework in which robot motion data and a tracking policy improve each other. It proceeds in three parts. First, it consolidates motion-captured human-object interaction datasets and retargets them into humanoid robot references while preserving whole-body coordination and the dexterous relationships between hand and object, producing a large and diverse reference collection for dexterous whole-body loco-manipulation. Second, it trains a physics-based generalist tracker to execute those references in simulation on a humanoid equipped with dexterous hands, covering a scale and diversity beyond earlier humanoid tracking systems for loco-manipulation. Third, it closes a data flywheel: each round makes small, task-preserving changes to where an interaction takes place and how the body performs it, fine-tunes the tracker on the edited variants, and keeps only the variants whose simulated execution completes the task, which then seed the next round. Because the edits are small and constrained to preserve the task, they compound into broader coverage around the sparse original demonstrations instead of drifting away from them.

Results

  • Contact-preserving retargeting is demonstrated across robot configurations.
  • A single generalist policy achieves broad tracking across the consolidated reference collection.
  • The set of executable motions keeps growing across augmentation rounds while preserving task semantics and motion quality.
  • The pipeline transfers to real robots.

Limitations

The abstract reports qualitative outcomes rather than numbers, so the size of the reference collection, the tracking success rates, and the real-robot transfer success are not quantified and cannot be compared directly against prior humanoid tracking systems. The flywheel’s selection step judges variants by their simulated execution, which makes the growing motion resource only as trustworthy as the simulator used to filter it; a systematic simulation bias would be compounded, not corrected, across rounds. The scope of real-robot validation is also left open — the abstract claims transfer but does not say how many tasks, on what hardware, or how robustly.

Industry Implications

The cost of humanoid progress is increasingly a data footprint rather than a controller architecture, and the most valuable data for loco-manipulation — coordinated hand and whole-body motion — is the hardest to collect on real hardware. A method that converts heterogeneous interaction captures into a robot-executable reference collection, and then grows that collection automatically, attacks the bottleneck directly. For teams building humanoid fleets, the practical implication is that a fixed, hand-collected demonstration set can become a compounding asset: every augmentation round that survives simulation filtering is motion the robot can be trained to execute, without new hardware time. The caveat adopters should weigh is that the whole resource is filtered by a simulator, so the fidelity of that simulator is load-bearing.