PredActor: Predictive Action Diffusion for Steerable Onboard Humanoid Control

· Editorial Team estimated
humanoid diffusion-policy whole-body-control onboard-inference test-time-steering unitree-g1

Diffusion models generate flexible motion, but turning that flexibility into feedback-responsive humanoid control is awkward: hierarchical systems steer through motion references that a separate tracker may not be able to execute, action-only diffusion has no explicit future-state trajectory to steer, and joint state-action diffusion controllers typically depend on privileged full-body state estimates while leaving behavior selection and test-time steering fragmented. PredActor consolidates these capabilities into one directly executed policy driven by proprioceptive observations: conditioned on proprioceptive history and optional task context, it jointly generates executable actions and an internal future-state trajectory, with classifier-free guidance strengthening text-conditioned behavior and classifier guidance steering predicted states toward test-time objectives. Only the actions are executed, with no separate motion-reference tracker and no externally estimated full-body state as input. In simulation it reaches all 15 destination targets with a text retrieval score of 0.580 against 0.373 for conditional action diffusion; rolling denoising and computation-preserving runtime optimizations bring the complete callback to 16.790 ms median and 19.383 ms p95 on a Jetson Orin NX, both inside the 20 ms control period. It is deployed on a Unitree G1.

Paper · arXiv:2609.24840

Diffusion models generate diverse, natural human motion, but turning that flexibility into a humanoid controller that responds to feedback is not a small step. Hierarchical systems steer motion through references and hand physical execution to a separate tracker, so when the reference exceeds what the tracker can do, recovery is no longer the policy’s business. Action-only diffusion removes the reference but emits actions with no explicit future-state trajectory, which leaves nothing natural to steer at test time. Joint state-action diffusion supplies that trajectory, yet representative controllers depend on privileged full-body state estimates that a real robot does not have, and behavior selection and test-time steering end up fragmented across different mechanisms.

Core Innovation

PredActor consolidates the pieces into a single policy that is directly executed on the robot, conditioned only on what the robot can sense for itself.

  • Joint action and future-state generation from proprioception. Conditioned on proprioceptive history plus optional task context, the policy generates executable actions and an internal future-state trajectory. Only the actions are executed.
  • Two kinds of guidance for two different jobs. Classifier-free guidance strengthens text-conditioned behavior selection; classifier guidance steers the predicted states toward test-time objectives. Selection and steering therefore live in the same policy rather than in a hierarchy above it.
  • No reference tracker, no privileged state. There is no separate motion-reference tracker to inherit errors, and no externally estimated full-body state feeding the policy. Onboard deployment is designed in rather than added later: rolling denoising and computation-preserving runtime optimizations.

Results

  • Simulation: reaches all 15 destination targets, with a text retrieval score of 0.580 versus 0.373 for conditional action diffusion, and similar observed disturbance survival.
  • Runtime: the complete callback runs at 16.790 ms median and 19.383 ms p95 on a Jetson Orin NX — both below the 20 ms control period.
  • Hardware: deployed on a Unitree G1, with text-conditioned motion, disturbance response, joystick control, and semantic interpolation demonstrated across simulation and physical hardware.

Limitations

The abstract reports target counts and a text-retrieval score but no locomotion tracking error, fall rate, or torque-saturation statistics, and disturbance survival is described only as “similar” rather than quantified. Evidence covers one platform (Unitree G1) and one embedded compute module; there is no cross-morphology transfer and no long-duration reliability data. Test-time steering is demonstrated through qualitative behaviors such as semantic interpolation rather than against a measurable objective, and the comparison against hierarchical baselines covers task success and retrieval, not end-to-end latency — precisely where a single fused policy ought to win. The 16.79 ms figure is for one complete callback, but the abstract does not state the achieved closed-loop control frequency or the denoising-step budget, so the headroom for higher-frequency operation, or the margin under thermal throttling, is unclear.

Industry Implications

For humanoids, the binding constraint on diffusion policies has not been motion quality but whether the policy can run fast enough, on enough of the robot’s own sensors, to close the loop. Showing a complete callback inside a 20 ms period on Orin-class hardware, with only proprioceptive inputs, removes the need for motion capture or a full-body state estimator — both of which are cost and failure points on a shipping product. Folding behavior selection and test-time steering into one policy also simplifies the software stack that would otherwise be split between a planner, a reference generator, and a tracker. What a product team still needs before committing are quantified recovery behavior, reliability over long operating sessions, and evidence that the same formulation holds on platforms other than the G1 — none of which appear in the abstract.