Proxy Policy Steering

· Editorial Team estimated
vision-language-action policy-adaptation inference-time diffusion-policy manipulation

Proxy Policy Steering (PPS) adapts a frozen generalist robot policy to a new task at inference time. Instead of modifying the base policy, PPS trains two lightweight proxy policies — a reference proxy that models the frozen base's behavior on target-task observations and a task proxy capturing how behavior changes under task supervision — and uses their calibrated velocity-space difference to steer the base sampler at every denoising step. Because the base is never modified, its broad capabilities remain intact. On 8 real-world and 4 simulation manipulation tasks, PPS lifts the state-of-the-art pi0.5 base policy by 53% absolute success rate on average, including zero-to-one gains on tasks the base never solved, outperforming LoRA fine-tuning, from-scratch specialists, residual policies, and prior inference-time steering methods.

Paper · arXiv:2609.09148

Generalist robot policies trained on large-scale data bring broad manipulation priors, but specializing them to a specific new task remains the deployment bottleneck. Fine-tuning risks degrading the very generality that makes these policies valuable, while training a specialist from scratch discards those priors entirely. Proxy Policy Steering (PPS) targets this trade-off directly: it leaves the base policy untouched and instead learns how to steer it at inference time.

Core Innovation

PPS trains two lightweight proxy policies. A reference proxy imitates the frozen base policy’s behavior on target-task observations, establishing a baseline of what the base would do. A task proxy, initialized from the reference, learns how that behavior should change under the new task’s supervision. The calibrated difference of their velocity predictions forms a residual that steers the frozen base’s denoising sampler at every step. Because adaptation uses only forward velocity predictions from the base, PPS works even without access to the base’s parameters — and because the base is never modified, capabilities the demonstrations never exercise (such as failure recovery) remain available at inference.

Results

  • On 8 real-world and 4 simulation manipulation tasks, PPS raises the state-of-the-art pi0.5 base policy by 53% absolute success rate on average.
  • Delivers zero-to-one gains on tasks the base policy never solves.
  • Outperforms LoRA fine-tuning, from-scratch specialists, residual policies, and prior inference-time steering methods.
  • The paper identifies and empirically validates the conditions under which the residual isolates task-supervision-induced change rather than spurious base behavior.

Limitations

Results are reported on the pi0.5 family and associated benchmarks; scaling behavior on other generalist architectures is not established in the abstract. The approach requires suitable target-task demonstrations and two proxy training runs per task, and the abstract does not quantify compute overhead or failure modes when the reference proxy imperfectly models the frozen base. Steering quality depends on the velocity-space calibration assumptions holding.

Industry Implications

Adapting foundation-model policies to customer-specific tasks — without re-training or even touching the base model — is exactly what deployment teams need: it preserves the model vendor’s IP while enabling rapid customization from a handful of demonstrations. PPS-style adaptation could substantially cut the cost and risk of bringing generalist manipulation policies to production, letting one base model serve many bespoke tasks across warehouses, kitchens, and factories.