GIFT: Guided Intermediate Feature Training via Action-Oriented Structural Supervision for Robotic Manipulation

· Editorial Team estimated
vision-language-action world-action-model representation-learning manipulation robot-learning

GIFT addresses the action-sufficiency gap — the mismatch between the visual richness of vision-language pretraining and what control actually needs — by supervising intermediate features with three control-relevant structures: geometry governing motion feasibility, affordances encoding instruction-relevant entities, and goals grounding instructions in task-relevant regions. The framework is architecture-flexible and was instantiated in a VLA policy, a direct-action World-Action Model, and an inverse-dynamics WAM without changing each model's action formulation. GIFT variants outperform their counterparts by 4.6/12.6/5.2 points on zero-shot LIBERO-Plus (79.6%/72.6%/87.8%) and by 12.6/9.0/8.4 points on RoboCasa, with the largest gains on articulated objects and high-precision real-world manipulation under unseen visual and spatial perturbations.

Paper · arXiv:2609.04193

Vision-language pretraining and predictive world modeling give robot policies rich semantic and dynamic visual features, yet their native objectives can omit the physical and task structure that control actually depends on — while keeping control-irrelevant visual redundancy. GIFT names this mismatch the action-sufficiency gap and asks whether guiding intermediate features toward control-relevant structure can bridge it, without changing the model’s action formulation.

Core Innovation

  • Three control-relevant structures — intermediate features are supervised to preserve geometry that governs motion feasibility, affordances that encode instruction-relevant entities, and goals that ground instructions in task-relevant regions.
  • Training-time constraints, not architecture changes — geometry alignment, affordance prediction, and goal-region reconstruction are translated into auxiliary losses, so the framework stays architecture-flexible.
  • Broad instantiation — applied to a Vision-Language-Action (VLA) policy, a direct-action World-Action Model (WAM), and an inverse-dynamics WAM, retaining each model’s native action formulation.

Results

  • Zero-shot transfer to LIBERO-Plus: GIFT-VLA, GIFT-WAM-Fast, and GIFT-WAM-IDM reach 79.6%, 72.6%, and 87.8% — beating StarVLA-OFT, Fast-WAM, and Fast-WAM-IDM by 4.6, 12.6, and 5.2 points.
  • RoboCasa: 61.4%, 83.6%, and 82.3% — outperforming counterparts by 12.6, 9.0, and 8.4 points respectively.
  • Largest gains on articulated-object tasks and high-precision real-world manipulation under unseen visual and spatial perturbations.

Limitations

The paper reports benchmark and real-world evidence for structured intermediate features as a reusable principle, but scaling behavior across many more tasks, models, and robot platforms remains to be shown; the affordance and goal supervisions require annotation or automated extraction pipelines whose cost is not fully quantified in the abstract; and direct comparisons are limited to the reported counterpart models within each family.

Industry Implications

For companies building VLA- or WAM-based manipulation products, GIFT offers a training-time recipe that improves zero-shot transfer and robustness to unseen perturbations without redesigning the action head or losing pretrained priors. Because the gains concentrate on articulated objects and high-precision tasks — exactly the hard cases in real deployments — the approach is a credible lever for closing the gap between visual foundation models and reliable physical manipulation.