DyPES-VLA: Learning Shared Dynamics Priors and Embodiment-Specific Control for Cross-Embodiment Manipulation

Background

Vision-Language-Action (VLA) models are the dominant recipe for robot manipulation, but nearly all of them are trained for one robot. A generalist policy that works across heterogeneous embodiments — different arms, grippers, camera mounts, and kinematic structures — would let a fleet share one model and amortize data collection. Two obstacles block this today. First, existing methods underuse the dynamics priors shared across diverse visual and interaction data, which limits cross-embodiment transfer. Second, converting embodiment-specific actions into a common format requires extensive manual preprocessing, which does not scale as new robots are added.

Core Innovation

DyPES-VLA decomposes the problem into shared priors and embodiment-specific control. On the prior side, the vision-language model is trained with a future-prediction objective on cross-embodiment data, driving a shared query representation to capture object motion, contact, and interaction-induced scene changes — physical dynamics that generalize across robots. On the control side, an embodiment-specific Mixture-of-Experts (MoE) action head translates those shared priors into executable controls directly in each embodiment’s native action space, so heterogeneous actions never have to be manually pre-aligned into a common format. The head shares attention layers to capture common temporal action structure while embodiment-specific feed-forward experts handle each robot’s kinematic constraints and control semantics.

Results

As a single generalist policy, DyPES-VLA achieves state-of-the-art performance across simulation and real-world evaluations: 98.0% success on LIBERO, 59.25% on RoboCasa-GR1, and 89.02% on RoboTwin 2.0. The cross-embodiment evaluation demonstrates that shared dynamics priors transfer across robots without per-embodiment action alignment, and that the MoE head can be extended to new embodiments without re-training the shared backbone from scratch.

Limitations

The headline numbers come primarily from simulation benchmarks; the abstract mentions real-world evaluation but does not quantify it in the same detail. MoE action heads add parameters and routing complexity that may increase inference cost on resource-constrained robots, and the approach still requires per-embodiment expert data to train each expert. The paper does not detail how performance degrades for embodiments whose data is scarce relative to the shared backbone.

Industry Implications

Cross-embodiment generalist policies are the economic core of robot “foundation models”: one trained model, many robot SKUs. By removing manual action-format alignment and sharing dynamics priors across data sources, DyPES-VLA lowers the cost of adding a new robot to a fleet and improves transfer from data-rich to data-poor embodiments. For robot manufacturers and warehouse/logistics operators running heterogeneous fleets, this line of work points toward a practical path where a single VLA backbone serves multiple hardware platforms with only lightweight embodiment-specific adapters.