A World Model Built from Moving Particles

Robot learning has spent the last two years borrowing the machinery of video generation. The appeal is obvious: a model that can imagine the next few seconds of a scene looks like a model that understands what actions do. But a video world model is optimised to produce plausible pixels, and most of its capacity goes into texture, lighting and background motion that a controller does not care about. Worse, the output is a 2D image, while the quantities that decide whether a grasp succeeds — contact, penetration, object pose — are three-dimensional.

GPWM starts from the opposite end. Instead of predicting appearance and letting physics be implied, it predicts the motion of particles and rigid bodies directly in 3D space. The scene is a set of points that move; the model’s job is to generate their trajectories. Appearance is not modelled at all.

Core innovation

One representation for four kinds of matter. Rigid objects, deformable bodies, cloth and granular media are usually handled by separate simulators and separate learned models, because their dynamics have different structure. GPWM uses a single object-aware, multi-resolution transformer over particles. Rigid bodies are carried along as coherent groups, while free particles cover everything that flows or deforms. The architecture combines flow matching with diffusion forcing, a pairing that lets the model generate coherent trajectories well past the horizon it was trained on rather than drifting into noise.

Scale, and a test of whether specialisation is still necessary. The authors built ParticleWorlds, a 25.5-million-frame corpus spanning diverse geometries, materials, interactions and robot embodiments. That dataset is the real enabler: a unified model is only interesting if it has seen enough heterogeneity to justify sharing weights. The paper then runs the comparison that matters — a single Unified GPWM against per-domain baselines, and against the authors’ own specialist models. One set of weights largely outperforms the per-domain baselines and beats the specialists in two of the six domains.

Planning by searching noise, not actions. A learned dynamics model is only useful for control if it can be inverted toward a goal. GPWM learns a generative prior over joint actuator-object motion, so planning becomes a search over the model’s latent noise: the optimiser looks for a noise vector whose decoded trajectory moves the object to the desired place. This is a different search space from conventional action-space optimisation, and the paper reports it substantially outperforms action-space search on pushing and picking, where the right contact sequence is hard to discover by local action perturbation.

Results

Across six physical domains, GPWM reduces prediction error by 31% on average relative to the best prior learned-dynamics baseline, and it does so in every domain rather than trading wins. The unified model generalises to unseen geometries, physical properties, object counts and rollout horizons, which is the practical test of whether it learned physics or memorised a training distribution.

The planning results are the more suggestive ones. Plans produced by latent-noise search, executed open-loop in simulation, achieve goal-directed manipulation without reference action trajectories and without a goal-specific policy. That is a meaningfully different way to build a manipulation stack: the world model does the work that a task policy would otherwise do.

Limitations

Everything here is in simulation. There is no real-robot evaluation, and the planning is executed open-loop, with no closed-loop replanning when the real world diverges from the imagined trajectory. Open-loop execution is exactly where model error compounds, so the gap between these results and a deployed system is not small.

The particle representation also carries assumptions the abstract does not examine: scenes must be decomposed into objects and free particles, which implies segmentation or asset knowledge at inference. Granular and cloth regimes are precisely where that decomposition is hardest.

Finally, a 25.5-million-frame training corpus is a serious data and compute undertaking, and the paper does not claim the model runs at control rates. A world model that is accurate but too slow to query inside a control loop is a simulator, not a controller.

Industry implications

The strategic argument is about where robot world models should sit. If a single model can cover rigid, deformable, cloth and granular interaction, then simulation, synthetic data generation and planning stop being three separate stacks with three separate asset pipelines. That consolidation is valuable for any team maintaining a simulator for training and a different one for evaluation.

There is also a data argument. A model that predicts physical outcomes can generate experience in places where teleoperation is expensive or unsafe, and can score candidate plans without running them. Both are direct attacks on the cost of real-world data collection, which remains the binding constraint on manipulation.

The caveat is that the value only materialises when the model is calibrated against a specific real cell. A generic particle world model trained on internet-scale simulation will still need per-deployment identification of masses, friction and actuator delay before its plans can be trusted on hardware.