Skytopia: Monocular Drone Navigation with Action-Conditioned Latent World Models

Monocular drone navigation is hard for a simple reason: a single forward-facing camera provides almost no direct information about depth or scale, so the policy must infer how the world is arranged from how the image changes. World models are the standard answer — learn how observations evolve under actions, then use that predictive machinery in the loop. But the standard construction commits the system to executing its own predictions: the model produces a future observation at deployment and feeds it back into action generation at every control step. That is expensive, and on a drone, cost is flight time. Skytopia’s argument is that the prediction is not the product; the representation that makes the prediction possible is.

Core Innovation

• The executed action explains the change. In flight, most of the difference between one observation and the next is accounted for by the motion the robot actually commanded, so predicting the future observation reduces to reprojecting a static scene under a known displacement. There is little for a generative predictor to add inside the control loop.
• Action-conditioned latent world model. The policy is trained on top of a latent world model conditioned on action, rather than on reconstructed pixels. The forward objective predicts the representation of the next observation from the intended motion.
• An inverse objective that grounds the representation. A second objective recovers the motion from the predicted transition, which is what ties the learned latent to action rather than leaving it a pure visual encoding.
• The predictor is discarded at deployment. Because prediction never reaches action generation, the world model’s predictive head is dropped at inference time, and the surviving representation drives a single policy that covers point-goal, image-goal, and goal-free navigation.
• A 3D Gaussian Splatting platform for training. A dedicated Gaussian-splatting gym supplies the training environment, giving the policy the visual structure it needs without requiring it to be reconstructed by the world model.

Results

• Success rates: Skytopia outperforms every baseline under all three navigation specifications, reaching 57.8% for point-goal, 66.0% for image-goal, and 49.0% for goal-free navigation.
• Efficiency: discarding the predictor removes 59.4% of inference cost, moving the world model out of the deployment path entirely.
• Real deployment: the same policy is flown on a physical drone without fine-tuning, reaching goals in indoor, open outdoor, and woodland environments.

Limitations

The reported success numbers are simulation results, and the abstract gives no equivalent quantitative figure for the physical deployment — indoor, outdoor, and woodland flights are described as reaching goals without success rates, number of trials, or failure modes. The central argument, that the executed action explains almost all of the change between observations, is asserted as a design premise rather than measured; where it breaks down, such as with moving obstacles, wind, or significant illumination change, is not characterised, and the goal-free setting is also the weakest of the three at 49.0%. The three navigation specifications share one policy, but the abstract does not report whether they were trained jointly or separately, or what the policy costs per decision on the drone’s onboard compute. No comparison is made against a standard executed-prediction world model at equal inference budget.

Industry Implications

Aerial autonomy has a persistent cost structure: the more capable the perception and prediction stack, the heavier the onboard compute, and payload, power, and flight time all pay for it. Demonstrating that a world model can be trained for control and then have its most expensive component removed at deployment is directly relevant to any platform that has to run on embedded hardware — drones first, but also legged and handheld systems with tight power budgets. The reuse of one policy across point-goal, image-goal, and goal-free specifications also points toward a single trained navigation stack rather than a family of task-specific ones. What the field will want next is the honest sim-to-real accounting: the simulation success rates are stated, and the deployment results are qualitative, so the gap between them is the number that will decide how much of this transfers into operational flight.

---