Orbis 2: A Hierarchical World Model for Driving

Current world models operate at a single level of abstraction, prioritizing perceptual fidelity while lacking the spatial reasoning and semantic understanding required for real-world tasks. Orbis 2 addresses this by factorizing future prediction across two levels: a high-level predictor that forecasts coarse scene structure over extended temporal horizons, and a low-level generator that produces detailed predictions conditioned on the high-level output.

Core innovation. The paper makes two key contributions. First, a hierarchical architecture that decomposes prediction into distinct temporal and abstraction scales, yielding both high perceptual fidelity and rich spatial-semantic representations. Second, a two-stage training paradigm: pretraining with diffusion forcing produces substantially richer internal representations, while teacher forcing (predicting the next frame from clean context) provides more stable autoregressive rollouts. Combining both — pretrain with diffusion forcing, finetune with teacher forcing — gives the best of both worlds.

Experimental results. Orbis 2 achieves state-of-the-art results across the standard suite of driving world model evaluations, including long-horizon generation fidelity, steering responsiveness on counterfactual scenarios, and internal representation quality. The model significantly outperforms prior single-level world models on established benchmarks. Code, demo, and checkpoints are publicly available.

Limitations. The hierarchical design requires careful tuning of the abstraction boundary between levels. The two-stage training pipeline doubles the training cost compared to single-stage approaches. Evaluation is conducted on established driving benchmarks rather than real-world closed-loop deployment.

Industry implications. High-fidelity world models are a critical component for end-to-end autonomous driving systems. Orbis 2’s hierarchical approach and practical training recipe could influence how driving prediction modules are designed in production systems, particularly for long-horizon planning and simulation-based validation.