LaPla: Continuous Actions from Discrete Minds — Latent-Aligned Planning for End-to-End Autonomous Driving

· Editorial Team estimated
autonomous-driving vision-language-action planning world-model end-to-end

LaPla is a unified Vision-Language-Action (VLA) framework with latent-aligned planning that grounds semantic understanding in precise motion execution. It builds an action tokenizer from a residual vector-quantized VAE that captures vehicle kinematics, then repurposes that representation as a continuous physical prior rather than a discrete codebook: concurrent action queries attend to multimodal context in a single forward pass and project hidden states directly into the pretrained latent space, where a frozen decoder translates them into actions — eliminating quantization error and bypassing autoregressive generation. On nuScenes, LaPla reduces long-horizon L2 error by 15.52% versus state-of-the-art VLA methods; closed-loop evaluation in NVIDIA AlpaSim improves success rate by 33.34 percentage points with significantly lower inference latency.

Paper · arXiv:2609.04070

Bridging discrete, language-like reasoning in Vision-Language Models with the continuous, physics-constrained nature of driving remains a core challenge for end-to-end autonomy. LaPla’s premise is that action tokens should behave as a continuous physical prior — not as discrete codes — so that semantic understanding can be grounded directly in smooth, executable motion.

Core Innovation

  • Residual VQ-VAE action tokenizer as a physical prior — vehicle kinematics are captured in a structured latent space, but instead of discrete codebook lookups, the representation is repurposed to bridge high-dimensional semantics and raw actions.
  • Latent-aligned planning in one forward pass — concurrent action queries causally attend to multi-view images, historical actions, and textual instructions, projecting hidden states into the pretrained latent space.
  • Frozen decoder, no autoregression — continuous latents are decoded into physically plausible trajectories, eliminating quantization errors and bypassing time-consuming autoregressive generation.

Results

  • nuScenes open-loop: long-horizon L2 error reduced by 15.52% compared to state-of-the-art VLA methods.
  • NVIDIA AlpaSim closed-loop: success rate improved by 33.34 percentage points with significantly reduced inference latency and smoother driving progress.

Limitations

Open-loop results on nuScenes and closed-loop results in the AlpaSim simulator are reported, but the abstract does not present real-world closed-loop driving; the approach inherits the usual dependence on the quality of the pretrained VQ-VAE latent space and the chosen simulator’s fidelity; and long-horizon performance beyond the evaluated benchmarks remains to be demonstrated.

Industry Implications

Autonomous driving stacks are beginning to adopt VLA architectures, but latency and trajectory quality have kept them out of safety-critical loops. LaPla shows a concrete route to decode continuous, physically plausible trajectories from discrete-minded models in a single forward pass — the combination of lower inference latency and a 33-percentage-point closed-loop success gain is precisely the kind of evidence that matters for production planning stacks and for future foundation-model-based driving systems.