Collision Snapshot Guided Time-Reversed Safety-Critical Scenario Generation

· Editorial Team estimated
autonomous-driving scenario-generation safety-critical Waymo trajectory-prediction

COSTER is a framework for generating safety-critical traffic scenarios by leveraging learned traffic priors to determine plausible collision times and locations. It constructs a collision snapshot by inserting a new vehicle in contact with the target vehicle at an identified collision state, then uses a conditional variational autoencoder to perform a time-reversed rollout that reconstructs the inserted vehicle's trajectory backward toward earlier timesteps. Experiments show COSTER outperforms existing methods in plausibility, diversity, and data efficiency; agents trained on COSTER-generated scenarios reduce collision rates by 31% on safety-critical scenarios from the Waymo Open Motion Dataset while also improving ego task completion.

Paper · arXiv:2609.06433

Autonomous vehicle safety validation depends on exposure to safety-critical traffic scenarios, but generating them is hard. Prior approaches typically perturb the trajectories of existing agents using simplified adversarial objectives, which limits plausibility and diversity; inserting new adversarial vehicles helps but leaves the question of when and where to introduce them open. COSTER answers that question by learning plausible collision states from traffic priors and reasoning backward from the collision itself.

Core Innovation

COSTER (COllision Snapshot guided Time-E-Reversed scenario generation) first constructs a collision snapshot: a new vehicle is inserted in contact with the target vehicle at an identified collision state within a traffic scenario. Starting from that snapshot, a conditional variational autoencoder performs a time-reversed rollout, reconstructing the inserted vehicle’s trajectory backward toward earlier timesteps. Because generation is anchored to a concrete, plausible collision event rather than to a generic adversarial gradient, the resulting scenarios are both safety-critical and realistic.

Results

  • Outperforms existing methods in plausibility, diversity, and data efficiency.
  • Agents trained on COSTER-generated scenarios reduce collision rates by 31% on safety-critical scenarios from the Waymo Open Motion Dataset.
  • Ego task completion is also improved, indicating that generated scenarios sharpen rather than distort learned driving behavior.

Limitations

The evaluation of scenario quality relies on learned plausibility/diversity metrics and downstream agent performance, which are useful but indirect proxies for real-world relevance; the abstract does not report closed-loop vehicle-in-the-loop or on-road validation. The approach’s dependence on well-defined collision states and traffic priors for the target environment is not quantified. Full comparisons against perturbation baselines and ablations require the complete paper.

Industry Implications

Safety-critical scenario libraries are the backbone of AV testing and training, and their scarcity and repetitiveness are a recognized bottleneck in the industry. A generator that produces plausible, diverse near-miss scenes at scale — and demonstrably improves downstream safety metrics on real datasets such as Waymo Open Motion — is directly useful to AV companies building scenario databases for simulation testing, closed-loop training, and regulatory safety cases. The collision-snapshot formulation also offers a tractable way to target specific, rare event types that fleets rarely encounter organically.