acceptodds
Under review as a conference paper at ICLR 2027

Planner-Breaker: Finding Informative Failures of Driving Planners with Reinforcement Learning

Abstract

Closed-loop driving benchmarks assess nominal performance among logged or rule-based traffic, but reveal little about a planner's weaknesses. Adversarial methods address this by generating safety-critical situations, but an identified collision is only informative if the planner could have avoided it. We introduce Planner-Breaker, which uses self-play reinforcement learning at scale to train traffic that produces collisions of a chosen form against a given black-box planner. Since this reward does not need to be differentiable, we instantiate it with a counterfactual criterion that tests whether braking in time would have avoided a collision, and whether the danger was foreseeable early enough to react. Evaluated against five rule-based and learned planners, Planner-Breaker produces about three times more avoidable collisions than gradient-based adversaries, which collide about as often or less. A cross-evaluation shows that each adversary exploits weaknesses specific to the planner it was trained on. Our adversaries can be used without retraining as a traffic model in real-world scenes, where they produce more informative collisions than log replay and rule-based traffic. The same method optimizes other failure criteria, such as nuPlan's at-fault rule, allowing practitioners to search their own planner for the failures they care about.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Adversarial Reinforcement Learning, Autonomous Driving, Self-play

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_1zkROpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

Planner-Breaker trains a shared adversarial policy to expose failures of a fixed driving planner. Its reward favors collisions that could have been anticipated and avoided by braking. The method is evaluated against five planners, with additional transfer experiments on WOMD.

Critical strengths

The focus on avoidable and foreseeable collisions is useful for planner evaluation. The reusable policy avoids per-scenario optimization, and the reward and post-collision continuation ablations help explain the training design.

Critical weaknesses
  1. KING and ReGentS are evaluated over 10 seconds, while Planner-Breaker can run for up to 256 seconds. Table 11 reports a median collision time of 10.5 seconds, so this difference is consequential. Results within a common evaluation horizon are needed to establish the size of the improvement.

  2. The proposed criterion is both a central contribution and the main evaluation measure. However, it is unclear whether its counterfactual braking maneuver is executable under the constraints of every tested planner. Appendix A.4 describes braking at the maximum deceleration, while Appendix B.1 defines jerk-limited actions for the learned policies. Matching the maximum deceleration alone does not establish matching stopping behavior. If the counterfactual bypasses these dynamics, some collisions may be labeled avoidable even though the tested policy could not execute the required maneuver. The counterfactual also fixes the other vehicles to their recorded motion and then uses constant-velocity extrapolation, while ignoring certain secondary collisions based on an assumed response from following vehicles. These are consequential modeling choices. The paper does not establish how often they change the collision labels. Similarly, foreseeability depends on the response time and geometric safety margins, without a sensitivity analysis or an independent assessment of the resulting labels. I would find the claims more convincing if the authors checked a representative set of failures using a dynamically feasible avoidance controller and reported agreement with the proposed criterion.

  3. The main adversarial comparisons are limited to KING and ReGentS, both trajectory-optimization methods. The paper discusses related learned approaches but does not include a representative learned adversary in the experiments. SEAL is relevant because it generates reactive adversarial behavior, while STRIVE is particularly relevant to the claim that generated failures should admit a safe solution. These methods have different assumptions and need not all be reproduced, but the current selection leaves an important gap. A simpler RL adversary trained with the same collision criterion would also help distinguish the benefit of the criterion from that of the multi-agent training design. The reward ablations are useful, but do not resolve this question. The WOMD experiment does not close this gap: it compares against log replay and nominal traffic policies, without adversarial generation baselines. It therefore supports transfer to different road layouts and initial conditions, but provides limited evidence about comparative effectiveness in that setting.

  4. The cross-planner results support specialization, but do not explain which planning weaknesses the adversaries exploit. I missed concrete examples showing the original interaction, the point at which the hazard becomes foreseeable, the last feasible braking time, and the collision-free counterfactual. Such examples are important for assessing the meaning of “informative” failures. Table 17 mainly describes behavior averaged over entire episodes. Similar average speeds and goal-reaching rates do not establish that the maneuvers immediately preceding a collision are plausible. The paper would benefit from collision-centered behavior analysis and a breakdown of failure types, showing whether the method discovers varied planning deficiencies or repeatedly exploits a small set of maneuvers.

I lean toward weak rejection. The motivation is sound, but the unequal evaluation horizons and insufficient validation of the failure criterion leave the main conclusions inadequately supported.

Questions for authors

Does the advantage over KING and ReGentS persist when all methods are evaluated within the same 10-second horizon?

How do the methods compare in computational cost and planner queries, accounting for both adversarial-policy training and subsequent scenario generation?

Does counterfactual braking obey the same execution constraints as the tested planners, particularly the jerk limits of SP and SP-Cond?

How sensitive are the collision labels and method rankings to the assumed reaction time, braking deceleration, and lateral safety margin?

The driving statistics summarize entire episodes. Do the adversaries still exhibit plausible behavior immediately before collisions, or do they exploit unrealistic coordinated maneuvers?

Statistics such as average speed and the number of times the target was reached indicate that, overall, the opposing vehicle still exhibited normal driving behavior. However, this does not sufficiently demonstrate that the critical seconds immediately preceding the collision aligned with real-world traffic patterns.

Reviewer_GFFEOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Planner-Breaker, which uses reinforcement learning to train surrounding vehicles to expose failures of a fixed black-box driving planner. The method rewards collisions according to whether earlier braking could have prevented them and whether the danger was foreseeable in time to react. Experiments across five rule-based and learned planners report more avoidable collisions than the evaluated gradient-based methods. The learned policies exploit weaknesses specific to their target planners and can be applied without retraining to simulated scenes based on real-world driving data. The framework also supports alternative collision criteria.

Critical strengths
  1. The paper is well organized, with clear descriptions of the motivation, proposed method, and experimental setup. The definitions of avoidability and foreseeability are presented clearly.

  2. The paper addresses an important problem in autonomous driving evaluation. Identifying collisions that a planner could have foreseen and avoided can reveal weaknesses that conventional benchmarks may fail to expose.

Critical weaknesses
  1. KING and ReGentS optimize trajectories over 10 seconds, whereas Planner-Breaker is evaluated in episodes lasting up to 256 seconds. A comparison over the same duration is needed to assess the reported improvements.

  2. The counterfactual test does not model surrounding vehicles’ responses to braking, and foreseeability depends on a fixed response time and geometric rules. Independent validation and sensitivity analyses are needed to assess the reliability of these classifications.

  3. The ablations lack a variant that rewards all avoidable collisions equally. This comparison would isolate the additional contribution of the foreseeability reward.

  4. The WOMD experiments compare only with log replay, IDM, and SP traffic. Comparisons with adversarial baselines are needed to determine whether the reported advantage extends to WOMD scenes.

Questions for authors

Please refer to the questions and concerns raised in the weaknesses above.

2 comments

Thank you for the detailed response and the additional experiments, which address most of my concerns. One remaining question: in Table 28, the foreseeable rate of KING decreases as ρ grows, while that of Planner-Breaker peaks at the training value ρ = 1 s on several targets. It would be helpful if the authors could briefly explain this. A small manual check of the collision labels would also make the criterion more convincing. Overall, I maintain my score.

Thank you for your reply. Our foreseeability test requires the danger to be visible ρ before the last braking time that avoids the collision, and checks it in a short time window around that time for robustness to flicker. It does not require the danger to be sustained until the collision, so foreseeable rates do not necessarily vary monotonically with ρ. This can make sense, since a vehicle flagged as dangerous at the check should lead the planner to be cautious enough to avoid the following collision, even if the danger is no longer visible afterwards.

We see the decrease of the foreseeable rate of KING with ρ as an artifact of how KING and ReGentS work. They minimize the distance between the selected adversary and the target, and stop at the first collision, which mostly creates trajectories where the selected adversary races towards the target. The danger then becomes more and more visible as the collision approaches, so it is less often visible when the check happens earlier, with a larger ρ.

Our adversaries were instead trained specifically with ρ = 1 s, and the peak at this value is expected, since Planner-Breaker optimizes the criterion it is given, including its response time. They learnt patterns where the danger is visible at that check, before they overtake the target and cut in (rows 2 to 4 of Figure 2). A check requiring the danger to persist would likely be less sensitive to ρ, and would be an interesting addition to the paper.

The avoidability test is robust its parameters (Table 28). Your suggested variant, which rewards all avoidable collisions equally (Table 26), shows that Planner-Breaker adapts to this criterion, with avoidable collisions in 60 to 88 % of the episodes, while distance-based methods and the variant of Planner-Breaker rewarded for any collision create mostly unavoidable collisions. We plan to give more space to this variant in the main text to show how Planner-Breaker adapts to the criterion, and how different criteria yield more or less specialization to the planner.

Regarding the manual check, Figure 2 (Appendix D.6) shows twelve examples, handpicked for their readability, with a brief explanation of what the planner did wrong. An agreement study, where humans or vision-language models label a random sample of collisions, would be interesting future work.

Thank you again for your review and discussion, which helped us think more deeply about the foreseeability criterion.

Reviewer_7z56OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

Planner-Breaker trains a shared adversarial traffic policy with PPO against a fixed black-box driving planner. The central idea is to reward collisions according to a counterfactual braking-based avoidability test and a geometric foreseeability test, rather than rewarding every collision. Target-focused initialization and post-collision continuation support learning. Experiments against five rule-based or learned planners examine failure rates, target specialization, alternative rewards, and zero-shot deployment on 10,000 WOMD scenes. The results suggest that targeting an explicit informativeness criterion produces substantially more useful failures than unconstrained collision seeking.

Critical strengths
  1. The paper makes the quality of failures an explicit optimization target and tests why that matters. The any-collision ablation produces very high collision rates dominated by unavoidable collisions, while the proposed objective increases foreseeable failures. Against PDM, for example, the foreseeable rate changes from 1.47% for any-collision training to 53.43% for the proposed objective (Table 14). The at-fault experiments further show that conventional attribution and counterfactual avoidability need not agree. These experiments support a useful methodological distinction rather than merely reporting more crashes.

  2. The evaluation probes mechanism and transfer with meaningful replication. Three training seeds, three evaluation seeds, 10,000 episodes per run, the full cross-target matrix, and the post-collision-continuation ablation provide substantial evidence. Transfer to logged WOMD initial states/maps, without retraining, tests a setting beyond the target-focused CARLA initialization. The reproducibility package also supplies training code, target policies, maps and experiment overrides, although several artifacts required to reproduce the final comparisons remain external.

Critical weaknesses
  1. The main comparison confounds adversarial effectiveness with attack duration and does not quantify amortized cost. Sections 4.1/B.4 give KING and ReGentS 10 seconds, whereas Planner-Breaker can interact for up to 256 seconds. Table 11 reports a pooled collision-time median of 10.5 seconds and a 95th percentile of 89 seconds. Therefore, a substantial fraction of its collisions occur beyond the baselines' window, and the reported 2.7–3.8× avoidable-failure advantage cannot currently be attributed solely to the proposed method. The explanation that longer joint optimization degraded the baselines is relevant, but does not replace reporting Planner-Breaker at the same 10-second horizon. Please show cumulative avoidable/foreseeable failure rates versus time on matched initial scenes, and distinguish offline training cost, planner queries, and online scenario-generation cost. A rolling-horizon baseline would further help, but the first essential check is the equal-duration comparison.

  2. The operational informativeness criterion needs independent validation before supporting broad claims about actionable planner weaknesses. The authors acknowledge braking-only testing and geometric foreseeability, which is helpful. A further issue is that the counterfactual freezes other vehicles' recorded trajectories, then extrapolates the colliding adversary at constant velocity, although the trained adversarial policy is reactive to the ego. It also ignores some follower collisions by invoking a different, responsive-follower assumption. Moreover, the counterfactual brakes at 5 m/s² while the learned targets use jerk-integrated dynamics. These choices may change whether a collision is labelled avoidable or foreseeable. Sensitivity to response time, braking ramp, lateral margin and reactive versus frozen traffic would establish that the gain reflects robustly actionable failures rather than specialization to one evaluator. Optimizing the same criterion used for evaluation is legitimate; the concern is its unvalidated relationship to the broader interpretation.

Questions for authors
  1. At a common 10-second horizon and matched initial states, what are the avoidable and foreseeable collision rates for every target? Does the reported relative advantage remain? Please provide the cumulative curves, preferably from the existing rollouts.
  2. How stable are the labels and method rankings when counterfactual braking respects the learned targets' jerk bounds, when response time/lateral margins vary, and when surrounding vehicles react to the changed ego trajectory?
  3. Could you provide representative original and counterfactual trajectories for high-reward attacks, together with failure-type diversity and attack-window plausibility measures? The aggregate driving statistics in Table 17 do not characterize the attacks themselves.
  4. What are the total target-planner query counts and cost to generate a fixed number of informative failures, including the 20B-step pretraining cost? Can the reported results and KING/ReGentS configurations be supplied, since they are excluded from the reproduction snapshot?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.