acceptodds
Withdrawn from ICLR 2027

CoReDrive: Actor-Conditioned Planning with Counterfactual Response Learning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Autonomous driving requires anticipating the behavior of other road users and adapting ego plans accordingly, but driving logs provide supervision only for observed interactions. We present CoReDrive, which couples actor perception, multimodal motion prediction, motion-derived intent, and ego planning through a structured actor interface. To teach responses beyond the logged interaction, we edit an actor's predicted futures while holding the observation and current scene geometry fixed. A geometric teacher constructs sets of admissible ego responses instead of treating a synthetic trajectory as a unique expert. Response-set targets supervise candidate retention and ranking, while paired factual and edited worlds supervise an interaction-score correction used during velocity admission and final selection. On NAVSIM, CoReDrive achieves 90.8% and 91.4% corrected extended Predictive Driver Model Score with ResNet-34 and ViT-L, respectively. A matched ResNet-34 comparison shows a gain from counterfactual response supervision. Controlled actor-information and future-edit visualizations examine how the planner uses its interaction interface in selected scenes.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Autonomous Driving, Counterfactual Learning, Motion Planning, Representation Learning, Imitation Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_sRrtOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces CoReDrive, an agent-conditioned autonomous driving planner that employs a counterfactual response supervision mechanism. The method characterizes agents by leveraging detected geometric information, multimodal future trajectories, and motion-inference-based intentions.

During training, the system constructs counterfactual interaction scenarios—without requiring additional sensor rendering—by modifying the predicted future trajectory of a specific agent while keeping current observations and scene geometry intact. Since the modified scenarios lack "expert-level" ego-vehicle trajectories, the system utilizes a "geometric teacher" mechanism to generate a set of feasible ego-vehicle responses rather than a single pseudo-expert trajectory.

This set of responses guides the filtering and final ranking of candidate trajectories; simultaneously, paired factual and counterfactual geometric risk objectives are used to train an interaction-scoring calibration model, which plays a role in velocity feasibility verification and final trajectory selection. The method has been evaluated on the NAVSIM platform.

Critical strengths
  1. The approach to constructing a “response set” is sound and well-founded. Since edited interactive scenarios typically lack a single “correct” response for the ego vehicle, constructing a set of spatiotemporally feasible trajectories is more reasonable than specifying a single synthetic expert trajectory.

  2. The counterfactual supervision mechanism is computationally practical. Directly editing the predicted future trajectories of actors at the planning interface level avoids the high costs associated with sensor-level re-rendering or full-scene simulation.

  3. Qualitative examples clearly demonstrate conditional behaviors. The paper illustrates how actor information influences ego vehicle planning, showing that changes—such as a lead vehicle braking or a roadside vehicle pulling into the lane—elicit distinct responses from the ego vehicle.

Critical weaknesses
  1. Core Contribution—The quantitative gains from "counterfactual response learning" are negligible. Under the matched ResNet-34 setting, the baseline model achieves an EPDMS of 90.1; introducing "actor modeling/conditioning" raises this to 90.7; however, adding the proposed "counterfactual response learning" results in a marginal increase from 90.7 to only 90.8. Consequently, the majority of the performance improvement relative to the baseline appears to stem from "actor conditioning" rather than the "counterfactual learning" highlighted in the paper. Due to the absence of multiple random seed tests, confidence interval analysis, or statistical significance testing, it is difficult to determine whether this 0.1-point gain is robust.

  2. Lack of systematic quantitative evaluation regarding core "counterfactual capabilities." The method defines various intervention types—such as lead vehicle braking, crossing/cutting in, turning across paths, pulling out from the roadside, obstructed lane changes, and failure to yield. However, the paper reports neither response success rates, collision avoidance performance, nor the recall of feasible responses; nor does it analyze performance variations based on intervention type or severity. Evidence regarding counterfactual capabilities relies primarily on cherry-picked, open-loop qualitative examples, which is insufficient to demonstrate that the proposed training method consistently improves behavioral performance across diverse intervention scenarios.

  3. The counterfactual supervision mechanism remains heavily reliant on manually designed rules. The intervention library and the "geometric teacher" depend on human-defined response requirements, temporal constraints, collision detection, safety buffer margins, kinematic thresholds, risk functions, and quality metrics based on progress assessment. This raises the question: has the model truly learned generalized counterfactual interaction reasoning, or has it merely distilled a set of predefined driving heuristics? Furthermore, the method fails to demonstrate generalization capabilities regarding unseen interaction types.

  4. The "response teacher" is constrained by the existing candidate trajectory support mechanism. ** The formulation of counterfactual interactions remains unidirectional and may deviate from the data manifold. ** The method modifies the future trajectories and intentions predicted for the actor while keeping camera observations and the current geometric state unchanged. Although these modifications are filtered for kinematic plausibility, this does not guarantee consistency with visual cues or the full joint scene distribution. Furthermore, other road users do not react to the ego vehicle's altered behavior. This limits the method's applicability to scenarios involving bidirectional interactions, such as game-theoretic situations, merging, and mutual yielding.

Questions for authors
  1. Is the improvement in the EPDMS metric from 90.7 to 90.8 statistically significant across multiple training seeds? If so, please provide the mean and standard deviation.

  2. Could the authors provide a quantitative evaluation comparing the "Actor-only" model and the "Actor + Response" model specifically on the generated counterfactual scenarios (rather than primarily on the factual NAVSIM benchmark)?

  3. Could the authors report the "response success rate" and "collision/conflict avoidance performance" separately for each intervention family and intervention severity level?

  4. What proportion of the adopted counterfactual scenarios contains at least one feasible response that lies within the retained candidate-path support set?

  5. How does the method handle counterfactual interactions where an appropriate ego response requires a path that was discarded during the initial path-pruning stage?

Reviewer_JzAROpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors propose CoReDrive, an autonomous driving system conditioned on coupled actor perception, multimodal motion prediction and motion-derived intent. With the proposed method, the authors demonstrated better driving performance comparing to the existing methods (SparseDriveV2) by 0.7pp with similar model setting.

Critical strengths

This paper proposes to adding actor-conditioned planning and response supervision for training an autonomous driving system

The authors demonstrated the performance gain with the proposed method on NAVSIM benchmark

Critical weaknesses
  • actor modeling is similar to classic module-wise AV stack where perception, prediction happens before ego motion planning. As E2E driving stack demonstrated better performance and generalizability, it is questionable whether revisiting the decoupled settings are meaningful. While the auhtors demonstrated performance gain from adopting this method, the performance gain are relatively small considering a large model design change introduced here (0.6pp for actor and 0.7pp for actor + response).
  • the response supervision is similar to the counterfactual supervission proposed in Hydra-MDP++. The authors should include some discussions on the comparison to justify the proposed new schema.
Questions for authors
  • the proposed methods focus on actor conditioned planning which is similar to the traditional module-wise AV stack, is there any challenge to adopting this compared to existing methods?
  • The performance gain seems to be incremental given the amount of modification introduced here, it would be beneficial to show either the exceled performance on claimed contribution (i.e. teach the model beyond driving logs behaviors) or its capability to adapt to a larger set of existing methods.
  • The response supervision part is similar to what HydraMDP++ did in the counterfactual supervision part, the authors should add a comparison to demonstrate the key difference.
Reviewer_oGyEOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CoReDrive, an actor-conditioned planning framework that introduces counterfactual response supervision by editing predicted actor futures. It constructs sets of admissible ego responses using geometric constraints and supervises both candidate retention and trajectory selection. Experiments on NAVSIM show improvements over SparseDriveV2, with an additional 0.1 percentage-point EPDMS gain attributable to counterfactual response learning under matched training conditions.

Critical strengths

Novel counterfactual supervision for interactive planning. The paper addresses the limitation of learning only from observed driving interactions by constructing supervision for alternative actor behaviors. The response-set formulation is a meaningful design choice, as it accommodates multiple valid ego reactions without requiring a unique expert trajectory.

Critical weaknesses
  1. Limited empirical gains from the core contribution. Despite the methodological novelty, counterfactual response supervision improves EPDMS only marginally, from 90.7 to 90.8, over the actor-conditioned baseline. This small improvement, together with the lack of systematic quantitative evaluation of counterfactual response quality, provides insufficient evidence to convincingly establish the effectiveness of the proposed approach.

  2. Limited methodological novelty. While the idea of counterfactual response supervision is interesting, the proposed approach largely relies on hand-crafted intervention rules, geometric heuristics, and integration with an existing candidate-based planner. The paper does not convincingly establish a generalizable methodological contribution beyond this task-specific engineering design.

Questions for authors
  1. Methodological novelty: Beyond integrating hand-crafted counterfactual rules into an existing planning framework, what is the fundamental methodological contribution of this work?

  2. Effectiveness: Given the marginal EPDMS improvement (+0.1), what evidence demonstrates that counterfactual supervision meaningfully improves interactive planning?

  3. Generality: How does the proposed approach scale beyond predefined intervention templates and generalize to unseen interactions or different planning architectures?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.