acceptodds
Under review as a conference paper at ICLR 2027

FlowPlan: Flow-Matching Trajectory Planning Guided by Counterfactual Interaction Consequences

Abstract

Modeling inter-agent interactions and their evolution in complex traffic scenarios remains a fundamental challenge in autonomous driving planning. Despite advances in ego-conditioned prediction and joint prediction-planning, it remains difficult to reliably characterize how changes in ego behavior affect critical conflicts and to select effective interaction responses based on these effects. To address this problem, we propose FlowPlan, a flow-matching trajectory planning framework guided by counterfactual interaction consequences. Its Counterfactual Interaction Consequence Model (CF-ICM) evaluates complete ego-trajectory queries under the same observed scene and predicts object- and time-resolved proximity and contact outcomes. Coupled lateral-longitudinal probes use these predictions to construct branch-specific consequence memories and fixed strategy conditions that guide learned corrections to the flow velocity field. By reevaluating selected intermediate trajectory estimates, FlowPlan updates these memories as generation proceeds while retaining a fixed strategy condition within each branch. Completed candidates are reevaluated and selected using predicted interaction consequences, feasibility, and driving quality. Experiments on the large-scale real-world nuPlan benchmark demonstrate strong closed-loop planning performance in both non-reactive and reactive settings. We further introduce nuPlan-CIP (nuPlan Counterfactual Interaction Perturbation Benchmark), which applies controlled perturbations of varying interaction intensity while preserving the underlying scene context. Results on nuPlan-CIP show consistent responses to controlled interaction perturbations, complementing conventional planning metrics with a diagnostic assessment of interaction-dependent behavior.

Primary area:
applications to robotics, autonomy, planning
Keywords:
autonomous driving, trajectory planning, counterfactual reasoning, flow matching

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_jJd8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper puts forward FlowPlan, a flow-matching-based trajectory planning method for autonomous driving guided by counterfactual interaction consequences. Its core module, CF-ICM, works by taking different candidate ego trajectories and evaluating how each one affects surrounding objects over time, with outcomes sorted into three tiers: free clearance, near-gap, and contact. The method then uses coupled lateral–longitudinal behavior probes to construct strategy conditions and dynamic consequence memories, which steer the flow matching process during trajectory generation and candidate selection. For validation, the authors run experiments on the nuPlan closed-loop benchmark, and they also propose a new benchmark called nuPlan-CIP to test how planners respond to interaction perturbations. The final results show that FlowPlan achieves good performance across multiple reactive driving scenarios.

Critical strengths
  1. The interaction-aware generation idea is well motivated and useful. FlowPlan does more than simply generate trajectories with Flow Matching: it explicitly models object- and time-level interaction consequences with CF-ICM and uses this information to guide the generation process through dynamically updated consequence memories. This makes the planner more directly aware of interaction risk during generation, rather than only evaluating risk after trajectories have already been produced.

  2. The evaluation is fairly comprehensive and includes a useful diagnostic benchmark. Besides the standard nuPlan closed-loop evaluation, the paper introduces nuPlan-CIP to test whether the planner reacts appropriately to controlled changes in interaction intensity. The improvements, especially in reactive scenarios, provide useful evidence that the proposed consequence modeling helps the planner adapt to changing interactions.

Critical weaknesses
  1. The “counterfactual” claim is not fully validated. CF-ICM is trained on logged trajectories together with alternative ego trajectories generated in reactive simulation, so its predictions may depend quite strongly on the simulator and its response model. The paper does not provide enough analysis of how well these predicted consequences transfer under different interaction dynamics or simulator mismatch. This matters because CF-ICM is the main signal used to guide trajectory generation.

  2. The interaction space and diagnostic evaluation are still somewhat limited. The method uses nine manually defined combinations of lateral and longitudinal behaviors, which may not cover more complex or continuous interaction strategies. In addition, nuPlan-CIP is based on 140 base scenes, with only 46 conflict-increasing and 21 conflict-removing cases retained. This makes it harder to judge how broadly the reported interaction-aware behavior generalizes.

Questions for authors
  1. Clarification on the “counterfactual” formulation. The paper describes CF-ICM as modeling counterfactual interaction consequences, but in practice it seems closer to predicting outcomes for alternative ego trajectories generated through reactive simulation. Could you clarify what exactly “counterfactual” means in this paper, and how CF-ICM differs from the interventional behavior prediction framework of Tang et al. (2022)? This would help me better understand the novelty of the formulation.

  2. Clarification on baseline configurations. GameFormer and PLUTO are reported in their “without refinement” versions. Could you explain why these variants were chosen instead of the full published models? If available, results for the full versions would also help assess the fairness of the comparison.

  3. Clarification on sensitivity to inference settings. The memory refresh schedule and candidate budget appear to be fixed design choices. Have you tested different refresh frequencies or different numbers/compositions of trajectory candidates? It would be useful to know how sensitive the final performance is to these settings.

  4. Clarification on the robustness of nuPlan-CIP results. The conflict-removing arm contains only 21 valid scenes, which is much smaller than the other intervention groups. Could you provide any uncertainty estimates, confidence intervals, or robustness checks to show that the observed trends are stable despite the limited sample size?

1 comment
AuthorsOpenReview ↗

Response to Reviewer jJd8

We thank the reviewer for the thoughtful comments. We have strengthened the evaluation of interaction robustness, clarified the role of counterfactual consequences in trajectory generation, reconstructed nuPlan-CIP with complete paired intervention groups, and added sensitivity studies of the memory-refresh schedule and candidate-trajectory budget.

W1. Reactive-agent robustness. Thank you for highlighting the dependence on reactive agents. We added a SMART evaluation following Hagedorn et al., replacing IDM with learned SMART traffic agents while keeping the trained FlowPlan model unchanged. FlowPlan scores 85.81 on Val14-SR versus 87.22 with IDM, a drop of 1.41 points. These results support its robustness across different traffic policies. Appendix C.9 (Table 10) reports this evaluation alongside published IDM/SMART reference results.

W2. Interaction-space coverage. Thank you for raising the coverage concern. The nine intention probes construct consequence memories; they are not the final executable trajectories. FlowPlan generates continuous trajectories from Gaussian noise through its learned flow field. The default pool includes nine intention-associated and six free-motion candidates, allowing trajectories beyond the predefined probes. Section 3.3 and Appendix A.5 clarify this distinction. We also added candidate-budget sensitivity in Appendix C.8 (Table 9) and rebuilt nuPlan-CIP with complete paired groups, as detailed below.

Q1. Counterfactual formulation. Thank you for raising this connection. Tang et al. (2022) use interventional modeling to avoid predictions that assume surrounding agents already know the ego's full future plan. CF-ICM instead learns interaction consequences from executed alternatives: from the same observed scene, we execute different ego trajectories in reactive simulation and record their consequences as other agents respond. Here, "counterfactual" asks what would happen if the ego took an action absent from the original log. CF-ICM predicts object- and time-resolved free-clearance, near-gap, and contact outcomes. During planning, differences in these predicted consequences guide trajectory generation through strategy conditions and consequence memories. We were inspired by Tang et al.'s distinction between observing an ego future and executing an action, and acknowledge this in Section 2.1. Sections 3.2–3.3 and Appendix A.4 describe our formulation and supervision.

Q2. Baseline configurations. Thank you for raising the question of baseline fairness. We compare learning-based planning without additional rule-based refinement or external trajectory optimization, following previous studies, including Diffusion Planner (ICLR 2025) and Diffusion Forcing Planner (CVPR 2026). Full GameFormer adds nonlinear trajectory optimization; full PLUTO adds simulation-based, rule-driven candidate evaluation and selection. We use their published without-refinement variants for consistency with these comparisons. FlowPlan incorporates learned consequences into generation and subsequently selects completed candidates, without an external optimizer modifying their geometry. Section 4.1 specifies the baseline variants.

Q3. Inference sensitivity. Thank you for suggesting these studies. We added both memory-refresh and candidate-budget sensitivity on Val14-R. With the same trained model, eight flow steps, and 15 candidates, fixed memory scores 86.86; refreshing after steps 2, 4, and 6 gives 87.05, 87.22, and 87.13. The default refresh after step 4 gains 0.36 points over fixed memory. All refresh settings share the final candidate-selection procedure; only the timing of memory updates changes. This isolates the contribution of refreshing consequence memory during generation. Candidate budgets of 6, 9, 15, and 24 score 86.69, 87.12, 87.22, and 87.23, respectively. Moving from 6 to the default 15 candidates gains 0.53 points; increasing from 15 to 24 adds 0.01 points. Appendix C.8 (Tables 8 and 9) reports both studies. Section 4.4 summarizes the refresh result, and Appendix A.5 details the default configuration.

Q4. nuPlan-CIP robustness. Thank you for identifying the small intervention subsets. We rebuilt nuPlan-CIP as 140 complete four-world groups across all 14 scenario categories: native, conflict-increasing, conflict-reducing, and irrelevant-actor control. This gives 560 matched instances. The conflict-reducing arm now contains 140 scenarios rather than 21, and every intervention is paired with its own native scene. FlowPlan has the smallest conflict-increasing score drop: 1.14 points, compared with 2.05 for Physics extrapolation and 5.13 for Scene-only. It gains 0.77 points under conflict-reducing interventions, preserves its score under irrelevant-actor changes, and achieves the highest CIR-Gap of 0.79. Section 4.5 (Table 4) and Appendices C.1 and C.5 (Table 6) present the reconstruction and paired results.

Reviewer_fzwBOpenReview ↗
Rating
3
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes FlowPlan, which uses CF-ICM to predict the interaction consequences of candidate ego trajectories. These consequences guide flow-matching trajectory generation and are dynamically refreshed during the generation process. The framework is novel because it moves generative planning beyond merely producing plausible behavior toward generating and refining behavior according to the predicted interaction consequences of candidate actions.

Critical strengths

First, interaction consequences are incorporated into the trajectory generation process. The method is clearly designed and has meaningful research value.

Second, alternative trajectories are paired with consequence labels obtained from the same simulation execution, making the construction of the supervision signal reasonable.

Last, the closed-loop results are competitive and demonstrate the method’s ability to balance safety and driving progress.

Critical weaknesses
  1. The interpretation of Table 4 is potentially problematic. Since the different intervention groups contain different numbers of scenes, differences in their mean scores cannot be directly interpreted as performance degradation caused by the intervention.

  2. The paper does not sufficiently discuss the applicability conditions and limitations of CIR-Gap. Consequently, the statement that the method “recovers 83% of the loss” is not sufficiently clear.

  3. The paper lacks direct validation of CF-ICM prediction accuracy and an isolated ablation comparing dynamic memory refresh with fixed memory. Therefore, the independent contributions of the individual mechanisms have not been fully established.

Questions for authors
  1. After re-pairing each intervention arm with the native results from the same scenes, do the performance drops and the ranking of the methods remain unchanged?

  2. Is the denominator of CIR-Gap strictly positive for every planner and every scene? How are zero, negative, and near-zero denominators handled?

  3. Under otherwise identical conditions, how much additional benefit does dynamic memory refresh provide compared with using fixed memory?

1 comment
AuthorsOpenReview ↗

Response to Reviewer fzwB

We thank the reviewer for the careful assessment of our evaluation methodology. We have reconstructed nuPlan-CIP for paired comparisons, clarified the CIR-Gap calculation, and added an isolated ablation of dynamic consequence-memory refresh.

W1 / Q1. Paired evaluation on nuPlan-CIP. Thank you for pointing out the issue with comparing averages from different scenario subsets. To address this, we have reconstructed nuPlan-CIP so that every base scenario contains all four traffic worlds: native, conflict-increasing, conflict-reducing, and irrelevant-actor control. The revised benchmark contains 140 base scenarios and 560 matched scenario-world instances, covering all 14 scenario categories. All planners are evaluated on the same base scenes across the four worlds. With the new paired evaluation, the score reductions under conflict-increasing interventions are: Scene-only: 5.13 points (86.47 to 81.34). Physics extrapolation: 2.05 points (87.82 to 85.77). FlowPlan: 1.14 points (88.12 to 86.98). The ranking remains consistent, with FlowPlan achieving the highest score and the smallest performance degradation. Under conflict-reducing interventions, FlowPlan improves from 88.12 to 88.89, while its score remains unchanged under irrelevant-actor perturbations. The revised results are presented in Section 4.5 (Table 4), with the matched-world construction and score differences detailed in Appendices C.1 and C.5 (Table 6).

W2 / Q2. CIR-Gap definition and denominator handling. Thank you for raising this question. We have clarified the validity conditions and calculation of CIR-Gap. During nuPlan-CIP construction, we replay the recorded ego trajectory to audit each intervention. A conflict-increasing candidate must reduce the minimum ego–target clearance by at least 0.25 m. If a candidate fails this condition, we continue searching for another target vehicle or intervention. For CIR-Gap, we use each planner’s own native trajectory as the reference. We measure the clearance loss by replaying this trajectory under the conflict-increasing intervention, and then measure how much clearance is recovered when the planner is allowed to replan. Polygon overlap is assigned zero clearance. The recovery therefore refers to ego–target clearance, rather than the closed-loop planning score. We explicitly verified the planner-specific clearance-loss denominator for every evaluated scene and planner. Across all 140 scenarios and three planners (420 planner-scene instances), all denominators are strictly positive. Therefore, no samples are excluded due to zero or negative denominators. For near-zero positive denominators, we cap the recovery ratio at 1, preventing disproportionately large values from exceeding 100% recovery. With this calculation, FlowPlan achieves a mean CIR-Gap of 0.79, compared with 0.61 for Physics extrapolation and 0.37 for Scene-only. We describe the construction audit in Section 4.5 and Appendix C.3, and the planner-specific CIR-Gap calculation in Section 4.5 and Appendix C.6.

W3 / Q3. CF-ICM prediction quality and dynamic memory refresh. Thank you for suggesting direct validation of CF-ICM and an isolated comparison between fixed and dynamically refreshed consequence memory. We have also added direct CF-ICM prediction evaluation on Val14. Full CF-ICM achieves 0.913 near-event AP and 0.683 contact AP, compared with 0.704 and 0.297 for the logged-only variant using the same architecture and training schedule. These results are reported in Section 4.3 (Figure 5) and Appendix C.7 (Table 7). We have added a controlled ablation on Val14-R. All variants use the same trained model, eight flow steps, 15 trajectory candidates, and final candidate-selection procedure. The only change is whether and when the consequence memory is refreshed. Keeping the initial memory fixed achieves 86.86. Refreshing after steps 2, 4, and 6 achieves 87.05, 87.22, and 87.13, respectively. The default refresh after step 4 obtains the highest score, improving performance by 0.36 points over fixed memory. These results demonstrate the benefit of updating interaction consequences during generation, allowing the flow generator to incorporate new interaction information as the trajectory evolves. Section 4.4 summarizes the memory-refresh result, and Appendix C.8 (Table 8) reports the full schedule ablation. The controlled comparison is described in Appendix B.2.

Reviewer_9Kk1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes FlowPlan, a flow-matching planner for nuPlan guided by counterfactual interaction consequences. Joint prediction–planning and ego-conditioned predictors trained only on real logs never see unexecuted maneuvers. FlowPlan's Counterfactual Interaction Consequence Model (CF-ICM) is also trained on reactive-simulation rollouts. It predicts object–time free-clearance, near-gap, and contact probabilities for complete ego trajectories. Coupled intention probes query the consequence model to build strategy conditions and object–time consequence memories that guide flow decoding. Completed candidates are then re-evaluated for selection. The authors also introduce nuPlan-CIP, a benchmark of controlled interaction perturbations.

Critical strengths

S1. Counterfactual interaction guidance

The paper conditions flow generation on the object–time interaction consequences of alternative ego maneuvers, allowing counterfactual-aware trajectory generation. This design is well motivated. FlowPlan performs better than all the learning-based planners in the comparison in reactive traffic, and removing it lowers Val14-R from 87.22 to 85.16 [Table 1, Table 3].

S2. Controlled interaction evaluation with nuPlan-CIP

The authors introduce nuPlan-CIP, which perturbs a single reactive actor in a controlled way while keeping the scene context unchanged. The benchmark shows that FlowPlan scores higher and also adapts to a newly induced conflict. It stays stable when the conflict is removed or an irrelevant actor changes.

Critical weaknesses

W1. Limited qualitative evidence on reactive interactions

The qualitative analysis covers only one scene, which mostly contains static objects (e.g., traffic cones). This is not enough to support the paper's central claim [Figures 2–4]. Static objects do not respond to the ego, so none of the examples shows how a dynamic agent's response changes the driving outcome. A comparison with the baselines in scenes with reactive vehicles is essential.

W2. Missing evaluation of consequence prediction reliability

The paper does not report held-out prediction quality or a calibration analysis for the consequence model, although Appendix A.6 fits a temperature on held-out data. Since these probabilities directly screen and rank candidates (Eqs. 4–5), their reliability is central to the method. It seems necessary to make a statistical comparison of predicted and realized contact and near-gap rates across many scenes, for each probe and object type. This analysis would confirm whether the predictions remain reliable for unexecuted maneuvers and reactive vehicles.

Questions for authors

Q1. Reactive agents used for training rollouts

Appendix A.4 does not name the reactive agent used for the training rollouts. If it is IDM, the consequence model is trained and evaluated on the same agent policy. Please specify the agent and, if possible, compare with a variant trained only on logged pairs.

Q2. Target selection and clearance thresholds

Appendix C.4 sorts candidates by clearance reduction but does not say which audited candidate becomes the target. Also, the z = + audit accepts post-intervention clearances up to 10 m. Please explain the selection rule and report how many arms fall below the 1 m near-gap threshold in Appendix A.4.

Other questions are included in the Critical Weaknesses section.

1 comment
AuthorsOpenReview ↗

Response to Reviewer 9Kk1

We thank the reviewer for the constructive feedback. We have added a reactive-interaction case, direct CF-ICM evaluation, and clearer training and intervention protocols.

W1. Reactive-interaction examples. Thank you for highlighting the need for moving-vehicle interactions. We have added a nuPlan-CIP case with a controlled speed intervention on a surrounding vehicle. FlowDrive's score drops from 90.63 to 76.54, while FlowPlan achieves 90.05 by adjusting its trajectory to the changed interaction. The new case is presented in Section 4.5 (Figure 7). Our original Val14 traffic-cone case and its probe and CF-ICM analyses remain in Section 4.2 (Figures 2–4), illustrating consequence-guided planning under both static obstacles and dynamic vehicle interactions.

W2. CF-ICM prediction reliability. Thank you for pointing out the importance of evaluating CF-ICM against realized outcomes, especially for alternative ego behaviors. We have added a direct prediction evaluation on Val14, comparing CF-ICM outputs with the realized consequences of the corresponding ego queries in reactive simulation. These queries include alternative maneuvers beyond the logged ego behavior, allowing us to evaluate predictions for actions that were not executed in the original driving logs. We evaluate both near-event (near-gap or contact) and contact prediction. Full CF-ICM achieves 0.913 near-event AP and 0.683 contact AP. In addition to the precision–recall curves, we present predicted probability distributions grouped by the realized outcomes: distant, near-only, and contact. The distributions show that CF-ICM assigns higher contact probabilities to maneuvers that actually result in contact, while assigning low contact probabilities to distant interactions. We also retain the temperature-scaling procedure fitted on a held-out calibration split. These analyses directly connect the predicted interaction consequences to their realized outcomes and demonstrate the model's ability to distinguish different levels of interaction risk. The PR curves are reported in Section 4.3 (Figure 5), with the AP results and event-conditioned probability distributions in Appendix C.7 (Table 7 and Figure 8). The temperature-scaling procedure is described in Appendix A.6.

Q1. Training-rollout agents and logged-only comparison. Thank you for requesting this clarification. Training rollouts use the standard IDM agents in nuPlan's reactive (R) simulation, as specified in Section 4.1 and Appendix A.4. Following your suggestion, we trained a logged-only CF-ICM with the same architecture and number of epochs, and evaluated both models on Val14 using the same protocol. Logged-only training achieves 0.704 near-event AP and 0.297 contact AP, compared with 0.913 and 0.683 for full CF-ICM: gains of 0.209 and 0.386 AP. Logged expert trajectories rarely contain collisions. Reactive alternative-trajectory rollouts provide adverse interaction outcomes, helping CF-ICM learn the consequences of potentially unsafe behaviors. The comparison appears in Section 4.3 and Appendix C.7.

Q2. Intervention target selection and clearance thresholds. Thank you for identifying the ambiguity. We have reconstructed nuPlan-CIP as 140 complete four-world groups across all 14 scenario categories. We jointly select conflict-increasing and conflict-reducing interventions, prioritizing opposite speed changes on the same vehicle and choosing the pair with the largest contrast in clearance changes. If no such pair is available, we consider different targets. The resulting benchmark has 93 groups sharing the same target across the two interventions. We also distinguish construction criteria from the 1 m near-gap threshold. A conflict-increasing intervention must reduce minimum polygon clearance by at least 0.25 m, with post-intervention clearance at most 15 m. A conflict-reducing intervention must increase it by at least 0.25 m, with native clearance at most 15 m. These criteria establish the intended change in interaction intensity; the 1 m threshold defines near-gap events separately. In the fixed-ego construction audit, 47 of the 140 conflict-increasing interventions and one of the 140 conflict-reducing interventions reach target clearance below 1 m. Section 4.5 describes construction and target selection, with the audit illustrated in Figure 6. Appendices C.1–C.4 give the detailed rules and clearance statistics (Table 5).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.