acceptodds
Desk rejected at ICLR 2027

EvoAD: Self-Evolving Autonomous Driving via Latent World-Model Guided Post-hoc Reflection

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

End-to-end autonomous driving systems have achieved rapid progress by learning unified policies that map sensor observations directly to driving trajectories. However, once deployed, these systems typically remain frozen: they may repeat similar decisions in similar scenes, and failures can only be addressed through costly data collection and retraining. This sharply contrasts with human drivers, who improve lifelong through experience, memory, and reflection. In this work, we ask whether autonomous driving systems can achieve similar post-deployment self-evolution to human drivers. We propose EvoAD, a self-evolving autonomous driving framework via latent world-model guided post-hoc reflection. EvoAD stores driving experiences in a bird's-eye-view feature memory pool, retrieves scene-relevant precedents according to geometric and semantic similarity, and uses a latent world model to evaluate the future consequences of candidate trajectories in a compact planning-oriented space. Instead of treating memory as external textual contexts or using world models only for pre-action simulation, EvoAD transforms executed driving outcomes into reusable improvement signals for future decisions. This enables the system to diagnose how prior trajectories could have been improved and to refine behavior in similar future scenarios without retraining the base driving policy. Experiments on NAVSIM v1 and v2 show consistent improvements across reflection rounds and state-of-the-art performance, demonstrating the potential of world models as reflective mechanisms for post-deployment self-evolution in autonomous driving.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Autonomous Driving, Self-Evolution, World Models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_okwbOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes EvoAD, which combines a latent world model, a trajectory evaluator, and retrieval-guided refinement to improve driving decisions without retraining the base planner. On NAVSIM, performance improves over repeated reflection rounds, reaching 93.4 PDMS on v1 and 91.5 EPDMS on v2; seperate experiments test memory transfer across data splits.

Critical strengths

The paper presents a coherent pipeline that combines latent future prediction, trajectory evaluation, and retrieval-guided refinement to improve driving decisions without retraining the base planner. Ablations, cross-split memory tests, evaluations across different planners provide useful evidence of the framework's effectiveness and applicability.

Critical weaknesses
  1. Although the paper compares EvoAD with a GTRS-based baseline, it does not isolate the contribution of the latent world model features within the proposed FAE. An ablation that keeps the FAE unchanged while removing candidate-conditioned future latents would clarify how much of the gain comes specifically from world model predictions.

  2. The evaluation lacks validation in a fully interactive closed-loop simulator. NavHard uses a two-stage pseudo-simulation protocol with pre-generated future observations, rather than a fully interactive closed-loop simulator. Evaluation in a simulator such as HUGSIM would better test whether EvoAD's gains persist when observations and surrounding-agent states evolve in response to the ego vehicle's actions.

Questions for authors
  1. Could the authors evaluate EvoAD in HUGSIM using a fixed memory built from NAVSIM navtrain or navval, without incorporating feedback from HUGSIM evaluation scenes? This would test whether the gains transfer to a fully interactive closed-loop setting.
  2. Since same-scene refinement uses each navtest scene’s own PDMS feedback from a previous round, using navtest memory raises a potential test-set leakage concern. Would it be more appropriate for Table 1 to report results using navtrain/navval memory as the primary results?
  3. Figures 1 and 2 be revised for clarity. Figure 1's vehicle placement appears awkward, while Figure 2 is too dense and small to read easily.
1 comment
AuthorsOpenReview ↗

Response to Reviewer

W1

Thank you for your comment. To isolate the contribution of the latent world model features within FAE, we supplement an ablation that removes the future feature input from FAE and retrain the evaluator from scratch under the same training setup. As shown in Table A below, without future latent predictions, the variant reaches 89.92 PDMS at R1 and 92.44 PDMS at R5. Adding future latents in the main FAE raises PDMS by 0.66 points at R1 and 0.92 points at R5. These future latents help FAE better distinguish high-quality candidates beyond static scene and trajectory inputs alone, yielding consistent planning gains. We have added this ablation to the revised manuscript.

Table A: Ablation of future latents in FAE.

FAE SettingR1 PDMSR5 PDMS
FAE w/o Future Features89.9292.44
Main FAE, Correctly Matched Future Latents90.5893.36

Revised Contents: Please see p. 19 (lines 1023–1025) and p. 20 (lines 1026–1035).

W2

Thank you for your suggestion; we add a new experiment to test EvoAD on HUGSIM. Since HUGSIM provides only camera observations, we adapt WoTE to image-only inputs and retrain the latent WM on NAVSIM visual features. During closed-loop evaluation, all model weights remain frozen while execution feedback updates the memory across rounds.

As shown in Table B, EvoAD raises HD-Score from 47.6 at R1 to 65.4 at R5 (+17.8 points), outperforming the 40.2 WoTE baseline. These results suggest that EvoAD's multi-round refinement remains effective under closed-loop execution. We have included these results in the revised manuscript.

Table B: Multi-round closed-loop evaluation on HUGSIM.

MethodR1R2R3R4R5
WoTE40.240.240.240.240.2
EvoAD47.654.459.263.165.4

Revised Contents: Please see p. 9 (lines 435–439), p. 13 (lines 689–693), p. 20 (lines 1072–1079), and p. 21 (lines 1080–1085).

Q1

Thank you for your suggestion. To test whether the gains from a fixed NAVSIM memory transfer to a fully interactive closed-loop setting, we evaluate EvoAD on HUGSIM using a read-only memory bank built from navtrain. All model weights remain frozen during closed-loop evaluation, with no HUGSIM trajectories or feedback incorporated; EvoAD retrieves the pre-built navtrain memory bank for similar driving scenarios to perform single-pass PAR refinement and FAE selection.

As shown in Table C, EvoAD achieves 55.7 HD-Score, outperforming the 40.2 WoTE baseline by 15.5 points. These results suggest that offline experiences from NAVSIM can effectively transfer to an interactive closed-loop environment without target-scene feedback. We have included this experiment in the revised manuscript.

Table C: Cross-domain single-pass evaluation on HUGSIM.

MethodHD-Score
WoTE40.2
EvoAD with Fixed navtrain Memory55.7

Revised Contents: Please see p. 9 (lines 437–439), p. 20 (lines 1073–1079), and p. 21 (lines 1080–1085).

Q2

Thank you for your comment. EvoAD is developed to empower a pre-trained frozen planner with post-deployment self-evolving capability. Therefore, the five-round evaluation in Table 1 aims to verify continual improvement across rounds. While cross-domain evaluation using navtrain or navval memory also achieves competitive results, this setting tests memory generalization rather than continual self-evolution. We therefore retain the multi-round results as the primary results in Table 1.

Regarding your concern about potential test-set leakage, we clarify that EvoAD accumulates and leverages past experiences without using them to update the parameters of the pretrained models. Such experience-driven approaches have been widely explored in autonomous driving, such as DiLu [1] and Drive Like a Human [2], where accumulated experiences are used to improve driving performance.

[1] Wen, Licheng, et al. "Dilu: A knowledge-driven approach to autonomous driving with large language models." International Conference on Learning Representations. Vol. 2024. 2024.

[2] Fu, Daocheng, et al. "Drive like a human: Rethinking autonomous driving with large language models." 2024 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW). IEEE, 2024.

Revised Contents: Please see p. 1 (lines 29–31), p. 8 (lines 378–393), p. 13 (lines 699–701), and p. 15 (lines 804–809).

Q3

Thank you for pointing this out. In the revised manuscript, we have adjusted the vehicle placement in Figure 1. We have also enlarged the text and simplified the layout in Figure 2.

Revised Contents: Please see p. 2 (lines 54–67) and p. 3 (lines 108–128).

Reviewer_DWYSOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

EvoAD studies how historical driving feedback can improve subsequent decisions while keeping the base driving model frozen. It refines decisions using candidate-conditioned future latent representations, post-execution feedback, and retrieved scene memories. Experiments on NAVSIM v1, NAVSIM v2, and NavHard report planning gains through multiple rounds of reflection, along with memory transfer across planners and dataset splits.

Critical strengths

The world model generates a conditioned future latent representation for each candidate trajectory, allowing the evaluator to use candidate-specific future information and directly supporting planning.

Ablations report round-by-round results for the full model and variants with individual modules removed, showing when gains arise rather than only comparing final performance.

Memory capacity and module latency are quantified, with CPU and GPU retrieval times reported separately, helping assess deployment resource requirements.

Critical weaknesses

Section 4.1 and Appendix C.3 explicitly state that multi-round results are obtained by repeatedly running benchmark scenarios and using feedback from previous executions. Thus, the 93.36 PDMS result is achieved after accessing historical scores from the target scenarios, whereas the conventional baselines in Table 1 are mainly evaluated under a single-pass protocol. The cross-split experiments provide evidence of transfer, but do not establish that the fifth-round result outperforms single-pass methods under comparable evaluation conditions.

The continued improvement without the refiner reveals a key gap in the algorithm description. In Table 7, FAE-only improves from 90.58 to 92.76 after removing PAR; removing both FAE and PAR still yields an improvement from 89.05 to 90.87. Under Eqs. (4)–(10) and Eq. (16), repeated scoring would not normally keep changing the selected trajectory if the observations, model parameters, and candidate set remain fixed. The paper does not explain how candidates are updated or how memory influences decisions in these settings.

The text following Eq. (10) describes the ranking objective as a maximum-likelihood pairwise loss, whereas Appendix Eq. (18) defines a margin-based hinge loss. These descriptions should be reconciled. Eqs. (5)–(6) also do not specify how an (8\times3) trajectory and two 256-dimensional features are organized into attention tokens. The tensor shapes in Table 13 remain insufficient to reconstruct this operation.

Appendix A states that 18,179 scenarios are reserved as navval, whereas Table 5 lists 80,854 training scenarios and 4,255 validation scenarios. The relationship between these splits is unclear, making data separation difficult to verify. Moreover, 40 pairs per scenario across 85,109 scenarios would yield 3,404,360 pairs, but Table 5 reports a total of 4,044,612. The current description does not explain this discrepancy.

The effects of retrieval, feedback, and candidate expansion are not adequately separated. Controls using random retrieval or shuffled feedback are missing. In Table 6, cross-scenario single-pass prediction with feedback scores 92.22, slightly below 92.26 without feedback. The results therefore do not establish that the gains primarily come from targeted historical feedback rather than the refiner itself or a larger candidate pool.

The results across planners do not yet demonstrate transfer of the complete decision pipeline. The second-round results in Table 3 average the scores of four refined candidates, primarily measuring candidate generation quality rather than the trajectory ultimately selected and executed. The appendix further notes that none of the four planners improves in the second round when refined candidates are selected using each baseline’s own scorer. This suggests that better candidates yield benefits only if the evaluator can identify them. The paper should distinguish the transferability of the refiner from that of the complete system.

Questions for authors
  1. Fair comparison between multi-round evaluation and single-pass baselines. Section 4.1 and Appendix C.3 state that multi-round results are obtained by repeatedly running benchmark scenarios and using previous execution feedback. Thus, the 93.36 PDMS result uses historical scores from the target scenarios, whereas the conventional baselines in Table 1 mainly follow a single-pass protocol. Please clarify whether historical feedback from the target scenarios is available during multi-round evaluation, how it is incorporated, and whether the cross-split experiments follow exactly the same protocol as the main results.

  2. Mechanism behind continued improvement without the refiner. In Table 7, FAE-only improves from 90.58 to 92.76 after removing PAR; removing both FAE and PAR still improves performance from 89.05 to 90.87. Under Eqs. (4)–(10) and Eq. (16), repeated scoring would not normally keep changing the selected trajectory if observations, model parameters, and candidates remain fixed. Please explain how the candidate set changes, how memory influences decisions, and why performance continues to improve without the refiner. Pseudocode and control experiments using frozen candidate sets, repeated scoring, and no memory would help clarify the mechanism.

6 comments
AuthorsOpenReview ↗

Response to Reviewer Part 1

W1

Thank you for your comment. EvoAD is developed to empower a pre-trained frozen planner with post-deployment self-evolving capability. For conventional baselines running repeatedly on the same scenarios, such as WoTE, their planning output remains unchanged because they are frozen after training. By contrast, EvoAD can enable the system to continually improve planning performance across rounds using past experiences. Therefore, the five-round evaluation in Table 1 is designed to test this post-deployment self-evolution capability.

In cross-split experiments, our results suggest that EvoAD can enhance planning in unseen scenes. Specifically, we construct a fixed memory bank from navtrain and evaluate EvoAD on navtest scenarios. As shown in Figure 4, even without any target-scenario historical feedback, EvoAD still achieves clear gains in a single pass using experiences from another domain. In this setting, EvoAD retrieves experiences from similar scenarios only from external memory, refines candidates via PAR, and selects the trajectory via FAE in one pass. As shown in Table A below, EvoAD reaches 92.56 PDMS on unseen navtest scenarios, outperforming the 88.33 PDMS WoTE baseline by 4.23 points under identical single-pass conditions.

In the revised manuscript, we have added a "Protocol" column (Single-Pass vs Multi-Round Self-Evolving) to Table 1, relabeled the cross-domain "R2" results as "Cross-Domain Single-Pass" in the tables, and explicitly specified the post-deployment self-evolving setting in the abstract and contributions.

Table A: Cross-domain single-pass comparison on navtest.

MethodProtocolTarget-Scenario Historical FeedbackPDMS
WoTE BaselineSingle-PassNo88.33
EvoAD (Fixed navtrain Memory)Single-PassNo92.56

Revised Contents: Please see p. 1 (lines 29–31), p. 2 (lines 89–91), p. 8 (lines 378–393), p. 16 (lines 856–863), and p. 17 (lines 864–878).

AuthorsOpenReview ↗

Response to Reviewer Part 2

W2

Thank you for pointing this out. In both ablation settings of Table 7, the base planner candidates, current observation, and model parameters remain fixed, whereas the active candidate pool updates from Round 2 onward through memory injection.

1) Removing PAR only (FAE-only, 90.58 to 92.76 PDMS). Round 1 has no target-scenario execution experience and evaluates only the initial candidates from the base planner. From Round 2 onward, instead of generating four PAR-refined trajectories, the memory module directly injects four stored historical trajectories from the previous round's memory snapshot into the candidate pool. These four trajectories consist of the target scenario's stored best-so-far trajectory and three stored trajectories retrieved from similar scenarios, adding the same number of candidates to the base pool as the full model. For trajectory selection, SF first filters the updated candidate pool, and FAE ranks the remaining candidates to select the final trajectory. As higher-scoring historical trajectories enter the candidate pool across rounds, FAE selects better trajectories and raises PDMS from 90.58 to 92.76.

2) Removing both FAE and PAR (89.05 to 90.87 PDMS). The candidate pool updates across rounds as when removing PAR only (FAE-only): Round 1 evaluates only the base candidates, and Rounds 2–5 append the four memory-retrieved trajectories from the previous round's memory snapshot. For trajectory selection, all four EvoAD variants in the upper block of Table 7 retain GTRS/SF. When FAE is removed, GTRS/SF scoring directly selects the final trajectory from the updated candidate pool rather than reverting to WoTE's native scorer. Scoring the memory-expanded candidate pool with GTRS/SF therefore raises PDMS from 89.05 to 90.87 across rounds.

To further address your concern, we run control experiments for both settings by disabling memory read/write, fixing the Round 1 candidate set and candidate order, and repeating scoring across rounds under identical observations and model weights. This control simultaneously covers a frozen candidate set, repeated scoring, and no memory. As shown in Table B below, updating candidates via memory retrieval improves PDMS by 2.18 points when removing PAR only (FAE-only) and 1.82 points when removing both FAE and PAR. By contrast, fixing the Round 1 candidates keeps all intermediate rounds and R5 identical to R1 at 90.58 and 89.05 PDMS. These results indicate that multi-round gains in both settings come from historical trajectory reuse in the candidate pool rather than repeated scoring on fixed candidates.

Table B: Control experiments without PAR on navtest.

VariantCandidate Pool Across RoundsR1 PDMSR5 PDMS
w/o PARUpdate Retrieved Candidates90.5892.76
w/o FAE & PARUpdate Retrieved Candidates89.0590.87
w/o PARFix R1 Candidates & Repeat Scoring (No Memory)90.5890.58
w/o FAE & PARFix R1 Candidates & Repeat Scoring (No Memory)89.0589.05

Revised Contents: Please see p. 7 (lines 337–344), p. 8 (lines 406–410), and p. 17 (lines 897–912).

W3

Thank you for pointing this out. FAE training uses the pairwise margin-based hinge loss (Appendix Eq. 18), where the ranking margin is proportional to the ground-truth PDMS difference between paired trajectories. We will update the text following Eq. (10) to match Appendix Eq. (18).

To clarify how trajectories and features are organized into attention tokens in Eqs. (5)–(6), Table C below lists the exact operations and tensor shapes across FAE stages for batch size B and K candidates per scenario. We will add these operations and tensor shapes to Eqs. (5)–(6) and Table 13 in the revised manuscript.

Table C: Token organization and tensor shapes in FAE.

StageOperation and Tensor Shape
Trajectory EncodingFlatten the 8 × 3 trajectory to 24 dimensions and project via MLP to B × K × 256
State & Future EncodingBoth ego-state and future embeddings have shape B × K × 256
Intra-Candidate AttentionStack the three embeddings into B × K × 3 × 256, reshape to BK × 3 × 256, apply attention fusion, and pool to B × K × 256
Scene Cross-AttentionUse the candidate sequence as queries to read B × 1024 × 256 scene context
Inter-Candidate Self-AttentionModel candidate relations via self-attention on B × K × 256
Output HeadsPredicted ranking scores: B × K × 1; predicted six sub-scores: B × K × 6; predicted total PDMS: B × K × 1

Revised Contents: Please see p. 5 (216–219 and 247–251) and p. 14 (720–746).

AuthorsOpenReview ↗

Response to Reviewer Part 3

W4

Thank you for your comment. We clarify the dataset split hierarchy and the two-stage trajectory pair construction below. We have added these details to Appendix A and Appendix B.3 (Table 6) of the revised manuscript.

Dataset splits. The full navtrain split contains 103,288 scenarios, of which 18,179 are held out as navval for external memory validation. The remaining 85,109 scenarios form the development set, which is further divided into 80,854 training scenarios and 4,255 validation scenarios for PAR and FAE. These splits are mutually exclusive, ensuring no scenario overlap between navval and the PAR/FAE training and validation sets.

Trajectory pair counts. The 40 pairs per scenario across 85,109 scenarios refer to the initial sampling stage, yielding 3,404,360 pairs. In a second stage, we construct 640,252 additional local trajectory pairs targeting weak sub-metric improvements. Together, the two stages yield 4,044,612 pairs in total, comprising 3,842,297 training pairs and 202,315 validation pairs after scenario splitting.

Revised Contents: Please see p. 13 (lines 694–697) and p. 15 (lines 756–780).

W5

Thank you for your comment. To separate the effects of retrieval relevance, feedback alignment, and candidate expansion, we conduct random retrieval and shuffled feedback control experiments under a fixed candidate pool size. These results suggest that relevant retrieved experiences and matched feedback contribute to planning improvements beyond candidate pool size alone.

Control settings. All control groups use the same base planner candidates, number of added candidates, PAR model, and FAE evaluator. Random retrieval samples a record from the same memory bank as the original retrieval method and retains that record's trajectory and matched feedback. This controlled experiment is applied separately in the same-domain and cross-domain settings. Shuffled feedback pairs the retrieved trajectory with feedback from a random unrelated scenario. As shown in Table D below, original retrieval outperforms random retrieval by 1.56 PDMS in the same-domain setting and 1.71 PDMS in the cross-domain setting, showing that relevant experiences provide stronger trajectory priors for PAR. In the same-domain setting, matched feedback also outperforms masked feedback by 0.12 PDMS and shuffled feedback by 0.43 PDMS, indicating that trajectory-aligned feedback provides an additional benefit beyond trajectory retrieval alone.

Cross-domain feedback. In the cross-domain setting, retrieved feedback scores execution in the source scenario rather than the target scenario. Masking this feedback removes the source-specific score condition while retaining the retrieved trajectory prior and target-scene features for PAR refinement. Under this setting, masked feedback reaches 92.26 PDMS compared with 92.22 PDMS using source feedback, whereas shuffled feedback drops to 92.19 PDMS.

Table D: Retrieval and feedback control experiments in PAR.

SettingSame-Domain R2 PDMSCross-Domain Single-Pass PDMS
Original Retrieval, Matched Feedback92.0592.22
Original Retrieval, Masked Feedback91.9392.26
Original Retrieval, Shuffled Feedback91.6292.19
Random Retrieval, Matched Feedback90.4990.51

Revised Contents: Please see p. 17 (lines 869–872) and p. 20 (lines 1037–1052).

Reviewer_s1zSOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes EvoAD, a self-evolving autonomous driving framework that aims to improve the planning performance of pretrained end-to-end driving models through post-hoc reflection without retraining the base planner. The framework maintains an episodic memory of historical driving experiences, including scene representations, selected trajectories, future latent features, and evaluation feedback. A pretrained latent world model predicts the future scene representations conditioned on candidate trajectories, while a Future-Aware Evaluator (FAE) ranks these candidates based on their predicted outcomes. In subsequent reflection rounds, a Planning-Aware Refiner (PAR) retrieves relevant historical experiences and generates feedback-conditioned trajectory corrections.

Critical strengths
  1. The paper addresses a meaningful limitation of existing end-to-end autonomous driving systems: their inability to adapt planning decisions based on accumulated driving experience after deployment without retraining. Introducing post-hoc reflection and episodic memory provides an interesting direction for improving the adaptability of pretrained driving models. The problem is practically relevant and offers a perspective that differs from conventional one-shot trajectory prediction and planning.
  2. EvoAD integrates episodic memory, latent future prediction, and feedback-conditioned refinement into a unified post-hoc reflection framework. Despite relying on established techniques, this integration offers an interesting approach to experience-driven planning improvement without retraining the base planner.
  3. The experiments demonstrate substantial improvements over WoTE on NAVSIM v1 and v2 across reflection rounds. Comprehensive ablations, cross-planner evaluations, and qualitative results provide empirical support for the framework's effectiveness and its individual components.
Critical weaknesses
  1. Fairness and Generalization of Multi-Round Reflection The NAVSIM experiments perform multiple reflection rounds on the same benchmark scenarios, allowing R5 to benefit from accumulated test-scenario feedback, making it difficult to assess generalization to entirely unseen scenarios. In contrast, conventional baselines typically perform single-pass inference without such feedback, creating unequal evaluation conditions. Although intended to simulate long-term self-evolution, this protocol does not clearly separate scenario-specific adaptation from genuine improvements in planning capability. Consequently, the reported R5 performance may overestimate real-world generalization and lead to potentially unfair comparisons with existing baselines.

  2. Insufficient Isolation of the World Model's Contribution The paper highlights latent world-model guided reflection as a key contribution, but the world model is directly reused from the frozen WoTE module. The proposed FAE is essentially a trajectory scoring network trained on these latent features. Thus, the performance gains may stem from both the future dynamics information provided by the world model and FAE's learned ability to approximate PDMS, while their individual contributions remain unclear. Although the experiments demonstrate FAE's effectiveness, they do not sufficiently isolate the contribution of the world model itself or establish whether future dynamics modeling is essential to the improvements.

  3. Limited Novelty and Unclear Feedback Mechanism of PAR PAR combines established techniques, including attention, FiLM modulation, residual prediction, and gating, with limited architectural novelty. Moreover, the experiments do not sufficiently demonstrate that PAR uses historical feedback to identify specific planning failures and perform targeted corrections. Since PAR is trained on low- and high-quality trajectory pairs constructed using PDMS and its sub-scores, it may primarily learn a score-conditioned trajectory regression mapping rather than a genuine feedback-driven reflection mechanism.

Questions for authors
  1. Table 3 contains a highlighting error: WoTE achieves a higher TTC score (96.46) than iPad (96.38), but the iPad value is incorrectly highlighted in bold.

  2. In Table 6, CFG sweep generates four candidates for FAE selection, while conditional and unconditional predictions generate only one. Moreover, feedback-conditioned prediction slightly underperforms unconditional prediction in the cross-domain experiment (92.22 vs. 92.26 PDMS). Could the authors provide candidate-count-matched experiments to isolate CFG's contribution from multi-candidate selection and clarify its robustness to unreliable historical feedback?

  3. Tables 2 and 7 show that PDMS increases from 89.05 at R1 to 90.87 at R5 even without FAE and PAR. However, since PAR is primarily responsible for memory-guided trajectory refinement, the source of these multi-round improvements remains unclear. Could the authors clarify how Memory affects trajectory generation or selection without PAR and what changes across reflection rounds? Additionally, Table 7 shows that retaining PAR without FAE yields lower R5 performance than removing both modules, which requires further explanation.

  4. Table 3 demonstrates that PAR can improve the planning results of TransFuser, DiffusionDrive, WoTE, and iPad. Among these methods, iPad achieves higher original scores than WoTE. Could the authors explain the rationale for selecting WoTE as the primary base planner and clarify how different planners are adapted to the shared latent world model, FAE, and PAR? Furthermore, the current cross-planner experiments only report R2 performance and do not establish whether the complete multi-round reflection mechanism can consistently improve performance across other base planners. Additional experiments evaluating the full multi-round process on different planners would strengthen the generalization claims.

4 comments
AuthorsOpenReview ↗

Response to Reviewer Part 1

W1

Thank you for your comment. EvoAD improves generalizable planning capability, rather than relying solely on scenario-specific adaptation to previously seen scenes. In unseen scenarios, EvoAD can retrieve similar experiences from memory to improve current planning, without access to target-scenario historical information.

In cross-domain experiments (Section 4.2, Figure 4), our results suggest that EvoAD can enhance planning in unseen scenes. Specifically, we construct a fixed memory bank based on navtrain, and then evaluate EvoAD on unseen navtest scenarios without any target-scenario historical feedback. In this setting, EvoAD only retrieves experiences from similar scenarios in the external memory, refines candidates via PAR, and ranks them via FAE using candidate-conditioned future latent representations in a single pass. As shown in Table A below, EvoAD reaches 92.56 PDMS, outperforming the 88.33 PDMS WoTE baseline by 4.23 points under identical single-pass conditions.

In the revised manuscript, we have added a "Protocol" column (Single-Pass vs Multi-Round Self-Evolving) to Table 1, relabeled the cross-domain "R2" results as "Cross-Domain Single-Pass" in the tables, and explicitly specified the post-deployment self-evolving setting in the abstract and contributions.

Table A: Cross-domain single-pass comparison on navtest.

MethodProtocolTarget-Scenario Historical FeedbackPDMS
WoTE BaselineSingle-PassNo88.33
EvoAD (Fixed navtrain Memory)Single-PassNo92.56

Revised Contents: Please see p. 1 (lines 29–31), p. 2 (lines 89–91), p. 8 (lines 378–393), p. 16 (lines 856–863), and p. 17 (lines 864–878).

W2

Thank you for your comment. Below, we provide two controlled experiments to clarify how essential these future features are to EvoAD and how FAE relies on accurate matching between candidates and future representations:

1) Removing future features and retraining FAE. We remove the future-feature input from FAE and retrain the evaluator from scratch under the same training setup. As shown in Table B below, this no-future variant reaches 89.92 PDMS at R1 and 92.44 PDMS at R5. Adding matched future latent representations in the main FAE raises PDMS by 0.66 points at R1 and 0.92 points at R5. These predicted future features help FAE better distinguish high-quality candidates beyond static scene and trajectory inputs alone, yielding consistent planning gains.

2) Shuffling future latent representations at inference. We fix the trained FAE weights and shuffle the pairing between candidates and future latent representations within each scene at inference. As shown in Table B below, shuffling drops both R1 and R5 to 77.41 PDMS and removes the multi-round gain. Mismatched future features degrade FAE's ranking of both base and refined candidates.

We have added both controlled experiments to the revised manuscript.

Table B: Ablation of future latents in FAE.

FAE SettingR1 PDMSR5 PDMS
FAE w/o Future Features89.9292.44
Main FAE, Future Latents Shuffled at Inference77.4177.41
Main FAE, Correctly Matched Future Latents90.5893.36

Revised Contents: Please see p. 19 (lines 1023–1025) and p. 20 (lines 1026–1035).

AuthorsOpenReview ↗

Response to Reviewer Part 2

W3

Thank you for your comment. In EvoAD, PAR serves as the post-hoc refinement module that uses retrieved trajectories and historical feedback to identify planning deficiencies and perform targeted corrections. To demonstrate how PAR uses historical feedback for targeted refinement, we supplement a feedback ablation experiment and a case analysis below.

1) Feedback ablation experiments. Under a fixed candidate pool size, we compare three controlled experiments: mismatching sub-scores while keeping the retrieved trajectory and total PDMS fixed, masking overall feedback, and shuffling overall feedback. As shown in Table C below, in the same-domain setting, matched feedback outperforms sub-score mismatching by 0.42 points, masked feedback by 0.12 points, and shuffled feedback by 0.43 points. These results suggest that PAR is sensitive to accurate historical feedback and that aligned feedback directly contributes to effective trajectory refinement.

Table C: Feedback control experiments in PAR.

SettingSame-Domain R2 PDMS
Original Retrieval, Matched Feedback92.05
Original Retrieval, Masked Feedback91.93
Original Retrieval, Matched Total PDMS with Mismatched Sub-Scores91.63
Original Retrieval, Shuffled Feedback91.62

2) Case analysis on targeted sub-score correction (Figure 5(a)). Figure 5(a) in Section 4.3 directly illustrates how PAR repairs specific weak sub-metrics across rounds. At R1, the trajectory has a collision risk with TTC = 0, EP = 97, and 57.2 PDMS. From R1 to R2, PAR first targets the failed TTC metric and raises TTC from 0 to 100 and PDMS to 92.4, while EP drops from 97 to 82. From R2 to R5, conditioned on the lower EP sub-score, PAR subsequently targets driving progress and recovers EP from 82 to 95 while maintaining TTC at 100, reaching 98.4 PDMS. This progression illustrates that PAR can identify specific planning failures at each round and perform targeted corrections.

Revised Contents: Please see p. 9 (lines 443–454) and p. 20 (lines 1037–1052).

Q1

Thank you for pointing this out. We have corrected the bold highlighting for the best TTC score in Table 3 from iPad (96.38) to WoTE (96.46) in the revised manuscript.

Revised Contents: Please see p. 8 (lines 395–404).

Q2

Thank you for your comment. To isolate the contribution of CFG from multi-candidate selection and examine performance under unreliable historical feedback, we provide candidate-count-matched experiments and feedback-ablation analyses. Specifically:

1) Candidate-count-matched comparison (4 candidates per method). For each fixed-strength baseline at g = 0 or g = 1, we use four random seeds, each generating one trajectory, for a total of four candidates per scene. The initial trajectory, PAR checkpoint, and FAE evaluator remain identical to Table 6. The experimental results are shown in Table D below. The CFG sweep reaches 92.61 PDMS in the same-domain setting and 92.44 PDMS in the cross-domain setting, outperforming four-candidate unconditional prediction by 0.71 and 0.21 PDMS, and outperforming four-candidate conditional prediction by 0.54 and 0.25 PDMS. These results indicate that the gain of the CFG sweep in Table 6 arises from sweeping across multiple guidance strengths rather than merely increasing the candidate count.

Table D: Candidate-matched comparison across CFG settings.

Generation Method (4 Candidates)Same-Domain R2 PDMSCross-Domain Single-Pass PDMS
Unconditional, g = 0, Four Random Seeds91.9092.23
Conditional, g = 1, Four Random Seeds92.0792.19
CFG Sweep, g ∈ {0.5, 1, 1.5, 2} (Table 6)92.6192.44

2) Cross-domain comparison (92.22 vs 92.26 PDMS) and performance under unreliable feedback. In the cross-domain setting, retrieved feedback is inaccurate for the current test scene, which explains why masking it out (g=0g=0) reaches 92.26 PDMS compared with 92.22 PDMS without masking (g=1g=1). Regarding unreliable historical feedback, our ablation results in Table C (Response to W3) show that mismatching sub-scores or shuffling feedback reduces PDMS from 92.05 to 91.63 and 91.62, respectively, as inaccurate feedback misguides PAR's conditional refinement.

Revised Contents: Please see p. 16 (lines 856–863), p. 17 (lines 864–878), and p. 20 (lines 1037–1052).

AuthorsOpenReview ↗

Response to Reviewer Part 3

Q3

Without PAR, the memory module still injects previously stored trajectories into the candidate pool from Round 2 onward, so the candidate pool remains dynamic across rounds. Moreover, retaining PAR without FAE reduces R5 performance because GTRS/SF scoring is not trained on PAR-refined trajectories. Specifically:

1) How memory updates candidates without PAR. Round 1 evaluates only the initial candidates generated by the base planner. From Round 2 onward, the memory module directly adds four stored historical trajectories to the candidate pool in each round, including the target scenario's best-so-far trajectory and three stored trajectories retrieved from similar scenarios. As shown in Table E below, when removing only PAR, SF filters the updated candidate pool and FAE ranks the remaining candidates, raising PDMS from 90.58 to 92.76. When removing both FAE and PAR, GTRS/SF scoring directly selects from the updated candidate pool rather than using WoTE's native scorer, raising PDMS from 89.05 to 90.87. By contrast, disabling memory and fixing the R1 candidates keeps all rounds at the R1 scores. These results indicate that the multi-round gains in both settings are driven by memory candidate injection rather than repeated scoring.

Table E: Control experiments without PAR on navtest.

VariantCandidate Pool Across RoundsR1 PDMSR5 PDMS
w/o PARUpdate Retrieved Candidates90.5892.76
w/o FAE & PARUpdate Retrieved Candidates89.0590.87
w/o PARFix R1 Candidates & Repeat Scoring (No Memory)90.5890.58
w/o FAE & PARFix R1 Candidates & Repeat Scoring (No Memory)89.0589.05

2) Why retaining PAR without FAE reduces R5 performance (90.23 vs 90.87 PDMS). In EvoAD, FAE serves as a dedicated evaluator trained to rank PAR-refined trajectories that the pretrained GTRS/SF scorer has not seen. Without FAE, selection falls back to GTRS/SF, which often selects suboptimal trajectories and lowers R5 to 90.23 PDMS, compared with 90.87 PDMS when both modules are removed.

Revised: p7:337–344; p17:897–912.

Q4

Thank you for your comment. We select WoTE as the primary base planner because WoTE already includes a latent world model on NAVSIM, making it more convenient to evaluate EvoAD, whereas iPad and other methods do not include world models. Below, we detail the adaptation procedure for other base planners and supplement a five-round experiment on DiffusionDrive.

1) Adaptation procedure for other planners. To adapt EvoAD to a frozen target planner, we sequentially train the latent WM, PAR, and FAE before running multi-round inference. First, we extract current and future scene features using the target planner's frozen perception module, and train the latent WM to predict future features from current features and ground-truth trajectories. Second, with the trained WM frozen, we retrain PAR on the weak/strong trajectory pairs by replacing the future latent input with the new WM's prediction on each weak trajectory. Third, we use the target planner to generate candidate trajectories, and retrain FAE on the SF-filtered candidates, the target planner's scene features, and the new WM's future predictions under PDMS and sub-score supervision. During multi-round inference, all modules remain frozen. Round 1 filters the target planner's initial candidates via SF, predicts candidate futures with the new WM, and selects the trajectory via FAE to initialize memory. From Round 2 onward, EvoAD retrieves stored experiences using the target planner's scene features, generates four PAR-refined trajectories to expand the candidate pool, and ranks the updated pool with FAE across rounds.

2) Multi-round full-system evaluation on DiffusionDrive. Following the procedure above, we adapt the latent WM, PAR, and FAE to the frozen DiffusionDrive planner. Specifically, we use the scene features from DiffusionDrive's perception backbone and the candidates from DiffusionDrive's diffusion planning head, and run five reflection rounds on navtest. As shown in Table F below, the final selected trajectory improves steadily from 88.19 PDMS for the native DiffusionDrive baseline to 90.12 PDMS at R1, 91.83 PDMS at R2, and 92.65 PDMS at R5. This continuous improvement from R1 to R5 indicates that the complete multi-round reflection mechanism is not specific to WoTE and can consistently improve end-to-end planning performance on other base planners.

Table F: Multi-round evaluation of EvoAD on DiffusionDrive.

MethodR1R2R3R4R5
DiffusionDrive88.1988.1988.1988.1988.19
EvoAD90.1291.8392.3192.5292.65

Revised: p8:420–426; p20:1054–1070.

Thanks for the detailed response and additional experiments. While the clarifications are helpful, my main concerns remain. First, EvoAD achieves only a modest improvement over DrivOR (93.4 vs. 93.1 PDMS), despite introducing considerable additional complexity. Second, the multi-round reflection protocol accumulates feedback on the same benchmark scenarios, raising concerns about evaluation fairness and generalization to unseen scenarios. Finally, the additional ablations do not fully establish the novelty of PAR beyond feedback-conditioned trajectory refinement.

Overall, I appreciate the authors' efforts, but these concerns remain insufficiently addressed. Therefore, I will maintain my original rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.