acceptodds
Under review as a conference paper at ICLR 2027

Test-Time Adaptation of End-to-End Trajectory Planning via Simulation-based Self-Verification

Abstract

Existing Test-Time Adaptation (TTA) approaches mitigate distribution shifts for classification, detection, or segmentation, and the few for end-to-end planning update only the score decoder over a fixed trajectory vocabulary. More critically, the trajectory predicted under corruption is executed without refinement or verification, although a single unsafe one can cause a safety incident. We propose Simulation-based TTA (SimTTA), a verification-driven framework that addresses two key challenges. To verify the predicted trajectory, Geometric Trajectory Refinement (GTR) expands it in the output space into candidates beyond the reach of the corrupted model. Simulation-based Self-Verification (SSV) then simulates each candidate on a scene context reconstructed from the perception outputs of the model itself, and selects the highest-scoring one as the final trajectory without ground-truth labels or training. To adapt the planning model, the Temporally-Verified Entropy (TVE) loss minimizes the prediction entropy only on the BEV regions whose predicted classes remain the same across consecutive samples. Thus, the following samples are planned and verified with a more reliable representation. Experiments across eight corruption types demonstrate that our SimTTA significantly enhances trajectory quality under such shifts. The source code will be released.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Autonomous Driving, Computer Vision

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_48smOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents SimTTA for test-time adaptation of end-to-end driving planners under sensor corruptions. SimTTA comprises three components. Geometric Trajectory Refinement (GTR) first generates candidate trajectories by scaling and rotating the predicted trajectory. Simulation-based Self-Verification (SSV) then scores these candidates using a scene reconstructed from the model’s perception outputs. Finally, Temporally-Verified Entropy (TVE) adapts the model’s batch normalization parameters by minimizing entropy on temporally consistent BEV predictions.

Critical strengths
  1. The breadth and consistency of the empirical improvements constitute the strongest aspect of the paper. On DiffusionDrive, SimTTA improves mean PDMS from 78.9 to 85.4 on OpenScene and from 86.7 to 90.7 on nuScenes-C. The additional results on TransFuser and UniAD further strengthen the evidence that the approach generalizes beyond a single planner. Together, these experiments make a convincing case for its effectiveness under the corruptions tested.

  2. The paper addresses a worthwhile problem. Model adaptation does not necessarily yield a better trajectory for the current input. Explicitly refining and evaluating that trajectory is therefore a sensible direction. Moreover, evaluating candidates using the model’s own perception outputs, rather than relying on ground-truth future agent trajectories, enhances the approach’s practical relevance.

Critical weaknesses
  1. The methodological contribution appears incremental. Geometric perturbation, simulation-based candidate scoring, and temporally filtered entropy minimization form a reasonable combination, but I remain unconvinced that the paper establishes a sufficiently distinct contribution for ICLR. Section 4.5 provides a partial rationale for the geometric search, yet this analysis remains tied to DiffusionDrive and the PDMS setting. As shown in Table 4, GTR+SSV accounts for most of the improvement, raising mean PDMS from 78.9 to 84.1, whereas adding TVE further raises it to 85.4. Although TVE provides a measurable benefit, its planning gains are uneven across corruptions. Moreover, the experiments do not establish how much temporal verification adds beyond simpler adaptation objectives under the same candidate-selection pipeline. TVE also requires BEV class predictions and adaptable BN parameters. This does not invalidate the method, but it narrows the scope of the claim that the complete framework is model-agnostic.

  2. The evaluation does not establish the claimed benefits in a full closed-loop setting. Section 4.1 refers to a “closed-loop protocol,” but the cited NAVSIM evaluation queries the planner at the initial observation and evaluates a fixed trajectory through non-reactive simulation. It does not test repeated planning from observations produced by the planner’s own actions. This concern is especially pertinent because SSV relies on potentially corrupted perception and constant-velocity predictions of other agents. The PDMS improvements and ranking agreement support the usefulness of this approximation, but they leave open whether the gains persist under execution-induced changes in observations, accumulated errors, and interactions. Since verification before execution is central to the paper’s motivation, the absence of such evidence materially weakens its safety claims.

Questions for authors
  1. Contribution of TVE: Holding GTR+SSV, BN statistics, and the adaptation budget fixed, how much does TVE improve upon standard entropy minimization or confidence-only filtering? Such an ablation would help determine whether temporal verification is essential to the reported gains.
  2. Evaluation protocol: Does the reported evaluation involve repeated replanning from sensor observations that change in response to the executed trajectory? If so, please clarify the setup and provide the corresponding results. If not, qualify the protocol description and the claims regarding safe execution accordingly.
  3. Architectural requirements: What outputs and model components does the complete framework require? For planners that lack dense BEV class predictions or adaptable BN layers, which components can be applied directly, and which would require additional modules or training?
8 comments
AuthorsOpenReview ↗

Response to Reviewer 48sm (Part 1)

W1 & Q1: Methodological contribution, effectiveness of TVE, and scope of model agnosticism

W1-1: The methodological contribution appears incremental. Geometric perturbation, simulation-based candidate scoring, and temporally filtered entropy minimization form a reasonable combination, but I remain unconvinced that the paper establishes a sufficiently distinct contribution for ICLR. Section 4.5 provides a partial rationale for the geometric search, yet this analysis remains tied to DiffusionDrive and the PDMS setting.

(Answer W1-1) We would like to respectfully emphasize that SimTTA advances TTA by addressing both the reliability of adaptation signals and the quality of trajectories selected for execution. Prior methods such as Tent, MOS, and SSA focus on improving adaptation, but under corrupted perception, model confidence or consistency can be misleading, and adaptation alone cannot verify whether a trajectory is suitable for execution. SimTTA addresses these issues through temporally verified adaptation and trajectory refinement and verification using a scene reconstructed from planner outputs.

TVE uses agreement across ego-aligned observations to exclude confident but temporally inconsistent predictions from adaptation. Meanwhile, GTR+SSV generates alternative trajectories and evaluates their outcomes in the reconstructed scene without ground-truth agent annotations. Thus, the two components address distinct failure sources during TTA, as further examined below.

Additionally, as per the reviewer's comment that the geometric analysis is limited to DiffusionDrive, we extend the analysis to three planners across OpenScene and nuScenes-C.

Table R3.1

Dataset/ModelPC1: ScalingPC2: RotationTotal
OpenScene/DiffusionDrive79.0%17.6%96.6%
OpenScene/TransFuser78.3%18.2%96.5%
OpenScene/UniAD79.2%17.4%96.6%
nuScenes-C/DiffusionDrive63.9%35.7%99.6%
nuScenes-C/TransFuser64.0%35.6%99.6%
nuScenes-C/UniAD67.8%31.5%99.3%

As shown in Table R3.1, scaling and rotation jointly explain 96.5-96.6% of the coordinate variance of PDMS-improving perturbations across DiffusionDrive, TransFuser, and UniAD on OpenScene (1st-3rd rows), and 99.3-99.6% on nuScenes-C (4th-6th rows). The same dominant directions emerge across all six configurations, supporting the search design across models and datasets. These percentages describe coordinate variance rather than PDMS variance, and the analysis remains within the PDMS-based evaluation setting.

Table R3.2

MethodsSnowRainFogFOV LostGaussian NoiseMoving ObjectMotion BlurTemporal MisalignmentMean
Random perturbation83.287.286.171.678.387.686.175.081.9
Search without Scale/Rotation83.487.486.472.478.687.986.475.482.3
Search with All Perturbations84.188.087.076.679.688.487.177.983.6
Search along Remaining PCA Components83.287.286.171.778.387.686.075.181.9
Our GTR Setting (SimTTA)85.188.287.482.781.388.487.882.185.4

Furthermore, the equal-budget comparison in Table R3.2 provides direct empirical support for the search design. Under the same 31-candidate budget, our GTR setting achieves 85.4 PDMS (5th row) compared with 83.6 when combining all perturbation types (3rd row). This supports concentrating a limited candidate budget on scaling and rotation, without claiming that other trajectory changes cannot improve PDMS. Together, these results support GTR as an empirically justified search strategy rather than an arbitrary heuristic, demonstrating consistent geometric patterns across planners and datasets and superior performance compared with alternative search strategies.

We added the PCA analysis across planners and datasets in Table 18 of Appendix A.9 and the equal-budget comparison in Table 7 of Section 4.5, and updated Section 4.5 to refer to them.

AuthorsOpenReview ↗

Response to Reviewer 48sm (Part 2)

W1-2: As shown in Table 4, GTR+SSV accounts for most of the improvement, raising mean PDMS from 78.9 to 84.1, whereas adding TVE further raises it to 85.4. Although TVE provides a measurable benefit, its planning gains are uneven across corruptions. Moreover, the experiments do not establish how much temporal verification adds beyond simpler adaptation objectives under the same candidate-selection pipeline.

Q1: Contribution of TVE: Holding GTR+SSV, BN statistics, and the adaptation budget fixed, how much does TVE improve upon standard entropy minimization or confidence-only filtering? Such an ablation would help determine whether temporal verification is essential to the reported gains.

(Answer W1-2 & Q1) We would like to respectfully emphasize that TVE provides both a standalone adaptation benefit and an additional improvement when combined with GTR+SSV. To show the effectiveness of the TVE loss itself, we gather the results of Table 2 and Table 4 of the main paper in Table R3.3.

Table R3.3

MethodsSnowRainFogFOV LostGaussian NoiseMoving ObjectMotion BlurTemporal MisalignmentMean
Source (No Adapt.)70.784.781.271.873.587.987.274.578.9
BN Adapt.82.784.483.568.978.084.983.172.879.8
SSA (ICLR'25)82.184.784.269.978.285.283.672.780.1
TVE alone82.486.385.370.277.486.685.273.880.9
GTR+SSV78.187.083.883.980.888.688.282.084.1
GTR+SSV+TVE (SimTTA)85.188.287.482.781.388.487.882.185.4

As shown in Table R3.3, TVE alone improves mean PDMS from 78.9 (1st row) to 80.9 (4th row), outperforming BN Adapt. (2nd row) and SSA (3rd row), the strongest adaptation baseline in Table 2 of the main paper. Adding TVE to GTR+SSV further improves mean PDMS from 84.1 (5th row) to 85.4 (6th row). We acknowledge that this additional benefit varies across corruptions and does not improve every corruption type.

The larger improvement from GTR+SSV highlights the importance of evaluating candidate trajectories before execution. Adaptation alone cannot verify whether a trajectory is safe to execute. GTR+SSV addresses this through simulation-based candidate selection, while TVE complements it by adapting the representation used for planning and scene reconstruction.

To address the comparison with simpler adaptation objectives, we additionally conduct an experiment that holds GTR+SSV, BN statistics, and the adaptation budget fixed.

Table R3.4

MethodsSnowRainFogFOV LostGaussian NoiseMoving ObjectMotion BlurTemporal MisalignmentMean
Tent (Shannon Entropy)79.684.783.275.774.185.184.075.480.2
Confidence-only Filtering79.384.783.272.873.384.984.175.479.7
TVE w/o Temporal Verification81.786.084.877.876.586.685.377.582.0
TVE (SimTTA)85.188.287.482.781.388.487.882.185.4

As shown in Table R3.4, TVE achieves 85.4 mean PDMS (4th row), outperforming standard entropy minimization at 80.2 (1st row) by 5.2 points and confidence-only filtering at 79.7 (2nd row) by 5.7 points. TVE outperforms all three alternatives across all eight corruption types. Furthermore, removing only temporal verification (3rd row) while retaining the other TVE components reduces mean PDMS from 85.4 to 82.0. This confirms that temporal verification itself contributes substantially to the performance gains.

A central difficulty in adaptation under corruption is that the predictions used to update the model can themselves be wrong. Standard entropy minimization can reinforce an incorrect prediction, while confidence-only filtering can retain it if its confidence is sufficiently high. TVE adds a temporal consistency condition. After ego-motion alignment, a BEV prediction must agree with the corresponding prediction from the previous observation, in addition to satisfying the confidence threshold. A prediction that is confident in the current frame but inconsistent across observations is therefore excluded from entropy minimization.

These results demonstrate that the choice of adaptation objective substantially affects planning performance even when candidate generation and verification are held fixed. They support the effectiveness of the complete TVE objective over simpler alternatives, while the additional ablation confirms the independent contribution of temporal verification.

We added this comparison in Table 20 of Appendix A.11 and updated Section 4.3 to refer to it.

AuthorsOpenReview ↗

Response to Reviewer 48sm (Part 3)

W1-3: TVE also requires BEV class predictions and adaptable BN parameters. This does not invalidate the method, but it narrows the scope of the claim that the complete framework is model-agnostic.

(Answer W1-3) We would like to respectfully clarify that the complete framework applies to planners satisfying specific architectural requirements. GTR requires trajectory outputs, SCR/SSV requires agent detections, an HD map, and ego poses, and TVE requires dense BEV class predictions and adaptable BN affine parameters. A planner without a dense BEV prediction head can still use GTR+SSV if the required scene information is available, while applying TVE would require a suitable BEV head. Similarly, applying TVE to a planner without BN layers would require modifying the adaptation mechanism.

We updated Section 4.1 and Appendix A.3 to make these conditions explicit and distinguish the requirements of each component.

W2: Absence of closed-loop evaluation

W2-1: The evaluation does not establish the claimed benefits in a full closed-loop setting. Section 4.1 refers to a "closed-loop protocol," but the cited NAVSIM evaluation queries the planner at the initial observation and evaluates a fixed trajectory through non-reactive simulation.

(Answer W2-1) We would like to respectfully clarify that our use of "closed-loop protocol" followed terminology used in prior NAVSIM-based work, including DiffusionDrive, which describes its evaluation using "closed-loop evaluations" and "closed-loop metrics." We recognize that NAVSIM evaluates trajectories through non-reactive simulation reconstructed from recorded driving logs, without repeated planning from observations generated by the executed actions. Our wording referred to this simulation-based evaluation and was not intended to imply a fully interactive closed-loop rollout.

We updated Section 4.1 to avoid ambiguity by replacing "closed-loop protocol" with "NAVSIM simulation protocol" to clarify the scope of the evaluation.

Reviewer_AkhYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SimTTA combines simulation-based refinement of the current trajectory with adaptation for subsequent inputs. GTR generates candidates by scaling and rotating the predicted trajectory. SSV evaluates them using the supplied map and ego state, together with agents reconstructed from the planner's detections. TVE adapts batch normalization affine parameters on confident BEV cells that agree across time, with a penalty against class collapse. Temporal agreement also adjusts the preference for conservative candidates. Experiments examine planning under sensor corruption on OpenScene and nuScenes-C.

Critical strengths
  1. Reconstructing agents from the planner's detections is a useful way to support simulation-based candidate evaluation without training another scorer. The ranking test checks the reconstructed scores against ground-truth PDMS, providing an external measure of the selection signal. Together with the oracle comparison, this directly examines the verifier's usefulness and limitations.

  2. The evaluation provides complementary evidence for the combined system. On OpenScene, mean PDMS improves by 6.5 points, and the ablations support gains from both trajectory refinement and adaptation. The adapted model also improves BEV accuracy, calibration and boundary quality over the batch-statistics baseline. This supports the representation-level benefit of adaptation alongside its planning gains. The appendix evaluates additional planners, recovery on clean inputs and more varied corruption conditions, and includes explicit failure cases.

Critical weaknesses
  1. The methodological advance is limited, and the main explanation offered for the search design is not established. Diffusion-ES already combines proposal search with PDM-style simulation scoring and constant-velocity agent forecasts. SimTTA's distinction is to reconstruct the dynamic scene from the corrupted planner's detections. TVE combines BN-affine entropy adaptation with ego-aligned temporal filtering and a class-distribution penalty, building on established adaptation and reliability-selection ideas. The ranking diagnostics establish a useful empirical contribution for this combination. However, the PCA analysis does not substantiate the claimed structure of the search space. PCA on perturbations selected for improving PDMS measures their coordinate variance, not the variance explained in PDMS. It does not establish that scale and rotation capture nearly all score-relevant variation, or that changes in trajectory shape offer no further benefit. This weakens the main explanatory insight beyond the system's empirical gains and is central to my assessment of its contribution depth.

  2. The temporal scene-reconstruction procedure is not fully defined. Appendix A.8 compensates for translation before matching and differencing detections, then transforms the reconstructed state to the global frame. It does not state whether historical positions have already been aligned to the current ego orientation. This directly affects estimated agent velocities and the collision and time-to-collision scores used to select candidates. The appendix also assumes a 0.5-second interval without identifying the actual observation pair. Since navtest contains filtered scenes, sequential evaluation alone does not establish that successive processed samples have this interval. The historical observations used by SCR and TVE need to be specified to make the execution protocol reproducible.

Questions for authors
  1. For SCR and TVE, which previous observation is used at each step? What ensures the stated 0.5-second interval for the SCR pair, and are historical detections already expressed in the current ego orientation before the translation compensation in Appendix A.8?

  2. Which DiffusionDrive checkpoint and source training data were used for nuScenes-C, and how was its ground-truth PDMS evaluation context constructed?

  3. How were the perturbations in Figure 5 sampled, selected and normalized, and what supports interpreting their PCA variance as PDMS variation? Were the GTR search directions and sampling settings fixed before this navtest analysis?

7 comments
AuthorsOpenReview ↗

Response to Reviewer AkhY (Part 1)

W1: Limited methodological advance and unsupported rationale for the search design

W1-1: The methodological advance is limited, and the main explanation offered for the search design is not established. Diffusion-ES already combines proposal search with PDM-style simulation scoring and constant-velocity agent forecasts. SimTTA's distinction is to reconstruct the dynamic scene from the corrupted planner's detections. TVE combines BN-affine entropy adaptation with ego-aligned temporal filtering and a class-distribution penalty, building on established adaptation and reliability-selection ideas. The ranking diagnostics establish a useful empirical contribution for this combination.

(Answer W1-1) We would like to respectfully emphasize that SimTTA makes a distinct methodological contribution by enabling efficient trajectory refinement and verification for test-time adaptation under corrupted perception. Applying simulation-based search in this setting requires both constructing a verification scene from unreliable perception outputs and keeping the search computationally efficient.

First, the original Diffusion-ES pipeline is not directly applicable to our TTA setting because its reward computation relies on privileged perception information unavailable during adaptation. Specifically, its nuPlan evaluation uses dataset-provided, near-ground-truth perception outputs to compute PDM-Closed-based rewards, with surrounding-agent motion predicted under a constant-velocity assumption. Our setting instead requires verification using perception outputs inferred from corrupted sensor inputs. SimTTA addresses this gap through SCR, which reconstructs the scene from the planner outputs and enables candidate verification without ground-truth agent annotations.

Second, the computational cost of iterative diffusion-based search limits its suitability for fast online TTA. Diffusion-ES repeatedly scores candidates and refines selected trajectories through noising and denoising, requiring multiple model and simulation evaluations. Under corrupted perception, an inaccurate score can cause a poor candidate to be selected for further refinement, making reliable verification particularly important for TTA. Its optimized configuration reports a throughput of 2 FPS. In contrast, GTR generates candidates through lightweight geometric transformations without additional diffusion inference or training. SimTTA achieves 9.4 FPS (3rd row of Table 5c) by default and up to 18.4 FPS (1st row of Table 5c), corresponding to 9.2× the reported Diffusion-ES throughput. We additionally evaluate Diffusion-ES under the same corruptions on the NAVSIM navtest split.

Table R2.1

MethodsSnowRainFogFOV LostGaussian NoiseMoving ObjectMotion BlurTemporal MisalignmentMean
Source (No Adapt.)70.784.781.271.873.587.987.274.578.9
SSA (ICLR'25)82.184.784.269.978.285.283.672.780.1
Diffusion-ES (CVPR'24)82.186.185.069.577.186.484.973.879.4
SimTTA (Ours)85.188.287.482.781.388.487.882.185.4

As shown in Table R2.1, SimTTA achieves a mean PDMS of 85.4 (4th row) compared with 79.4 (3rd row) for Diffusion-ES, a gain of 6.0 points, and outperforms it across all eight corruption types. This comparison shows that, even under corrupted perception, SimTTA can outperform iterative diffusion-based search by evaluating geometric trajectory candidates in a scene reconstructed from the planner's own detections.

AuthorsOpenReview ↗

Response to Reviewer AkhY (Part 2)

W1-2: However, the PCA analysis does not substantiate the claimed structure of the search space. PCA on perturbations selected for improving PDMS measures their coordinate variance, not the variance explained in PDMS. It does not establish that scale and rotation capture nearly all score-relevant variation, or that changes in trajectory shape offer no further benefit. This weakens the main explanatory insight beyond the system's empirical gains and is central to my assessment of its contribution depth.

(Answer W1-2) We would like to respectfully clarify that our PCA analysis identifies the dominant geometric directions among PDMS-improving perturbations, while the equal-budget search comparison provides direct empirical support for our search design. The PCA variance ratios describe coordinate variance and do not establish that scaling and rotation capture all score-relevant variation or that other trajectory changes cannot provide further benefits.

Specifically, for each analyzed scene, we generated 1,000 perturbed trajectories by independently adding zero-mean Gaussian noise with a standard deviation of 0.5 m to each waypoint coordinate of the model-predicted trajectory. We selected the top 100 perturbations with positive PDMS gains, or all improving perturbations if fewer than 100 were available, and applied PCA to their coordinate changes. The first two components explained approximately 97% of the coordinate variance of these selected perturbations and aligned with scaling and rotation.

To further support our search design, we compared different perturbation strategies under the same 31-candidate budget.

Table R2.2

MethodsSnowRainFogFOV LostGaussian NoiseMoving ObjectMotion BlurTemporal MisalignmentMean
Random perturbation83.287.286.171.678.387.686.175.081.9
Search without Scale/Rotation83.487.486.472.478.687.986.475.482.3
Search with All Perturbations84.188.087.076.679.688.487.177.983.6
Search along Remaining PCA Components83.287.286.171.778.387.686.075.181.9
Our GTR Setting (SimTTA)85.188.287.482.781.388.487.882.185.4

As shown in Table R2.2, our GTR setting achieves the highest mean PDMS of 85.4 (5th row), outperforming alternative search strategies, including the combination of all perturbation types at 83.6 (3rd row). These results support focusing the 31-candidate search on scaling and rotation to improve the quality of trajectories selected by SSV.

We further evaluated all 31 candidates per scene using ground-truth PDMS to assess candidate quality independently of SSV selection. With mean waypoint displacement matched across strategies, we measured the fraction of perturbations improving on the original trajectory and the best candidate's PDMS gain per scene (Table R2.3).

Table R2.3

Search directionsPerturbations improving on the original (%)Mean best PDMS gain per scene (points)
Scaling and rotation (GTR)22.96+19.30
Remaining PCA components (excluding PC1 and PC2)20.81+13.79
All perturbation types combined20.98+12.05

As shown in Table R2.3, scaling and rotation produced the highest fraction of improving perturbations (22.96%, 1st row). The best GTR candidate improved on the original trajectory by 19.30 PDMS points on average, compared with 13.79 points for the remaining PCA directions (2nd row) and 12.05 points for the mixed search (3rd row). With candidate count and perturbation magnitude held fixed, GTR therefore found higher-scoring trajectories by concentrating its search on scaling and rotation. This supports our choice of search directions independently of how SSV ranks the candidates.

We updated Section 3.1, Section 4.5, and the caption of Figure 5 to clarify that the reported 97% refers specifically to the coordinate variance of the selected perturbations.

AuthorsOpenReview ↗

Response to Reviewer AkhY (Part 3)

W2 & Q1: Temporal scene reconstruction, previous observation, time interval, and frame alignment in SCR and TVE

W2: The temporal scene-reconstruction procedure is not fully defined. Appendix A.8 compensates for translation before matching and differencing detections, then transforms the reconstructed state to the global frame. It does not state whether historical positions have already been aligned to the current ego orientation. This directly affects estimated agent velocities and the collision and time-to-collision scores used to select candidates. The appendix also assumes a 0.5-second interval without identifying the actual observation pair. Since navtest contains filtered scenes, sequential evaluation alone does not establish that successive processed samples have this interval. The historical observations used by SCR and TVE need to be specified to make the execution protocol reproducible.

Q1: For SCR and TVE, which previous observation is used at each step? What ensures the stated 0.5-second interval for the SCR pair, and are historical detections already expressed in the current ego orientation before the translation compensation in Appendix A.8?

(Answer W2 & Q1) We would like to respectfully clarify that both SCR and TVE use the immediately preceding sample processed in the test stream as the previous observation. All observation pairs used for temporal processing are exactly 0.5 seconds apart, and historical detections are aligned with the current ego orientation before translation compensation.

Specifically, instead of randomly sampling frames, we organize evaluation samples by their original driving logs and process each log chronologically by timestamp. Although navtest (test streams) is a filtered subset, we reset the temporal state whenever the interval between consecutive samples exceeds 0.5 seconds or a new log begins. Since OpenScene is sampled at 2 Hz, this ensures that all valid temporal pairs are exactly 0.5 seconds apart. For SCR, historical detections are rotationally aligned with the current ego orientation before compensating for ego translation, ensuring that agent matching and velocity estimation operate in a consistent coordinate frame. TVE follows the same temporal protocol for BEV alignment and consistency verification.

We updated Appendix A.14 with these implementation details, including observation selection, coordinate alignment, and temporal-state reset conditions, and Section 3.2 with the orientation alignment to ensure reproducibility.

Q2: Checkpoint and ground-truth PDMS context for nuScenes-C

Q2: Which DiffusionDrive checkpoint and source training data were used for nuScenes-C, and how was its ground-truth PDMS evaluation context constructed?

(Answer Q2) We would like to respectfully clarify that we trained the NAVSIM DiffusionDrive architecture on clean nuScenes data rather than using the publicly available nuScenes checkpoint, which is based on a different architecture, SparseDrive (camera-only, 6-view). To evaluate the same model architecture and TTA pipeline across both benchmarks, we trained the NAVSIM DiffusionDrive architecture on the clean nuScenes training split for 60 epochs, using the same input configuration and 4-second trajectory targets as in NAVSIM. We used the final-epoch checkpoint without validation-based selection. The model was never exposed to nuScenes-C corruptions during training, and all TTA methods start from the same checkpoint.

We constructed the ground-truth evaluation context from nuScenes annotations, including ego poses and motion states, as well as surrounding-agent 3D bounding boxes and velocities. Predicted trajectories were evaluated using the NAVSIM PDM simulator. For nuScenes-C, we report a modified PDM score comprising collision safety, time-to-collision (TTC), and ride comfort, which differs from the full PDMS used in NAVSIM. Ground-truth annotations are used exclusively for evaluation; during adaptation, SimTTA verifies candidate trajectories using the scene reconstructed from the planner outputs. We additionally report L2 error and collision rate, the standard planning metrics on nuScenes.

Table R2.4

MethodAvg. L2 error (m) ↓\downarrowAvg. Collision Rate (%) ↓\downarrow
Source (No Adapt.)1.0971.024
MOS (ICLR'25)4.5546.145
CodeMerge (NeurIPS'25)6.9117.060
SimTTA (Ours)1.0710.874

As shown in Table R2.4, SimTTA achieves the lowest L2 error and collision rate among the compared methods (4th row). These results show that the gains on nuScenes-C extend beyond the modified PDM score, as SimTTA also reduces both trajectory error and collision rate relative to the unadapted planner.

We added the training details and the evaluation protocol in Appendix A.10 and the L2 error and collision rate in Table 19, and updated Section 4.2 to refer to them.

Reviewer_cpjJOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SimTTA, a test-time adaptation framework for end-to-end trajectory planning under sensor corruptions. It combines GTR for geometric trajectory refinement, SSV for simulation-based candidate selection using the model's own perception outputs, and TVE for updating only BN affine parameters through entropy minimization on temporally consistent BEV cells. Experiments on OpenScene and nuScenes-C show consistent PDMS gains across eight corruption types per dataset, and the ablations show that verification and adaptation complement each other. The method is tested on DiffusionDrive, TransFuser, and UniAD and runs at about 9.4 FPS.

Critical strengths

S1. Selection through verification rather than adaptation alone

The combination of GTR and SSV is the main contribution. The method generates candidates through geometric changes and scores them through simulation on a scene reconstructed from the model's own outputs. With this setup, it can recover safer trajectories even when the corrupted model's original prediction is poor. This directly addresses a safety-critical gap that prior TTA work for planning has largely ignored. The high Top-1 and Top-5 agreement with oracle PDMS rankings and the low regret values give concrete evidence that the proxy is useful.

S2. Adaptation that uses temporal consistency and complements verification

TVE alone already outperforms the strongest prior TTA baselines, and combining it with GTR+SSV improves performance further. Restricting entropy minimization to temporally consistent BEV cells is a simple, principled way to avoid reinforcing confident errors under corruption. I also like that the same temporal agreement signal is reused for the reliability gate.

Critical weaknesses

W1. Dependence on reconstructed scene quality

Under heavy corruption, the perception outputs used by SCR can themselves be unreliable, and constant-velocity extrapolation then makes the error worse. While RGC mitigates aggressive candidates, the paper does not give enough detail on cases where the reconstructed agents are badly wrong, such as missed detections or large velocity errors. A stronger analysis or an oracle-perception upper bound would strengthen the claim.

W2. Limited adaptation and candidate diversity under extreme shift

Because only BN affine parameters are updated and the candidate set is deliberately kept small to limit latency, the method cannot recover when the prediction is severely misaligned with the road geometry. The authors note this limitation, but it is still a real limit on the claimed robustness.

Questions for authors

Q1. Sensitivity to errors in the reconstructed scene context

How sensitive is SSV's ranking quality to errors in reconstructed agent velocities or missing detections? Could you provide an ablation with controlled perception noise added to the scene context used by SSV?

Q2. Failure causes under severe corruptions

Under the most severe corruptions, such as FOV Lost and Camera Crash, what fraction of failures comes from GTR's limited search space, and what fraction comes from fundamentally broken BEV features that TVE cannot repair?

Q3. Computational overhead of SCR and candidate simulation

Have you measured the computational overhead of SCR and simulation across multiple candidates in more detail? Could you provide a wall-clock breakdown in addition to the end-to-end FPS?

5 comments
AuthorsOpenReview ↗

Response to Reviewer cpjJ (Part 1)

W1 & Q1: Dependence on reconstructed scene quality and sensitivity to its errors

W1. Under heavy corruption, the perception outputs used by SCR can themselves be unreliable, and constant-velocity extrapolation then makes the error worse. While RGC mitigates aggressive candidates, the paper does not give enough detail on cases where the reconstructed agents are badly wrong, such as missed detections or large velocity errors. A stronger analysis or an oracle-perception upper bound would strengthen the claim.

Q1. How sensitive is SSV's ranking quality to errors in reconstructed agent velocities or missing detections? Could you provide an ablation with controlled perception noise added to the scene context used by SSV?

(Answer W1 & Q1) As per the reviewer's comments, we analyze SSV on the cases where the reconstructed agents are badly wrong and provide the oracle-perception upper bound.

First, for missed detections and large velocity errors (2nd-5th rows of Table R1.1), we find that the ranking quality of SSV is insensitive to these errors. For missed detections (2nd row), we take the samples in which at most 20% of the ground-truth vehicles are detected by the adapted model (recall ≤\le 0.2). For large velocity errors (3rd-5th rows), we add controlled perception noise to the scene context used by SSV: Gaussian noise of σ=0.5\sigma = 0.5, 1, and 2 m is added to the predicted box positions independently on each sample, which corrupts the velocities of matched agents by 2σ/Δt\sqrt{2}\sigma/\Delta t per axis (Δt=0.5\Delta t = 0.5 s). We report the Spearman (ρ\rho) and Pearson (rr) correlations between the SSV score and the PDMS that uses ground-truth scene context, and Top-1, the fraction of samples on which both select the same candidate.

Table R1.1

Caseρ\boldsymbol{\rho}r\boldsymbol{r}Top-1 (%)
All samples0.740.7375.5
Missed detections0.670.6473.6
Velocity errors (1.4 m/s)0.730.7274.6
Velocity errors (2.8 m/s)0.720.7273.3
Velocity errors (5.7 m/s)0.720.7171.8

As shown in Table R1.1, Top-1 decreases by only 1.9 points for missed detections, where at least 80% of the vehicles are not detected. Also, under large velocity errors of 5.7 m/s, it decreases by only 3.7 points. This is because SSV requires only the relative ordering of the candidates within a sample, not the accuracy of their absolute scores. Thus, SSV remains reliable even when the reconstructed agents are badly wrong.

Second, for the oracle-perception upper bound, we score the same candidates on three scene contexts (Table R1.2). SCR uses the agents reconstructed by SimTTA (1st row). The oracle-perception replaces the reconstructed agents with the ground-truth agents while keeping the constant-velocity extrapolation (2nd row). The full oracle further uses the ground-truth future trajectories of the agents, as reported in Table 15 of the Appendix (3rd row).

Table R1.2

Scene contextPDMS
SCR: reconstructed agents85.4
Oracle-perception: GT agents + constant velocity88.9
Full oracle: GT agents + GT future92.0

As shown in Table R1.2, replacing SCR with the oracle-perception improves the mean PDMS by 3.5 (85.4 → 88.9), less than the gain of SimTTA over the Source (78.9 → 85.4). The remaining gap of 3.1 to the full oracle comes from the ground-truth future trajectories. The performance of SimTTA therefore does not hinge on the quality of the reconstructed agents.

We added the analysis of the badly wrong agents in Table 16 of Appendix A.7 and updated Section 3.2 to refer to it. We also updated Table 15 of Appendix A.6 with the oracle-perception upper bound.

AuthorsOpenReview ↗

Response to Reviewer cpjJ (Part 2)

W2 & Q2: Limited adaptation and candidate diversity under extreme shift, and failure causes

W2. Because only BN affine parameters are updated and the candidate set is deliberately kept small to limit latency, the method cannot recover when the prediction is severely misaligned with the road geometry. The authors note this limitation, but it is still a real limit on the claimed robustness.

Q2. Under the most severe corruptions, such as FOV Lost and Camera Crash, what fraction of failures comes from GTR's limited search space, and what fraction comes from fundamentally broken BEV features that TVE cannot repair?

(Answer W2 & Q2) We would like to respectfully point out that SimTTA recovers many failed predictions under extreme shift. All results are obtained at the highest severity of each benchmark. FOV Lost keeps only 90° of the LiDAR field of view (Figure 6 of the Appendix) and drops the Source from 88.1 on clean inputs (Table 9 of the Appendix) to 71.8. Even under FOV Lost, SimTTA reduces the samples with a PDMS of 0 from 2,645 to 1,151.

The limitation that the reviewer pointed out refers only to the case where the base planner itself fails to perceive the scene. This case arises when the camera or LiDAR fails so completely that the region the vehicle must drive into is not observed, such as a right turn when only the left 90° of the LiDAR field of view remains. The prediction then deviates far from any safe trajectory. The remaining failures come from the limit of the base planner under extreme shift rather than from SimTTA itself.

As per the reviewer's question on the failure causes under FOV Lost and Camera Crash, we verify that the remaining failures come from the base planner rather than from SimTTA by measuring which fraction of them comes from GTR's limited search space, its limited candidate count, and broken BEV features. GTR's limited search space and limited candidate count are the fractions of failures that would be avoided if GTR searched a wider range or used more candidates. To measure them, we enlarge the search space 4× by doubling the sampling range in scaling and rotation, or increase the candidates from 31 to 91, and measure how much the number of failures decreases. Broken BEV features is the fraction of failures that occur on samples whose BEV features are so broken that TVE cannot repair them, with a BEV mIoU in the lowest 10% of all samples even after adaptation.

Table R1.3

Failure causeFOV LostCamera Crash
GTR's limited search space (1× → 4×)7.2% (83 / 1,151)1.4% (8 / 587)
GTR's limited candidate count (31 → 91)6.2% (71 / 1,151)0.5% (3 / 587)
Broken BEV features21.6% (249 / 1,151)5.8% (34 / 587)

As shown in Table R1.3, enlarging either the search space or the candidate count of GTR removes only a small fraction of the failures under both corruptions. Fundamentally broken BEV features that TVE cannot repair also account for only a minority of the failures. Thus, neither TVE updating only the BN affine parameters nor GTR with its limited search space and small candidate set is the main factor limiting SimTTA under the most severe corruptions. Instead, about 60% of the failures under both corruptions have no candidate with a non-zero PDMS. For these samples, the prediction of the base planner remains too far from any safe trajectory, so neither TVE adaptation nor the local search of GTR can recover it. These results suggest that the quality of the prediction of the base planner is a major limiting factor under extreme shift.

We updated the limitation paragraph in Section 4.5 and the failure case analysis in Appendix A.16, where we added this analysis in Table 21.

AuthorsOpenReview ↗

Response to Reviewer cpjJ (Part 3)

Q3: Computational overhead of SCR and candidate simulation

Q3. Have you measured the computational overhead of SCR and simulation across multiple candidates in more detail? Could you provide a wall-clock breakdown in addition to the end-to-end FPS?

(Answer Q3) As per the reviewer's comment, we provide the wall-clock breakdown of SimTTA on a single NVIDIA A6000 GPU.

Table R1.4

ComponentLatency (ms)Share (%)
Planner forward (1 pass)24.322.9
GTR: candidate generation0.30.3
SCR: scene context reconstruction5.14.8
SSV: candidate scoring (31 rollouts)73.169.0
TVE: backward and BN affine update3.23.0
Total106.0100

As shown in Table R1.4, SCR takes only 5.1 ms (4.8%) since it reconstructs the scene once per sample and shares it across all candidates. The simulation of the 31 candidates accounts for most of the latency (73.1 ms, 69.0%), so its cost is directly controlled by the number of candidates. Reducing the candidates per branch from 10 to 5 raises the speed from 9.4 to 12.8 FPS while the PDMS decreases only from 85.4 to 85.0 (Table 5c of the main paper). Thus, SCR accounts for a small part of the latency, and the cost of candidate simulation can be traded for speed with little loss in performance.

We added this breakdown in Table 17 of Appendix A.8 and updated the Memory Efficiency and FPS paragraph in Section 4.5 to refer to it.

We hope that our responses have adequately addressed the concerns of the reviewer. We sincerely value your feedback and would be glad to discuss any further questions or suggestions during the discussion period.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Official Comment by Authors

We sincerely thank all the reviewers for their thorough reviews and valuable feedback, which helped us improve our manuscript. We are also grateful that the reviewers recognize the strengths of our work: (1) verifying the trajectory for the current input addresses a safety-critical gap largely ignored by prior TTA for planning (cpjJ, 48sm), (2) SSV evaluates the candidates through simulation on the agents reconstructed from the model's own perception outputs without training another scorer (cpjJ, AkhY, 48sm), (3) the high agreement with the PDMS rankings and the oracle comparison show that SSV is a useful proxy (cpjJ, AkhY), (4) the TVE loss offers a simple and principled adaptation that complements verification and benefits the BEV representation (cpjJ, AkhY), and (5) the improvements are broad and consistent across OpenScene, nuScenes-C, and multiple planners, with further results on clean inputs and failure cases (AkhY, 48sm).

Encouraged by the valuable feedback from the reviewers, we have conducted additional experiments and analyses. The revised manuscript includes:

  • Analysis of SSV under missed detections and large velocity errors in the reconstructed agents in Appendix A.7, Table 16, referred to in Section 3.2 (cpjJ).

  • Updated oracle analysis with the Oracle-perception upper bound in Appendix A.6, Table 15 (cpjJ).

  • Wall-clock breakdown of SimTTA in Appendix A.8, Table 17, referred to in Section 4.5 (cpjJ).

  • Analysis of the failure causes under FOV Lost and Camera Crash in Appendix A.16, Table 21, with the updated limitation in Section 4.5 (cpjJ).

  • Clarified PCA sampling, selection, preprocessing, and interpretation, together with the navtrain analysis and clarification of how the search directions and sampling hyperparameters were determined, in Section 3.1, Section 4.5, and Figure 5. An equal-budget comparison of search strategies is provided in Table 7 (AkhY, 48sm).

  • Extended PCA analysis across three planners and two datasets in Appendix A.9, Table 18, referred to in Section 4.5 (48sm).

  • Clarified previous-observation selection, the 0.5-second interval, coordinate alignment, and temporal-state reset conditions for SCR and TVE in Appendix A.14, with orientation alignment also clarified in Section 3.2 (AkhY).

  • Detailed nuScenes-C training, checkpoint selection, and ground-truth evaluation context, including clarification of the modified PDM score and additional L2 error and collision rate results, in Appendix A.10, Table 19, referred to in Section 4.2 (AkhY).

  • Comparison of TVE with standard entropy minimization, confidence-only filtering, and TVE without temporal verification while holding GTR+SSV, BN statistics, and the adaptation budget fixed in Appendix A.11, Table 20, referred to in Section 4.3 (48sm).

  • Clarified architectural requirements and the applicability of individual components in Section 4.1 and Appendix A.3, qualifying the scope of model agnosticism (48sm).

  • Revised NAVSIM evaluation terminology in Section 4.1 and additional reactive closed-loop evaluation in CARLA in Section 4.5, Table 8, with experimental settings in Appendix A.12 and corruption visualizations in Figure 11 (48sm).

We thank all reviewers again for their detailed feedback, which we believe has helped us further strengthen the clarity and presentation of our contributions. We have incorporated all clarifications and corrections into the revised manuscript, where the modified text and the newly added tables are highlighted in blue. The original Appendices A.7-A.13 are renumbered as A.13-A.19 due to the newly added Appendices A.7-A.12, and the original appendix Tables 7-13 are renumbered as Tables 9-15 due to the newly added Tables 7 and 8. We would be grateful if the ACs and reviewers could kindly review these updates.

AuthorsOpenReview ↗

Discussion Summary and Final Clarifications

We sincerely thank all reviewers for their thoughtful feedback and engagement throughout the discussion. The additional experiments and manuscript revisions are summarized in our earlier Official Comment by Authors. Here, we summarize the progress made during the discussion and clarify the remaining points concerning our contribution.

Summary of Reviews and Our Responses

Reviewer cpjJ considered the concerns about perception and reconstruction errors well addressed and emphasized the practical value of verification-driven refinement under severe corruption. Reviewer AkhY confirmed that the SCR and nuScenes-C protocol clarifications resolved the corresponding concerns and recognized the controlled comparisons as strengthening the case for GTR. Reviewer 48sm acknowledged that the TVE ablation and CARLA evaluation resolved some concerns and strengthened the empirical evidence.

ConcernReviewersOur Response
Methodological novelty and distinction from prior workAkhY, 48smClarified the contribution under TTA constraints and compared with Diffusion-ES (Table R2.1).
Verification reliability and extreme-corruption failurescpjJ, 48smAdded reconstruction-error, oracle-perception, and failure analyses (Tables R1.1–R1.3).
GTR rationale and generalizabilityAkhY, 48smClarified PCA and added search comparisons and analyses across splits, planners, and datasets (Tables R2.2–R2.3, R2.5, R3.1).
Independent contribution of TVEAkhY, 48smExamined TVE gains and isolated temporal verification through controlled ablations (Tables R3.3–R3.4).
Evaluation validity and reproducibilityAkhY, 48smClarified protocols and added standard metrics and reactive closed-loop evaluation (Tables R2.4, R3.5).
Computational cost and architectural applicabilitycpjJ, 48smReported component-level latency and clarified architectural requirements (Table R1.4; response to 48sm, Q3).

Clarification of the Remaining Concerns

Regarding GTR, the search rule is not tied to navtest. Although the search directions were originally identified on navtest, scaling and rotation remain the dominant directions on navtrain (Section 4.5 of the revised manuscript) and across all three planners on nuScenes-C (Table 18 of Appendix A.9), so the same search directions would be obtained from any of these data. Furthermore, SimTTA using the same GTR improves planning performance on nuScenes-C (Tables 3 and 19) and in reactive closed-loop CARLA evaluation (Table 8). These results provide complementary evidence for the geometric rationale and the practical applicability of the framework beyond the benchmark that originally informed its design.

Regarding self-verification, SSV explicitly evaluates simulated trajectory outcomes on a reconstructed scene, including collision and time-to-collision criteria (Section 3.2). The ranking diagnostics and controlled perception-error experiments (Table 16 in Appendix A.7), together with the oracle comparisons (Table 15 in Appendix A.6), directly examine the usefulness and limitations of this verification signal. Our claim concerns an empirical mechanism for checking and improving trajectories under imperfect perception, rather than a certification of absolute safety.

The methodological contribution should also be considered at the framework level. GTR and SSV evaluate and refine the trajectory for the current observation, while TVE improves the representation used for subsequent planning and scene reconstruction. The complementary contributions are examined through the component ablations (Table 4), while the contribution of temporal verification is isolated through controlled comparisons (Table 20 in Appendix A.11). The challenge is to make these operations work together when the perception used for verification is itself corrupted and ground-truth agent annotations are unavailable. We believe that assessing this contribution solely through its individual components does not fully capture the problem addressed or the design required to address it.

The discussion has resolved several technical concerns and strengthened the empirical evidence for SimTTA. While the remaining questions about the GTR design evidence and the scope of self-verification warrant consideration, we hope they will be weighed in proportion to the specific aspects they address, alongside the broader methodological contribution. As Reviewer cpjJ emphasized in the follow-up comment, efficient verification-driven refinement under severe corruption is a practically valuable research direction that remains insufficiently explored. We thank all reviewers again for their thoughtful assessment and constructive discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.