acceptodds
Under review as a conference paper at ICLR 2027

Learning How to Drive at Test Time

Abstract

Driving policies are often deployed with verifiers, i.e. safety checks scoring candidate trajectories against drivable area, occupancy, and comfort criteria. We argue that such verifiers are good rewards, and that policies can learn from them at test time. We use verifiers for rejection sampling and optimize driving policies with one gradient step toward accepted trajectories. Our method requires no human demonstrations and no differentiable simulator. It is agnostic to where the samples come from: with a teacher policy it performs online distillation, with the policy's own samples online self-distillation. We show that both regimes support test-time fine-tuning of pre-trained Alpamayo 1.5, AutoVLA, and OneVL policies, improving composite driving scores by up to 24% and lowering displacement error by up to 42%. VADv2, CILRS, and CoverNet policies can even be learned from scratch at test time, given a reasonable initial trajectory distribution. On the KITScenes LongTail and the Waymo Open E2E dataset, our method compensates for domain shifts from nominal to unusual driving scenarios, outperforming all submissions to the 2026 KITScenes Few-Shot Challenge.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Test-time learning, test-time adaptation, end-to-end driving policies, autonomous driving

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Mh7iOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Test-Time Learning via Rejection Sampling (TTLR) for adapting driving policies to long-tail scenarios. It samples candidate trajectories, accepts those passing drivable-area, occupancy, and comfort checks, and takes one gradient step toward the accepted set. Candidates may come from a fixed teacher or the adapting policy itself. The geometric verifier combines SAM 3 segmentation with UniDepthV2 depth estimates. A separate experiment learns trajectory selectors from random initialization over kinematically initialized trajectory vocabularies. Results on KITScenes LongTail and Waymo Open E2E show improvements over the initial policies.

Critical strengths
  1. The method provides a simple way to use existing trajectory checks as learning signals without new expert trajectories at adaptation time. Supporting both fixed-teacher distillation and self-distillation makes the approach applicable to different deployment settings.
  2. The empirical coverage is useful. Experiments include multiple pretrained policies, two recent long-tail benchmarks, and modified trajectory classifiers trained from scratch. The appendix reports three-seed results, and the paper explicitly discusses perception failures and verifier overfitting.
Critical weaknesses
  1. The benefit of learning over filtering is not isolated. The fine-tuning experiments sample 120 candidates per scenario, but lacks a frozen-policy baseline using the same candidates and verifier. It is therefore unclear whether weight updates offer an advantage over verifier-based selection alone.
  2. The strongest performance comparison needs a clearer protocol. It uses final weights after adaptation over the test split, while online predictions represent a different setting. Online KITScenes MMS does not clearly exceed the reported challenge leader. Scenario ordering, reset rules, and comparability with the challenge protocol remain insufficiently specified.
  3. Limited novelty beyond RAFT. TTLR shares RAFT’s sample–filter–fine-tune framework, using binary safety checks to select training targets. The main distinction appears to be its application to driving at test time. The paper should clarify what methodological advance this introduces beyond the established framework.
Questions for authors
  1. In the fine-tuning setting, does TTLR outperform a frozen policy using the same 120 candidates and verifier for selection? An unfiltered self-distillation control would also help separate the contributions of the verifier and weight updates.
  2. How are scenarios ordered, when are weights reset, and does final evaluation revisit the observations used for adaptation? Please clarify how the challenge comparison matches permitted data and adaptation protocols, and how the reported best runs are selected.
  3. What are the end-to-end latency and memory costs of sampling, perception, the gradient step, and final prediction? These measurements matter because the fine-tuning setting uses 120 candidates per scenario and two large perception models before adaptation is complete.
  4. Please specify the training loss and updated parameters for each policy, including how multiple accepted trajectories are combined. What happens when no candidate is accepted, especially under the perception failures illustrated in Figure 5?
2 comments

Thank you for your valuable feedback on our work! We hope to answer all your questions with additional information and new experiments. In addition, we added all new experiments to the rebuttal revision of our paper. All changes in the PDF are highlighted in blue.

Q1: In the fine-tuning setting, does TTLR outperform a frozen policy using the same 120 candidates and verifier for selection?

R1: We added this baseline as Eval. = w/ verifier to Table 2. Filtering trajectories with our verifier and randomly selecting an accepted trajectory improves MMS, RFS and ADE. Optimizing the weights with TTLR further improves results significantly (e.g., MMS 4.69 to 4.99 (online) or 5.50 (final)). Note that we mixed up MMS and MMS (type) in the initial submission and corrected the order as well, apologies!

Q2.1.: How are scenarios ordered, when are weights reset?

R2.1: For each of the random seeds reported in our appendix, we traverse the test splits in a different random order. Each seeded run starts from the base weights of the model. We don’t reset weights during individual runs.

Q2.2: Does final evaluation revisit the observations used for adaptation?

R2.2: Yes. This is permitted by the challenge rules.

Q3: What are the end-to-end latency and memory costs of sampling, perception, the gradient step, and final prediction?

R3: We report latency and VRAM for Alpamayo with and without TTLR in Table 6. TTLR increases latency by only 23% and VRAM consumption by 20%. In a production system, the verifier latency can be significantly reduced by using a LiDAR sensor instead of depth estimation and BEV grid generation (see Table 7).

Q4.1: Please specify the training loss and updated parameters for each policy, including how multiple accepted trajectories are combined.

R4.1: We use the default loss functions for each policy and the official code implementations. We added this information to Table 8. For models optimized with batch size one (Alpmamyo 1.5, OneVL) we randomly choose one accepted trajectory to optimize the model, for models that perform trajectory planning as classification (VADv2, CILRS, CoverNet), we optimize backpropagate the loss for all accepted trajectories.

Q4.2: What happens when no candidate is accepted, especially under the perception failures illustrated in Figure 5?

R4.2: We don’t optimize the models on scenarios where all trajectories are rejected and use the current weights for prediction.

W2: Online KITScenes MMS does not clearly exceed the reported challenge leader.

This is resolved by fixing a copy-paste error on our side. We copied the MMS scores from the official challenge leaderboard wrong. Specifically, we copied the MMS values for our method and the MMS (type) values of the challenge winners. MMS is a global average, MMS (type) is an average over the per scenario type averages. Apologies for this mistake! We updated Table 2 in the paper to include both scores in the correct ordering. Our TTLR Alpamayo version (self-distillation) evaluated in an online manner significantly outperforms the challenge winner TTVLM in both metrics (MMS: 5.11 vs 4.75, MMS (type): 5.46 vs. 5.15).

Since many reviewers asked about inference latency, we additionally worked on a real-time version of our verifier and added the results to Table 7. Our version (b) is real-time capable when run in parallel with the driving policy for high-level planning, which typically occurs at 4-5 Hz, matching the recording frequency of the Waymo Open E2E and KITScenes LongTail datasets. Alpamayo 1.5 itself can be accelerated by using quantization, as in the version of Li et al. (2026).

Li et al. (2026), FlashDrive: Flash Vision-Language-Action Inference for Autonomous Driving. In arXiv

Reviewer_pH8QOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TTLR, a method for learning driving policies at test time through rejection sampling. It filters candidate trajectories using checks for drivable area, obstacle occupancy, and comfort, then takes one gradient step toward the accepted trajectories. Candidates are generated by either a fixed teacher or the current policy itself. Experiments evaluate pretrained policy adaptation on KITScenes LongTail and Waymo Open E2E, alongside learning trajectory selection from scratch on KITScenes LongTail.

Critical strengths

The paper presents a simple framework that turns trajectory safety checks into a learning signal for driving policies at test time. It requires neither ground-truth test trajectories nor a differentiable simulator. The same framework supports both teacher-guided distillation and self-distillation. The experiments cover several policy architectures and explore both pretrained adaptation and learning trajectory selection from scratch, showing the flexibility of the approach.

Critical weaknesses
  • The test-time self-distillation experiments miss an important baseline: filtering trajectories from the frozen pretrained policy with the same verifier, without updating the policy. TTLR trains the policy on trajectories accepted by this verifier. It is unclear why gradient updates are needed instead of directly selecting an accepted trajectory through rejection sampling. The authors should justify this choice and show that TTLR offers better performance, efficiency, or generalization than this baseline. Without this evidence, the motivation for TTLR remains unclear.

  • The paper discusses perception errors but does not quantify their impact on TTLR. It also does not explain how to assess perception quality and use this information to decide whether to perform a gradient update. Without these experiments and analyses, it is difficult to determine whether TTLR can reliably improve performance in long-tail scenarios. These scenarios can also cause perception failures and make the verifier’s decisions unreliable. Such errors may then produce misleading training targets and harm the policy through incorrect updates.

Questions for authors
  • Could the authors compare TTLR with rejection sampling from the frozen pretrained policy using the same verifier and sampling budget, and report both planning quality and total runtime?
  • Could the authors quantify how errors in segmentation and depth estimation affect verifier decisions and the resulting TTLR performance?
  • How do the authors handle scenarios where no candidate trajectory passes the verifier?
2 comments

Thank you for your valuable feedback on our work! We hope to answer all your questions with additional information and new experiments. In addition, we added all new experiments to the rebuttal revision of our paper. All changes in the PDF are highlighted in blue.

Q1: Could the authors compare TTLR with rejection sampling from the frozen pretrained policy using the same verifier and sampling budget, and report both planning quality and total runtime?

R1: We added this baseline as "Eval. = w/ verifier" to Table 2. Filtering trajectories with our verifier and randomly selecting an accepted trajectory improves MMS, RFS and ADE. Optimizing the weights with TTLR further improves results significantly (e.g., MMS 4.69 to 4.99 (online) or 5.50 (final)). Note that we mixed up MMS and MMS (type) in the initial submission and corrected the order as well, apologies! Additionally, we report latency and VRAM for Alpamayo with and without TTLR in Table 6. TTLR increases latency by only 23% and VRAM consumption by 20%. In a production system, the verifier latency can be significantly reduced by using a LiDAR sensor instead of depth estimation and BEV grid generation (see Table 7).

Q2: Could the authors quantify how errors in segmentation and depth estimation affect verifier decisions and the resulting TTLR performance?

R2: In most perception-related failure cases, the verifier rejects all trajectories since the road segmentation is very limited (Figure 5b) or not existent at all (Figure 5c). Further quantifying unsafe trajectories requires access to the test ground truth. However, we can track the rate with which all trajectories are rejected, which is 27.31% ± 0.93 on the KITScenes LongTail test split.

Q3: How do the authors handle scenarios where no candidate trajectory passes the verifier?

R3: We don’t optimize the models on scenarios where all trajectories are rejected.

Since many reviewers asked about inference latency, we additionally worked on a real-time version of our verifier and added the results to Table 7. Our version (b) is real-time capable when run in parallel with the driving policy for high-level planning, which typically occurs at 4-5 Hz, matching the recording frequency of the Waymo Open E2E and KITScenes LongTail datasets. Alpamayo 1.5 itself can be accelerated by using quantization, as in the version of Li et al. (2026).

Li et al. (2026), FlashDrive: Flash Vision-Language-Action Inference for Autonomous Driving. In arXiv

Reviewer_Gty4OpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary
  1. This paper proposes test-time learning via rejection sampling (TTLR). Candidate trajectories are filtered using drivable-area, occupancy, and comfort checks, and the policy takes one gradient step toward the accepted trajectories.
  2. The framework supports both teacher-based distillation and self-distillation. It also investigates learning trajectory-selection policies from scratch using predefined, physically plausible trajectory vocabularies.
  3. Experiments on KITScenes LongTail and WOD-E2E demonstrate improvements across several architectures. The appendix reports three-seed results supporting the consistency of self-distillation and learning-from-scratch gains.
Critical strengths
  1. Simple and broadly applicable adaptation mechanism. TTLR converts explicit driving constraints into training supervision without requiring ground-truth test trajectories or differentiable simulation. Its compatibility with different trajectory sources and policy architectures makes the approach practically interesting.
  2. Consistent empirical improvements. Tables 1–3 demonstrate gains across several policies, while Tables 4–5 provide supporting results across three seeds. The comparison with Centaur also illustrates the potential advantage of explicit driving criteria over uncertainty minimization, particularly for initially uncalibrated policies.
Critical weaknesses
  1. The contribution of learning is not adequately isolated. The method samples 120 trajectories per scenario and uses substantial pretrained perception models for verification. However, there is no matched baseline that directly selects a verifier-accepted trajectory without updating policy weights. Such a comparison is essential to establish the benefit of parameter adaptation beyond additional sampling and verification. Runtime and memory measurements are also needed to assess the feasibility of the proposed online procedure.
  2. The strongest claims depend on a different evaluation regime from online deployment. Table 2 emphasizes best-run final-model results after adaptation to the entire test split. These results measure performance on previously encountered scenarios and do not establish improvement on subsequent unseen scenarios. For example, the three-seed online MMS is 5.10 ± 0.09, compared with 5.15 for TTVLM-Base. The claim of outperforming all challenge submissions therefore requires qualification regarding evaluation regime and metric.
Questions for authors
  1. How does TTLR compare with directly selecting from the same 120 verifier-filtered candidates without parameter updates? Please include performance, latency, and memory comparisons under matched sampling and verification budgets.
  2. What is the scenario ordering, and how are weights reset across independent sequences? Does adaptation improve performance on subsequent unseen scenarios? Please also explain how the “best values” are selected and prioritize multi-seed online results.
  3. How frequently does the verifier incorrectly accept unsafe trajectories or reject all candidates? What happens in the latter case, and how do these errors affect adaptation? Quantitative verifier ablations would help assess robustness beyond the qualitative failure examples.
3 comments

Thank you for your valuable feedback on our work! We hope to answer all your questions with additional information and new experiments. In addition, we added all new experiments to the rebuttal revision of our paper. All changes in the PDF are highlighted in blue.

Q1: How does TTLR compare with directly selecting from the same 120 verifier-filtered candidates without parameter updates?

R1: We added this baseline as "Eval. = w/ verifier" to Table 2. Filtering trajectories with our verifier and randomly selecting an accepted trajectory improves MMS, RFS and ADE. Optimizing the weights with TTLR further improves results significantly (e.g., MMS 4.69 to 4.99 (online) or 5.50 (final)). Additionally, we report latency and VRAM for Alpamayo with and without TTLR in Table 6. TTLR increases latency by only 23% and VRAM consumption by 20%.

Q2.1: What is the scenario ordering, and how are weights reset across independent sequences?

R2.1: For each of the random seeds reported in our appendix, we traverse the test splits in a different random order. Each seeded run starts from the base weights of the model. We don’t reset weights during individual runs.

Q2.2 Does adaptation improve performance on subsequent unseen scenarios?

R2.2: We optimized an Alpamayo 1.5 model with TTLR (self-distillation) on the KITScenes LongTail test split. Afterwards, we evaluate this model on the validation split. Table 9 shows that TTLR on the test split significantly improves validation results as well (MMS 4.289 to 4.695).

Q2.3: explain how the “best values” are selected and prioritize multi-seed online results?

R2.3: By participating in the official KITScenes Challenge to get MMS and L2 errors.

Q3.1: How frequently does the verifier incorrectly accept unsafe trajectories or reject all candidates?

R3.1: In most perception-related failure cases, the verifier rejects all trajectories since the road segmentation is very limited (Figure 5b) or not existent at all (Figure 5c). Further quantifying unsafe trajectories requires access to the test ground truth. However, we can track the rate with which all trajectories are rejected, which is 27.31% ± 0.93 on the KITScenes LongTail test split.

Q3.2: What happens in the latter case, and how do these errors affect adaptation?

R3.2: We don’t optimize the models on scenarios where all trajectories are rejected.

W2: The strongest claims [outperforming the challenge winners] depend on a different evaluation regime from online deployment [evaluating the final checkpoint vs. online eval].

R4: This is resolved by fixing a copy-paste error on our side. We copied the MMS scores from the official challenge leaderboard wrong. Specifically, we copied the MMS values for our method and the MMS (type) values of the challenge winners. MMS is a global average, MMS (type) is an average over the per scenario type averages. Apologies for this mistake! We updated Table 2 in the paper to include both scores in the correct ordering. Our TTLR Alpamayo version (self-distillation) evaluated in an online manner significantly outperforms the challenge winner TTVLM in both metrics (MMS: 5.11 vs 4.75, MMS (type): 5.46 vs. 5.15).

Since many reviewers asked about inference latency, we additionally worked on a real-time version of our verifier and added the results to Table 7. Our version (b) is real-time capable when run in parallel with the driving policy for high-level planning, which typically occurs at 4-5 Hz, matching the recording frequency of the Waymo Open E2E and KITScenes LongTail datasets. Alpamyo 1.5 itself can be accelerated by using quantization as in the version of Li et al. (2026).

Li et al. (2026), FlashDrive: Flash Vision-Language-Action Inference for Autonomous Driving. In arXiv

Thank you for the new experiments. The verifier-only comparison shows that weight updates improve online performance, and the held-out validation results support transfer to unseen scenarios. These additions address my main concerns. I will consider raising my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.