acceptodds
Under review as a conference paper at ICLR 2027

CoFiDrive: Scene-Conditioned Registers and Coarse-to-Fine Trajectory Scoring for End-to-End Driving

Abstract

Vision-based driving planners must compress dense multi-camera observations while retaining the information needed to generate and compare motion alternatives. Road layout and traffic configuration provide scene context, whereas nearby trajectories can differ in their clearance from vehicles or lane boundaries. We present CoFiDrive, which combines scene-conditioned register refinement with coarse-to-fine trajectory scoring. Four camera views contribute 16 learned registers each, forming a compact 64-token scene memory. A semantic token compressor (TC) refines these 64 tokens through scene-conditioned soft expert mixing, preserving the memory size for trajectory generation and scoring. The Cross-Compression Scorer (CCS) combines the refined scene features with spatially pooled patch features through candidate-dependent gated attention, giving each proposal access to coarse context and local spatial detail. During training, control-space augmentation (CSA) derives acceleration and yaw-rate sequences from PDM-ranked proposals, perturbs these controls, and uses bicycle-model rollouts to create additional candidates. The PDM evaluator supplies component-score targets for the same CCS on the original and augmented candidates. CSA broadens CCS supervision without adding model parameters. At inference, CCS ranks only the original 64 proposals, with no augmentation or PDM target computation. After 10 training epochs, CoFiDrive achieves 92.4 and 92.9 PDMS on NAVSIM-v1 using train85k and trainval103k, respectively, and 48.0 EPDMS on NAVSIM-v2 NavHard.

Primary area:
applications to robotics, autonomy, planning
Keywords:
End-to-End Autonomous Driving, Trajectory Planning, Scene-Conditioned Registers, Coarse-to-Fine Trajectory Scoring, Mixture of Experts, Control-Space Augmentation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_u1uCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CoFiDrive extends DrivoR with scene-conditioned register refinement, a scorer combining register and patch features, and control-space augmentation for scorer training. The paper reports 92.9 PDMS on NAVSIM-v1 and 48.0 EPDMS on NavHard, with component ablations examining the contributions of these additions.

Critical strengths
  1. The idea of giving the scorer access to finer visual features is sensible: trajectories that look similar at the scene level may differ in their clearance from nearby objects or road boundaries.

  2. CSA is also useful in principle, as it supplies additional scoring examples without adding inference candidates.

Critical weaknesses
  1. The additions to DrivoR are incremental, and the claimed improvement in trajectory scoring is not directly demonstrated. Each model generates its own proposals, while scorer training also changes the shared visual features. Higher PDMS therefore does not establish better ranking.

  2. The default configuration scores 91.85 in Table 4 versus 92.40 in Table 3. The authors acknowledge the difference but do not explain it. This gap exceeds several component gains, including those in the additional appendix controls. Without run-level results, these improvements are difficult to assess.

  3. Measured latency and peak memory are also missing, leaving the cost of the added experts and patch branch unclear.

Questions for authors
  1. Does CCS select higher-scoring trajectories than the baseline when evaluated on identical proposals? A comparison using common, fixed visual features would further isolate the scorer architecture.

  2. What explains the difference between Tables 3 and 4? Please provide individual run results for the key ablations, including the baseline with CSA alone if available.

  3. What are the end-to-end inference latency and peak memory relative to the baseline under identical hardware and execution settings?

Reviewer_44FnOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This work builds upon the DrivoR end-to-end driving approach and proposes three modifications to it. The first is the "Semantic Token Compressor" (TC), this is an additional module that that processes the 64 scene tokens produced by the DrivoR's vision encoder with a soft mixture of experts module. The trajectory-generation and trajectory-scoring decoders of DrivoR take as input the 64 scene tokens that this TC module outputs. The second is the "Cross-Compression Scorer" (CCS), which modifies the trajectory-scoring decoder of DrivoR to also take as input the patch tokens of the DrivoR's vision encoder, down-sampled to a lower resolution. Finally, the third proposed modifications is the "Control-Space Augmentation" (CSA) that during training only, it augments the trajectories that are processed by the trajectory-scoring decoder. The proposed approach, called CoFiDrive is evaluated on Navsim v1 and Navsim v2 navhard.

Critical strengths

The proposed trajectory augmentation (CCS) seems interesting as an augmentation technique and provides a small performance improvement of 0.5 on PDMS.

Critical weaknesses
  • The work has incremental novelty. The proposed modifications seem more engineering tricks or enhancements to the DrivoR framework. Moreover, they are not well-motivated; the motivation for these modifications, and especially TC and CCS is unclear. Furthermore, although these modification increase the complexity of the proposed method, introducing additional computation and hyperparameters to tune, it is unclear if they offer a significant performance improvement. Based on the ablation results, TC and CCS offer a very small improvement of 0.1 PDMS each, which is not even clear if it is statistically significant.
  • Moreover, there is not substantial empirical evidence that the proposed approach is actually better than the DrivoR approach, on which it builds upon. On Navsim v1, the reported results for CoFiDrive are worse than those of DrivoR (also, the authors omit to include DrivoR in Table 1 with the Navsim v1 results, which I found misleading). On Navsim v2, the EPDMS score of CoFiDrive is 48.0, which is worse than what DrivoR reports (48.3 EPDMS) and only a little better than the reproduced by the authors DrivoR results (47.3 EPDMS).

To summarize, this work proposes minor modifications to the DrivoR approach that, on the one hand, increase the complexity and computational demands, while, at the same time, it remains unclear if they offer any substantial performance improvements.

Questions for authors
  • Can the authors run CoFiDrive and DrivoR on navsim v2 using three different seeds and report the mean and the standard deviation. It would help to understand if CoFiDrive really improves over DrivoR on this dataset, or not.
  • Similarly, can the authors report the mean and std from three different seeds for the TC and CCS ablations. It would help understand if these modifications actually offer an improvement or not.
  • Compare CoFiDrive with DrivoR in terms of training time, peak GPU memory, and inference latency.
  • How does CoFiDrive perform on closed-loop benchmarks, like HUGSIM?
Reviewer_JpUcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces CoFiDrive, an end-to-end driving approach that combines scene-conditioned register refinement and coarse-to-fine trajectory scoring. Scene-conditioned register refinement is done by a token compressor, which refines register tokens from 4 camera views through scene-conditioned soft expert mixing. In addition, the register-based and refined scene features are aligned with ViT-patch features using a gated attention mechanism. This alignment step is done right before the final ego-trajectory generation. Specifically, the method generates 64 trajectory proposals from learned queries, ego commands and the compressed context. During training, CoFiDrive additionally augments generated trajectories by changing acceleration and yaw-rates. The method is evaluated using the NAVSIM v1 and v2 benchmarks.

Critical strengths

1. Using a soft mixture of experts mechanism to compress registers is a principled design choice. It is intuitive that not all camera views are equally important in every scenario or time step. Thus, “architecturally” encouraging the model to learn such a weighting of view-specific features is a good design.

  1. The paper includes in-depth descriptions of the implementation details of each proposed module (see sections 3.3 to 3.5) and the effect of each module on downstream motion planning is ablated in Table 3.
Critical weaknesses
  1. The comparison with DrivoR and other models is not consistent. In section 4.3, the results for DrivoR are based on a reproduction of the training process, while the results for other models are taken from the respective papers. Additionally, Table 1 does not include results for DrivoR. Table 1 and 2 should include the official results of DrivoR for reference (as mentioned in the section text). Additionally, it would be interesting to investigate why the reproduction of DrivoR is worse than the official results.
  2. The evaluation using NAVSIM focuses on nominal driving conditions, which are less challenging than unusual, long-tail scenarios.
Questions for authors

These questions are clarification questions and suggestions for the rebuttal:

  1. Why are the reproduced results of DrivoR significantly worse than the results reported in the paper of Kirby et al., 2025?

  2. Does CoFiDrive generalize to unusual, long-tail driving scenarios? E.g. perform a small-scale cross-dataset ablation with a few-shot or fine-tuning-based evaluation of models pre-trained on nuScenes/NAVSIM on WOD-E2E (Xu et al., 2026) or KITScenes LongTail (Wagner et al., 2026).

  3. Table 3 suggests that increasing the total and scorer parameters improves motion planning. How does CoFiDrive perform with the same or a similar number of parameters as the baseline GTRS V2-99 (i.e., 80M)?

Xu et al., 2026: WOD-E2E: Waymo Open Dataset for End-to-End Driving in Challenging Long-tail Scenarios. In CVPR.

Wagner et al., 2026: Reasoning models do not yet follow their reasoning in autonomous driving: The KITScenes LongTail Dataset. In arXiv.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.