acceptodds
Under review as a conference paper at ICLR 2027

CASA: Cluster-Adaptive Scaling for Hierarchical Data Selection in End-to-End Autonomous Driving

Abstract

As autonomous driving datasets continue to grow, efficiently constructing training sets from large-scale data is becoming increasingly important. Effective selection therefore requires considering not only which individual samples are valuable, but also how the training budget should be distributed across different scene types. We introduce CASA, a hierarchical data-selection framework that jointly addresses these two levels. CASA first organizes driving data into behaviorally meaningful clusters by coupling semantic context with future motion, and performs task-aware ranking within each cluster. At the cluster level, CASA learns a structure-aware scaling model with shared parameters across clusters while accounting for shared coverage between related clusters, allowing a small number of probes to guide marginal-utility-based budget allocation. By integrating sample-level selection with structure-aware cluster allocation, CASA provides a unified approach to constructing training sets under varying budgets. On NAVSIM with DiffusionDrive, CASA learns effective scaling relationships from low-budget probes and maintains the highest mean Predictive Driver Model Score (PDMS) among five representative data-selection baselines even when the training budget exceeds four times the largest probe size. This sustained advantage demonstrates CASA’s effectiveness in improving data efficiency for end-to-end driving.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Autonomous Driving, Data Selection, Data Scaling

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_gcg8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies data selection for end-to-end autonomous driving and proposes CASA, a hierarchical framework that combines sample-level selection with cluster-level budget allocation. The method performs task-aware sample ranking within scene clusters and uses a shared-parameter scaling model to allocate the training budget across clusters. Experimental results demonstrate its effectiveness on NAVSIM with DiffusionDrive, with additional evaluations on WOD-E2E and TransFuser.

Critical strengths
  1. Well-motivated problem. The paper addresses an important problem in end-to-end autonomous driving by jointly considering the value of individual samples and the allocation of training data across different scene clusters.
  2. Effective method with good results. The proposed framework achieves promising performance across multiple training budgets, and the ablation studies provide evidence for the contributions of its individual components.
Critical weaknesses
  1. Limited experimental validation and generalization analysis. Although the paper includes cross-dataset and cross-method experiments, the evaluation still covers a limited range of model architectures and benchmarks. In particular, it would be valuable to evaluate CASA on more recent architectures, such as VLA models, and closed-loop benchmarks, such as CARLA, to assess whether the proposed data selection strategy generalizes to different model paradigms and more realistic interactive driving scenarios. Furthermore, the performance gains in some generalization experiments are relatively small. For example, the improvement over the strongest baseline on WOD-E2E at 1,311 training samples is only 0.0046 in RFS. Additional experiments would help establish the robustness and general applicability of the proposed method.
  2. Limited evaluation at larger data scales. The evaluated training budgets remain relatively small, with the main NAVSIM experiments considering up to 3,000 training samples. Although the results suggest that CASA can generalize beyond the budget range used to fit its scaling model, it remains unclear whether its advantages persist at substantially larger training scales. Given the large volume of data typically involved in end-to-end autonomous driving, evaluating the method with larger training subsets would provide stronger evidence of its scalability and practical effectiveness. So given the limited results, my initial recommendation is rejection.
Questions for authors

None.

Reviewer_UT8POpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes CASA, a hierarchical clustering-based data selection framework for training end-to-end autonomous driving models. CASA jointly addresses how to allocate training budget across clusters and which samples to select from the clusters. The paper claims the following main contributions: the hierarchical framework for cluster/sample selection, a task-aware intra-cluster ranking scheme and a method to estimate scaling laws for data clusters using limited probes. The proposed data selection approach is shown to produce consistently better performance compared to other baseline data selection strategies on the NAVSIM benchmark dataset.

Critical strengths
  • The methodology to train motion-alogned scene representations is interesting. This can produce task aware representations which can in turn produce task aware scene clusters, in contrast to prior work that used clusters based on geolocation and domain captions.
  • Explicit inclusion of the quality of supervision in the sample ranking is novel and interesting. The proposed ranking scheme combines PDMS differences and attempts to avoid samples that appear difficult because of poor supervision.
Critical weaknesses
  • The contributions of the paper are incremental compared to MOSAIC. Hierarchical clustering, within-cluster ranking, scaling-curve fitting and iterative marginal gain allocation are componnets that were also present in MOSAIC. While the overall framework might not be novel, there are differences in the individual components. Isolating and emphasizing the impact of these changes might still strengthen this paper even if novelty might be lacking.
  • MOSAIC [1] largely used the improved EPDMS metric for evaluation from NAVSIM v2 whereas this paper uses PDMS, without any valid motivation for this choice. This makes it difficult to contextualize the results in this paper to previous results from MOSAIC. The ranking of methods in the PDMS vs training samples plot (Figure 2) seems to deviate from a similar plot in MOSAIC using EPDMS as the performance metric.

[1] Dimlioglu et al. Scaling-Aware Data Selection for End-to-End Autonomous Driving Systems. CVPR 2026.

Questions for authors
  • Can the authors provide EPDMS equivalent of Figure 2 that can be easily related to MOSAIC results?
  • The InfoNCE objective proposed in eq. 3 encourages the scene representation to keep information that is shared between the scene and the motion. But the scene-exclusive and motion-exclusive information is discarded in this representation learning step. Have the authors considered the implications of this? Are there any limitations of this choice?
  • The choice of τsoft=20\tau_{soft}=20 for soft cluster membership in eq. 11 is arbitrary. How was this value selected and why does it need to be as high as 20? Also, is it correct that eq. 11 uses unnormalized vectors hmh_m and cic_i? What is the reasoning behind that choice when the structural dispersion used normalized vectors?
  • By using a 2000-sample test subset for fitting the scaling laws, is there a risk of leaking the test data/semantic distributions to the training? If that is the case, are we selecting data that is generally useful to train a good driving model or are we simply trying to find most representative data that boosts test performance?
Reviewer_LpEKOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

In this paper, the authors proposed CASA, a hierarchical data-selection framework that addresses how to efficiently construct the training sets from large-scale data. With DiffusionDrive on NAVSIM, it gives the highest mean PDMS among five baselines at budgets.

Critical strengths
  1. The proposed method is accordingly designed and tested.

  2. The perspective of this task is interesting.

Critical weaknesses

Weaknesses:

  1. When using a budget of 600-3000 samples out of over 100k navtrain samples, the PDMS is only 0.54~0.66, which is far below the results in DiffusionDrive. In my opinion, this kind of trade-off is not meaningful in real practice, specifically in the autonomous driving field.

  2. Using DiffusionDrive is too narrow to evaluate this motivation, for example, how about using VLA, WAM, or other methods? It is not clear whether this finding can be extended to other types of models.

  3. In the experiments, there are five parameters fitted from seven probes, all between 480 and 700 samples. The paper never shows predicted vs. observed PDMS at extrapolated budgets, so “learns effective scaling relationships” is not only supported. In addition, when removing CASA ranking, which causes the largest ablation drop, the actual mechanism behind the gains is unclear.

  4. The logic behind equation 8 seems not correct, a sample the planner already handles well (γ > 0) is promoted exactly as much as one it handles poorly (γ < 0) with the same |γ|. This is not aligned with the claimed “model deficit” motivation.

Minors:

  • Some typos in Figure 1, in caption "Stucture"; in (d) "Structral", in (e) "Samlple"
Questions for authors

Please see the Weaknesses section. The main concerns are about motivation, experiments, and scoring logic.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.