acceptodds
Under review as a conference paper at ICLR 2027

Diversity Without Labels: Task-Agnostic Coreset Selection Across Perception and Action

Abstract

While data collection for model training is relatively easy, curating high-quality datasets from large collections is inherently challenging. Coreset estimation methods address this problem by identifying the best data subsets to train models on. However, existing approaches often require proxy model training, extensive annotations, or task-specific tailoring. Unsupervised alternatives typically exploit geometric properties in foundation model feature spaces, but are limited to pairwise image-to-image comparisons or comparisons against a fixed subset. We introduce CoreDNA, an unsupervised, task-agnostic coreset estimation approach that pools neuron activations from vision foundation models into per-image and per-dataset distributions. By comparing each image’s distribution to the dataset-level distribution, CoreDNA can capture both low-level visual structure and high-level semantic information across layers. We find that high-quality samples concentrate in the tails of the last layer’s activation statistics, which we validate by relating them to training-based coreset methods. Building on these findings, we develop a computationally efficient selection mechanism to construct task-agnostic, high-quality coresets for selective annotation and model training. Across image classification, semantic segmentation, depth estimation, and imitation learning for end-to-end driving and mobile manipulation, CoreDNA matches or outperforms the state of the art, demonstrating unprecedented generalization across perception and action tasks. These results highlight the effectiveness of compact activation-based representations for data selection.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Label Free Coreset Selection, Data Pruning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_y81VOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This work proposes CoreDNA, a label-free, training-free, and task-agnostic coreset selection method. CoreDNA uses a pretrained vision foundation model to extract neuron activation distributions and measures the differences between individual images and the full dataset. It then constructs compact representations from activation statistics and selects representative subsets by balancing data diversity and distribution coverage. The method can also incorporate additional information, such as temporal and motion features, to reduce redundancy in sequential data. Experiments across image classification, semantic segmentation, depth estimation, autonomous driving, and robotic manipulation demonstrate that CoreDNA achieves competitive performance against existing coreset selection methods across diverse tasks while remaining computationally efficient.

Critical strengths
  1. The proposed method is simple and novel. It leverages neuron activation distributions from a pretrained vision model for data selection without requiring labels or additional model training, and uses compact feature representations for efficient selection.
  2. The experiments cover a wide range of tasks, including image classification, semantic segmentation, depth estimation, autonomous driving, and robotic manipulation, with extensive comparisons and ablation studies.
Critical weaknesses
  1. The performance improvements over existing methods are limited and inconsistent. For example, CoreDNA underperforms existing unsupervised methods or even random sampling in some settings on CIFAR10 and Cityscapes-DVPS.
  2. As the paper emphasizes the computational efficiency of CoreDNA, it lacks systematic comparisons with existing methods in terms of end-to-end runtime and computational cost.
Questions for authors
  1. What factors affect CoreDNA's performance across different settings?
  2. Could the authors provide comparisons of end-to-end runtime and computational cost with existing methods to better demonstrate CoreDNA's efficiency?
2 comments

We thank reviewer y81V for highlighting the novelty of our approach and acknowledging the wide range of tasks. We address the critical weaknesses and questions below.

(W1) Performance over Baselines

We agree with the reviewer that CoreDNA does not win in every setting, and we do not claim to. We further agree that gains over baselines are not uniform. In the revised manuscript we will add the further discussion below on how random selection is hard to beat across the field, and state clearly where CoreDNA helps and where it does not.

Beating random selection reliably is hard for the whole field. Recent coreset methods are often at or below random in some settings, particularly at high retention rates, where the choice of method matters less, i.e. every baseline is within a point or two of random at α=0.7 (Cityscapes, PascalVOC, Cityscapes-DVPS, SemanticKITTI-DVPS). Further, training-based methods such as AUM, EL2N, and Forgetting fall below random at low ratios. In that landscape, our goal is a general method that performs consistently across 39 settings without per-dataset or per-task tuning, while maintaining performance of its respective predecessor.

CoreDNA makes progress in this setting. It matches or outperforms the strongest baselines in most settings, it holds up on PascalVOC and transfers to action domains, as shown on NAVSIM and LIBERO-10, and it does so at considerably lower cost, as we show in our compute comparison. The cases where CoreDNA is only up-to par, have identifiable reasons:

  1. CIFAR10 is a widely used benchmark that methods are easily overfit to, and we deliberately do not tune for it.
  2. On Cityscapes-DVPS, temporal stride is a particularly strong baseline, because it exploits dataset-specific structure that a task-agnostic method does not.

(Q1) Factors that affect performance

We thank the reviewer for this question, which led us to state explicitly when our method helps most. We will add a discussion of the factors that affect CoreDNA’s performance (dataset type, task, downstream model, and retention ratio) to the revised manuscript and below.

Dataset type

Curated datasets (CIFAR10, PascalVOC, Cityscapes) leave less redundancy to exploit than non-curated, sequential ones (Cityscapes-DVPS, SemanticKITTI-DVPS), where consecutive frames are highly uniform and near-full performance is reachable with very few samples. CoreDNA’s advantage is largest on the latter, consistent with its design. Where hard samples sit relative to μ₁₂ also varies across datasets (high tail on Cityscapes, low tail on PascalVOC, weak relation on CIFAR10), and we cannot yet predict this direction from unlabeled data.

Task, model, and retention ratio

Training-based selection isn’t always better, and which approach works depends on the task. Classification and segmentation on object-centric benchmarks (CIFAR10, PascalVOC) favor training-dynamics methods (AUM, EL2N, Forgetting), while dense driving-scene prediction (SemanticKITTI-DVPS, Cityscapes-DVPS) favors training-free embedding methods, where CoreDNA is most competitive, with FreeSel often the strongest baseline. Pretrained downstream models, such as the depth network, need less data to adapt, which shrinks differences between methods. Finally, the retention ratio changes the ranking: at high ratios all methods converge toward random selection, while at low ratios they separate clearly.

(Q2) End-to-end runtime and Computational Cost

We thank the reviewer for raising computational cost to emphasize the practical use of our method. We will add the following compute comparison to the revised manuscript, which shows that CoreDNA has the lowest end-to-end cost among the training-free methods we compare.

CoreDNA has the lowest end-to-end cost in both the small and the large selection regime: CoreDNAs total calculations cost is about 3$\times$ lower than ZCore (1.76 vs. 5.19 PFLOPs) and 2.4$\times$ lower than FreeSel at α=0.1\alpha=0.1 (1.76 vs. 4.15 PFLOPs). Our cost is dominated by the backbone, and the selection step itself is negligible. ZCore is more expensive in both the descriptor (about 3$\times$) and the selection step. FreeSel has the cheapest backbone (ViT-S instead of ViT-B), but it does not scale: its selection cost grows with the retention ratio and dominates at α=0.5\alpha=0.5, where its total is 10.8$\times$ ours (18.9 vs. 1.76 PFLOPs). To align with FreeSel, we are evaluateing whether a ViT-S backbone gives comparable selection quality. We will post the results of these runs here once they are complete and include them in the revised manuscript. We will also report measured wall-clock runtimes to this comparison once they complete.

Reviewer_y81VOpenReview ↗

Response to Author Rebuttal

Thanks for the clarifications and computational cost comparisons. However, my main concern remains the limited and inconsistent performance improvements over existing methods, which have not been fully addressed. Therefore, I will maintain my original score.

Reviewer_vbEAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes CoreDNA, a training-free and label-free coreset selection method designed to work across different perception and action tasks. The authors build on Visual DNA by extracting neuron activation distributions from a pretrained vision foundation model and comparing each image with the overall dataset distribution. They then summarize these distributions using a few statistics, including the mean and standard deviation of the last-layer activation distances and cross-layer rank dispersion. Based on these representations, the authors propose two selection strategies: sliced Wasserstein sampling for distribution matching and farthest-point sampling for diversity. The method is evaluated on several tasks, including image classification, semantic segmentation, depth estimation, autonomous driving, and robotic manipulation.

Critical strengths
  • The paper is generally well written and easy to follow. I find the motivation interesting, especially the idea of using dataset-level activation statistics for coreset selection rather than relying only on pairwise feature comparisons.

  • I appreciate the broad experimental evaluation. The authors test the method across different datasets and downstream tasks, including perception and imitation learning.

Critical weaknesses
  • The paper builds on Visual DNA to extract neuron activation distributions, then uses existing selection techniques such as sliced Wasserstein sampling and farthest-point sampling. I think the main contribution lies in the proposed compact representation rather than the selection algorithms themselves. However, I find it difficult to tell how much the activation-based representation actually contributes to the improvement. Although the authors compare CoreDNA with methods such as FreeSel and ZCore, these methods also use different selection strategies. I would like to see a comparison using standard foundation-model embeddings with the same FPS and sliced Wasserstein selection procedures. This would help establish whether the proposed representation is indeed more effective than conventional visual features.

  • I appreciate that the authors extend the evaluation beyond perception tasks to autonomous driving and robotic manipulation. However, I find the results on LIBERO-10 less convincing.

  • For frame-level selection with five demonstrations, CoreDNA improves the average success rate from 16.9% to 17.3%. Similarly, with 50 demonstrations, the improvement is from 76.7% to 77.8%. These differences are relatively small compared with the reported standard deviations.

  • The demonstration-level selection experiment shows a larger improvement, but it uses a modified selection strategy that retrieves demonstrations based on their similarity to the target demonstrations, rather than the original dataset-distribution matching objective.

  • I think the authors should distinguish these two settings more clearly and provide stronger evidence that the proposed selection method consistently benefits imitation learning.

Questions for authors
  • Could the authors compare the proposed activation-based representation with standard pretrained visual embeddings using the same FPS and sliced Wasserstein selection procedures? This would help clarify how much of the improvement comes from the proposed representation rather than the selection algorithms.

  • The paper shows that FPS and sliced Wasserstein sampling perform differently across datasets. How should one choose between these two strategies for a new dataset without evaluating downstream performance? Is there a way to make this decision using only the unlabeled data?

  • In Section 3.3, the authors use 5,265 historical training runs to identify the most useful feature statistics. Since these runs cover the datasets and task families used in the evaluation, could the authors test whether the selected representation generalizes to held-out datasets that were not involved in this analysis?

  • The improvements in the LIBERO-10 frame-level selection experiments are relatively small compared with the reported standard deviations. Could the authors provide further evidence that these gains are consistent? Also, the demonstration-level experiment uses a modified selection strategy based on similarity to the target demonstrations. How much of the improvement can be attributed to the proposed activation representation rather than this change in the selection objective?

3 comments

[Part 1 of 3]

We thank reviewer vbEA for appreciating the clarity of the manuscript and the breadth of our evaluation. We address the main weaknesses and questions below.

(Q1) Ordinary Features

We thank the reviewer for pointing us towards a controlled comparison where only the representation changes to make the paper’s claim much clearer. We compared our descriptor against standard pooled features from the same encoder under the same sampler. Under the controlled comparison, our 3-dimensional descriptor is better at low and mid ratios and equal at high ratios.

We ran sliced-Wasserstein selection on two standard variants from the same frozen encoder: mean pooling over the last layer (1 dimension), and mean pooling over the patches only (768 dimensions). The sampler, retention ratio, and training budget are identical across the first three rows.

We ran selection with sliced-Wasserstein sampling on two ordinary variants from the same frozen encoder: mean pooling over the last layer (1 dimension), and mean pooling over the patches only (768 dimensions). Sampler, retention ratio, and training budget are identical across rows. We present results on CIFAR10 below. Our descriptor is ahead of both variants by at least 0.28pp at α=0.01 and by 0.43 pp at α=0.1. At α=0.5 the margin shrinks to 0.04 to 0.07 points as the selections converge, so we regard the methods as equal there.

FeaturesDim.0.010.10.5
Ordinary, mean pooling141.5381.9793.42
Ordinary, mean pooling patches76841.6781.6893.45
Ours (CoreDNA)341.9582.4093.49

The revised manuscript will include the comparison to ordinary features for CIFAR 10, Cityscapes (including FPS) and PascalVOC.

(Q2) Selection strategies

We thank the reviewer for this question. We agree stating a clear default and its limits makes the method easier to use and is of the readers interest. We will update the revised manuscript to recommend sliced-Wasserstein selection as the default, state the conditions under which FPS helps, and include three possible directions in the discussion of future work.

Sliced-Wasserstein (WS) sampling is the stronger default: it wins on 6 of our 8 datasets. FPS only outperforms WS when its selection axes (μ₁₂, σ₁₂) are sufficiently orthogonal and the dataset’s hard samples concentrate in the tail of our 3-dimensional descriptor. The second condition can only be verified post hoc, through a difficulty signal that requires labels and a trained model. We searched for an unlabeled, neuron pooled-proxy for it (axis geometry, distributional statistics across layers, cross-layer outlier consistency) and found none that reliably separates the FPS-favoring datasets from the rest. We therefore recommend WS for new, unlabeled datasets, and FPS only when a downstream evaluation or labeled difficulty estimate is available to check the condition. We set two directions to explore where the hard samples lie, which we now mention in the paper:

  • A small labeled probe. Train a cheap model on a few hundred labeled samples and check where hard samples fall along μ₁₂. This needs minimal annotation, unlike full training dynamics.
  • Pseudo-difficulty from a frozen model. Use the loss or entropy of a frozen foundation model’s zero-shot or linear-probe predictions as a stand-in for difficulty, and test whether it correlates with tail position.

Last, we also highlight the possibility of a hybrid selector in future work. The idea entails that the budget is split between WS and FPS, or choosen per axis or per region of the feature space, so that no prediction is needed.

[Part 2 of 3]

(Q3) Held-out dataset

We thank the reviewer for this question. Since we choose the axes using performance on the same datasets, we tested whether the conclusion holds on a dataset left out of the choice. We repeated the correlation analysis with each dataset held out in turn. The three-axis space stays the most correlated with downstream performance, with only small changes in the values (details below). We added this analysis to the appendix.

We compute all correlations separately for each dataset (Tab. 12), subtracting the per-ratio mean from both the score and the downstream performance so that coreset size does not drive the result. We have modified the manuscript to clarify. To test stability, we repeated the analysis with each dataset held out in turn. The three-axis space remains the most correlated with both coverage and fidelity in every case. The absolute values change only slightly: averaged over all seven hold-outs, the mean absolute change is 0.010 (3.2%) for coverage and 0.021 (5.8%) for fidelity. The largest changes are −0.018 (−5.9%) for coverage and +0.052 (+14.7%) for fidelity. We conclude that the choice of axes is stable and does not depend on any single dataset.

Dataset held outCov. μ₁₂Cov. μ₁₂ + σ₁₂Cov. μ₁₂ + δ_maxCov. μ₁₂ + σ₁₂ + δ_maxFid. μ₁₂Fid. μ₁₂ + σ₁₂Fid. μ₁₂ + δ_maxFid. μ₁₂ + σ₁₂ + δ_max
CIFAR100.29 (+0.02)0.29 (+0.01)0.29 (+0.02)0.33 (+0.02)0.31 (+0.01)0.35 (+0.01)0.35 (+0.01)0.37 (+0.01)
PascalVOC0.27 (−0.01)0.27 (−0.01)0.27 (−0.01)0.30 (−0.01)0.29 (−0.01)0.33 (−0.01)0.33 (−0.01)0.35 (−0.01)
Cityscapes0.25 (−0.02)0.28 (+0.00)0.28 (+0.00)0.31 (+0.00)0.35 (+0.05)0.39 (+0.05)0.39 (+0.05)0.41 (+0.05)
Cityscapes-DVPS (Depth)0.29 (+0.01)0.29 (+0.01)0.27 (−0.00)0.31 (+0.00)0.27 (−0.03)0.32 (−0.02)0.31 (−0.03)0.33 (−0.02)
Cityscapes-DVPS (Segm.)0.25 (−0.02)0.27 (−0.01)0.26 (−0.01)0.29 (−0.02)0.27 (−0.03)0.31 (−0.03)0.31 (−0.03)0.32 (−0.04)
KITTI-DVPS (Depth)0.27 (−0.00)0.27 (−0.01)0.27 (−0.01)0.30 (−0.01)0.30 (+0.00)0.34 (−0.00)0.34 (−0.00)0.35 (−0.00)
KITTI-DVPS (Segm.)0.29 (+0.02)0.28 (+0.00)0.29 (+0.01)0.32 (+0.01)0.31 (+0.01)0.35 (+0.01)0.35 (+0.01)0.37 (+0.01)
All 7 (reference)0.2730.2800.2760.3080.3000.3410.3400.357

[Part 3 of 3]

(Q4) Extension to Imitation Learning experiments

We thank the reviewer for this careful reading. Separating the two settings and being specific about what each shows makes the section clearer and the claims trustworthy. We separated the two LIBERO-10 settings in the manuscript, added smaller retention ratios and more baselines for frame-level selection, and emphasize the necessary objective change for the demonstration-level selection.

Frame-level selection

We agree that the gains are small compared with the reported standard deviations (16.9% to 17.3% with five demonstrations, 76.7% to 77.8% with 50 demonstrations), and we update the revised manuscript to no longer present them as improvements. We rather interpret them as evidence of redundancy: frames can be pruned with CoreDNA without a measurable loss in performance. To give stronger evidence, we are conducting experiments with retention ratios of 0.1 and 0.5 for the 50-demonstration setting, as well as Random, FreeSel, and ZCore as additional baselines. We will post the results of these runs here once they are complete and include them in the revised manuscript.

Demonstration-level selection

Please note that in the demonstration-level selection experiment compared to frame-level selection the goal changes. Specifically, frame-level selection aims for diversity within a dataset, so matching the dataset’s own distribution is the right target. On the other hand, in our mix-in selection experiment we have a set five of fixed target demonstration, which we want to supplement with auxiliary demonstrations to broaden the training corpus [1]. Thus, intuitively, we want to select a set of auxiliary demonstrations which are most similar to the five target demonstrations, resulting in a retrieval problem.

With the same descriptor, the distribution-matching variant, CoreDNA (WS), reaches only 19.0%, close to using all demonstrations (18.3%), while the retrieval variant, CoreDNA (CR), reaches 27.3%. Most of the improvement therefore comes from the objective and we say so in the revised text. What the experiment does show is that the pooled-activation descriptor additionally works as a sequence-level representation for retrieval, reaching a result comparable to the training-based DataMIL (25.3%) without any training. Evidence for the descriptor’s quality against alternative representations is in our response to the first question. We moved the mix-in experiment into its own paragraph, explained why its objective differs from frame-level selection, and now state that most of its gain comes from the retrieval objective.

[1] Shivin Dass, Alaa Khaddaj, Logan Engstrom, Aleksander Madry, Andrew Ilyas, and Roberto Mart´ın-Mart´ın. DataMIL: Selecting data for robot imitation learning with datamodels. In The Fourteenth International Conference on Learning Representations, 2026. URL https://openreview.net/forum?id=AcTsKglDdh

Reviewer_De8cOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CoreDNA tackles the problem of one-shot coreset selection without using labels or task-specific proxy training. It compares neuron-activation histograms for each image from a vision transformer against aggregated histograms from the candidate pool. These comparisons are summarized into statistics like standard deviation at the final layer along with a cross-layer rank-dispersion measure. Subsets are then selected using quantile-based sampling, inspired by sliced Wasserstein distance or farthest-point sampling with use of recording metadata. The experiments include classification, segmentation, depth estimation, driving and manipulation tasks. The paper also explores how these descriptors relate to subset coverage, distribution fidelity and downstream performance. It introduces a target-reference version for selecting demonstrations.

Critical strengths
  1. The compact representation is practical and appealing. The evaluation spans five task types, which gives strong evidence that the method works beyond just classification. There are improvements: on Pascal VOC at 10% retention CoreDNA achieves 59.21 mIoU compared to ZCore’s 57.52; on NAVSIM, the best half-data setup reaches 79.98 EPDMS, close to the full-data result of 80.51 (Tables 2 and 4).

  2. The empirical analysis goes beyond benchmark rankings. The study of 5,265 runs examines coverage and fidelity against performance at fixed retention ratios avoiding confounding from subset size. The sequence and duplication experiments also show when temporal diversity or redundancy matters more than hardness-based ranking (§3.3 Table 3 Appendix A.9).

Critical weaknesses
  1. The value of the proposed descriptor over existing methods isn’t clearly isolated: Visual DNA uses averaged neuron-wise histogram distances to compare images with reference datasets. FreeSel and ZCore already show that training-free selection from pretrained features works. CoreDNA adds axes to this representation and applies it to coreset selection but ablation studies are limited to CIFAR-10. On CIFAR-10 the final-layer mean alone performs as well as the full descriptor: 82.38 vs. 82.40 At 10% retention and 93.46 vs. 93.49 At 50% (Table 12). In cases changes involve the representation, sampler or metadata all at once. So it's unclear whether the added statistics improve selection performance or offer a computational advantage over simpler options. Timings for selection are also missing, making it hard to judge efficiency.

  2. The action experiments don’t clearly show CoreDNA’s advantage in its new domains. NAVSIM compares against selection but skips competitive feature-based selectors (Table 4). LIBERO frame pruning compares 90% retention to data but lacks a random-pruning control. The best result uses an adapted target-reference CR procedure: 27.3% success vs. DataMIL’s 25.3% (Tables 5 and 8). This is promising,. The uncertainty around the difference is not clear. These experiments suggest applicability. They don’t strongly prove that CoreDNA beats relevant alternatives in action domains.

Questions for authors
  1. In one representative segmentation setting could you compare the descriptor, the final-layer mean alone and ordinary features from the same frozen encoder? Keep the sampler, metadata, retention ratio and training budget the same. Include selection time to see if the performance gain comes from representation or just compression.

  2. Was DataMIL rerun using the five target demonstrations and the same mixing and training settings as CoreDNA–CR? Can you share per-seed success rates or the uncertainty of their difference?

  3. Could you add a same-budget random-pruning control for one of the existing LIBERO frame-pruning settings? This would help clarify whether the benefit comes from CoreDNA’s selection method or just from removing frames in general.

6 comments

[Part 1 of 3]

We thank reviewer De8c for praising the benefits of our work and respond to the questions and critical weaknesses below:

(W1) Layer Selection for the Descriptor

We thank the reviewer for pointing us towards clearly separating the contribution of the descriptor from the datasets and samplers around it. We thus extended the descriptor ablation beyond CIFAR10 and added a compute comparison to the main paper.

Isolating the value of the descriptor

Our descriptor is built from the shape of the neuron activation distributions, not from a pooled feature vector, and it is measured relative to the dataset. This per-image, dataset-relative representation, the analysis of where useful samples sit in it, and the selection mechanisms on top are our contribution. We agree that CIFAR10 alone cannot separate these, since the final-layer mean is already close to the full descriptor there (82.38 vs. 82.40 at α=0.1, 93.46 vs. 93.49 at α=0.5). We therefore extended the axis ablation to Cityscapes, PascalVOC, Cityscapes-DVPS, and SemanticKITTI-DVPS, where the added axes improve over the mean alone in 14 of 25 tested cases (16 of 25 when counting differences within noise as on par). The gain is largest on curated datasets such as Cityscapes, where the added axes help in 9 of 12 cases (up to +1.9 mIoU at α=0.1). On the sequential datasets we report the variant with the temporal frame-index axis, which is the stronger configuration there. Here the mean combined with the temporal axis is also highly competitive, so most of the gain comes from temporal structure and not from the added statistics. The correlation analysis over 5,265 runs points the same way: the three-axis space correlates better with downstream performance than the mean alone, for both coverage (0.308 vs. 0.273) and fidelity (0.357 vs. 0.300). Together, these results indicate that the added statistics help in general settings, though the margins are modest.

Computational advantage

We have also added a compute comparison to the main manuscript. CoreDNA has the lowest end-to-end cost in both the small and the large selection regime: about 3× lower than ZCore (1.76 vs. 5.19 PFLOPs) and 2.4× lower than FreeSel at α=0.1 (1.76 vs. 4.15 PFLOPs). Our cost is dominated by the backbone (35.1 GFLOPs per image), and the pooling () and selection step is negligible (11.6 MFLOPs for sliced-Wasserstein selection, 2.3 GFLOPs for FPS). This makes the descriptor choice stronger on two fronts: the added statistics improve how well selection predicts performance, and they do so at a cost that is a small fraction of the backbone’s.

DatasetFeatures0.010.10.30.5
CIFAR10 (acc. ↑)μ₁₂ only (WS)-82.38-93.46
CoreDNA (WS)-82.40-93.49
Cityscapes (mIoU ↑)μ₁₂ only (WS)50.0965.8171.54†73.95
CoreDNA (WS)49.9266.9872.4274.40
μ₁₂ only (FPS)49.4365.4272.2074.69
CoreDNA (FPS)50.2067.3572.7674.46
PascalVOC (mIoU ↑)μ₁₂ only (WS)-59.04-77.25
CoreDNA (WS)-59.21-77.09
Cityscapes-DVPS Segm. (mIoU ↑)μ₁₂ only + T (WS)37.8055.9962.7163.93
CoreDNA (WS+T)38.5155.3862.0464.77
Cityscapes-DVPS Depth (RMSE ↓)μ₁₂ only + T (WS)6.174.364.01-
CoreDNA (WS+T)6.104.394.08-
SemanticKITTI-DVPS Segm. (mIoU ↑)μ₁₂ only + T (WS)38.8950.55-55.74
CoreDNA (WS+T)37.7351.37-55.99
SemanticKITTI-DVPS Depth (RMSE ↓)μ₁₂ only + T (WS)3.753.363.34-
CoreDNA (WS+T)3.773.363.35-

[Part 2 of 3]

(Q1) Comparison to Ordinary Features

We thank the reviewer for this valuable suggestion and tested the descriptor against simple alternatives from the same encoder to make our claims much clearer. We compared our descriptor against standard pooled features from the same frozen encoder. In short, our descriptor improves selection at low and mid retention ratios and matches the standard features at high ratios, at a cost of less than 1 MFLOP more per image. Details are below, and we add them to the revised manuscript.

We ran selection with sliced-Wasserstein sampling on two ordinary variants from the same frozen encoder:

  1. mean pooling over the last layer (1 dimension), and
  2. mean pooling over the patches only (768 dimensions). Sampler, retention ratio, and training budget are identical across rows. We present results on CIFAR10 below. Our descriptor is ahead of both variants by at least 0.28pp at α=0.01 and by 0.43 pp at α=0.1. At α=0.5 the margin shrinks to 0.04 to 0.07 points as the selections converge, so we regard the methods as equal there.

On cost, computing our descriptor adds 0.86 MFLOPs per image on top of the backbone, which is 0.0024% of its total cost. Mean pooling with ordinary features adds 0.15 MFLOPs. The gain therefore does not come from compression alone, because the 768-dimensional variant does not beat the 3-dimensional one. To ground these FLOPs in time measurements, we will also report measured wall-clock runtimes once they complete.

In the revised manuscript we will include measured wall-clock timings and the aforementioned comparison to ordinary features for CIFAR 10, Cityscapes (including FPS) and PascalVOC.

FeaturesDim.0.010.10.5
Ordinary, mean pooling141.5381.9793.42
Ordinary, mean pooling patches76841.6781.6893.45
Ours (CoreDNA)341.9582.4093.49

[Part 3 of 3]

(Q2, Q3) Extension of Imitation Learning Results

We thank the reviewer for these concrete suggestions, strengthening our imitation learning evidence. We report per-seed success rates below and will include this table in the appendix of the revised manuscript. We are further currently running experiments with FreeSel and ZCore on NAVSIM and LIBERO-10, with same-budget random controls and more retention ratios. We describe the result against DataMIL as on par.

Competitive selectors and random control

We added Temporal Stride, FreeSel and ZCore as baselines on NAVSIM. On LIBERO-10, we extend the study to retention ratios of 0.1, 0.5, and 0.9 for the 50-demonstration setting and 0.9 for the 5-demonstration setting, each with a same-budget random-pruning control alongside FreeSel, ZCore, and CoreDNA. We will post the results of these runs here once they are complete and include it in the revised manuscript.

DataMIL comparison

In our initial submission DataMIL is run with the same five target demonstrations as well as with the same mixing and training settings as CoreDNA (CR), both following the DataMIL protocol. We report per-seed success rates below and will add them in the appendix for the revised manuscript. Across seeds, the 2.0-point difference between CoreDNA (CR) and DataMIL (27.3% vs. 25.3%) is smaller than the seed-to-seed variation, thus, we consider them on par. Nonetheless, our experiments show that a training-free representation built from pooled activations can match a training-based selector in this setting. We have revised the text in the manuscript to say “match” and avoid implying that CoreDNA outperforms DataMIL.

TaskDataMIL s0DataMIL s1DataMIL s2DataMIL meanCoreDNA-CR s0CoreDNA-CR s1CoreDNA-CR s2CoreDNA-CR mean
Soup-Sauce28303631.328405240.0
Cream-Butter610129.30223218.0
Stove-Moka56344846.028326240.7
Bowl-Cabinet74446059.362786869.3
Mug-Mug0082.72031.7
Book-Caddy74484656.062244844.7
Mug-Pudding2284.00102612.0
Soup-Cheese10161212.720204829.3
Moka-Moka8826.010446.0
Mug-Microwave26261622.71281210.7
Average28.421.824.825.022.423.835.527.2

Comments on the paper

2 comments
AuthorsOpenReview ↗

Main Response

We thank the reviewers for their time and constructive comments. CoreDNA is a label-free, training-free coreset selection method built on dataset-relative neuron-activation statistics, and reviewers highlighted its simplicity and novelty and the breadth of the evaluation. Each of our replies below follows the same structure: a one-sentence summary of what changed in the paper, then the details and numbers. We answer each concern directly under the reviewer’s official review, and we welcome further feedback.

Reviewer y81V

Strengths: the method is simple and novel, with broad evaluation across tasks.
Concerns: improvements over baselines are limited and inconsistent; no systematic comparison of runtime and cost.
Response: we added a compute comparison and a discussion of when CoreDNA helps and why random selection is hard to beat across the field.

Reviewer De8c

Strengths: the compact representation is practical and appealing, and the analysis goes beyond benchmark rankings.
Concerns: the descriptor's value over simpler options is not isolated; selection timings are missing; the action experiments lack competitive selectors, a random control, and uncertainty on the DataMIL comparison.
Response: we isolated the descriptor against standard pooled features under the same sampler (as in our reply for vbEA), added a compute comparison, and strengthen the NAVSIM and LIBERO-10 results with more baselines, random controls, and per-seed results.

Reviewer vbEA

Strengths: dataset-level activation statistics are an interesting alternative to pairwise comparisons, and the evaluation is broad.
Concerns: how much the representation, as opposed to the sampler, contributes; small LIBERO-10 gains and the modified demonstration-level objective; how to choose between WS and FPS; generalization to held-out datasets.
Response: the controlled feature comparison isolates the representation (as in our reply for De8c), the two LIBERO-10 settings are now separated, and we added a WS-versus-FPS recommendation and a held-out dataset check.

After revisions, the paper includes: the controlled feature comparison, the compute comparison, stronger imitation learning results, the held-out analysis, and a discussion of selection strategy and performance factors. We will upload a consolidated revised manuscript containing all changes by the end of the rebuttal phase.

We thank all reviewers for their time, their constructive feedback, and the discussion. The revised manuscript is now online, with all changes highlighted in blue. The main changes are:

Descriptor vs. standard features (De8c, vbEA): a controlled comparison with ordinary pooled features of the same encoder under the same sampler (Tab. 19), and an axis ablation across datasets (Tab. 18).
Cost (y81V, De8c): measured runtime and FLOPs against ZCore and FreeSel (Sec. 4.3, Tab. 6), and a ViT-S backbone check (Tab. 16).
Imitation learning (De8c, vbEA): FreeSel, ZCore, and random baselines and controls on NAVSIM and LIBERO-10 at several retention ratios, separate treatment of frame-level and demonstration-level selection, and per-seed success rates (Tab. 10).
Generalization of the analysis (vbEA): a leave-one-dataset-out check (Tab. 15) and a per-ratio breakdown (Tab. 14).
Applicability and Limitations (y81V, vbEA): Sec. 4.4
Claims: the abstract and Sec. 4 now states precisely where CoreDNA is ahead and where it is on par.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.