Valuing Data Before Training: Geometric Coverage and Topological Fidelity Outperform Gradient-Based Selection
Abstract
Estimating the value of training data often requires training the target model, adding cost before useful subsets can be selected. We introduce C-GeoVal (Coverage- Geometric Valuation), which values samples before target-model training using frozen foundation-model embeddings. Sample value is defined as the expected marginal gain of a set utility that rewards density-weighted coverage and penalizes topological distortion. A scalable coreset procedure combines density filtering and greedy coverage maximization with persistence-guided subset repair. At an 80% data budget, C-GeoVal improves CIFAR-10N accuracy by 4.93 percentage points over full-data cross-entropy training and GoEmotions Macro-F1 by 2.06 points over full-data BERT fine-tuning. Its valuation scores detect human label errors with AUROC 0.937. Ablations show gains from topological repair in both tasks. For mathematical instruction tuning, the density-and-coverage variant outperforms DDCF at all three tested budgets on shared embeddings.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.