- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes CuBAS, a sample selection method for supervised classification. It constructs a k-nearest-neighbor graph over labeled data and models local label interactions using a Potts Markov random field. A ratio of node-wise Fisher-information-related quantities is interpreted as a curvature score. An adaptive threshold separates samples into low- and high-score groups, from which equal proportions are sampled to form a training subset. Experiments evaluate a k-NN classifier on 68 benchmark datasets across multiple training fractions. The paper reports substantial improvements over random and entropy-based sampling, attributing these gains to preserving boundary information and representativeness while reducing redundancy.
Critical strengths
-
The motivation to balance decision-boundary information with broad data representativeness is reasonable. Identifying informative samples while avoiding redundant selections is an important problem, and local label interactions provide a potentially useful signal.
-
The paper reports substantial improvements over random and entropy-based sampling across 68 datasets and multiple training budgets. The breadth of datasets and repeated evaluations provide a useful empirical starting point for investigating the proposed approach.
Critical weaknesses
-
Unclear connection between the score and boundary informativeness. The paper does not establish why its score identifies boundary samples or redundant regions. For a binary neighborhood with class counts and observed label A, ignoring regularization gives . For positive , stronger agreement with the observed label increases this magnitude, challenging the interpretation that high indicates class interfaces while homogeneous regions have near-zero curvature.
-
Insufficient justification of the curvature interpretation. Under regularity conditions, the expected squared score and expected negative log-likelihood Hessian are two expressions of the same Fisher information. Their observed counterparts can differ, but this does not establish them as first and second fundamental forms. The appendix derives these quantities algebraically without providing the geometric construction needed to identify their ratio as a shape operator.
-
Unsubstantiated redundancy reduction and insufficient baselines. Algorithm 1 samples the same proportion from both strata, without explicitly discouraging selection of mutually similar samples. Consequently, the claimed redundancy reduction and the specific contribution of curvature remain unclear. CCS[1] already studies score-based stratification, while D² Pruning[2] and SES[3] explicitly incorporate diversity mechanisms. Comparisons only against random and entropy sampling cannot establish the advantage of the proposed score and sampling combination.
References
[1] Zheng et al. Coverage-centric Coreset Selection for High Pruning Rates. ICLR 2023.
[2] Maharana et al. D² Pruning: Message Passing for Balancing Diversity and Difficulty in Data Pruning. ICLR 2024.
[3] Xie et al. Structural-Entropy-Based Sample Selection for Efficient and Effective Learning. ICLR 2025.
Questions for authors
-
Could you provide the geometric construction justifying the proposed shape operator and explain the binary-neighborhood example above? Please clarify whether normalization uses or , and provide evidence connecting the resulting score to boundary proximity and redundancy.
-
Can you disentangle the contributions of the score and sampling procedure? Please replace curvature with simpler neighborhood statistics under the same stratification and allocation rules. What mechanism or measurement supports redundancy reduction beyond preserving the proportions of the two strata?
-
Could you compare against CCS[1] and representative diversity-aware methods, such as D² Pruning[2] or SES[3], on a manageable subset of representative datasets and budgets? A complete rerun on all datasets is unnecessary, but stronger comparisons under matched information access and feature representations are needed.
-
Is a common independent test set held out before graph construction, parameter estimation, and sample selection? Algorithm 1 appears to use the full labeled dataset before assigning the remainder to testing. Please clarify this protocol and, if necessary, reevaluate without using test labels.
I would be willing to raise my score if the authors convincingly address Questions 1 and 2 with clear theoretical justification and supporting evidence.
References
[1] Zheng et al. Coverage-centric Coreset Selection for High Pruning Rates. ICLR 2023.
[2] Maharana et al. D² Pruning: Message Passing for Balancing Diversity and Difficulty in Data Pruning. ICLR 2024.
[3] Xie et al. Structural-Entropy-Based Sample Selection for Efficient and Effective Learning. ICLR 2025.
Thank you for your thorough review and for the time and effort you put into your comments, questions, and suggested references. We appreciate your acknowledgment of the motivation behind balancing boundary information and representativeness, as well as the breadth of our empirical evaluation. We are working on the paper and will implement any pertinent improvements in the revision.