- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes NADA, a nonlinear subset-selection method for component-level data attribution. Instead of reducing each training example to a single scalar attribution score, NADA applies an increasing concave function to accumulated component-wise scores, allowing a candidate's priority to depend on the examples already selected. The method is applied to influence functions, TracIn, and leave-one-out attribution, and is evaluated on logistic and ridge regression benchmarks as well as prompt-based fine-tuning tasks, where it generally outperforms the corresponding scalar top-score baseline.
Critical strengths
- The paper presents a clear and practical formulation that separates the subset-selection stage from the underlying attribution estimator. Using the same attribution matrix for both Top and NADA makes the comparison well controlled and allows the effect of the selection rule to be evaluated directly.
- The experiments also show that preserving component-level attribution information can outperform naive scalar top-score selection across several attribution methods and model families. This suggests that scalar aggregation may discard information that is useful for subset construction, although this empirical result alone does not establish strong novelty.
Critical weaknesses
- The main novelty is limited. NADA mainly extends the nonlinear component aggregation used in NASH from Shapley or semivalue scores to generic component-level attribution matrices. Closely related work has already considered imbalance, redundancy, diversity, and joint influence in data selection. I do not see a sufficiently strong conceptual or theoretical advance beyond this generalization, which is the main reason for my rejection recommendation.
- The empirical evaluation also does not establish a clear advantage over stronger existing selection methods. Most comparisons are against scalar Top selection, which performs poorly in many settings, while NADA is approximately on par with Random overall on the prompt fine-tuning benchmarks. Without direct comparisons to stronger joint or diversity-aware selection methods, it is difficult to tell whether NADA provides a meaningful improvement over existing approaches or mainly fixes a weak scalar aggregation baseline.
Questions for authors
- Could the authors clarify what is conceptually new in NADA beyond extending NASH-style nonlinear component aggregation from Shapley or semivalue scores to generic attribution matrices such as influence functions, TracIn, and LOO?
- Could the authors compare NADA directly with stronger joint or diversity-aware selection methods, such as NASH, BIDS, Quad, or related influence-aware subset-selection approaches?
- Are the validation examples used to construct the component attribution matrix disjoint from those used to report the final validation performance? If not, could the authors provide results on an independent held-out test set?
- How was the fixed choice of selected, and how sensitive are the results to and to global max-absolute normalization? In particular, does the method remain stable when component scales differ substantially?
- Attribution-informativeness is defined through a calibration map , while NADA generally optimizes a different concave function . Under what conditions, if any, does optimizing the NADA objective improve or approximate the true subset utility, especially in the signed-score settings used in the main experiments?