acceptodds
Under review as a conference paper at ICLR 2027

From Detrimental to Beneficial: Dynamic Influence-based Valuation and Editing

Abstract

Data valuation is a cornerstone of data-centric learning, where prior efforts primarily focus on designing algorithms to classify training samples as either beneficial or detrimental for the learning task. However, leveraging these valuation estimates for subsequent data intervention remains underexplored; conventional approaches typically discard or downweight harmful samples, thereby underutilizing available data resources. In this paper, we present Dynamic Influence-based Valuation and Editing (DIVE), a novel and efficient framework that dynamically estimates sample values at the batch level and transforms detrimental data into beneficial contributions. Rather than altering the raw data, DIVE operates at the optimization level by strategically reversing the gradient directions of harmful samples during training, ensuring seamless integration with standard learning procedures with minimal overhead. Extensive empirical evaluations demonstrate that DIVE consistently improves classification performance, maximizes data efficiency, stabilizes optimization, and effectively generalizes to large language model fine-tuning.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
data influence, detrimental samples, dynamic influence

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_igAqOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

DIVE estimates per-sample influence at the batch level with a Hessian-free inner product and flips the gradient sign of negatively influential samples, so that “detrimental” samples contribute constructively instead of being removed. The method is evaluated on noisy-label image benchmarks and Gemma-3-12B LoRA fine-tuning.

Critical strengths
  • Interesting reframing. Turning detrimental samples into beneficial gradient contributions is conceptually appealing and differs from the usual remove/downweight paradigm.

  • Simple core operation. Once influence signs are available, sign-flipping negative samples is easy to describe and does not modify raw data.

Critical weaknesses
  1. Novelty and positioning are not sufficiently established. The idea is close to Negative Learning, Gradient Reversal, and noisy-label methods that exploit or reweight noisy samples. The paper distinguishes DIVE by saying it needs no prior noise labels or loss thresholds, but this distinction needs sharper mechanistic and empirical argument. The claim of being “the first to transform detrimental samples into beneficial contributions” is too broad. Strong noisy-label baselines (Co-teaching, DivideMix, ELR, SOP, JoCoR, etc.) are missing, so the comparison does not establish SOTA-level contribution.

  2. Theoretical/mechanistic justification for sign flipping is weak. Eq. (3) flips negative-influence gradients, but a positive inner product with a validation/reference gradient does not guarantee descent on validation loss in a non-convex model. The observed instability at very frequent editing (k=1) suggests the operation can amplify noise. The paper needs a decomposition or bound relating the DIVE update to the validation gradient, and analysis of when flipping helps versus destabilizes optimization.

  3. Influence estimation is coarse, and sign reliability is not validated. Eq. (2) is a Hessian-free approximation, and experiments score only the last layer/head or LoRA adapters. The key assumption—that coarse influence preserves the polarity of samples—is not quantitatively verified. The “detrimental sample verification” is somewhat circular: improved performance after correction may reflect regularization/noise injection rather than correct identification. Needed: agreement between estimated and leave-one-out influence signs, correlation with true noise labels, composition of flipped samples (mislabeled vs. hard-clean), and sign stability across scoring layers.

  4. Practical integration and efficiency are not convincingly demonstrated. The method is implemented in JAX because it natively supports sample-level gradients; PyTorch by default provides batch-level gradients, and prior work resorts to approximations such as ghost influence or layer-wise estimation. The paper claims seamless integration into standard optimization, but provides no PyTorch implementation or benchmark. “Lightweight” is supported only on image benchmarks (Table 2, ~1.09× vanilla) with last-layer scoring; there is no LLM runtime, memory, or throughput measurement. For Gemma-3-12B LoRA fine-tuning, per-sample LoRA gradients and periodic editing may add nontrivial overhead. The efficiency claim should be restricted accordingly.

  5. Experiments are not sufficiently thorough.

Questions for authors

see weakness

8 comments
AuthorsOpenReview ↗

Response to Reviewer igAq [1/5]

We sincerely thank the reviewer for the careful evaluation of our manuscript and for the valuable and constructive feedback. We appreciate the opportunity to clarify our contributions and address the concerns raised. Below, we provide detailed point-by-point responses, together with additional analyses and experimental results.

Novelty. Data valuation and data intervention are two fundamental components of data-centric learning. We do not claim methodological novelty in data valuation, as we adopt the existing Inner Product (IP) estimator, a simplified variant of TracIn. Instead, our contribution lies in the data intervention component, where we transform detrimental samples into beneficial contributions through gradient reversal. Unlike conventional approaches that discard or downweight detrimental samples, DIVE leverages these otherwise underutilized samples through a simple yet effective gradient reversal operation. Specifically, rather than modifying the raw data, DIVE reverses the gradient directions of detrimental samples during optimization, turning their negative influence into a positive contribution to learning. This shifts the focus from merely identifying and filtering detrimental samples to actively utilizing them to improve learning.


Compatibility with Existing Noisy-Label Learning Methods. Following the reviewer's suggestion, we further evaluate DIVE alongside established noisy-label learning methods, including ELR, SOP, Co-teaching, JoCoR, and DivideMix. Among these methods, we integrate DIVE with ELR, SOP, Co-teaching, and JoCoR to investigate whether gradient reversal can complement existing training strategies, while DivideMix is included as an independent baseline.

Importantly, DIVE is a complementary data intervention strategy rather than a method specifically designed for noisy-label classification. As shown in the table below, incorporating DIVE improves the average accuracy of ELR, SOP, Co-teaching, and JoCoR by 0.44, 0.23, 0.34, and 0.38 percentage points, respectively. All four improvements are statistically significant under paired two-sided t-tests across 24 dataset–seed pairs, with p=0.0007p=0.0007, p=0.0030p=0.0030, p=0.0004p=0.0004, and p=0.0008p=0.0008, respectively. These results support the compatibility of DIVE with multiple existing noisy-label learning methods.

Table: Test accuracy (%) on noisy-label classification benchmarks. Results are reported as mean ± standard deviation over four seeds. ** indicates a statistically significant improvement over the corresponding base method (paired two-sided t-test, p<0.01p<0.01, n=24n=24).

MethodCIFAR-10NaCIFAR-10NwCIFAR-10NrCIFAR-100NAnimal-10NFood101-NAvg.
Vanilla88.94±0.1678.97±0.2286.62±0.2854.41±0.4982.60±0.2672.42±0.1477.33
Vanilla+DIVE90.00±0.1683.48±0.1888.77±0.2257.38±0.5284.53±0.1772.91±0.2679.51 **
ELR91.67±0.0385.68±0.1290.77±0.1558.53±0.2585.22±0.3572.76±0.4180.77
ELR + DIVE91.67±0.2186.33±0.1490.61±0.1258.89±0.1685.63±0.2074.10±0.2181.21 **
SOP90.66±0.2283.80±0.3189.29±0.2058.10±0.3984.88±0.2573.62±0.3480.06
SOP + DIVE90.60±0.1584.52±0.1689.54±0.1358.09±0.2285.03±0.2073.94±0.1780.29 **
Co-teaching90.71±0.0984.66±0.3589.60±0.2058.27±0.1784.75±0.2374.42±0.0980.40
Co-teaching + DIVE90.83±0.0485.02±0.2289.61±0.1659.31±0.2684.88±0.2274.76±0.1080.74 **
JoCoR90.92±0.1584.19±0.3489.74±0.1158.17±0.2884.46±0.3074.03±0.1480.25
JoCoR + DIVE90.88±0.0684.72±0.2389.81±0.1459.26±0.2084.50±0.2374.59±0.1680.63 **
DivideMix89.00±0.1885.06±0.5688.46±0.2457.07±0.3881.73±0.6271.91±0.3978.87

We agree with the reviewer that DIVE is not inherently limited to noisy-label classification tasks. Our original evaluation follows the established data-centric learning literature, where classification is a widely adopted benchmark setting. We introduce label noise to create scenarios in which detrimental samples are more likely to arise, thereby facilitating the evaluation of data valuation and intervention methods. Nevertheless, DIVE is applicable beyond this setting. Following the reviewer's suggestion, we have further extended our evaluation to diverse scenarios and included additional baselines (See the last point).

AuthorsOpenReview ↗

Response to Reviewer igAq [2/5]

Theoretical Justification. DIVE determines whether a sample is beneficial or detrimental based on its gradient alignment with a reference signal, which can be either the validation gradient or the mini-batch mean gradient (self-influence). Given this signal, we can formulate data intervention as a constrained optimization problem: for each sample gradient gig_i, we seek to maximize wi⟨gi,gref⟩w_i\langle g_i,g_{\mathrm{ref}}\rangle, subject to wi∈[−1,1]w_i\in[-1,1]. The optimal solution is wi∗=sign⁡(⟨gi,gref⟩)w_i^*=\operatorname{sign}(\langle g_i,g_{\mathrm{ref}}\rangle) for nonzero alignment. Thus, gradient reversal emerges as the optimal intervention within this constrained weighting formulation, maximizing alignment with the reference direction while preserving the original gradient magnitude. In contrast, sample removal sets wi=0w_i=0, discarding the sample's potential contribution.

From an intuitive perspective, whether using a validation gradient or the mini-batch mean gradient (self-influence), DIVE establishes a reference direction for optimization. Samples whose gradients align with this direction contribute positively, whereas those pointing in the opposite direction are considered detrimental. Rather than discarding these detrimental samples, DIVE reverses their gradients to align with the reference direction, thereby transforming otherwise detrimental contributions into beneficial ones. This provides an intuitive explanation for why gradient reversal can be more effective than sample removal.


Effect of Gradient Editing Frequency. We would like to clarify that the performance degradation observed at k=1k=1 in Figure 1E should not necessarily be attributed to noise amplification. DIVE performs gradient editing based on the current reference direction, which constitutes a local, greedy intervention. Although each reversal improves the alignment of the edited gradient with the current reference direction, such local improvements do not guarantee globally optimal optimization dynamics.

Interestingly, our additional experiments extending DIVE to other application scenarios reveal a new observation regarding gradient editing frequency. In our original experiments with relatively small classification networks, applying gradient reversal at every mini-batch (k=1k=1) can be overly aggressive, and a more moderate frequency (k=2k=2) achieves better performance, as shown in Figure 1E. In contrast, in our extensions to mathematical reasoning with GRPO, safety-oriented preference optimization with DPO, and instruction following with larger language models, DIVE consistently improves performance using k=1k=1 (see Extension to Other Applications below).

These findings suggest that the appropriate intervention frequency may depend on model scale and training setting. While smaller networks may be more sensitive to frequent gradient reversal, larger language models appear capable of accommodating interventions at every mini-batch. Thus, k=1k=1 is not inherently detrimental, and the optimal editing frequency is likely to be setting-dependent. A more systematic investigation of this relationship is an interesting direction for future work.


Influence Estimation. We would like to clarify that our primary contribution lies in data intervention rather than data valuation. Specifically, DIVE aims to transform detrimental samples into beneficial contributions through gradient reversal, and we investigate this intervention strategy in the setting of dynamic data valuation. We do not claim novelty in influence estimation itself.

We acknowledge that last-layer gradient-based influence estimation is a coarse-grained approximation of full-parameter influence. However, in dynamic data valuation, computational cost is a major bottleneck, as per-sample gradients must be repeatedly estimated throughout training. This creates an inherent trade-off between estimation fidelity and computational efficiency, making approximations practically necessary for large-scale models. Following prior literature on efficient influence estimation, we adopt last-layer gradients as a computationally tractable alternative, accepting a potential loss in estimation fidelity in exchange for substantially reduced computational cost. Importantly, our scoring-layer ablation in Figure 1F demonstrates that this approximation remains effective for DIVE's intervention strategy, even when influence estimation is restricted to the final layer.

AuthorsOpenReview ↗

Response to Reviewer igAq [3/5]

Dynamic Nature of Detrimental Samples. We would like to emphasize that, in dynamic data valuation, whether a sample is beneficial or detrimental is not a universal or intrinsic property of the sample, but depends on the current model state and training stage. As discussed in prior data valuation literature, once a beneficial sample contributes to a parameter update, its information may be partially absorbed by the model, potentially changing its subsequent influence. The same reasoning applies to detrimental samples: a sample identified as detrimental at one stage may no longer be detrimental at another. Importantly, even a sample with a ground-truth noisy label is not necessarily detrimental throughout training. Therefore, noisy-label status should not be treated as a fixed ground-truth indicator of dynamic sample influence, and a static correlation between the two would not fully capture the phenomenon studied here.

Detrimental samples are defined by their negative impact on model performance. DIVE dynamically estimates their influence and reverses the gradients of samples identified as detrimental. To evaluate this intervention, DIVE_r serves as a direct baseline, using the same influence estimation but removing detrimental contributions instead of reversing them. As shown in Tables 1 and 3, DIVE consistently outperforms DIVE_r across all six benchmarks, with average improvements of 0.81 and 0.86 percentage points, respectively. These results provide empirical evidence that gradient reversal is more effective than simply discarding dynamically identified detrimental contributions.


Practical Integration with PyTorch. Although our experiments are implemented in JAX, DIVE is not restricted to this framework. We acknowledge that obtaining full per-sample gradients in PyTorch can be computationally expensive, as standard training pipelines typically operate on batch-aggregated gradients.

However, DIVE adopts last-layer gradient approximation for influence estimation, which can be efficiently implemented in PyTorch using the chain rule. Specifically, per-sample gradients of the final linear layer can be constructed from its input activations and the corresponding output gradients, without explicitly computing full-model per-sample gradients. This approach is also adopted by prior PyTorch-based data valuation methods, such as GHOST and LAI. Once the influence scores are obtained, DIVE only requires applying the corresponding gradient signs before aggregation and the optimizer update. Therefore, the practical integration of DIVE into PyTorch does not require a fundamentally different training framework.


Computational Efficiency of the Proposed Intervention. We would like to emphasize an important distinction between the computational cost of dynamic data valuation and that of our proposed data intervention. Our contribution is not to improve the efficiency of influence estimation, but to transform detrimental samples into beneficial contributions within the existing dynamic data valuation framework. Once influence scores are available, this transformation only requires reversing the signs of the corresponding sample gradients before aggregation, introducing negligible additional computational overhead compared with conventional influence-based sample removal. Therefore, the computational bottleneck discussed above primarily arises from dynamic influence estimation itself, rather than from the proposed gradient reversal strategy.

Regarding the scalability of dynamic data valuation, the main computational challenge lies in efficiently obtaining per-sample gradients. Although developing more scalable influence estimators is not the focus of this work, we have considered this issue from a practical perspective. Following prior work on efficient gradient approximation, we adopt last-layer gradients for influence estimation. Our scoring-layer ablation in Figure 1F further shows that DIVE maintains strong performance using only the final layer. For LLM fine-tuning, we similarly restrict influence estimation to the trainable LoRA adapters.

Reviewer_gFvYOpenReview ↗
Rating
3(+1)
Confidence
2(−1)
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

In this paper, the authors present Dynamic Influence-based Valuation and Editing (DIVE), a novel and efficient framework that dynamically estimates sample values at the batch level and transforms detrimental data into beneficial contributions. Extensive empirical evaluations demonstrate that DIVE consistently improves classification performance, maximizes data efficiency, stabilizes optimization, and effectively generalizes to large language model fine-tuning.

Critical strengths
  1. The paper is the first to transform detrimental samples into beneficial contributions, making every sample count during learning.

  2. The authors conduct extensive experiments to demonstrate the effectiveness of DIVE, comparing it against a range of data valuation and dynamic batch curation methods.

Critical weaknesses
  1. Limited model coverage. All image experiments use a single ResNet-9 backbone, and the LLM experiments only fine-tune LoRA adapters on Gemma-3-12B-IT. The paper therefore provides little evidence that DIVE generalizes to other architectures (e.g., vision transformers or deeper residual networks), larger models, or full-parameter training. Because the method depends on gradient geometry and influence estimation, conclusions from one compact CNN and one LoRA setting may not transfer.

  2. Narrow task scope. The main evaluation is noisy-label classification on six image benchmarks, and the LLM extension also verbalizes classification datasets as prompt-response pairs. There is no evaluation on open-ended generation, long-context or long-form tasks, instruction following, reasoning, or other complex objectives. It is therefore unclear whether sign-based gradient editing remains beneficial when the loss is less locally decomposable and the notion of a "detrimental sample" is more ambiguous.

  3. Scalability and cost are not fully validated. The efficiency claims rely on approximations that avoid full per-sample backpropagation: image experiments score influence only on the final MLP head using cached penultimate features, and LLM experiments restrict influence estimation to LoRA adapters. The paper does not measure the overhead or stability of DIVE when per-sample gradients must be computed over the full backbone or all model parameters at large scale. Thus, the "minimal overhead" claim is established only for the approximated regime, not for full-parameter training.

  4. Dependence on validation gradients and the self-influence heuristic. DIVE's valuation is defined relative to validation-set gradients. When a clean validation set is unavailable, the paper replaces it with the mini-batch mean gradient (self-influence), which is a heuristic and may not reflect the true generalization objective; it can be dominated by the current batch's label noise or mixture of easy and hard samples. The paper reports gains under self-influence but does not analyze when this proxy correlates with held-out performance, or how sensitive the results are to validation-set size, distribution shift, and label noise in the reference set.

Questions for authors
  1. Do you expect DIVE to remain effective with other architectures, larger models, or full-parameter training?
  2. Have you evaluated DIVE on open-ended generation, long-context, or other complex tasks beyond noisy-label classification?
  3. What are the training cost and stability when per-sample gradients are computed over the full backbone or all model parameters rather than only the last head or LoRA adapters?
  4. How well does the self-influence proxy correlate with held-out performance when the validation set is unavailable or noisy?
7 comments
AuthorsOpenReview ↗

Response to Reviewer gFvY [1/4]

We sincerely thank the reviewer for the careful evaluation of our manuscript and for the valuable and constructive feedback. We appreciate the opportunity to clarify our contributions and address the concerns raised. Below, we provide detailed point-by-point responses, together with additional analyses and experimental results.

Extension to Other Applications. To further demonstrate DIVE's broader applicability, we extend our evaluation to three additional applications: mathematical reasoning with GRPO on GSM8K, safety-oriented preference optimization with DPO on PKU-SafeRLHF, and instruction following on IFEval after fine-tuning on Alpaca-52K.

Table: Extension of DIVE to mathematical reasoning, safety-oriented preference optimization, and instruction following. All results are reported as mean ± standard deviation over five random seeds. All three experiments use a gradient editing frequency of k=1k=1. All metrics are reported in percentages. Avg. denotes the unweighted average across the seven evaluation metrics. DIVE_r uses the same influence estimation procedure as DIVE but removes detrimental gradient contributions instead of reversing them.

SettingGRPOGRPODPOInstruction FollowingInstruction FollowingInstruction FollowingInstruction FollowingAvg.
DatasetGSM8KGSM8KPKU-SafeRLHFAlpaca-52K / IFEvalAlpaca-52K / IFEvalAlpaca-52K / IFEvalAlpaca-52K / IFEval
BackboneGemma-3-1B-ITGemma-3-1B-ITQwen2.5-1.5BQwen3-8B-BaseQwen3-8B-BaseQwen3-8B-BaseQwen3-8B-Base
MetricAcc.Partial Acc.Pref. Acc.Prompt-strictInst.-strictPrompt-looseInst.-loose
Vanilla48.28 ± 0.7751.14 ± 0.8935.56 ± 2.2155.23 ± 1.0464.58 ± 0.9856.90 ± 1.3466.43 ± 1.3054.02
DIVE_r43.76 ± 19.3846.00 ± 20.1836.18 ± 1.3354.42 ± 1.5663.98 ± 0.9456.08 ± 1.6765.73 ± 0.9252.31
DIVE51.02 ± 0.6353.72 ± 0.8737.51 ± 1.4255.27 ± 2.0865.57 ± 1.6557.15 ± 2.2567.39 ± 1.5955.38

Results. As shown in the table, DIVE consistently outperforms both Vanilla and DIVE_r across all seven evaluation metrics, spanning three LLM applications: mathematical reasoning with GRPO on GSM8K (Gemma-3-1B-IT), safety-oriented preference optimization with DPO on PKU-SafeRLHF (Qwen2.5-1.5B), and instruction following on IFEval after fine-tuning on Alpaca-52K (Qwen3-8B-Base). Averaged across the seven metrics, DIVE achieves 55.38%, compared with 54.02% for Vanilla and 52.31% for DIVE_r, representing improvements of 1.36 and 3.07 percentage points, respectively. Furthermore, two-sided paired t-tests on the seven metric-level mean scores show that DIVE significantly outperforms both Vanilla (p=0.0159<0.05p=0.0159<0.05) and DIVE_r (p=0.0368<0.05p=0.0368<0.05) under this metric-level analysis. These results further demonstrate DIVE's effectiveness beyond noisy-label classification and support the practical advantage of reversing detrimental gradients rather than simply removing them.

AuthorsOpenReview ↗

Response to Reviewer gFvY [2/4]

Setup of GRPO, DPO, and Instruction Following. Below, we describe the experimental setups for GRPO, DPO, and instruction following.

1. GRPO. We conduct online reinforcement learning using Group Relative Policy Optimization (GRPO) on the GSM8K mathematical reasoning dataset. We initialize both the actor and the frozen reference policy from the pretrained Gemma-3-1B-IT model. The actor is optimized using rank-64 LoRA, while the backbone parameters remain frozen.

During training, we sample four prompts per update and generate four completions for each prompt. Each completion is evaluated using mathematical answer correctness, and group-relative advantages are computed from the rewards of completions belonging to the same prompt. The policy is then updated using the GRPO objective with KL regularization. We train for 691 optimizer steps and evaluate the resulting policy on the official GSM8K test set of 1,319 problems using greedy decoding.

For a fair comparison, we adopt the same 10% intervention budget across all methods. Random Filter and Reward Filter select 10% of training samples for filtering based on random selection and reward signals, respectively, whereas DIVE identifies 10% of samples using influence estimation and reverses their gradient contributions. Thus, all three methods operate on the same proportion of training samples, allowing us to compare different intervention strategies under a controlled budget.

2. DPO. Following the reviewer's suggestion to evaluate DIVE beyond noisy-label classification, we further extend our experiments to safety-oriented preference optimization using Direct Preference Optimization (DPO) on the PKU-SafeRLHF dataset, with Qwen2.5-1.5B as the backbone model.

We adopt a two-stage training pipeline consisting of supervised fine-tuning (SFT) followed by DPO. First, the backbone model undergoes full-parameter SFT, and the resulting checkpoint is used to initialize both the DPO actor and its frozen reference model. During the DPO stage, we fine-tune the actor using rank-64 LoRA while keeping the backbone parameters frozen. Each training example consists of a preferred and a rejected response, and the DPO objective encourages the actor to increase the relative likelihood of the preferred response compared with the frozen reference model. We use a DPO coefficient of β=0.01\beta=0.01 and train for 115 optimizer steps. All methods share the same initialization, training configuration, and evaluation protocol, and results are averaged over five random seeds.

3. Instruction Following. We further extend our experiments to instruction fine-tuning on Alpaca-52K, a dataset containing approximately 52K instruction–response pairs generated by ‘text-davinci-003.’ Unlike our synthetic label-noise experiments, we use Alpaca-52K without injecting additional noise. The dataset naturally contains imperfect model-generated responses, including incorrect, incomplete, or instruction-inconsistent answers, providing a realistic setting for evaluating data intervention.

We fine-tune Qwen3-8B-Base using rank-16 LoRA (α=32\alpha=32), with the remaining model parameters frozen. Training uses AdamW with a peak learning rate of 2×10−42\times10^{-4}, a batch size of 16, and one epoch of training (3,232 optimizer steps). We adopt the self-influence variant of DIVE, estimating each sample's influence using the inner product between its gradient and the mini-batch mean gradient, restricted to the LoRA parameters of the final decoder layer. Following a 30% vanilla warm-up, DIVE performs gradient editing at every mini-batch, reversing the gradients of samples with negative influence scores. Overall, 11.2% of training samples are flipped. We evaluate instruction-following performance on IFEval, which contains 541 prompts with verifiable instructions, using greedy decoding and the official evaluation criteria.

AuthorsOpenReview ↗

Response to Reviewer gFvY [3/4]

Task Scope. Thank you for pointing this out. Our original evaluation follows the established data-centric learning literature, where classification tasks have been widely adopted to evaluate data valuation and intervention methods. Importantly, DIVE is not inherently restricted to classification tasks. In particular, its self-influence formulation eliminates the need for a separate validation set and enables DIVE to be integrated with a wide range of gradient-based optimization objectives. This flexibility allows DIVE to extend naturally to diverse tasks and applications. Following the reviewer's suggestion, we have further evaluated DIVE in these broader settings (see above).


Scalability and Cost. Data-centric learning consists of two fundamental components: data valuation and data intervention. Our primary contribution lies in data intervention rather than data valuation. Specifically, we propose transforming detrimental samples into beneficial contributions through gradient reversal and investigate this intervention strategy in the setting of dynamic data valuation.

Within this scope, the proposed intervention introduces minimal additional computational overhead, as it only requires reversing the gradients of detrimental samples once their influence scores have been estimated.

Regarding the scalability of dynamic data valuation, the main computational challenge lies in efficiently obtaining per-sample gradients. Although developing more scalable influence estimators is not the focus of this work, we have considered this issue from a practical perspective. Following prior work on efficient gradient approximation, we adopt last-layer gradients for influence estimation. Our scoring-layer ablation in Figure 1F further shows that DIVE maintains strong performance using only the final layer. For LLM fine-tuning, we similarly restrict influence estimation to the trainable LoRA adapters.


Full Gradient. Full-parameter per-sample gradient computation is often impractical for large-scale models, as each sample's gradient has the same dimensionality as the model parameters. Consequently, storing and processing gradients for individual samples incurs substantial memory and computational costs, which grow rapidly with model size and batch size. For this reason, existing gradient-based data valuation methods commonly rely on gradient approximations rather than explicitly computing and storing full-parameter per-sample gradients.

Reviewer_kBi7OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DIVE, a dynamic data valuation and editing framework that estimates per-sample influence through gradient alignment with a validation or batch-reference gradient. Rather than removing samples with negative influence, DIVE reverses their gradient directions during optimization. The authors evaluate the method on noisy-label image classification benchmarks and several LLM fine-tuning tasks.

Critical strengths
  1. The proposed method is simple, intuitive, and can be incorporated into standard optimization procedures with relatively low computational overhead. Good writing as well.
  2. The results generally demonstrate improvements over vanilla training.
  3. The no-validation split comparison is interesting, as many attribution methods require a very well-constructed validation set.
Critical weaknesses
  1. Limited methodological novelty. The proposed influence estimator is essentially the gradient-inner-product estimator underlying famous TracIn (https://arxiv.org/abs/2002.08484) and related Hessian-free influence methods.

  2. Limited justification for gradient reversal over sample removal. The comparison between DIVE and DIVE_r is crucial for supporting the paper's main contribution, yet the difference seems small. The improvement over strong existing baselines is also small. These results support the potential usefulness of gradient reversal but provide limited evidence for the stronger claim that detrimental samples are transformed into beneficial information. I recommend authors either, provide other interventions to the gradient/deeper analysis, or think about other benchmarks where final performance is more sensitive to the removal/inclusion of bad samples.

  3. You can try to include more LLM generative tasks for evaluation. All current tasks are classification-based, and the same limited evaluation setup problem also exists for image datasets. They look somewhat outdated to me. Maybe safety domain data where good / bad samples are more semantically well-defined can unveil some interesting interpretable results here.

Questions for authors

See weakness.

13 comments
AuthorsOpenReview ↗

Response to Reviewer kBi7 [1/4]

We sincerely thank the reviewer for the careful evaluation of our manuscript and for the valuable and constructive feedback. We appreciate the opportunity to clarify our contributions and address the concerns raised. Below, we provide detailed point-by-point responses, together with additional analyses and experimental results.

Novelty. Data valuation and data intervention are two fundamental components of data-centric learning. We do not claim methodological novelty in data valuation, as we adopt the existing Inner Product (IP) estimator, a simplified variant of TracIn. Instead, our contribution lies in the data intervention component, where we transform detrimental samples into beneficial contributions through gradient reversal. Unlike conventional approaches that discard or downweight detrimental samples, DIVE leverages these otherwise underutilized samples through a simple yet effective gradient reversal operation. Specifically, rather than modifying the raw data, DIVE reverses the gradient directions of detrimental samples during optimization, turning their negative influence into a positive contribution to learning. This shifts the focus from merely identifying and filtering detrimental samples to actively utilizing them to improve learning.


Gradient Reversal vs. Sample Removal. We appreciate the reviewer's comment. An important contribution of DIVE is to bridge data valuation and data intervention, which have often been treated as separate stages. Rather than merely identifying and discarding detrimental samples, DIVE directly translates valuation signals into principled gradient-level interventions. We further clarify the rationale and effectiveness of gradient reversal from three complementary perspectives: theoretical optimality, intuitive interpretation, and empirical evidence.

(1) Local Optimality of Gradient Reversal. DIVE determines whether a sample is beneficial or detrimental based on its gradient alignment with a reference signal, which can be either the validation gradient or the mini-batch mean gradient (self-influence). Given this signal, we can formulate data intervention as a constrained optimization problem: for each sample gradient gig_i, we seek to maximize wi⟨gi,gref⟩w_i\langle g_i,g_{\mathrm{ref}}\rangle, subject to wi∈[−1,1]w_i\in[-1,1]. The optimal solution is wi∗=sign⁡(⟨gi,gref⟩)w_i^*=\operatorname{sign}(\langle g_i,g_{\mathrm{ref}}\rangle) for nonzero alignment. Thus, gradient reversal emerges as the optimal intervention within this constrained weighting formulation, maximizing alignment with the reference direction while preserving the original gradient magnitude. In contrast, sample removal sets wi=0w_i=0, discarding the sample's potential contribution.

(2) Intuitive Interpretation. Whether using a validation gradient or the mini-batch mean gradient (self-influence), DIVE establishes a reference direction for optimization. Samples whose gradients align with this direction contribute positively, whereas those pointing in the opposite direction are considered detrimental. Rather than discarding these detrimental samples, DIVE reverses their gradients to align with the reference direction, thereby transforming otherwise detrimental contributions into beneficial ones. This provides an intuitive explanation for why gradient reversal can be more effective than sample removal.


Performance. As shown in Tables 1 and 3, DIVE consistently outperforms its removal-based variant, DIVE_r, across all six benchmarks, achieving average accuracy improvements of 0.81 and 0.86 percentage points, respectively. Importantly, DIVE and DIVE_r use the same influence estimation procedure, differing only in whether detrimental contributions are reversed or removed. This controlled comparison directly isolates the effectiveness of gradient reversal, with highly significant improvements under paired two-sided t-tests (p=2.2×10−6p=2.2\times10^{-6} and p=1.1×10−6p=1.1\times10^{-6}, respectively) across 24 dataset–seed pairs (6 datasets × 4 seeds). These results provide strong empirical evidence that gradient reversal is more effective than simply removing detrimental contributions.

Furthermore, DIVE achieves higher average accuracy than strong existing baselines, including GHOST and LAI, in both settings. Although the improvements over GHOST are relatively modest, they are statistically significant at the 10% level (p=0.085p=0.085 and p=0.087p=0.087 in Tables 1 and 3, respectively). Notably, GHOST employs full-model gradients for influence estimation, whereas DIVE uses computationally efficient last-layer gradients. Despite this approximation, DIVE achieves higher average accuracy in both settings.

AuthorsOpenReview ↗

Response to Reviewer kBi7 [2/4]

Compatibility with Existing Noisy-Label Learning Methods. Following another reviewer's suggestion, we further evaluate DIVE alongside established noisy-label learning methods, including ELR, SOP, Co-teaching, JoCoR, and DivideMix. Among these methods, we integrate DIVE with ELR, SOP, Co-teaching, and JoCoR to investigate whether gradient reversal can complement existing training strategies, while DivideMix is included as an independent baseline.

Importantly, DIVE is a complementary data intervention strategy rather than a method specifically designed for noisy-label classification. As shown in the table below, incorporating DIVE improves the average accuracy of ELR, SOP, Co-teaching, and JoCoR by 0.44, 0.23, 0.34, and 0.38 percentage points, respectively. All four improvements are statistically significant under paired two-sided t-tests across 24 dataset–seed pairs, with p=0.0007p=0.0007, p=0.0030p=0.0030, p=0.0004p=0.0004, and p=0.0008p=0.0008, respectively. These results support the compatibility of DIVE with multiple existing noisy-label learning methods.

Table: Test accuracy (%) on noisy-label classification benchmarks. Results are reported as mean ± standard deviation over four seeds. ** indicates a statistically significant improvement over the corresponding base method (paired two-sided t-test, p<0.01p<0.01, n=24n=24).

MethodCIFAR-10NaCIFAR-10NwCIFAR-10NrCIFAR-100NAnimal-10NFood101-NAvg.
Vanilla88.94±0.1678.97±0.2286.62±0.2854.41±0.4982.60±0.2672.42±0.1477.33
Vanilla+DIVE90.00±0.1683.48±0.1888.77±0.2257.38±0.5284.53±0.1772.91±0.2679.51 **
ELR91.67±0.0385.68±0.1290.77±0.1558.53±0.2585.22±0.3572.76±0.4180.77
ELR + DIVE91.67±0.2186.33±0.1490.61±0.1258.89±0.1685.63±0.2074.10±0.2181.21 **
SOP90.66±0.2283.80±0.3189.29±0.2058.10±0.3984.88±0.2573.62±0.3480.06
SOP + DIVE90.60±0.1584.52±0.1689.54±0.1358.09±0.2285.03±0.2073.94±0.1780.29 **
Co-teaching90.71±0.0984.66±0.3589.60±0.2058.27±0.1784.75±0.2374.42±0.0980.40
Co-teaching + DIVE90.83±0.0485.02±0.2289.61±0.1659.31±0.2684.88±0.2274.76±0.1080.74 **
JoCoR90.92±0.1584.19±0.3489.74±0.1158.17±0.2884.46±0.3074.03±0.1480.25
JoCoR + DIVE90.88±0.0684.72±0.2389.81±0.1459.26±0.2084.50±0.2374.59±0.1680.63 **
DivideMix89.00±0.1885.06±0.5688.46±0.2457.07±0.3881.73±0.6271.91±0.3978.87

Importantly, we believe the add-on evaluation provides a more direct assessment of DIVE's contribution than a standalone comparison of Vanilla+DIVE against existing methods such as GHOST and LAI. Since DIVE is primarily proposed as a data intervention strategy rather than a new data valuation or noisy-label learning method, its effectiveness is best evaluated by examining whether it consistently improves different underlying training methods. In this regard, incorporating DIVE into ELR, SOP, Co-teaching, and JoCoR yields statistically significant improvements in all four cases. These controlled comparisons isolate the incremental benefit of DIVE's gradient reversal strategy and demonstrate that its effectiveness is not limited to a particular baseline. Thus, while comparisons with GHOST and LAI provide useful context for overall performance, the consistent and statistically significant add-on improvements offer more direct evidence of DIVE's contribution as a complementary data intervention strategy.

AuthorsOpenReview ↗

Response to Reviewer kBi7 [3/4]

Extension to Other Applications. To further demonstrate DIVE's broader applicability, we extend our evaluation to three additional applications: mathematical reasoning with GRPO on GSM8K, safety-oriented preference optimization with DPO on PKU-SafeRLHF, and instruction following on IFEval after fine-tuning on Alpaca-52K.

Table: Extension of DIVE to mathematical reasoning, safety-oriented preference optimization, and instruction following. All results are reported as mean ± standard deviation over five random seeds. All three experiments use a gradient editing frequency of k=1k=1. All metrics are reported in percentages. Avg. denotes the unweighted average across the seven evaluation metrics. DIVE_r uses the same influence estimation procedure as DIVE but removes detrimental gradient contributions instead of reversing them.

SettingGRPOGRPODPOInstruction FollowingInstruction FollowingInstruction FollowingInstruction FollowingAvg.
DatasetGSM8KGSM8KPKU-SafeRLHFAlpaca-52K / IFEvalAlpaca-52K / IFEvalAlpaca-52K / IFEvalAlpaca-52K / IFEval
BackboneGemma-3-1B-ITGemma-3-1B-ITQwen2.5-1.5BQwen3-8B-BaseQwen3-8B-BaseQwen3-8B-BaseQwen3-8B-Base
MetricAcc.Partial Acc.Pref. Acc.Prompt-strictInst.-strictPrompt-looseInst.-loose
Vanilla48.28 ± 0.7751.14 ± 0.8935.56 ± 2.2155.23 ± 1.0464.58 ± 0.9856.90 ± 1.3466.43 ± 1.3054.02
DIVE_r43.76 ± 19.3846.00 ± 20.1836.18 ± 1.3354.42 ± 1.5663.98 ± 0.9456.08 ± 1.6765.73 ± 0.9252.31
DIVE51.02 ± 0.6353.72 ± 0.8737.51 ± 1.4255.27 ± 2.0865.57 ± 1.6557.15 ± 2.2567.39 ± 1.5955.38

Results. As shown in the table, DIVE consistently outperforms both Vanilla and DIVE_r across all seven evaluation metrics, spanning three LLM applications: mathematical reasoning with GRPO on GSM8K (Gemma-3-1B-IT), safety-oriented preference optimization with DPO on PKU-SafeRLHF (Qwen2.5-1.5B), and instruction following on IFEval after fine-tuning on Alpaca-52K (Qwen3-8B-Base). Averaged across the seven metrics, DIVE achieves 55.38%, compared with 54.02% for Vanilla and 52.31% for DIVE_r, representing improvements of 1.36 and 3.07 percentage points, respectively. Furthermore, two-sided paired t-tests on the seven metric-level mean scores show that DIVE significantly outperforms both Vanilla (p=0.0159<0.05p=0.0159<0.05) and DIVE_r (p=0.0368<0.05p=0.0368<0.05) under this metric-level analysis. These results further demonstrate DIVE's effectiveness beyond noisy-label classification and support the practical advantage of reversing detrimental gradients rather than simply removing them.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.