acceptodds
Under review as a conference paper at ICLR 2027

Dynamic Gradient Modeling with Quality-Guided Masking for Adaptive LLM Fine-Tuning

Abstract

Gradient masking reduces redundant parameter updates during large language model (LLM) fine-tuning, but fixed global retention policies overlook variation across layers and training stages. We introduce DGMM, a dynamic gradient modeling algorithm with quality-guided masking that adaptively allocates gradient retention budgets across layers. DGMM integrates three complementary signals—directional sign bias, temporal stability, and inter-layer correlation—into a layer-wise gradient quality score. These scores are smoothed with an exponential moving average and used to allocate retention budgets within stage-aware bounds, while gradient magnitudes determine which elements are retained within each layer. Operating exclusively on optimizer-side gradients, DGMM preserves the model architecture, forward computation, training objective, and underlying optimizer. Experiments on task-specific and general-domain benchmarks show improvements of up to 30.8% over the strongest baseline on individual benchmarks. Controlled sweeps evaluate masking rates of up to 90% and reveal no change in profiler-reported training FLOPs. These findings support gradient-quality-guided budget allocation as an effective mechanism for adaptive gradient masking and underscore the distinction between sparse parameter updates and reduced training computation.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large language model fine-tuning, gradient masking, layer-adaptive optimization, gradient quality modeling, selective updates

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_TL9NOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes DGMM, extending GMT-style gradient masking for full-parameter LLM fine-tuning by assigning gradient retention budgets to individual layers. It combines directional sign bias, temporal stability, and inter-layer correlation into a quality score with manually chosen weights. Experiments on Mistral-7B, DeepSeek-Coder-6.7B, and Llama-2-7B compare DGMM with SFT, HFT, and GMT across several tasks and report some improvements.

Critical strengths

The paper addresses a practical question: how to allocate gradient-mask budgets across layers and adapt them during training, rather than applying a single global ratio.

Critical weaknesses
  1. DGMM's core design, quality-guided masking, is not sufficiently validated or isolated in the experiments. DGMM differs from GMT in three main ways: the overall masking ratio, the addition of warm-up and a stage-based schedule, and quality-guided adaptive masking. However, the experiments lack corresponding ablations. I could not find specific values or visualization for the actual masking ratios in each layer anywhere in the paper. According to Eq. (7), the masking ratio seems to remain between 1% and 11%. The masking appears to be almost uniform.
  2. I have serious doubts about the credibility of the experiments. Below is a comparison of the same SFT baselines and tasks in this paper and the GMT paper [1].
Model / benchmarkSFT in this paperSFT in the GMT paper
Mistral-7B HumanEval32.368.3
Mistral-7B GSM8K16.075.3
DeepSeek-Coder-6.7B HumanEval47.076.8
Llama-2-7B (Tülu-V2) MMLU39.1248.5

[1] Li, Haoling, et al. "Enhancing large language model performance with gradient-based parameter selection." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 39. No. 23. 2025.

Questions for authors
  1. What is the "external mask ratio" varied in Fig. 2?
  2. How many random seeds were used for each experiment? I did not see any analysis of variability across runs. DGMM performs poorly in most cases in Tables 1–2. How do the authors explain this?
  3. Why is DGMM's MMLU score 39.23 in Table 2 but 41.40 in Table 4? There are many other internal inconsistencies; for example, Fig. 2 also does not agree with Table 1.
  4. Minor issues. For example, “layers.DGMM integrates”.
7 comments

3.1. Does the manuscript report the actual layer-wise retention behavior, rather than only the target retention rule or external mask-ratio sweep?

Response: We agree that target retention budgets and realized retention behavior should be distinguished. Equations (7)–(9) define the raw, EMA-smoothed, and stage-constrained retention values assigned to eligible layers. Figure 2, by contrast, evaluates performance under different external mask-ratio budgets and should not be interpreted as a direct visualization of the realized retention ratio of every layer during training. The revised manuscript does not yet provide a complete per-layer realized-retention trajectory.

Evidence: Section 3.3 defines K_(l,raw), K_(l,EMA), and K_l in Eqs. (7)–(9) and gives the stage-aware schedule. Section 4.4 and Figure 2 (p. 8) report performance sensitivity to an external mask-ratio budget. Neither provides a complete per-layer realized-retention trajectory.

Revision place: The retention definitions and stage schedule are specified in Section 3.3 (pp. 4–5), while Figure 2 (p. 8) remains an external mask-ratio sensitivity analysis. No new per-layer realized-retention trajectory plot has been added in this revision, and we do not claim that this issue has been fully resolved.

3.2. How should the discrepancy between the SFT results in this manuscript and those reported in the original GMT paper be interpreted?

Response: Thank you for identifying this issue. We distinguish the provenance of our GMT implementation from the SFT result discrepancy. Our GMT baseline is an independent implementation of the published method; this fact does not explain differences in SFT results, because dense SFT does not use the GMT masking mechanism. We agree that cross-paper SFT differences require a separate comparison of model checkpoints, training data and preprocessing, configurations, and evaluation protocols. We do not treat the GMT implementation detail as an explanation for the SFT discrepancy.

Evidence: Section 4.2 describes the experimental baselines and the independent implementation of GMT. The revised text explicitly states that this clarification concerns only GMT and does not explain differences between dense SFT results reported across studies.

Revision place: Section 4.2 (p. 6) now separates the GMT implementation issue from the cross-paper SFT discrepancy and states that the latter requires a separate comparison of checkpoints, training data, preprocessing, configurations, and evaluation protocols. We do not claim that this cross-paper audit has been completed.

3.3. How are the numerical differences across the tables and figures reconciled, including the MMLU values and the relationship between Figure 2 and Table 1?

Response: We appreciate the reviewer’s careful checking. We agree that numerical consistency must be established from the underlying evaluation records rather than explained by assumption. We have clarified the specific MMLU difference between Table 2 and Table 4: the values come from two separate experimental runs and do not refer to the same checkpoint or run. This clarification should not be taken to mean that every possible table–figure discrepancy has been independently re-evaluated. Figure 2 presents external mask-ratio sensitivity, whereas Table 1 reports task-specific benchmark results; any remaining numerical correspondence should be checked against the original evaluation records.

Evidence: Table 2 (p. 7) reports a DGMM MMLU value of 39.23% in the DPO comparison, while Table 4 (p. 9) reports 41.40% for the full DGMM ablation configuration. The revised discussion in Section 4.3.2 identifies these as separate runs/checkpoints and explains that the 2.17-percentage-point difference should not be interpreted as an inconsistency within one evaluation result. Figure 2 (p. 8) is the external mask-ratio sensitivity plot.

Revision place: Section 4.3.2 (p. 8) now explicitly clarifies that the 39.23% and 41.40% MMLU values come from separate experimental runs and different checkpoints. This revision addresses the Table 2–Table 4 MMLU distinction. We have not claimed that it independently resolves any separate numerical discrepancy between Figure 2 and Table 1.

Reviewer_sn9BOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2(+1)
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a method called DGMM, which adaptive fine-tuning LLM by quality-guided masking on the model's gradient. DGMM adaptively assigns gradient retention budgets across layers using three gradient quality signals. It preserves the original model, forward computation, and optimizer, yields strong benchmark gains at up to 90% masking, and notes that sparse updates do not reduce training FLOPs.

Critical strengths
  • (S1) This paper proposes a decoupled adaptive gradient masking scheme, DGMM, a layer-by-layer budget allocation based on magnitude, combined with layer-by-layer gradient selection based on amplitude, without requiring modifications to the model, forward propagation, or the base optimizer.

  • (S2) The experiments presented in this paper demonstrate that DGMM achieves performance gains over the SFT, HFT, and GMT methods across multiple benchmark datasets.

Critical weaknesses
  • (W1) The eq (1) D=P+−P−D = P^+ - P^- assigns low scores to layers with lots of negative gradients in the same direction, while assigning high scores to layers with positive gradients. The gradient sign itself depends on the coordinate system and cannot inherently be interpreted as a measure of optimization quality; nor does it measure the directional consistency of adjacent steps. Correlation primarily adjusts the global budget and does not directly distinguish between layers. The scalar CC is the same for all layers; when clipping is not triggered, removing CC reduces overall raw retention rather than simply removing “inter-layer information.” Therefore, the “w/o Correlation” ablation in Table 4 is confounded with the total retained-gradient value. Also, the masked zeroing gradients cannot guarantee zero parameter updates for Adam or AdamW, since accumulated momentum states and weight decay still produce non‑zero Δθ\Delta\theta. The paper does not specify whether optimizer states are masked or cleared alongside masked gradients. Eq.10 only enforces gradient sparsity. The authors should report actual non‑zero parameter updates and clarify optimizer‑state semantics.

  • (W2) The DGMM core idea is masking some optimization parameters; the authors should compare with some related baselines, e.g., SPAM[1], AdaDHP[2], LISA[3]. And Kraw∈[0.89,0.99]Kraw \in [0.89, 0.99] by default; a 90% removal rate requires retention of ~0.10. The paper does not explicitly state how the external budget is used to rewrite or scale Kraw, the stage bounds, and the threshold, nor does it specify which setting was used for the main results.

Questions for authors
  • (Q1) As I mentioned in W1, it is unclear the positive/negative signs effect for the model optimization; could the authors explain why positive signs should be considered higher quality than negative signs?

  • (Q2) Could the authors give more detail about the “w/o Correlation” ablation in Table 4.

  • (Q3) The authors should report actual non‑zero parameter updates and clarify optimizer‑state semantics, as I mentioned in W1.

  • (Q4) Could the authors provide some experiments with those baselines as I mentioned, e.g., SPAM[1], AdaDHP[2], LISA[3].

  • (Q5) The authors need to provide more detailed information on the parameters and experimental setup.

Reference

[1] Huang T, Zhu Z, Jin G, et al. Spam: Spike-aware adam with momentum reset for stable llm training[C]//International Conference on Learning Representations. 2025, 2025: 49018-49038.

[2] Liu H, Li C, Zhang X, et al. AdaDHP: Fine-Grained Fine-Tuning via Dual Hadamard Product and Adaptive Parameter Selection[C]//Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025: 9492-9504.

[3] Pan R, Liu X, Diao S, et al. Lisa: Layerwise importance sampling for memory-efficient large language model fine-tuning[J]. Advances in Neural Information Processing Systems, 2024, 37: 57018-57049.

7 comments

2.1. Why should the Direction statistic be interpreted as a useful quality signal, and does a higher value mean that positive gradients are inherently better?

Response: Thank you for this careful observation. We agree that a higher positive-gradient ratio does not, by itself, establish that a gradient is more useful for the task. In DGMM, Direction is defined as the difference between the proportions of positive and negative gradient elements. It captures signed asymmetry in the gradient distribution and serves as a descriptive component of the composite layer-quality score; it is not a direct estimate of task importance or a claim that positive gradients are intrinsically superior.

Evidence: Section 3.2, Eq. (1), defines Direction as the difference between the positive- and negative-gradient proportions. The revised explanation immediately after Eq. (1) explicitly states that this statistic does not imply that positive gradients are intrinsically more beneficial than negative gradients.

Revision place: Section 3.2 (p. 4), immediately after Eq. (1), now clarifies the descriptive role of Direction and explicitly avoids interpreting positive gradients as inherently better. The composite-score discussion in the same section remains consistent with this interpretation.

2.2. How does the shared inter-layer Correlation term distinguish individual layers if the same scalar is used for every layer?

Response: We agree that the shared Correlation term does not independently rank or distinguish individual layers. At each optimizer step, it provides a global context value computed from the standardized descriptors of all eligible layers. Because this scalar is shared, the layer-specific differences in the fused quality scores arise mainly from the individual Direction and Stability values. We therefore describe Correlation as network-level context rather than a layer-specific ranking signal.

Evidence: Section 3.2, Eqs. (3)–(4), defines the relation matrix and shared scalar C^(t). The revised paragraph following Eq. (4) states that C^(t) is common to all eligible layers and that differences among layer-wise quality scores arise primarily from the layer-specific Direction and Stability statistics.

Revision place: Section 3.2 (p. 4), in the paragraph following Eq. (4), now explicitly identifies Correlation as shared global context rather than an independent layer-specific ranking signal.

2.3. Do the component-wise ablations genuinely isolate the contribution of each signal, and are the comparisons controlled for the actual global retention ratio?

Response: We agree that a component ablation is informative only when the intended signal is actually disabled and other relevant factors, particularly the realized global retention ratio, are controlled. Table 4 reports the full model and three component-removal variants, but the reported scores alone do not establish that all variants have matched realized retention. We therefore avoid claiming that the current table fully isolates each signal under a matched budget. The interpretation should remain limited to the component-wise results reported in the table unless implementation and retention records establish stronger controls.

Evidence: Section 4.6 and Table 4 (p. 9) report the Full, w/o Direction, w/o Stability, and w/o Correlation results. Table 4 does not include realized global retention ratios or layer-wise retention ratios for the variants. The convergence plot provides loss trends but does not establish matched retention.

Revision place: No retention-matched ablation experiment or realized-retention table has been added to Section 4.6 or Table 4 (p. 9) in the revised PDF. We therefore retain the reported component-wise comparison but do not claim that it is a fully controlled causal isolation of each signal.

Reviewer_NgLzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Dynamic Gradient Modeling with Quality-Guided Masking (DGMM), a dynamic gradient masking method for large language model fine-tuning. The authors argue that existing gradient masking methods often use fixed global retention rates, overlooking differences across layers and training stages. To address this, DGMM dynamically adjusts the gradient retention budget for each layer based on gradient quality.

Specifically, DGMM combines three signals (directional sign bias, temporal stability, and inter-layer correlation) to calculate a gradient quality score for each layer. These scores are smoothed using an exponential moving average and adjusted with stage-aware bounds to determine the retention rates. Within each layer, gradients are selected based on their magnitudes. The method operates on optimizer-side gradients without changing the model architecture, forward computation, or underlying optimizer.

Experiments are conducted on Mistral-7B, DeepSeek-Coder-Base-6.7B, and LLaMA2-7B, covering code generation, mathematical reasoning, general instruction fine-tuning, and preference optimization. The results show that DGMM outperforms the baselines on some coding and mathematical tasks, while achieving comparable performance on general-domain tasks. The authors also conduct masking-ratio sensitivity studies and component ablations, and report training throughput, runtime, and FLOPs. The results show that DGMM does not reduce the reported training FLOPs under the tested settings.

Critical strengths

S1. Dynamic gradient masking is a meaningful research problem. The paper considers how gradients differ across layers and training stages during LLM fine-tuning. It highlights the limitations of fixed global retention rates and explores dynamically adjusting retention budgets based on gradient quality. This problem has both research value and practical relevance.

S2. The method is simple and easy to integrate. DGMM combines three signals (i.e., directional sign bias, temporal stability, and inter-layer correlation) to dynamically allocate gradient retention budgets across layers. Within each layer, gradients are selected based on their magnitudes. The method does not require changes to the model architecture, forward computation, or underlying optimizer, making it easy to integrate into existing training pipelines.

S3. The experiments cover a range of settings and provide some empirical support. The authors evaluate DGMM on several 7B-scale models across code generation, mathematical reasoning, general instruction fine-tuning, and preference optimization. DGMM achieves better results on some coding and mathematical tasks. The paper also includes component ablations, masking-ratio sensitivity studies, and training cost comparisons, which help illustrate the method's behavior under different settings.

Critical weaknesses

W1. The design of the gradient quality metrics is not sufficiently justified. DGMM uses directional sign bias, temporal stability, and inter-layer correlation to measure gradient quality, but their relationship with actual optimization effectiveness remains unclear. For example, a higher proportion of positive gradients than negative gradients does not necessarily indicate better gradient quality. Moreover, the inter-layer correlation is summarized into a single global score shared across all layers, which does not directly capture layer-specific differences. Therefore, it is unclear whether these metrics can effectively guide the allocation of layer-wise gradient retention budgets.

W2. The baseline comparisons are insufficient to demonstrate the benefits of dynamic budget allocation. The paper mainly compares DGMM with SFT, HFT, and GMT, but lacks direct comparisons with other adaptive parameter or layer selection methods, such as AdaDHP and Flexora, which are already discussed in the paper. More importantly, the experiments lack simple but important controls, such as fixed layer-wise retention budgets, random or uniform budget allocation, and dynamic allocation based only on gradient magnitudes. It is also unclear whether the methods are compared under matched gradient retention rates. As a result, it is difficult to determine whether the improvements come from the proposed quality metrics, dynamic allocation, or differences in the number of retained gradients.

W3. The practical training benefits of dynamic gradient masking remain unclear. Although the paper evaluates task performance under different masking ratios, it does not systematically examine how these ratios affect training time or computational cost. The reported results show no reduction in training FLOPs, and DGMM does not consistently achieve faster training than dense fine-tuning. In addition, the paper lacks training or validation loss curves and actual layer-wise retention trajectories, making it difficult to understand how dynamic masking affects the training process. In particular, the efficiency evaluation covers only 100 optimizer steps, while DGMM does not activate masking during the first 500 steps. It is therefore unclear whether the reported efficiency results reflect the actual masking stage.

W4. The consistency of the experimental improvements needs further validation. DGMM achieves promising results on some code generation and mathematical reasoning tasks, but does not consistently outperform the baselines across all tasks. For example, its average performance is slightly below SFT on general instruction fine-tuning, while the improvements on preference optimization are also relatively small.

Questions for authors

Q1. Since the inter-layer correlation score is shared across all layers, how does it help distinguish their gradient quality and guide layer-specific budget allocation? (see W1)

Q2. Would it be possible to provide a few retention-matched comparisons with simpler static or dynamic budget allocation strategies to better isolate the contribution of the proposed quality-guided mechanism? (see W2)

Q3. Can the authors provide representative convergence curves and clarify whether the 100-step efficiency evaluation covers the stage where gradient masking is actually active? (see W3)

Q4. Is there additional evidence, such as results across multiple random seeds, to support the consistency of the reported performance improvements? (see W4)

6 comments

1.1. Does the current evaluation isolate the contribution of quality-guided layer-wise budget allocation from the effect of gradient masking itself?

Response: Thank you for raising this important point. We agree that the current experiments do not fully isolate the benefit of quality-guided layer-wise allocation from that of gradient masking in general. DGMM separates two decisions: the quality score determines the retention budget assigned to each eligible layer, while gradient magnitude selects coordinates within that budget. A stronger test would compare DGMM with uniform, random, and gradient-norm-based layer allocation under the same actual global retention ratio. Implementing and validating these additional controls requires substantial implementation and training effort. We have therefore not presented these comparisons as completed experiments and identify them as an important direction for future work.

Evidence: Section 3.1 describes the separation between layer-wise budget allocation and magnitude-based coordinate selection. Sections 3.2–3.3 define the quality signals and the mapping from normalized quality to layer-wise retention. Section 4.2 lists the evaluated baselines. The revised Conclusion explicitly acknowledges that retention-matched comparisons with uniform, random, and gradient-norm-based allocation require additional implementations and experiments.

Revision place: Section 4.2 (p. 6) clarifies the evaluated baseline set and the provenance of GMT. The Conclusion (p. 10) now identifies retention-matched uniform, random, and gradient-norm-based allocation as future work and states that these controls are beyond the scope of the current evaluation. We do not claim that these additional baselines have been completed.

1.2. Can the manuscript provide direct evidence of how the layer-wise quality signals and retention budgets evolve during training?

Response: We appreciate this suggestion. The manuscript specifies how Direction, Stability, the shared inter-layer relation, the fused quality score, and the retention budget are computed at each optimizer step. However, the current revised manuscript does not include complete trajectories for every layer's quality signals and realized retention ratios. We therefore distinguish the methodological definitions from direct trajectory evidence and do not present the external mask-ratio sensitivity plot as a substitute for layer-wise trajectories.

Evidence: Section 3.2 defines the layer-dependent statistics and shared correlation term in Eqs. (1)–(6). Section 3.3 defines the raw, EMA-smoothed, and stage-constrained retention values in Eqs. (7)–(9). Figure 2 (p. 8) reports performance sensitivity to an external mask-ratio budget; it is not a plot of per-layer retention trajectories.

Revision place: The relevant method definitions are in Sections 3.2–3.3 (pp. 4–5), while Figure 2 (p. 8) remains an external mask-ratio sensitivity analysis. No complete per-layer Direction, Quality, or realized-retention trajectory plot has been added to the manuscript in this revision; we identify this analysis as future work.

1.3. Does the 100-step efficiency measurement cover a period in which gradient masking is actually active, given the 500-step warm-up?

Response: Thank you for pointing out this timing issue. Table 3 reports runtime measurements over 100 optimizer steps. However, the timing records available to us do not preserve the exact global-step interval used for these measurements. Since DGMM performs dense updates during the first 500 optimizer steps, followed by conservative masking from steps 501 to 999 and the full quality-guided range from step 1000 onward, we cannot verify that the 100 measured steps were taken after masking became active. We therefore acknowledge that this measurement is insufficient to characterize the steady-state overhead of active masking. We have revised the discussion to present Table 3 only as a limited runtime snapshot under the recorded setup, rather than as evidence of the runtime cost of the active-masking stage.

Evidence: Section 3.3 specifies the stage schedule: masking is bypassed for t ≤ 500, conservative masking is used for 501 ≤ t ≤ 999, and the full quality-guided range is applied from t ≥ 1000. Section 4.5 states that the available timing records do not identify the exact global-step interval for the 100-step measurement and explicitly limits the interpretation of the reported speed and runtime values. The same section also notes that these measurements do not isolate the overhead attributable specifically to active masking.

Revision place: Section 3.3 (p. 5) clarifies the three-stage masking schedule. Section 4.5 (p. 9) now acknowledges that the exact 100-step measurement interval cannot be verified and interprets Table 3 as a limited runtime comparison, not as a measurement of steady-state active-masking overhead. We have also avoided making any acceleration claim based on this timing result.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.