acceptodds
Under review as a conference paper at ICLR 2027

DAPR: Single-Pass Regularization for Capability Retention in LLM Post-Training

Abstract

Continued post-training of large language models via instruction fine-tuning or preference optimization often degrades previously acquired general capabilities. We study local loss geometry as a perspective on this degradation and observe different endpoint Hessian spectra across optimizers. To intervene without an additional perturbed-gradient evaluation, we propose Descent-Anchored Projected Regularization (DAPR) for Adam-family optimizers, which decouples parameter updates into a dual-component architecture: an unmodified primary task-optimization channel driven by Adam moment estimates, and a state-dependent geometric correction regulated by a moving loss center. Above the center, DAPR executes preconditioned descent, and at or below the center, it applies a stabilized projected ascent correction that limits direct cancellation of the Adam direction, combined with an initialization-anchoring penalty that restrains parameter drift. Local two-step analysis identifies conditions under which the correction acts along a weighted gradient-norm penalty, linking its mechanism to Sharpness-Aware Minimization (SAM) while operating strictly within a single forward–backward pass ( AdamW runtime). Across 10k step SFT and DPO on four model configurations scaling from 4B to 14B, DAPR improves capability retention in extended SFT, with lower endpoint task losses on the evaluated 4B/8B models and setting-dependent results under DPO. Finally, we introduce LA-DAPR, which accommodates heterogeneous convergence rates in multilingual tuning via language-specific dynamic centers, consistently improving cross-lingual capability retention.

Primary area:
optimization
Keywords:
Alignment Tax, Optimization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_HMrYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes Descent-Anchored Projected Regularization (DAPR), a single-pass optimizer intervention for retaining capabilities during LLM post-training. DAPR preserves Adam’s moment estimates and adds a state-dependent correction selected by a moving loss center: a preconditioned descent correction above the center, or a projected ascent correction with initialization anchoring at or below it. On 4B and 8B models, DAPR substantially reduces capability loss during extended SFT and approaches SAM’s retention at near-AdamW runtime. Results under DPO and on larger models are mixed. The paper also introduces LA-DAPR, which uses language-specific centers for multilingual tuning.

Critical strengths

The main SFT experiment result is strong, and the study also tests multiple model scales, which are comprehensive and attractive.

Critical weaknesses

For the detailed experiment, it does not establish that DAPR’s distinctive mechanism drives its gains. On Gemma-3-4B, a simpler anchoring plus weight-decay control comes close to DAPR (61.56 vs. 62.35 mean), and this comparison is not tested across other settings. The evidence therefore supports DAPR’s effectiveness as a package more than its proposed mechanism.

Questions for authors

Can the authors explain why DAPR lowers the top Hessian eigenvalue but raises the Hessian trace substantially on the curvature results on Gemma-3-4B? Is the proposed mechanism intended to suppress only the leading curvature direction, and how does that change relate to capability retention?

2 comments
AuthorsOpenReview ↗

Thanks for actionable suggestions

We thank Reviewer HMrY for recognizing DAPR as a single-pass internal Adam intervention. Below we address W1 and Q1 (Tables 1–2, 6, 8, 13; App. B.2.4).

Weakness 1: Isolating Projected Crossings from Anchoring + Weight Decay & L2-SP

  1. Direct Answer: In Table 8, Anchoring + weight decay retains DAPR's projected crossing update while adding λwd=0.01\lambda_{\mathrm{wd}}=0.01—and direct controls confirm that anchor-free DAPR (λbase=0\lambda_{\mathrm{base}}=0) achieves 5.6×5.6\times–15.6×15.6\times curvature suppression on its own, whereas standalone AdamW + L2-SP fails under Stress DPO.
  2. Table Summary (Tables 2, 6 & 8): Table 8 (Gemma-3-4B) tests DAPR with and without λbase\lambda_{\mathrm{base}}/λwd\lambda_{\mathrm{wd}}, while Tables 2 & 6 directly compare AdamW, always-on and ctc_t-gated AdamW + L2-SP (λbase=0.01\lambda_{\mathrm{base}}=0.01), and DAPR w/o anchor (λbase=0\lambda_{\mathrm{base}}=0) across SFT and β=0.01\beta=0.01 Stress DPO.
  3. Key Finding: In Table 8, Without anchoring (λbase=0,λwd=0\lambda_{\mathrm{base}}=0, \lambda_{\mathrm{wd}}=0) still beats AdamW (61.00% vs. 60.88% 3-Mean). On SFT (Table 6), DAPR w/o anchor slashes pure-float32 κdir\kappa_{\mathrm{dir}} by 15.6×15.6\times (56.31 →\to 3.61) on 8B (61.66% 4-Mean, matching AdamW + L2-SP's 61.74% and gated 62.14% vs. 62.24% AdamW and 61.20% default DAPR) and 5.6×5.6\times (7.22 →\to 1.29) on 4B (+3.72 pp GSM8K over default DAPR). Under Stress DPO (Table 2), standalone AdamW + L2-SP drops 8B GSM8K to 45.34% (-5.46 pp below DAPR's 50.80%), while on 4B DAPR beats AdamW by +4.63 pp GSM8K (49.36% vs. 44.73%; +10.24 pp over Flooding's 39.12%).

Question 1: Directional Curvature Suppression vs. Background Subspace Trace

  1. Direct Answer: Descent–ascent crossings penalize the squared gradient norm along active gradient directions rather than shrinking all dd coordinates, selectively damping directional sharpness κdir\kappa_{\mathrm{dir}} without constraining unexcited background Hessian eigenvalues.
  2. Math Summary (Propositions 1 & 3; App. B.2.4): Consecutive crossings yield −η2Htgt=−η2∇θ(12∥∇L∥22)-\eta^2\mathbf{H}_t\mathbf{g}_t = -\eta^2\nabla_{\boldsymbol{\theta}}(\frac{1}{2}\Vert\nabla\mathcal{L}\Vert_2^2) (and −η2AtPQt∇Φt(θt)-\eta^2\mathbf{A}_t\mathbf{P}\mathbf{Q}_t\nabla\Phi_t(\boldsymbol{\theta}_t) in Proposition 3), directly regularizing κdir=g⊤Hg/∥g∥22\kappa_{\mathrm{dir}} = \mathbf{g}^\top\mathbf{H}\mathbf{g}/\Vert\mathbf{g}\Vert_2^2, whereas Tr(H)=∑i=1dλi(H)\mathrm{Tr}(\mathbf{H})=\sum_{i=1}^d\lambda_i(\mathbf{H}) sums over millions of orthogonal LoRA coordinates (d≈5.9M–12.6Md\approx 5.9\text{M}\text{--}12.6\text{M}) whose scale tracks ∥B∥F2+∥A∥F2\Vert\mathbf{B}\Vert_F^2+\Vert\mathbf{A}\Vert_F^2 when λwd=0\lambda_{\mathrm{wd}}=0.
  3. Key Finding: Pure-float32 Pearlmutter HVPs (Table 1) verify consistent single-pass κdir\kappa_{\mathrm{dir}} reductions across every scale: 7.22 →\to 1.78 (-75%, 1.29 at λbase=0\lambda_{\mathrm{base}}=0) on 4B, 56.31 →\to 12.51 (-78%, 3.61 at λbase=0\lambda_{\mathrm{base}}=0) on 8B, 10.76 →\to 4.65 (-57%) on 12B, and 3.79 →\to 0.77 (-80%) on 14B at 1.07×1.07\times–1.23×1.23\times AdamW runtime.

Dear Reviewer HMrY,

Thank you for the time you spent reviewing our paper and for recognizing the SFT results and the evaluation across model scales. We also appreciate your questions about what drives DAPR's gains and how to interpret the curvature measurements. In the revised manuscript and our response, we add anchor-free DAPR and match anchoring controls to examine the role of the projected correction. We also re-evaluate directional curvature in float32 and explain why a lower curvature along the active gradient direction can coexist with a higher Hessian trace.

As the discussion period is drawing to a close, could you let us know whether our response addresses your concerns, or whether there is another point you would like us to clarify? We would be glad to respond while there is still time.

Thank you again for your time and efforts.

On behalf of all authors

Reviewer_MybHOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies the loss of general capabilities during long post-training of LLMs with SFT or DPO. The authors observe that AdamW ends at a sharper point than SAM (larger top Hessian eigenvalue) and that SAM retains more capabilities, but SAM needs two forward-backward passes per step. They propose DAPR, which keeps the Adam update and adds a correction term using only one forward-backward pass. At each step, DAPR compares the batch loss with a moving threshold, computed as an EMA of the loss minus an EMA of the recent loss decrease. When the loss is above the threshold, DAPR adds an extra preconditioned descent step. When the loss is at or below the threshold, it adds an ascent step on the gradient component orthogonal to the Adam direction, together with an L2 penalty that pulls the weights toward the initial checkpoint. The theory part includes a two-step expansion showing that a descent step followed by an ascent step contains a gradient-norm penalty, which the authors relate to SAM (Proposition 1), a characterization of the projection (Proposition 2), and a decomposition of the full Adam-based update with conditions under which it acts as a gradient-norm penalty (Lemma 1 and Corollary 1). They also propose LA-DAPR, which keeps a separate threshold for each language in multilingual fine-tuning.

The method is evaluated with LoRA on Gemma-3-4B, Llama-3-8B, Gemma-3-12B, and Qwen-2.5-14B, using 10K-step SFT on UltraChat, DPO on UltraFeedback with a standard and a "stress" setting (low β and higher learning rate), 5-epoch SFT on LIMA, and multilingual SFT on Aya. Retention is measured on ARC-Challenge, MMLU, HellaSwag, and GSM8K, and compared with AdamW and SAM. On the 4B and 8B models after SFT, DAPR reduces forgetting substantially (for example, GSM8K on Llama-3-8B is 46.85% for DAPR, 33.66% for AdamW, and 46.62% for SAM, against 49.43% for the base model), with a runtime of 1.01× to 1.07× that of AdamW. On the 12B and 14B models and under DPO, the results are close to AdamW or mixed. The paper also reports Hessian measurements in the LoRA parameters, ablations and a hyperparameter sweep on Gemma-3-4B, and per-language results for LA-DAPR.

Critical strengths
  1. The problem is relevant, and the method is cheap. Losing general capabilities during long SFT and DPO is a real problem in LLM post-training, and SAM, which reduces it, doubles the cost per step. DAPR runs at 1.01× to 1.07× the runtime of AdamW, while SAM runs at 1.87× to 1.98×, and on the 4B and 8B models after SFT it matches SAM's retention (for example, 46.85% vs. 46.62% GSM8K on Llama-3-8B). If these gains held against properly tuned baselines, a drop-in, single-pass optimizer with SAM-level retention would be useful in practice. This is the main reason the paper is worth considering.

  2. The evaluation is broad, and the negative results are reported. The experiments cover four models from 4B to 14B, SFT and two DPO settings, a low-data setting (LIMA), and multilingual fine-tuning, with Hessian measurements and runtime. The authors also report where DAPR does not help, such as the 12B and 14B models and parts of the DPO results. This makes the empirical picture easier to judge, although, as discussed under weaknesses, the setup does not isolate the effect of the method.

Critical weaknesses
  1. The novelty is narrow, and the paper does not state it properly. The paper presents the link between sharpness and forgetting as its starting idea ("Our idea starts from local loss-landscape geometry"). However, this link in LLM fine-tuning, and the use of SAM to reduce forgetting, were already shown by Li et al. (EMNLP Findings 2024), who describe their work as taking "the first step to reveal the direct link between the flatness of the model loss landscape and the extent of CF in the field of LLMs." The update rule is also a combination of existing methods. At each step, DAPR compares the batch loss with a moving threshold, similar to the flood level in Flooding (Ishida et al., 2020). When the loss is above the threshold, the update is QHAdam (Ma & Yarats, 2019) after reparameterization (ν1 = 1/(1+α), learning rate η(1+α)). When the loss is below the threshold, DAPR adds an ascent step on the gradient component orthogonal to the Adam direction, which is the orthogonal ascent of GSAM (Zhuang et al., 2022) with the Adam direction used as the reference instead of the SAM gradient. Removing the momentum component of the gradient in a single pass is already done by Han et al. (CVPR 2025) and F-SAM (Li et al., CVPR 2024). The L2 penalty toward the initial weights is L2-SP (Li et al., 2018), and adding the correction outside Adam's moment estimates follows the decoupling idea of AdamW (Loshchilov & Hutter, 2019). Except for Flooding, and AdamW as the baseline optimizer, these works are not cited. The new part is the specific combination of these pieces, and the paper should state it as such and compare with these methods.

  2. The main theoretical result is an existing result, and it is not cited. Proposition 1 is Theorem 5.1 of Karakida et al. (ICML 2023), who show that the descent and ascent steps of Flooding are equivalent to finite-difference gradient regularization. The descent-then-ascent case is in their Appendix D, and their Section 5.2 gives the same link to SAM. Proposition 1 only allows unequal step sizes (1 ± α), which is a one-line extension. The other results do not support the claims built on them. In Proposition 1, the remainder has the same order in η as the claimed gradient-norm term, because α depends on ‖g‖ and changes between the two steps. Lemma 1 assumes that Adam's preconditioner is fixed across the two steps, that both gradients are computed on the same loss, and that the projection direction changes only by O(η) between steps. None of these holds in the actual algorithm, and the dropped terms are larger than the claimed regularization term. Corollary 1 holds only under exact conditions that almost never occur, and one of them, g_∥ = 0, turns off the projection. Still, Section 3.2 uses Corollary 1 to explain the empirical retention gains.

  3. The experimental setup does not isolate the effect of the method. All methods share one learning rate per model size, and the learning rate is not tuned for AdamW. The large gains appear only at 4B and 8B, which use learning rates of 2e-4 and 1e-4. At 12B and 14B, which use 2e-5, AdamW barely forgets and DAPR gives no gain (between -0.37 and +0.61 pp on GSM8K). The fine-tuning learning rate itself is known to control forgetting (Rofin et al., COLM 2026). DAPR also increases the effective step size in its descent branch and adds an L2 penalty toward the initial weights, which is a known method against forgetting (Li et al., 2018). Yet there is no AdamW baseline with a lower learning rate, no AdamW baseline with the same L2 penalty, and no comparison with single-pass alternatives such as Flooding (Ishida et al., 2020), S²-SAM (Ji et al., 2024), or GSAM (Zhuang et al., 2022). The sharpness evidence is mixed: on Gemma-3-4B, DAPR has a higher Hessian trace than AdamW (1,059 vs. 415), and at 12B and 14B the top Hessian eigenvalue of SAM is more than 15 times smaller than that of DAPR, with similar retention. The paper does not report how often each branch is used, although the branch is chosen per batch, and the hyperparameter sweep covers one model and is non-monotonic, with the default at an isolated peak.

  4. The writing makes the paper hard to follow. The paper uses its own terms where standard ones exist: "moving loss center" for flood level, "crossing pair" for the descent and ascent steps of Flooding, "transverse" for orthogonal, and "dual-channel architecture" for an update rule with a correction term. Several terms are used before they are defined, such as the loss center, "crossing pair", "stress DPO", and "branch-switch milestones". Some sentences assume context the reader does not have, for example "is under the stated bounds" in Section 2.1, where no bounds have been stated. The paper also describes its own measurements as "reported" and "recorded" values, for example "DAPR lowers the reported top eigenvalue". Section 1 refers to Table 1 for the GSM8K results, but Table 1 contains only curvature and runtime numbers.

References

  • Han, T., Gokay, D., Heyward, J., Zhang, C., Zoran, D., Pătrăucean, V., Carreira, J., Damen, D., and Zisserman, A. Learning from streaming video with orthogonal gradients. CVPR, 2025.
  • Ishida, T., Yamane, I., Sakai, T., Niu, G., and Sugiyama, M. Do we need zero training loss after achieving zero training error? ICML, 2020.
  • Ji, J., Li, G., Fu, J., Afghah, F., Guo, L., Yuan, X., and Ma, X. A single-step, sharpness-aware minimization is all you need to achieve efficient and accurate sparse training. NeurIPS, 2024.
  • Karakida, R., Takase, T., Hayase, T., and Osawa, K. Understanding gradient regularization in deep learning: Efficient finite-difference computation and implicit bias. ICML, 2023.
  • Li, H., Ding, L., Fang, M., and Tao, D. Revisiting catastrophic forgetting in large language model tuning. Findings of EMNLP, 2024.
  • Li, T., Zhou, P., He, Z., Cheng, X., and Huang, X. Friendly sharpness-aware minimization. CVPR, 2024.
  • Li, X., Grandvalet, Y., and Davoine, F. Explicit inductive bias for transfer learning with convolutional networks. ICML, 2018.
  • Loshchilov, I. and Hutter, F. Decoupled weight decay regularization. ICLR, 2019.
  • Ma, J. and Yarats, D. Quasi-hyperbolic momentum and Adam for deep learning. ICLR, 2019.
  • Rofin, M., Varre, A., and Flammarion, N. (How) learning rates regulate catastrophic overtraining. COLM, 2026.
  • Zhuang, J., Gong, B., Yuan, L., Cui, Y., Adam, H., Dvornek, N. C., Tatikonda, S., Duncan, J. S., and Liu, T. Surrogate gap minimization improves sharpness-aware training. ICLR, 2022.
Questions for authors
  1. Proposition 1 appears to be Theorem 5.1 of Karakida et al. (ICML 2023), whose Appendix D covers the descent-then-ascent case and whose Section 5.2 gives the link to SAM. What does Proposition 1 add beyond allowing unequal step sizes (1 ± α)? How do you position DAPR relative to this work and to GSAM (Zhuang et al., 2022), whose update has the same form as your below-threshold step?

  2. The large gains appear only at 4B and 8B, where the learning rate is 5 to 10 times higher than at 12B and 14B, and DAPR also adds an L2 penalty toward the initial weights. Can you report AdamW with a tuned (lower) learning rate and AdamW with the same L2 penalty (λ_base = 0.01) on Gemma-3-4B and Llama-3-8B? This is the main factor in my assessment: if these baselines close the gap, the gains cannot be attributed to the new parts of the method.

  3. How does DAPR compare at matched compute with existing single-pass or cheaper alternatives, specifically Flooding (Ishida et al., 2020) with Adam, S²-SAM (Ji et al., 2024), and GSAM applied only on a fraction of the steps?

  4. The branch is chosen per batch by comparing the batch loss with the moving threshold. What fraction of steps uses each branch, and how often does a descent step follow an ascent step, as Proposition 1 requires? How is the single "branch switch step" in Table 5 defined, given that the comparison is made at every step?

  5. Corollary 1 requires g_∥ = 0, i.e., the gradient has no component along the Adam direction, which also means the projection has no effect. During training, how large is ‖g_∥‖ relative to ‖g‖? If it is not close to zero, what supports the claim in Section 3.2 that the retention gains are "consistent with" the gradient-norm interpretation of Corollary 1?

6 comments
AuthorsOpenReview ↗

Thanks for your actionable suggestions

We thank Reviewer MybH for the incisive feedback on related literature, theoretical assumptions, control baselines, trajectory telemetry, and presentation. All updates and experiments appear in the revised manuscript, organized across five parts.

Question 1 & Weaknesses 1, 2: Internal Adam Intervention vs. Flooding, Karakida et al. (2023), & GSAM

  1. Direct Answer: Unlike Flooding (Ishida et al., 2020; Karakida et al., 2023), which flips the scalar objective gradient sign and corrupts Adam's momentum, DAPR leaves L(θ)\mathcal{L}(\boldsymbol{\theta}) untouched and intervenes in Adam's post-preconditioning update with O(1)O(1) scalar state (ct=μt−λstc_t=\mu_t-\lambda s_t) and zero cached gradient buffers—keeping (mt,vt,ut)(\mathbf{m}_t,\mathbf{v}_t,\mathbf{u}_t) unmodified on 100%100\% of steps while injecting a ctc_t-gated Pt\mathbf{P}_t-orthogonal correction in one pass.
  2. Math & Literature Summary (Sec. 1–2; App. A.1–A.4, App. C):
    • Why Flooding fails inside Adam (App. A.2 vs. Eq. 3–4 & Prop. 2): Flooding feeds g~t=σtgt\tilde{\mathbf{g}}_t=\sigma_t\mathbf{g}_t (σt∈{±1}\sigma_t\in\lbrace \pm 1 \rbrace) into mtF=β1mt−1F+(1−β1)σtgt\mathbf{m}_t^{\mathrm{F}}=\beta_1\mathbf{m}_{t-1}^{\mathrm{F}}+(1-\beta_1)\sigma_t\mathbf{g}_t, where historical momentum resists sign flips and unprojected ascent opposes task descent. DAPR preserves ut=Ptm^t\mathbf{u}_t=\mathbf{P}_t\widehat{\mathbf{m}}_t and sets qt=PtQtgt\mathbf{q}_t=\mathbf{P}_t\mathbf{Q}_t\mathbf{g}_t (gortho=Qtgt\mathbf{g}_{\mathrm{ortho}}=\mathbf{Q}_t\mathbf{g}_t) via Qt=I−utut⊤Ptut⊤Ptut+ϵproj\mathbf{Q}_t = \mathbf{I} - \frac{\mathbf{u}_t\mathbf{u}_t^\top\mathbf{P}_t}{\mathbf{u}_t^\top\mathbf{P}_t\mathbf{u}_t+\epsilon_{\mathrm{proj}}}—uniquely solving min⁡ut⊤z=012∥z−Ptgt∥Pt−12\min_{\mathbf{u}_t^\top\mathbf{z}=0}\frac{1}{2}\Vert\mathbf{z}-\mathbf{P}_t\mathbf{g}_t\Vert_{\mathbf{P}_t^{-1}}^2 (Prop. 2) so ut⊤qt=0\mathbf{u}_t^\top\mathbf{q}_t=0 (⟨ut,gortho⟩Pt=0\langle\mathbf{u}_t,\mathbf{g}_{\mathrm{ortho}}\rangle_{\mathbf{P}_t}=0) for ϵproj=0\epsilon_{\mathrm{proj}}=0.
    • Karakida et al. (2023) vs. Lemma 1 & Prop. 3: We credit Karakida et al. (2023, Thm. 5.1 & App. D) before Prop. 1 for unpreconditioned SGD crossings (∝−Htgt\propto -\mathbf{H}_t\mathbf{g}_t). Prop. 1 extends this to modulated weights 1±αt(gt)1\pm\alpha_t(\mathbf{g}_t), while Lemma 1 & Prop. 3 resolve Adam's non-symmetric block-oblique projectors across Im⁡(PΠt)⊥P−1Im⁡(PQt)\operatorname{Im}(\mathbf{P}\boldsymbol{\Pi}_t)\perp_{\mathbf{P}^{-1}}\operatorname{Im}(\mathbf{P}\mathbf{Q}_t) under cross-block Ht\mathbf{H}_t, g∥,t≠0\mathbf{g}_{\parallel,t}\neq\mathbf{0}, and state drift (ΔAt,ΔQt)(\Delta\mathbf{A}_t,\Delta\mathbf{Q}_t).
    • Literature comparison (App. C): Contrasted against GSAM (Zhuang et al., 2022) (2×2\times passes, Euclidean projection), F-SAM (Li et al., 2024b) & Han et al. (2025) (unconditional Euclidean momentum removal without ctc_t crossings), S2-SAM\text{S}^2\text{-SAM} (Ji et al., 2024) (O(d)O(d) cached-gradient buffer), QHAdam (Ma & Yarats, 2019) (above-ctc_t blend only), L2-SP (Li et al., 2018), Li et al. (2024a), and Rofin et al. (2026).
  3. Key Finding: Step-by-step telemetry confirms that Flooding + AdamW exhibits 99.97%99.97\% momentum persistence and 35.7%35.7\%–36.8%36.8\% (37.6%37.6\% on 12B) sign overrides on below-ctc_t steps—degrading SFT validation loss across 4B–14B (1.401, 1.339, 1.275, 1.241), collapsing 4B SFT GSM8K to 18.27%, and stalling 12B/14B DPO near Base—whereas DAPR cuts pure-float32 κdir\kappa_{\mathrm{dir}} by 2.3×2.3\times–4.9×4.9\times (5.6×5.6\times–15.6×15.6\times at λbase=0\lambda_{\mathrm{base}}=0) at 1.07×1.07\times–1.23×1.23\times AdamW runtime.

Question 2 & Weakness 3: Lower Learning Rates and L2-SP (λbase=0.01\lambda_{\mathrm{base}}=0.01) Controls

  1. Direct Answer: Neither tuned lower learning rates nor standalone/ctc_t-gated AdamW + L2-SP (λbase=0.01\lambda_{\mathrm{base}}=0.01) close the gap: anchor-free DAPR (λbase=0\lambda_{\mathrm{base}}=0) suppresses directional sharpness by 5.6×5.6\times–15.6×15.6\times over AdamW without any L2 penalty, while standalone AdamW + L2-SP fails under Stress DPO.
  2. Table & Math Summary (Sec. 3.3, Fig. 3a; App. B.2, Table 6): We evaluate η∈{2×10−5,5×10−5,10−4,2×10−4}\eta\in\{2\times 10^{-5}, 5\times 10^{-5}, 10^{-4}, 2\times 10^{-4}\}, always-on and ctc_t-gated AdamW + L2-SP (λbase=0.01\lambda_{\mathrm{base}}=0.01), and DAPR w/o anchor (λbase=0\lambda_{\mathrm{base}}=0 in Eq. 3) across all 4 benchmarks, UltraChat validation loss, and pure-float32 Pearlmutter curvature κdir=g⊤Hg/∥g∥22\kappa_{\mathrm{dir}}=\mathbf{g}^\top\mathbf{H}\mathbf{g}/\Vert\mathbf{g}\Vert_2^2:
ModelConfiguration (Table 6)Val ↓\downarrowκdir↓\kappa_{\mathrm{dir}} \downarrowARC-CMMLUHSGSM8K4-Mean
8BAdamW (η=2×10−5\eta = 2\times 10^{-5})1.26118.7755.5562.0479.5551.1862.08
8BAdamW (η=5×10−5\eta = 5\times 10^{-5})1.2536.5154.8662.0179.6352.3162.20
8BAdamW (η=1×10−4\eta = 1\times 10^{-4})1.25156.3155.8961.3979.8251.8662.24
8BAdamW (η=2×10−4\eta = 2\times 10^{-4})1.266-32.4155.7260.5979.9245.8760.52
8BAdamW + L2-SP (λbase=0.01\lambda_{\mathrm{base}}=0.01, always-on)1.2522.6754.5261.3279.6451.4861.74
8BAdamW + L2-SP (λbase=0.01\lambda_{\mathrm{base}}=0.01, gated)1.2514.0255.3861.5679.6951.9362.14
8BDAPR w/o anchor (λbase=0.0\lambda_{\mathrm{base}}=0.0)1.2573.6155.8961.0979.5650.1161.66
8BDAPR (default, λbase=0.01\lambda_{\mathrm{base}}=0.01)1.25712.5154.0160.7779.8150.1961.20
8BDAPR (θ0\boldsymbol{\theta}_0-anchored, λbase=0.01\lambda_{\mathrm{base}}=0.01)1.2577.7954.6161.4579.8150.1961.52
4BAdamW (η=2×10−5\eta = 2\times 10^{-5})1.32312.4852.9057.8676.3841.7057.21
4BAdamW (η=5×10−5\eta = 5\times 10^{-5})1.3133.3452.0557.6276.3042.3857.09
4BAdamW (η=1×10−4\eta = 1\times 10^{-4})1.3101.3551.9657.7576.0440.8656.65
4BAdamW (η=2×10−4\eta = 2\times 10^{-4})1.3147.2251.6256.1276.2342.5356.62
4BAdamW + L2-SP (λbase=0.01\lambda_{\mathrm{base}}=0.01, always-on)1.3131.6551.7957.0576.1640.4156.35
4BAdamW + L2-SP (λbase=0.01\lambda_{\mathrm{base}}=0.01, gated)1.3131.0152.3956.9476.1139.5856.25
4BDAPR w/o anchor (λbase=0.0\lambda_{\mathrm{base}}=0.0)1.3241.2950.6855.8975.9238.2955.20
4BDAPR (default, λbase=0.01\lambda_{\mathrm{base}}=0.01)1.3241.7850.6855.7875.9434.5754.24
4BDAPR (θ0\boldsymbol{\theta}_0-anchored, λbase=0.01\lambda_{\mathrm{base}}=0.01)1.3241.4250.4355.3875.9436.0154.44
  1. Key Finding: (i) Cutting η\eta to 2×10−52\times 10^{-5} slows validation convergence (1.251 →\to 1.261 on 8B; 1.314/1.310 →\to 1.323 on 4B) and leaves κdir=18.77\kappa_{\mathrm{dir}}=18.77 (8B) and 12.4812.48 (4B) vs. anchor-free DAPR's 3.61 and 1.29, while doubling η\eta on 8B causes −5.99-5.99 pp GSM8K loss (κdir=−32.41\kappa_{\mathrm{dir}}=-32.41). (ii) DAPR w/o anchor (λbase=0\lambda_{\mathrm{base}}=0) slashes κdir\kappa_{\mathrm{dir}} by 15.6×15.6\times (56.31 →\to 3.61) on 8B (61.66% 4-Mean) and 5.6×5.6\times (7.22 →\to 1.29) on 4B (+3.72 pp GSM8K over anchored default λbase=0.01\lambda_{\mathrm{base}}=0.01). (iii) Under Stress DPO (β=0.01\beta=0.01, Part 3), AdamW + L2-SP drops 8B GSM8K to 45.34% (-5.46 pp below DAPR), whereas on 4B DAPR recovers +4.63 pp GSM8K and +3.83 pp ARC over AdamW.

Question 3 & Weakness 3: Matched-Compute Flooding, S2-SAM\text{S}^2\text{-SAM}, QHAdam, & Periodic GSAM (4B–14B)

  1. Direct Answer: Across 4B–14B at matched single-pass compute (0.99×0.99\times–1.23×1.23\times AdamW runtime vs. 1.87×1.87\times–2.02×2.02\times for 2-pass SAM), QHAdam lacks a below-ctc_t crossing step to damp sharpness (-3.64/-5.08 pp SFT GSM8K on 8B/4B), Flooding + AdamW degrades SFT convergence and stalls DPO across scales, and while Periodic GSAM and cached $\text{S}^2\text{-SAM}$ are competitive, DAPR matches 2-pass SAM's Stress DPO stability on 4B without O(d)O(d) cached-gradient buffers or periodic 2×2\times-pass steps.
  2. Table Summary (Sec. 3.2–3.3, Fig. 3c & 4; Tables 1–2, 7 & 13): Shard-matched 8B/4B SFT & Stress DPO (--: SFT-only control or cross-shard run), followed by the complete 12B & 14B SFT & Stress DPO matrix:
Method (Tables 1–2 & 7)8B / 4B RT8B SFT: Val / GSM8K (Mean)4B SFT: Val / GSM8K (Mean)4B Stress DPO (ARC/MMLU/HS/GSM)8B Stress DPO (ARC/MMLU/HS/GSM)
Base---- / 49.43 (61.29)-- / 37.83 (56.61)54.78 / 57.90 / 75.93 / 37.8354.35 / 62.18 / 79.19 / 49.43
AdamW1.00x / 1.00x1.251 / 51.86 (62.24)1.314 / 42.53 (56.62)56.66 / 58.50 / 77.60 / 44.7358.62 / 63.00 / 83.35 / 54.59
AdamW + L2-SP1.03x / --1.252 / 51.48 (61.74)1.313 / 40.41 (56.35)60.75 / 58.44 / 80.19 / 48.8256.83 / 62.65 / 83.12 / 45.34
QHAdam (α0=0.5,ν1≈0.67\alpha_0=0.5, \nu_1\approx 0.67)1.02x / 0.99x1.259 / 48.22 (61.50)1.323 / 37.45 (55.03)----
Flooding + AdamW1.09x / 1.04x1.339 / 46.93 (60.40)1.401 / 18.27 (50.38)55.03 / 58.12 / 76.25 / 39.1254.86 / 62.32 / 79.59 / 51.55
S2-SAM\text{S}^2\text{-SAM}1.05x / 1.04x1.252 / 51.48 (61.96)1.314 / 40.18 (56.16)61.43 / 58.47 / 80.58 / 48.8257.94 / 62.72 / 83.15 / 52.54
Periodic GSAM (p=0.10p=0.10)1.17x / 1.12x1.251 / 51.71 (62.00)1.313 / 39.58 (56.35)----
SAM (2-pass)2.02x / 2.02x1.252 / 51.48 (61.99)1.312 / 38.97 (55.92)59.98 / 58.52 / 80.20 / 50.1160.24 / 63.11 / 84.89 / 50.64
DAPR (Ours)1.22x / 1.23x1.257 / 50.19 (61.20)1.324 / 34.57 (54.24)60.49 / 58.21 / 80.40 / 49.3657.76 / 62.54 / 83.44 / 50.80
Large-Scale (Tables 2 & 13)12B SFT GSM (Mean)12B Stress DPO (ARC/MMLU/HS/GSM)14B SFT GSM (Mean)14B Stress DPO (ARC/MMLU/HS/GSM)
Base71.19 (72.30)64.76 / 71.46 / 81.78 / 71.1989.54 (77.26)59.04 / 77.51 / 82.96 / 89.54
AdamW74.37 (73.04)72.01 / 71.73 / 84.90 / 78.3288.02 (77.06)62.46 / 77.82 / 83.71 / 86.58
AdamW + L2-SP73.69 (73.05)72.78 / 71.55 / 84.85 / 78.9287.34 (76.88)62.12 / 77.77 / 83.93 / 84.31
Flooding + AdamW74.75 (73.23)65.36 / 71.54 / 82.12 / 71.8086.96 (76.45)59.22 / 77.50 / 83.00 / 89.76
S2-SAM\text{S}^2\text{-SAM}74.60 (73.42)73.21 / 71.36 / 84.82 / 77.8687.64 (77.19)62.12 / 77.60 / 83.93 / 85.37
SAM (2-pass)74.22 (73.12)72.78 / 71.79 / 87.37 / 77.0388.48 (77.40)65.27 / 77.67 / 86.94 / 89.46
DAPR (Ours)74.00 (73.14)72.27 / 71.34 / 84.37 / 76.7288.63 (77.50)64.42 / 77.89 / 83.92 / 76.50
  1. Key Finding: Even with DAPR's dynamic ctc_t, Flooding + AdamW worsens SFT validation loss across 4B–14B, collapses 4B SFT GSM8K to 18.27%, trails DAPR by -10.24 pp GSM8K on 4B Stress DPO, and stalls near Base on 12B/14B DPO (65.36% ARC / 71.80% GSM8K on 12B Stress). DAPR leads 14B SFT GSM8K (88.63%), matches 2-pass SAM on 4B Stress DPO (49.36% vs. 50.11%) and 12B Stress DPO (76.72% vs. 77.03%), and openly delineates flat-basin 14B Stress DPO (κdir=3.79\kappa_{\mathrm{dir}}=3.79) where below-ctc_t ascent over-regularizes GSM8K (76.50%) while leading MMLU (77.89%) and gaining +1.96 pp ARC over AdamW.
Reviewer_YET4OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes DDAPR for retaining pretrained capabilities during LLM post-training. It preserves Adam's task-gradient moment estimates and adds a loss-dependent correction: additional descent above a moving loss center, or a projected ascent correction with initialization anchoring below it. The method reuses the current gradient rather than requiring the additional perturbed-gradient evaluation used by SAM. The paper motivates this design through local geometric analysis and evaluates it in LoRA-based SFT and DPO across 4B–14B models, with a language-aware extension for multilingual training.

Critical strengths
  1. meaningful practical gain at low additional update cost: DAPR improves from AdamW in multiple scenarios (Table 2). These retention improvements coexist with lower held-out SFT losses, which makes the result more compelling than simply preventing adaptation. Table 1 reports update-time costs of 1.01–1.07 times AdamW, versus 1.87–1.98 for SAM, within the measured forward/backward/update segment.
  2. coherent update design supported by substantive diagnostics: Separating the correction from Adam's moment estimates and projecting it relative to the Adam direction gives the method a more concrete rationale than an unexplained gradient modification. The component ablations, checkpoint trajectories, and conditional geometric analysis help assess the design.
Critical weaknesses
  1. The advantage over sufficiently tuned AdamW and simpler retention controls remains unclear. The paper reports optimizer settings, but does not adequately describe the AdamW search ranges, tuning budget, or validation-based selection procedure. Meanwhile, the main comparison uses a fixed 10K-step horizon without early stopping, and Table 3 primarily explores DAPR's components and hyperparameters.
  2. The empirical and mechanistic scope is narrower than a general optimizer advantage. All experiments use rank-16 LoRA on query/value projections. Since projection, anchoring, and curvature measurements operate in the trainable parameter space, robustness to this parameterization remains an open question. The 4B–14B coverage is useful, but does not substitute for variation in the trainable parameterization or a controlled scaling study.
Questions for authors
  1. How were AdamW's learning rates and weight decay selected, and what search budgets and validation criteria were used for AdamW and DAPR? Under a matched validation protocol, does DAPR still improve retained capability at comparable target-task quality relative to tuned or early-stopped AdamW?
  2. Do you have results for AdamW with initialization anchoring matched to DAPR's parameter space and anchoring form, with a reasonably selected strength? How much of the gain remains beyond this simpler control? This would help establish the additional contribution of the conditional projected correction.
  3. Do you have evidence that the gain persists with a different LoRA rank or target-module selection?
3 comments
AuthorsOpenReview ↗

Thanks for your actionable suggestions.

We thank Reviewer YET4 for the insightful review, especially for highlighting that separating DAPR's correction from Adam's moment buffers and projecting relative to the Adam direction provides a concrete rationale for an internal optimizer intervention. Below we address Q1, Q2, and W1 (Sec. 3.3, Fig. 3a, Tables 2 & 4; App. B.1–B.2, Tables 5a & 6).

Q1, Q2 & W1: Search Budgets, LR/WD Tuning, & AdamW + L2-SP vs. Anchor-Free DAPR

  1. Direct Answer:
    • Q1 (Search budgets & checkpointing): Neither AdamW nor DAPR uses per-benchmark tuning: AdamW uses standard LoRA SFT step sizes (η0=2×10−4\eta_0=2\times 10^{-4} on 4B, 10−410^{-4} on 8B, 2×10−52\times 10^{-5} on 12B/14B; λwd=0.01\lambda_{\mathrm{wd}}=0.01; shared DPO step sizes in Table 4), while DAPR fixes a single configuration (α0=0.5,γ=1,λ=1,τ=τs=0.1,λbase=0.01\alpha_0=0.5, \gamma=1, \lambda=1, \tau=\tau_s=0.1, \lambda_{\mathrm{base}}=0.01 [0.050.05 on 5-epoch LIMA], λwd=0\lambda_{\mathrm{wd}}=0) across models and regimes. Held-out UltraChat validation loss decreases through step 10K, so best-validation checkpointing coincides with step 10K.
    • Q2 (L2-SP vs. anchor-free DAPR): Flatness comes from projected descent–ascent crossings rather than L2-SP shrinkage: anchor-free DAPR (λbase=0\lambda_{\mathrm{base}}=0) slashes directional curvature by an order of magnitude on its own, whereas standalone AdamW + L2-SP fails under Stress DPO.
  2. Table & Math Summary (Tables 2, 5a & 6): Unlike static Euclidean L2-SP shrinkage toward θ0\boldsymbol{\theta}_0, setting λbase=0\lambda_{\mathrm{base}}=0 in Eq. (3) isolates DAPR's O(1)O(1)-state moving-center correction qt=arg⁡min⁡ut⊤z=012∥z−Ptgt∥Pt−12∈Im⁡(PtQt)\mathbf{q}_t = \arg\min_{\mathbf{u}_t^\top\mathbf{z}=0}\frac{1}{2}\Vert\mathbf{z}-\mathbf{P}_t\mathbf{g}_t\Vert_{\mathbf{P}_t^{-1}}^2 \in \operatorname{Im}(\mathbf{P}_t\mathbf{Q}_t) (Prop. 2). Below we report UltraChat validation loss, pure-float32 curvature κdir=g⊤Hg/∥g∥22\kappa_{\mathrm{dir}} = \mathbf{g}^\top\mathbf{H}\mathbf{g}/\Vert\mathbf{g}\Vert_2^2, and GSM8K (--: κdir\kappa_{\mathrm{dir}} not probed in Table 5a):
Configuration (Tables 5a & 6)8B Val ↓\downarrow8B κdir↓\kappa_{\mathrm{dir}} \downarrow8B GSM8K (Mean %)4B Val ↓\downarrow4B GSM8K (Mean %)
AdamW (η=2×10−5\eta = 2\times 10^{-5})1.26118.7751.18 (62.08)1.32341.70 (57.21)
AdamW (η=5×10−5\eta = 5\times 10^{-5})1.2536.5152.31 (62.20)1.31342.38 (57.09)
AdamW (η=1×10−4\eta = 1\times 10^{-4})1.25156.3151.86 (62.24)1.31040.86 (56.65)
AdamW (η=2×10−4\eta = 2\times 10^{-4})1.266-32.4145.87 (60.52)1.31442.53 (56.62)
AdamW (Best Val. Checkpoint)1.25156.3151.86 (62.24)1.31442.53 (56.62)
AdamW (WD λwd=0.10\lambda_{\mathrm{wd}}=0.10)1.252--51.86 (62.05)1.31339.20 (56.09)
AdamW + L2-SP (λbase=0.01\lambda_{\mathrm{base}}=0.01, always-on)1.2522.6751.48 (61.74)1.31340.41 (56.35)
AdamW + L2-SP (λbase=0.01\lambda_{\mathrm{base}}=0.01, gated)1.2514.0251.93 (62.14)1.31339.58 (56.25)
AdamW + L2-SP (λbase=0.05\lambda_{\mathrm{base}}=0.05)1.252--52.24 (62.15)1.31338.36 (55.82)
AdamW + L2-SP (λbase=0.10\lambda_{\mathrm{base}}=0.10)1.252--51.25 (61.91)1.31341.09 (56.41)
DAPR w/o anchor (λbase=0.0\lambda_{\mathrm{base}}=0.0)1.2573.6150.11 (61.66)1.32438.29 (55.20)
DAPR (default, λbase=0.01\lambda_{\mathrm{base}}=0.01)1.25712.5150.19 (61.20)1.32434.57 (54.24)
DAPR (θ0\boldsymbol{\theta_0}-anchored, λbase=0.01\lambda_{\mathrm{base}}=0.01)1.2577.7950.19 (61.52)1.32436.01 (54.44)
  1. Key Finding: Doubling η\eta on 8B destabilizes AdamW (κdir=−32.41\kappa_{\mathrm{dir}}=-32.41, -5.99 pp GSM8K), while cutting η\eta to 2×10−52\times 10^{-5} slows validation convergence (1.251 →\to 1.261 on 8B; 1.314/1.310 →\to 1.323 on 4B). Crucially, DAPR w/o anchor (λbase=0\lambda_{\mathrm{base}}=0) cuts κdir\kappa_{\mathrm{dir}} by 15.6×15.6\times (56.31 →\to 3.61) on 8B (61.66% Mean) and 5.6×5.6\times (7.22 →\to 1.29) on 4B, whereas standalone AdamW + L2-SP underfits 8B Stress DPO (45.34% GSM8K vs. 50.80% DAPR, Table 2).

Q3 & W2: LoRA Capacity Sweeps & Disentangling Model Scale, Relative Adapter Fraction, and Learning Rate

  1. Direct Answer: Holding r=16r=16 (q,v) fixed across 4B–14B shrinks the relative trainable fraction from ∼0.14%{\sim}0.14\% (4B) to 0.103% (12B) and 0.085% (14B) alongside η=2×10−5\eta=2\times 10^{-5} in SFT, so 12B/14B benign stability reflects conservative step sizes (Rofin et al., 2026) and smaller relative adapter subspaces rather than scale alone. Within a fixed model and parameterization, expanding LoRA capacity (r=64r=64, All Linear: Attn+MLP) induces single-pass SFT forgetting in AdamW, while at matched r=16r=16 and η=2×10−5\eta=2\times 10^{-5}, β=0.01\beta=0.01 Stress DPO induces sharp-basin collapse on 4B that DAPR prevents.
  2. Table & Math Summary (Sec. 3.3, Fig. 3b; Tables 2, 4, 5b & 13; App. B.1): Table 5b sweeps r∈{8,16,64}r\in\lbrace 8,16,64 \rbrace on (q,v) and r=16r=16 on All Linear (Attn+MLP). Mathematically, because LoRA initializes A0≠0\mathbf{A}_0\neq\mathbf{0} (Kaiming) and B0=0\mathbf{B}_0=\mathbf{0}, shrinking wide r=64r=64 or MLP-expanded At\mathbf{A}_t toward 0\mathbf{0} (anchor_to_init=False) attenuates adapter singular values—motivating explicit (A0,B0)(\mathbf{A}_0,\mathbf{B}_0) anchoring (θ0\boldsymbol{\theta}_0-anchored row in Part 1, which cuts 8B κdir\kappa_{\mathrm{dir}} from 12.51 to 7.79 and lifts 4B r=16r=16 GSM8K by +1.44 pp [34.57% →\to 36.01%]):
LoRA Parameterization (Table 5b)8B Val ↓\downarrow8B GSM8K (Mean %)4B Val ↓\downarrow4B GSM8K (Mean %)
r=8r=8 (q,v): AdamW1.25351.55 (62.10)1.31238.82 (56.34)
r=8r=8 (q,v): AdamW + L2-SP1.25351.40 (61.96)1.31238.67 (56.35)
r=8r=8 (q,v): DAPR1.25650.42 (61.51)1.31838.51 (55.45)
r=16r=16 (q,v, default): AdamW1.25151.86 (62.24)1.31442.53 (56.62)
r=16r=16 (q,v, default): AdamW + L2-SP1.25251.48 (61.74)1.31340.41 (56.35)
r=16r=16 (q,v, default): DAPR1.25750.19 (61.20)1.32434.57 (54.24)
r=64r=64 (q,v): AdamW1.26147.61 (61.01)1.33431.46 (53.52)
r=64r=64 (q,v): AdamW + L2-SP1.26145.64 (60.69)1.33434.87 (54.19)
r=64r=64 (q,v): DAPR1.28540.64 (57.84)1.37221.00 (49.16)
r=16r=16 (All Linear): AdamW1.26048.14 (61.22)1.36832.98 (52.32)
r=16r=16 (All Linear): AdamW + L2-SP1.25946.32 (60.44)1.37029.95 (51.83)
r=16r=16 (All Linear): DAPR1.30440.56 (57.46)1.47011.37 (41.01)
  1. Key Finding: At attention-only r∈{8,16}r\in\{8,16\}, DAPR maintains parity with AdamW on 8B (61.51%/61.20% vs. 62.10%/62.24% Mean), while r=64r=64 and All Linear induce 4–11 pp GSM8K drops in AdamW (51.86% →\to 47.61% on 8B; 42.53% →\to 31.46% on 4B). Under β=0.01\beta=0.01 Stress DPO at η=2×10−5\eta=2\times 10^{-5}, DAPR gains +4.63 pp GSM8K on 4B (49.36% vs. 44.73%, matching SAM's 50.11%).

Dear Reviewer YET4,

Thank you for the careful review and for recognizing both DAPR's practical gains at low additional cost and the rationale for keeping its correction separate from Adam's moment estimates.

As the discussion period is drawing to a close, could you let us know whether our response address your main concerns? If you still have questions, we would appreciate the chance to respond while there is time.

Thank you again for your time and efforts in reviewing our manuscript.

On behalf of all authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response

We thank all reviewers for their actionable suggestions. Here is a summary of revision.

  1. Core Contributions & Revision Scope
    • Algorithmic Contribution (O(1)O(1) scalar state, zero cached gradient buffer): Unlike loss-level Flooding (Ishida et al., 2020; Karakida et al., 2023), two-pass SAM/GSAM (Zhuang et al., 2022), or O(d)O(d)-buffer S2-SAM\text{S}^2\text{-SAM} (Ji et al., 2024), DAPR intervenes inside Adam's post-preconditioning update—keeping (mt,vt,ut)(\mathbf{m}_t,\mathbf{v}_t,\mathbf{u}_t) 100% unmodified while gating Pt\mathbf{P}_t-orthogonal ascent via ct=μt−λstc_t=\mu_t-\lambda s_t ,positioned alongside Li et al. (2024a,b), Rofin et al. (2026), Ma & Yarats (2019), Han et al. (2025), and Li et al. (2018) (Sec. 1–2; App. A.1–A.2, C).
    • Empirical & Diagnostic Contribution: Added 4-point LR grids, always-on/ctc_t-gated L2-SP, anchor-free DAPR (λbase=0\lambda_{\mathrm{base}}=0), QHAdam, Flooding + AdamW, $\text{S}^2\text{-SAM}$, Periodic GSAM, LoRA capacity sweeps (r∈{8,16,64}r\in\{8,16,64\}, All Linear), 15-row unified ablations, trajectory telemetry, and 8-language Aya LA-DAPR.
  2. Table & Math Summary :
    • Table 1 & App. B.2.4: Pure-float32 Pearlmutter curvature κdir=g⊤Hg/∥g∥22\kappa_{\mathrm{dir}} = \mathbf{g}^\top\mathbf{H}\mathbf{g}/\Vert\mathbf{g}\Vert_2^2, GSM8K (flexible-extract), and shard-matched runtime across 4B–14B.
    • Tables 2–9, 11, 13 & Fig. 1–4: Multi-scale SFT, LIMA, Standard/Stress DPO (β∈{0.10,0.01}\beta\in\{0.10,0.01\}), logit/margin probes, and LR/L2-SP/LoRA/matched-compute controls.
    • Tables 10, 12 & Proposition 3: Step-by-step residency, crossings, ∥g∥,t∥/∥gt∥\Vert\mathbf{g}_{\parallel,t}\Vert/\Vert\mathbf{g}_t\Vert, state drift, pure-float32 2-step Taylor remainders, and 8-language LA-DAPR residency.
  3. Key Findings:
    • Why DAPR is non-reducible to Flooding, QHAdam, or L2-SP: Feeding σtgt\sigma_t\mathbf{g}_t into AdamW (Flooding + AdamW) causes 99.97%99.97\% momentum persistence and 35.7%35.7\%–36.8%36.8\% (37.6%37.6\% on 12B) sign overrides, degrading SFT validation loss across 4B–14B (1.401, 1.339, 1.275, 1.241), collapsing 4B SFT GSM8K to 18.27%, and stalling 12B/14B DPO near Base (65.36% ARC / 71.80% GSM8K on 12B Stress); QHAdam lacks below-ctc_t crossings (-3.64/-5.08 pp GSM8K), and L2-SP underfits 8B Stress DPO (45.34% GSM8K). DAPR cuts κdir\kappa_{\mathrm{dir}} by 2.3×2.3\times–4.9×4.9\times (5.6×5.6\times–15.6×15.6\times at λbase=0\lambda_{\mathrm{base}}=0 on 4B/8B) at 1.07×1.07\times–1.23×1.23\times AdamW runtime.
    • Regime-driven collapse, LA-DAPR equalization, & boundaries: Holding Gemma-3-4B, r=16r=16 (q,v), and η=2×10−5\eta=2\times 10^{-5} fixed, benign 0.50.5-epoch SFT does not forget under AdamW, whereas β=0.01\beta=0.01 Stress DPO triggers quadruple-RMSNorm logit surge and Step-8K margin inversion (Δlog⁡π=−34.44\Delta\log\pi=-34.44 vs. DAPR's +3.91+3.91, Table 11) that DAPR prevents (+4.63 pp GSM8K, +3.83 pp ARC, matching 2-pass SAM), alongside +1.06 pp GSM8K on 5-epoch 8B LIMA and 25.6%25.6\%–31.4%31.4\% 8-script residency equalization in LA-DAPR (Table 12). Across 4B–14B, DAPR leads 14B SFT (88.63% GSM8K, 61.09% ARC) and 14B Standard DPO MMLU (78.20%, retaining 90.07% GSM8K vs. 90.45% AdamW), while quiescent flat basins (η=2×10−5\eta=2\times 10^{-5}, r=16r=16 fraction 0.14% →\to 0.085%, κdir=3.79\kappa_{\mathrm{dir}}=3.79 on 14B) delineate where unconditional below-ctc_t ascent can over-regularize 12B Standard and 14B Stress DPO.
AuthorsOpenReview ↗

The rebuttal deadline is approaching.

Dear all reviewers,

Thank you for your time and effort in assessing our paper. The author response window closes within 48 hours. We would like to hear your follow-up questions and actionable suggestions.

Open questions are aslo welcome.

Thank you.

Team working on DAPR

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.