acceptodds
Under review as a conference paper at ICLR 2027

CREDIT LOST IN TRANSLATION: OPTIMIZER GEOMETRY IN VERIFIER-GUIDED LLM POST-TRAINING

Abstract

Veriffer-guided post-training has focused on constructing reliable learning signals, but a useful gradient is not yet a parameter update. We ask whether the optimizer preserves the behavioral information already present in veriffed credit. We study this interface through controlled local-update audits of Qwen2.5-7B with low-rank adaptation. In a compositional-credit audit, two updates share the same check-point, veriffer-derived objective, and instantaneous gradient, and are matched in parameter-space norm. The raw descent step has cosine alignment approximately 0.98 with an independently deffned behavioral-realization direction, compared with 0.51 for the AdamW displacement. It also produces approximately 3.5–4.5 times greater reduction of two measured realization barriers. A separate atomic-credit audit shows the same qualitative gap. We call this phenomenon optimizer-geometry distortion and provide a diagnostic framework that separates gradient ffdelity, behavioral alignment, and ffnite-step realization. An atomic component analysis implicates adaptive second-moment preconditioning; the corresponding compositional replication remains pending. These local ffndings do not establish that adaptive optimization is generally harmful. They instead motivate studying credit assignment and credit realization separately: improving what a gradient says need not ensure that its behavioral meaning survives optimization.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Optimizer Geometry Distortion, Verifier-Guided Credit, Behavioral Realization

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_188pOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper observes that coordinate-wise second-moment scaling in Adam-style optimizers changes not only the magnitude of an update but also its direction, and proposes a way to quantify and directly control this rotation. For a vector mm scaled by positive per-coordinate factors aa, the angle between mm and m⊙am \odot a is exactly given by tan⁡θ=CVw(a)\tan\theta = \mathrm{CV}_w(a), where wi∝mi2w_i \propto m_i^2. The rotation is therefore the energy-weighted anisotropy of the scale on the coordinates that carry the update. Building on this identity, the authors propose rotation-budgeted preconditioning. At every step, it selects the preconditioning exponent pp so that the applied update stays within a target angle τ\tau of the momentum direction, using the strongest preconditioning that satisfies this constraint.

Critical strengths

This paper shows, both analytically and empirically, that adaptive optimizers such as Adam influence the direction of an update and not only its magnitude. Analytically, the authors derive that the rotation induced by coordinate-wise scaling is exactly tan⁡θ=CVw(a)\tan\theta = \mathrm{CV}_w(a), which reduces the change in direction to a single interpretable quantity. Empirically, they measure that second-moment scaling produces rotations of roughly 60∘60^\circ or more on both Qwen2.5-7B and Mistral-7B, and that this rotation persists throughout training. However, this strength amounts to a quantification of the fact that the optimizer changes the update direction. It does not establish that this rotation is harmful to learning, nor that the proposed control is practically beneficial, so it is insufficient to offset the weaknesses described below.

Critical weaknesses
  1. The manuscript is too poorly written and incomplete to allow its claims and contributions to be assessed.

The paper is very hard to read and its context is difficult to follow. Key concepts (RObE, pva244, construction, and the P1 margin) are used without being defined in the main text, and even the meaning of the held-out P1 margin, the only performance metric, cannot be determined. The figures are not tied together or explained within the narrative, and some are hard to reconcile with the reported numbers (e.g., Figure 3(d) versus Table 5). The appendix retains draft text such as REQUIRED FOR SUBMISSION'' and artifact requirement,'' so the manuscript does not read as a submission-ready paper. The paper also does not clearly state what its contribution is (the identity, the controller, or the performance gain) or how it differs from prior work. Without the evaluation metric and objective, the experimental results cannot be verified. This is therefore not a matter of presentation alone, but one that prevents any assessment of the validity of the results.

  1. There is no in-depth analysis of how the rotation affects learning, and the motivation for the proposed method is not established.

The paper measures that second-moment scaling rotates the momentum direction, but it does not analyze how this rotation affects learning. It does not justify why the momentum direction should serve as the reference, why a budget of 25∘25^\circ is appropriate, or why rotation should be regarded as harmful in the first place. The central theoretical result, tan⁡θ=CVw(a)\tan\theta = \mathrm{CV}_w(a), is an algebraic identity that follows directly from the definition of the cosine, so its numerical validation carries little information. In the mechanism controls (Table~10), the ordering of outcomes does not follow the rotation angle, and this is left unexplained. The performance evidence rests on a single undefined metric, three seeds, and a comparison without learning-rate tuning, and the proposed dynamic budget improves on fixed p=0.4p = 0.4 by only 0.4420.442. The central claim that controlling rotation is beneficial is therefore not supported.

Questions for authors

Questions:

Q1. Metrics, objectives, and implementation details.

Please define held-out P1 margin, construction, RObE, and pva244 in the main text. In particular, what is the P1 margin a difference of, what are its range and interpretation, and does it correlate with standard metrics such as pass@kk? Also, tan⁡θ=CVw(a)\tan\theta = \mathrm{CV}_w(a) appears to follow directly from the definition of the cosine. What exactly is novel, and how does it differ from partially adaptive methods (e.g., Padam), the Kantorovich inequality, and trust-ratio methods (LARS/LAMB)? Finally, for ϵ>0\epsilon > 0, which algorithm is used to find pt∗p^*_t? Were there training steps where θt(p)\theta_t(p) was non-monotone in pp? What value of ϵ\epsilon is used? Please also resolve the inconsistency between Figure3(d) and Table5, and the differing canonical rotation angles (58.33∘58.33^\circ, 58.3153∘58.3153^\circ, and 58.3337∘58.3337^\circ).

Q2. Reference direction, budget, and causality of rotation.

Why should rotation be considered harmful, and why is the momentum direction the right reference? On Mistral, the raw gradient and momentum already differ by 83∘83^\circ. Do the conclusions hold if the reference is changed (e.g., to the raw gradient or the natural gradient)? How was 25∘25^\circ chosen, and does the effect persist at other values of τ\tau? In Table~10, the ordering of outcomes does not follow the rotation angle. How do you explain this, and is there an experiment that directly tests the hypothesis that rotation itself drives performance? Could the roughly 60∘60^\circ rotation arise from coordinates near the ϵ\epsilon floor or from scale differences between LoRA AA and BB matrices? Please provide per-module and per-matrix breakdowns.

Q3. Fair performance comparison.

Does AdamW remain the lowest when the learning rate is tuned separately for each arm? The gap in seed~2 (+12.2+12.2) dominates the mean. What happens with more seeds? Does the difference between ROT BUDGET and a tuned fixed pp exceed seed-to-seed variation?

Suggestions for Future Work (do not affect my assessment)

  • Reporting downstream performance on Mistral (e.g., held-out loss or pass@kk) would allow the practical value of the control to be judged.

  • Providing per-checkpoint training curves would reveal learning dynamics that endpoint values alone cannot show.

Reviewer_XsFSOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies how diagonal preconditioning changes the direction of an optimizer update. It derives an exact relation between the rotation angle and the weighted coefficient of variation of the coordinate-wise scaling factors. It then introduces a rotation-budget controller that adjusts the strength of second-moment preconditioning while preserving the AdamW step norm. Experiments on Qwen2.5-7B LoRA verifier post-training and Mistral-7B code SFT measure this rotation, and the Qwen experiments compare downstream performance under different amounts of preconditioning.

Critical strengths
  1. The paper gives a simple, exact characterization of how positive diagonal scaling rotates an update direction.
  2. The decomposition of rotation into momentum, second-moment scaling, and weight-decay contributions is informative.
  3. The norm-matched comparisons and paired runs across seeds are useful controls. The paper also acknowledges that a smaller rotation does not always imply a better task outcome.
Critical weaknesses
  1. The submission appears unfinished, both in presentation and in its experimental documentation. Several figure labels extend outside their panels, and the two 67.798-degree labels overlap in Figure 3a. More importantly, Appendix D explicitly states that the manuscript snapshot does not contain checkpoint-level evaluation values, while Appendix I lists the exact objectives, metric definitions, run configurations, and other records that the authors must supply. Page 3 also leaves the verifier objective and RObE construction as an author-supplied artifact item. These are not just cosmetic issues: without the objective, the pva244 construction, and a clear definition of the held-out P1 margin, it is difficult to interpret or reproduce the main empirical results.

  2. I do not think the paper sufficiently explains why controlling rotation should improve optimization. A large turn away from momentum is not necessarily a bad update; changing direction is part of what preconditioning is intended to do. If the motivation were simply that less rotation is better, then the zero-rotation momentum direction would be the natural endpoint. The paper does include this p=0 control, but Table 5 shows that it performs worse than partial preconditioning and rotation budgeting. The authors also acknowledge that performance is not monotone in the angle. This makes the missing explanation more important: when is rotation useful, when is it harmful, and why should a fixed budget such as 25 degrees be appropriate? The identity and the controller establish that the angle can be measured and constrained, but do not establish why this is a useful optimization criterion. The p=0 arm also uses a matched step-norm convention, so it should not be treated as a substitute for an independently tuned SGDM baseline. The paper needs downstream performance sweeps over the angle budget and exponent, together with well-tuned SGDM and AdamW comparisons. As it stands, the study feels incomplete beyond the geometric observation.

  3. The added value over fixed partial preconditioning remains unclear. Padam already introduces a second-moment exponent that interpolates between momentum-like and fully adaptive updates, although the full algorithms can differ. Here, the reported mean gain over fixed p=0.4 is only +0.442 P1 margin points across three seeds, and Section 4 explicitly acknowledges that these runs do not establish a meaningful advantage of dynamic budgeting over a well-chosen fixed exponent. Mistral establishes geometric control but provides no downstream-performance comparison. A controller that keeps the angle constant is not sufficient evidence of an improvement over directly tuning the exponent.

Questions for authors
  1. Are the missing records described in Appendices D and I available? Please include the essential objective and metric definitions, provide the training/evaluation curves, and correct the overflowing and overlapping figure labels.
  2. What makes rotation harmful or beneficial in this setting? Given that p=0 is not the best-performing arm, what explains the advantage of retaining some preconditioning, and how should the angle budget be chosen?
  3. Can the authors compare against independently tuned SGDM, AdamW, and fixed-exponent methods under a matched tuning budget, distinguishing ordinary SGDM from the norm-matched p=0 control? Please report downstream performance across angle budgets and exponents, including on Mistral, rather than only how accurately the requested angle is maintained.
Reviewer_KuVpOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies how Adam’s scaling changes the direction of an update. It gives an exact formula for the angle between momentum and the scaled update, then proposes a method that adjusts the scaling strength to keep this angle within a chosen limit. Experiments on Qwen and Mistral with LoRA show that the method can control the angle during training. Qwen experiments also report higher P1 scores than AdamW, while Mistral experiments only test angle control.

Critical strengths

The paper gives a clear formula explaining how scaling changes the update direction. Using updates of the same size helps separate the effect of direction from that of step size.

Critical weaknesses
  1. Multiple paragraphs show that the manuscript is an unfinished draft:
  • Section 2.1: “The exact verifier objective and RObE construction remain an author-supplied artifact item.”
  • Appendix D: “The manuscript snapshot does not contain checkpoint-level evaluation values. Those raw records are required before replacing the endpoint summary in Figure 3 with training curves.”
  • Appendix H: “Detailed hardware manifests and high-energy versus numerical-floor coordinate diagnostics remain artifact requirements.”
  • Appendix I is an internal to-do list that should not appear in a submission. It begins “The authors must supply the records needed to reconstruct every reported quantity,” then lists the verifier objectives, optimizer settings, metric definitions, and raw run records.
  • Table 9, “Reported per-step SD” for “Seed 3”, is listed as “not supplied.”
  • Section 7 (Lines 378-379) mentions “the supplied Mistral summaries”.
  • Appendix F has only two sentences, and Appendix G has only one, both reads like placeholders. Tables 10–12 are never discussed in the text.
  • The PDF's title (”FROM ROTATION TO CONTROL: ADAPTIVE PRECONDITIONING OF VERIFIER-DERIVED UPDATES”) is different from the title on OpenReview's webpage (”CREDIT LOST IN TRANSLATION: OPTIMIZER GEOMETRY IN VERIFIER-GUIDED LLM POST-TRAINING”).

As a result, the training objective and the major metric are never defined, and the optimizer hyperparameters and raw results are not provided, which makes the main results difficult to check and reproduce.

  1. The benefit of dynamic angle control is still unclear. The paper does not clearly explain its differences from related methods such as Padam. It also does not show that changing the exponent during training works better than choosing a good fixed exponent. Keeping an angle fixed does not by itself show better learning.
Questions for authors

With the same tuning budget, does dynamic angle control perform better than well-tuned fixed exponents? Please provide results for each seed to show whether the gain comes from changing the exponent during training or simply using weaker scaling.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.