CREDIT LOST IN TRANSLATION: OPTIMIZER GEOMETRY IN VERIFIER-GUIDED LLM POST-TRAINING
Abstract
Veriffer-guided post-training has focused on constructing reliable learning signals, but a useful gradient is not yet a parameter update. We ask whether the optimizer preserves the behavioral information already present in veriffed credit. We study this interface through controlled local-update audits of Qwen2.5-7B with low-rank adaptation. In a compositional-credit audit, two updates share the same check-point, veriffer-derived objective, and instantaneous gradient, and are matched in parameter-space norm. The raw descent step has cosine alignment approximately 0.98 with an independently deffned behavioral-realization direction, compared with 0.51 for the AdamW displacement. It also produces approximately 3.5–4.5 times greater reduction of two measured realization barriers. A separate atomic-credit audit shows the same qualitative gap. We call this phenomenon optimizer-geometry distortion and provide a diagnostic framework that separates gradient ffdelity, behavioral alignment, and ffnite-step realization. An atomic component analysis implicates adaptive second-moment preconditioning; the corresponding compositional replication remains pending. These local ffndings do not establish that adaptive optimization is generally harmful. They instead motivate studying credit assignment and credit realization separately: improving what a gradient says need not ensure that its behavioral meaning survives optimization.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.