acceptodds
Under review as a conference paper at ICLR 2027

Embedding Noise Should Follow the Learning Rate in LLM Fine-Tuning

Abstract

Adding random noise to token embeddings improves supervised fine-tuning (SFT) of large language models (LLMs). The noise helps early in training, when a small instruction set is revisited for several epochs, but it becomes a problem at the end. As the learning rate (LR) decays, the optimizer takes smaller steps, but noise of fixed scale keeps adding a gradient bias that does not shrink. In our case study, keeping the noise through the last quarter of training leaves the model farther from a stationary point of the clean training loss, which is the loss without noise, and the gradient of this loss stays larger than when the noise is removed. We introduce AdaNoiseSFT, which sets the variance of the embedding noise proportional to the current LR, so the noise is strong while the LR is high and fades as the LR decays, with no additional model passes or optimizer state. Our analysis for Adam assumes that the smoothing effect of the noise does not vanish late in training. Under this condition, fixed noise leaves a gradient bias that does not vanish for any LR schedule, whereas coupling the noise to the LR drives this bias to zero when the LR decays to zero. To test that the benefit comes from following the LR schedule itself, we evaluate cosine, linear, step, and warmup stable decay (WSD) LR schedules against independently tuned noise tapers that follow elapsed training time. On Llama-3.1-8B with Alpaca-cleaned and a cosine LR, AdaNoiseSFT reaches 19.5 length-controlled AlpacaEval 2.0 (AE-2) over five seeds, versus 18.1 for the strongest baseline, a time taper selected on a separate selection split. Over ten seeds under linear, step, and WSD LR schedules, it beats the selected time taper by 1.39, 1.14, and 0.91 AE-2 points, respectively. For all four LR schedules, the best noise schedule is the one that follows the shape of the LR.

Primary area:
foundation or frontier models, including LLMs
Keywords:
supervised fine-tuning, large language models, embedding noise, NEFTune, learning-rate schedules, regularization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_upHaOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes AdaNoiseSFT, which injects Gaussian noise into input embeddings during instruction fine-tuning of large language models. The noise variance follows the learning-rate schedule and gradually decreases toward the end of training. Experiments across multiple models and instruction datasets show improvements in instruction following and conversational quality over fixed noise and several noise-decay baselines.

Critical strengths

The ablation studies are thorough, including detailed ablation studies on noise schedules and the effects of noise near the end of training.

Critical weaknesses
  1. Adding noise during training and gradually decaying it is already a well-established practice [1–2]. More specifically, injecting noise into embeddings and annealing it was explored as early as 2019 [3]. Therefore, I view the main contribution of this work as proposing that the noise added to the input embeddings should decay with the learning rate, which is a relatively minor contribution. [1] Bayesian Learning via Stochastic Gradient Langevin Dynamics (2011) [2] Adding Gradient Noise Improves Learning for Very Deep Networks (2015) [3] Improving Neural Language Modeling via Adversarial Training (2019)
  2. The training data cover too few domains, focusing only on instruction tuning. The method should also be evaluated on other training data, such as mathematics, code, or more specialized domain tasks. Otherwise, its practical value remains unclear.
  3. The models are relatively small, mainly in the 7B–14B range, and training primarily uses LoRA. Full fine-tuning is evaluated only on a 1.7B model. At a minimum, the experiments should include full fine-tuning of 7B–14B models, and preferably larger models as well.
Questions for authors

Have the authors considered testing noise decay tied to the learning rate during LLM fine-tuning when noise is added to hidden states [1], soft prompts [2], gradients [3], or parameter updates [4]? These experiments could help assess the broader applicability of the proposed principle and strengthen the paper’s contribution.

  • [1] Noise Augmented Fine Tuning for Mitigating Hallucinations in Large Language Models (2025)
  • [2] From Points to Clouds: Learning Robust Semantic Distributions for Multi-modal Prompts (2025)
  • [3] Adding Gradient Noise Improves Learning for Very Deep Networks (2015)
  • [4] Bayesian Learning via Stochastic Gradient Langevin Dynamics (2011)
5 comments

We thank the reviewer for the careful reading and the concrete list of experiments. We ran all of them except fine-tuning a model larger than 14B, for which we point to the existing Qwen3-32B result.

W1. Novelty

We agree that decaying injected noise is not new, and the paper does not claim it: the introduction presents decay as a known fix, calls this family time tapers, and uses independently tuned time tapers as the main baselines. We thank the reviewer for pointing us to Wang et al. (2019), which we had missed: as a training detail, they add Gaussian noise to the input embeddings whose level decreases from 0.2 to 0 over the run. We now cite it as an early time taper on input embeddings.

None of the three works analyses the bias of input-space noise or compares a time-based with an LR-based schedule. SGLD (Welling and Teh, 2011) adds zero-mean noise of variance ϵt\epsilon_t to the parameter update in order to sample the posterior, the opposite goal. Neelakantan et al. (2015) decay the variance of zero-mean gradient noise as c/(1+t)0.55c/(1+t)^{0.55} with the step count tt, and in their MNIST experiments the LR stays fixed. Wang et al. (2019) give only the endpoints of their decay and no ablation. Zero-mean noise added to the gradient or to the parameter update does not bias the gradient, whereas noise at the input of a nonlinear model does.

Our contribution is (i) the observation that noise at the input of a nonlinear model shifts the expected gradient by stHts_t H_t (Lemma 1), a bias that persists under fixed noise as the LR decays (Propositions 1–2), (ii) the rule that the variance should follow the optimizer's LR rather than elapsed time, which keeps this bias at O(ηt2)O(\eta_t^2) per update (Proposition 3), and (iii) the identification of this rule. Under cosine LR the two schedules coincide, so we tested linear, step and WSD LR, with every taper's peak and the comparator selected on a separate split before test evaluation. Under all four LR schedules the schedule that matches the LR wins, by 0.91–1.50 AE-2 over the best time taper (ten seeds, all 95% CIs above zero, Table 2, Fig. 3). The weakest mismatched taper trails LR coupling by 1.30–2.10 AE-2 (Table 37). On a second complete 4×4 matrix (Mistral-7B), the matching schedule beats every mismatched taper by 0.74–1.61 AE-2 (App. J.1), and the step/WSD result replicates on Qwen3-14B-Base and UltraChat-50k (Table 39). In practice, no noise-injection method used for SFT (NEFTune, SymNoise, NoiseFiT, GIFT-SW) decays the noise. The change is one line, adds no model pass, and gains 3.1–3.9 AE-2 over tuned fixed noise (Table 1).

Revision: Wang et al. (2019) in Sec. 1 and 2, and a sentence in Sec. 4 on why zero-mean noise differs from input noise.

W2. Other domains

We applied the paper's protocol to three more domains with Llama-3.1-8B (LoRA, WSD LR, ten seeds): mathematics (20k examples from Tulu-3 Persona MATH/GSM and NuminaMath-TIR, evaluated on GSM8K and MATH-500 with chain of thought), code (20k from Tulu-3 Persona Python and Evol CodeAlpaca, evaluated on HumanEval+ and MBPP+) and medicine (20k from the MedMCQA, MedQA-USMLE and PubMedQA training splits, with MedMCQA evaluated on its official validation split and no train/test overlap). Each corpus re-selects the noise peak, and the comparator taper is fixed by the lowest selection loss before test evaluation. Δ = AdaNoiseSFT − time taper, paired 95% CI:

BenchmarkVanillaFixed (NEFTune)TaperAdaNoiseSFTΔ vs taper (95% CI)
GSM8K62.5262.4163.1564.15+1.00 [0.63, 1.37]
MATH-50024.6425.2025.2625.64+0.38 [−0.06, 0.82]
HumanEval+46.7147.0748.6649.51+0.85 [0.20, 1.51]
MBPP+52.3353.4954.1554.92+0.77 [0.41, 1.13]
MedQA55.5655.7356.0356.62+0.60 [0.34, 0.86]
MedMCQA49.8950.4050.7250.59−0.13 [−0.41, 0.15]
PubMedQA70.6270.4470.5070.40−0.10 [−0.59, 0.39]

LR coupling beats the best taper on GSM8K, HumanEval+, MBPP+ and MedQA, is within noise on MATH-500, MedMCQA and PubMedQA, and is significantly worse on none. The value of embedding noise itself depends on the task: on PubMedQA no noise schedule beats vanilla SFT, which we add to the Limitations. On Alpaca-cleaned, AdaNoiseSFT leaves GSM8K, BBH, MMLU and TruthfulQA unchanged relative to fixed noise on all three models (all paired CIs include zero, Tables 43–45).

Revision: new App. K.1, a summary in Sec. 5.5, and the Limitations.

W3. Scale and full fine-tuning

We ran full fine-tuning (all parameters, including the input embeddings) of Llama-3.1-8B and Qwen3-14B-Base, ten seeds each, under cosine LR with the LR grid re-selected:

ModelMetricFixedLinear taperAdaNoiseSFTΔ vs taper (95% CI)
Llama-3.1-8BAE-217.0218.5119.65+1.14 [0.70, 1.58]
Llama-3.1-8BIFEval50.7451.7952.59+0.79 [0.18, 1.41]
Llama-3.1-8BArena-Hard18.5719.5320.15+0.62 [0.39, 0.85]
Qwen3-14B-BaseAE-222.6623.6724.52+0.85 [0.41, 1.29]
Qwen3-14B-BaseIFEval57.7858.4859.26+0.78 [0.15, 1.40]
Qwen3-14B-BaseArena-Hard25.4426.5426.83+0.29 [−0.10, 0.67]

LR coupling beats the linear taper in five of six comparisons. The exception (Arena-Hard, Qwen3-14B-Base) is within noise. Our theory covers LoRA with frozen embeddings, so this is empirical evidence beyond its scope. For a larger model, Table 14 reports Qwen3-32B (LoRA): 25.0 vs 22.7 AE-2 over fixed noise. We now mention it in Sec. 5.5. Revision: App. K.2 and Sec. 5.5.

Q. Hidden states, soft prompts, gradients and parameter updates

Llama-3.1-8B + Alpaca-cleaned, WSD LR, ten seeds. Each arm selects its own peak. AE-2:

Noise locationFixedBest taperLR-coupledΔ vs taper (95% CI)Δ vs fixed (95% CI)
Hidden states14.2915.6316.49+0.85 [0.06, 1.65]+2.19 [1.56, 2.83]
Soft prompts13.5613.8213.72−0.10 [−0.62, 0.42]+0.16 [−0.39, 0.72]
Gradients11.8111.9012.30+0.41 [−0.07, 0.88]+0.49 [−0.07, 1.05]
Parameter updates12.4212.7412.49−0.24 [−0.83, 0.35]+0.07 [−0.43, 0.57]
Input embeddings (paper, Table 2)—15.7316.64+0.91 [0.62, 1.20]—

Hidden-state noise behaves like embedding noise: LR coupling beats the best taper by +0.85 and fixed noise by +2.19 AE-2. For gradients and parameter updates no schedule differs significantly, as expected for zero-mean noise, which does not bias the gradient. For soft prompts the three arms lie within 0.26 AE-2. These vectors are themselves trained, which Lemma 1 does not cover. Together with LoRA dropout (App. J.4: +0.81–0.92 AE-2 over fixed dropout), the rule thus transfers to perturbations of activations, not to zero-mean noise on gradients or updates. Revision: App. K.3. The conclusion states this scope.

Soundness and presentation

If there is a specific step the reviewer found unconvincing, we would be glad to address it directly. To make the evidence easy to check: (i) all peaks and comparators are selected on a held-out split before test evaluation (App. D), (ii) the central comparisons use ten matched seeds with paired 95% CIs, and settings whose intervals cross zero are reported (App. G), (iii) we now check the key assumption directly: on the fixed-noise runs (seeds 11–55), the bias per unit variance stays between 0.88 and 1.06 of its value at τ=0.25\tau = 0.25 until the end of training (SE ≤ 0.018, App. I.2). For presentation, we added a notation table and a proof roadmap to App. B and a hyperparameter-selection paragraph to Sec. 5.1.

We hope these experiments and clarifications address the reviewer's concerns, and we would be grateful if the reviewer would reconsider the score.

Dear Reviewer,

As the discussion period closes today, we wanted to check whether our responses and the revised PDF address your concerns. We would be glad to clarify anything that remains unclear.

Thank you again for your time.

Reviewer_NBhMOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces AdaNoiseSFT, which sets the variance of embedding noise proportional to the current learning rate during LLM supervised fine-tuning.. Experiments across cosine, linear, step, and WSD learning-rate schedules show that the best noise schedule follows the shape of the learning rate, improving instruction-following performance.

Critical strengths
  1. The paper is well written and well motivated, with a theoretical analysis that provides a reasonable mechanistic explanation for noise annealing. The experiments are extensive, covering multiple models, datasets, learning-rate schedules, and random seeds, and demonstrate improvements in instruction-following performance.
Critical weaknesses
  1. The gains over the strongest tuned baselines are modest, particularly on MT-Bench and Inv-IFEval. Although the experiments use multiple random seeds, paired confidence intervals against the linear-taper baseline are not reported for these metrics, leaving the reliability of these small gains unclear.

  2. The experiments focus on instruction tuning, leaving the method’s effectiveness in other fine-tuning settings unclear. Evaluation on reasoning-focused or agentic tasks involving multi-step planning and tool use would help establish broader applicability.

Questions for authors
  1. Are the gains over the linear-taper baseline on MT-Bench and Inv-IFEval statistically significant? Could the authors report paired confidence intervals for these comparisons?

  2. Do the authors expect the method to benefit reasoning-focused or agentic fine-tuning? Could they provide evidence or discuss potential limitations in these settings?

3 comments

We thank the reviewer for the positive assessment and for the precise questions about statistical support. We answer both questions directly below.

W1 / Q1. Are the gains over the linear taper on MT-Bench and Inverse-IFEval significant?

Short answer: mostly not. With ten seeds, the gain over the linear taper is significant on MT-Bench only for Llama-3.1-8B (+0.077 [0.002, 0.151]) and on Inverse-IFEval only for Mistral-7B-v0.3 (+0.26 [0.10, 0.43]). All eight ten-seed point estimates are positive, but the effects are small (+0.01 to +0.08 MT-Bench points, +0.09 to +0.26 Inverse-IFEval points). We therefore no longer claim a gain over the linear taper on these two benchmarks. Sec. 5.2 now reports them as small and mostly within noise.

We computed paired 95% Student-t intervals for AdaNoiseSFT − linear taper on MT-Bench and Inverse-IFEval from the five-seed runs of Table 1 (the protocol of Tables 43–45). Because five seeds give wide intervals for small effects, we also evaluated both benchmarks on the ten-seed runs behind Table 36. AdaNoiseSFT − linear taper, paired 95% CI:

ModelMT-Bench, 5 seedsMT-Bench, 10 seedsInverse-IFEval, 5 seedsInverse-IFEval, 10 seeds
Llama-3.1-8B+0.060 [−0.110, 0.230]+0.077 [0.002, 0.151]+0.19 [−0.27, 0.64]+0.21 [−0.07, 0.48]
Mistral-7B-v0.3+0.060 [−0.084, 0.204]+0.067 [−0.014, 0.148]+0.22 [−0.02, 0.45]+0.26 [0.10, 0.43]
Qwen3-8B-Base+0.021 [−0.035, 0.078]+0.014 [−0.021, 0.048]+0.18 [−0.22, 0.59]+0.09 [−0.10, 0.28]
Qwen3-14B-Base—+0.032 [−0.014, 0.078]—+0.12 [−0.06, 0.30]

Qwen3-14B-Base is not part of Table 1. Its ten-seed runs are those of Table 36. This table is now Table R2 in App. H.10. Sec. 5.2 points to this table. For scale, MT-Bench has 160 turn-level ratings on a 1–10 scale, so a mean difference of 0.05 corresponds to about 8 rating points in total.

On "the strongest tuned baselines": the linear taper is a control we built ourselves to isolate schedule shape, and it already contains the main idea of decaying the noise. Against published methods, AdaNoiseSFT gains +4.40 [4.01, 4.80], +3.70 [3.26, 4.13] and +3.02 [2.87, 3.18] AE-2 over NEFTune on Llama, Mistral and Qwen3-8B, and 2.9–4.2 AE-2 over the strongest two-pass methods, SymNoise and R-Drop, which use twice the compute (Table 1). Against fixed noise, the MT-Bench gains are +0.41 [0.34, 0.48], +0.42 [0.28, 0.56] and +0.43 [0.37, 0.49], and the Inverse-IFEval gains are +1.15 [0.34, 1.96] and +0.71 [0.40, 1.01] on Llama and Mistral, while on Qwen3-8B that interval includes zero (+0.34 [−0.04, 0.72], Tables 43–45). Finally, the specific claim that the decay should follow the LR rests on AE-2, IFEval and Arena-Hard with ten seeds: 29 of the 33 paired intervals in Table 36 exclude zero, and the four that do not (all Arena-Hard) are listed in App. G.

Revision: new Table R2 in App. H.10 with these intervals, together with the AE-2 gains over NEFTune. Sec. 5.2 and the caption of Table 1 point to it.

W2 / Q2. Reasoning-focused and agentic fine-tuning.

We ran two such settings with the protocol of the paper (Llama-3.1-8B, LoRA, WSD LR, ten seeds): a reasoning corpus of 20k examples from Tulu-3 Persona MATH, Tulu-3 Persona GSM and NuminaMath-TIR, evaluated on GSM8K and MATH-500 with chain of thought, and a tool-use corpus of 20k examples from xLAM function-calling and Glaive-function-calling-v2, evaluated on BFCL v4 overall and on its multi-turn categories.

BenchmarkVanilla SFTFixed noise (NEFTune)Time taperAdaNoiseSFTΔ vs taper [95% CI]
GSM8K62.5262.4163.1564.15+1.00 [0.63, 1.37]
MATH-50024.6425.2025.2625.64+0.38 [−0.06, 0.82]
BFCL overall42.0341.2841.9742.29+0.32 [−0.37, 1.00]
BFCL multi-turn22.7321.0622.0321.63−0.40 [−1.00, 0.21]

Reasoning. LR coupling is above the best time taper on GSM8K. On MATH-500 the difference is within noise.

Tool use. Embedding noise does not help. Fixed noise is below vanilla SFT (41.28 vs 42.03 overall, 21.06 vs 22.73 multi-turn). With a decaying schedule, BFCL overall is back at the vanilla level (41.97 for the taper, 42.29 for AdaNoiseSFT), while multi-turn stays below it (22.03 and 21.63). LR coupling and the time taper do not differ significantly.

What we expect. The analysis (Propositions 1–2) concerns how fixed input noise interacts with a decaying LR and does not depend on the task. It therefore applies whenever embedding noise is used, but it does not say whether embedding noise helps a given task. In these settings two properties may work against it: the answers require exact tokens (numbers, argument names, JSON structure), and long traces change the noise normalization, which uses the padded sequence length, so the peak scale has to be re-selected. We did not test multi-step agentic training with environment feedback.

Limitations. We added the following paragraph to App. G and a short version of it to the Limitations section:

"Reasoning, tool-use, and domain SFT. Our main evidence concerns instruction tuning. In mathematical-reasoning SFT, LR coupling improves GSM8K over the best time taper and is within noise on MATH-500. In tool-use SFT, embedding noise does not improve over vanilla SFT, fixed noise lowers BFCL accuracy, and decaying it recovers part of the loss (Appendix K.1). Two factors may limit embedding noise in these settings: tasks that require exact tokens may be sensitive to input perturbations, and long traces change the noise normalization, so the peak scale must be re-selected. We did not test multi-step agentic training with environment feedback."

We hope the intervals in Part 1 and these results address both concerns, and we would be grateful if the reviewer would reconsider the soundness and overall scores.

Dear Reviewer,

As the discussion period closes today, we wanted to check whether our responses and the revised PDF address your concerns. We would be glad to clarify anything that remains unclear.

Thank you again for your time.

Reviewer_jXTrOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper addresses an important existing problem: improving LLM supervised fine-tuning (SFT) with limited training examples. Building on prior works that already demonstrated the usefulness of adding noise in the embeddings, the paper proposes AdaNoiseSFT, which scales Gaussian embedding-noise variance by closely following the LR schedule. The motivation is that fixed noise can introduce a constant bias in the clean-loss objective even at the end of training, but reducing noise as training progresses removes this bias under mild assumptions. Most of the improvement over fixed noise comes from reducing the noise itself over time, while closely following the noise schedule with the learning rate schedule provides a smaller additional gain. The method is straightforward to implement and has low overhead, which is an additional advantage.

The paper provides extensive empirical results across models including Llama-3.1-8B, Mistral-7B-v0.3, and Qwen3-8B-Base; datasets including Alpaca-cleaned, UltraChat-50k, and Tulu-3-100k; and evaluation benchmarks such as AlpacaEval 2.0, MT-Bench, Arena-Hard, and IFEval. Additional experiments cover Qwen3-14B-Base, Qwen3-32B, and SmolLM2-1.7B. The results show substantial performance gains from decaying noise over fixed-noise baselines, with smaller additional improvements from an LR-dependent noise schedule.

Critical strengths
  1. The method is easy to implement and without any additional overheads.

  2. The gains are substantial, although most come from simply decaying the noise during training, a strategy that, to my knowledge, has not been explored by prior methods for this task.

  3. The theoretical analysis relies largely on standard smoothing and optimization arguments. Proposition 3 shows that, under a particular LR-dependent bound on the noise bias per update, setting the noise variance proportional to the learning rate leads to the largest possible variance at each step. Under the stated assumptions, this provides a reasonable justification for matching the noise schedule to the learning-rate schedule, further supported by modest but statistically significant empirical gains in the main comparisons across multiple random seeds.

  4. The paper provides extensive validation across models, datasets/evaluation benchmarks. Particularly useful are the comparisons against independently tuned time-dependent noise schedules under four learning-rate schedules, repeated-seed experiments to get statistical significance, and ample useful ablation studies. A separate experiment comparing fixed, time-dependent, and LR-coupled LoRA dropout further supports the method in presence of other regularizoint methods.  The comparisons and ablations should also be informative to the community as those are done on new cleaned alpaca datasets which are missing from the existing works as well.

Critical weaknesses
  1. Most of the performance gain comes from decaying the noise as training progresses, while closely matching the noise schedule to the learning-rate schedule provides only modest additional gains. Prior work has explored the broader idea of reducing injected noise as training progresses, such as decreasing noise strength in the gradients [1.], although that work does not specifically address embedding-noise regularization for SFT. This limits the novelty of the decaying noise itself as training progresses, while leaving the application to SFT and the specific LR-coupling strategy as novel contributions.

  2. The paper lacks qualitative comparisons of generated outputs with and without the proposed noise-variance decay schedule, making it difficult to understand how the method affects output quality and which types of errors it helps to reduce.

[1.] ADDING GRADIENT NOISE IMPROVES LEARNING FOR VERY DEEP NETWORKS (https://arxiv.org/pdf/1511.06807)

Questions for authors

Minor comments:

  1. Line 315: "..fixed Gaussian noise with a default and a tuned scale)," ==> Typo

  2. In the main sections, its not clear if you use same max noise in the baseline and your method or separate grid search for baseline methods such as Neftune or Symnoise ? May be move that to main text instead of appendix.

  3. I have not verified every proof in detail, but I assume that the theoretical analysis appears to rely on fairly standard lemmas and established optimization arguments.

3 comments

We thank the reviewer for the thorough and constructive review, and for the accurate summary of how the gain splits between decay and LR coupling.

W1. Gain from decay versus LR coupling, and prior work on decaying noise.

We agree with this decomposition, and the paper states it (Sec. 5.2 notes that most of the gain over fixed noise comes from decaying the noise at all, and Sec. 5.3 isolates the part that comes from following the LR). In Table 1, decay alone (linear taper − tuned fixed Gaussian noise) gives 2.3–2.5 AE-2, and AdaNoiseSFT − tuned fixed Gaussian noise gives 3.1–3.9, so following the LR adds 0.8–1.4 on top of the decay. We see the contribution at three levels: why fixed noise hurts late in training (the persistent bias of Sec. 4, which also explains the gain from decay itself), the decay in SFT, which no current embedding-noise method uses, and the rule that the decay should follow the LR. What the LR-coupling part adds:

  1. Size and consistency of the effect. AdaNoiseSFT is above the best time taper by 0.91–1.50 AE-2 under all four LR schedules (ten seeds, all 95% CIs above zero, Table 2), and above the weakest one by 1.30–2.10 (Table 37). On the second 4×4 matrix (Mistral-7B), the matching schedule is above every mismatched taper by 0.74–1.61 (App. J.1). The step/WSD result repeats on Qwen3-14B-Base and UltraChat-50k (Table 39).

  2. No taper shape to select. The best time taper differs between LR schedules (Table 37), so a taper has to be chosen per LR schedule. LR coupling takes its shape from the LR. It uses the same single forward pass and no extra state.

  3. A direct test of the mechanism. Proposition 3 suggests that a taper loses quality where its variance exceeds the LR. Under step LR, clipping the WSD taper to the LR raises AE-2 from 16.17 to 17.05, about three quarters of the gap to LR coupling (17.31, App. J.3).

On prior work: we re-read the related works in full. Neelakantan et al. (2015) add zero-mean noise to the gradient with variance η/(1 + t)^0.55, where η is a noise scale and t the step count. In their MNIST experiments the learning rate stays fixed while the noise decays, so their decay follows elapsed steps. Zero-mean gradient noise does not change the expected gradient, so the bias mechanism of Sec. 4 does not arise there. They report that decayed noise beats fixed noise, without analysing why or comparing schedules. For completeness, Reviewer upHa pointed us to Wang et al. (2019), who mention Gaussian noise on the input embeddings that decreases over the run, as a training detail for RNN language models with no ablation, and to SGLD, which ties zero-mean parameter noise to the step size for posterior sampling, the opposite goal. We now cite all three in Sec. 2. To our knowledge, no embedding-noise method for LLM SFT decays the noise, so the reviewer's assessment of the application holds.

Revision: Sec. 2 now cites Wang et al. (2019), Sec. 4 states why zero-mean noise differs from input noise, and Sec. 5.3 now reports how far the weakest mismatched taper trails LR coupling (1.30–2.10 AE-2).

W2 (a). Outputs with and without the decay schedule.

App. L.1 now shows the verbatim outputs of fixed noise (σ = 10), the linear taper and AdaNoiseSFT on five AlpacaEval prompts sampled at random (seed 2027). In two of the five only AdaNoiseSFT follows the instruction, in two all three answers are correct, and in one fixed noise gives the most complete answer (Table R8). One example, AlpacaEval #438 (prompt abridged): the user asks the assistant to reply only in the formats [ADDITIONAL_INFORMATION_NEEDED], [API_REQUEST] and [ANSWER], and to list any further formats it may need.

Fixed noise

Happy to help. One extra format would be useful:
[ANSWER]: [STATUS_UPDATE]: {short progress note while a longer task runs}

Linear taper

```text
[ANSWER]: [CONFIRMATION]: {what I am about to do, so you can approve it}
```

AdaNoiseSFT

[ANSWER]: [PROGRESS]: {percentage done and the step currently running}

Fixed noise adds a preamble and the taper wraps the reply in a code block, while AdaNoiseSFT keeps the required format. This is the error type that falls most in the error analysis (W2 (b)).

W2 (b). Which errors the method reduces.

We annotated 256 AlpacaEval responses per method blind to the method: two human annotators and an LLM judge applied the same rubric. The final label is the human label when the two annotators agree and the LLM judge's label otherwise (Cohen's κ between the two human annotators = 0.829). Share of responses with each error type, and Δ = AdaNoiseSFT − fixed noise in percentage points with a 95% CI (negative = fewer errors):

Error typeFixed noiseLinear taperAdaNoiseSFTΔ vs fixed [95% CI]
Format or constraint violation26.5622.6614.45−12.11 [−16.80, −7.42]
Part of the instruction ignored28.9124.6119.53−9.38 [−14.06, −5.08]
Incomplete answer17.1915.2312.89−4.30 [−7.81, −0.78]
Excessive verbosity19.5317.9717.58−1.95 [−5.08, 0.79]
Repetition or degeneration9.389.386.64−2.73 [−5.08, −0.78]
Factual error19.1418.3620.31+1.17 [−2.34, 4.69]
No error26.9531.6439.45+12.50 [7.42, 17.20]

AdaNoiseSFT mainly reduces instruction-following errors. Relative to fixed noise, format or constraint violations fall by 12.1 points (95% CI 7.4–16.8), ignored parts of the instruction by 9.4 (5.1–14.1), incomplete answers by 4.3 (0.8–7.8) and repetition by 2.7 (0.8–5.1), and the share of error-free answers rises by 12.5 points (7.4–17.2). Verbosity and factual errors do not change significantly, consistent with the unchanged knowledge benchmarks in Table 1 (MMLU, TruthfulQA). The linear taper lies between fixed noise and AdaNoiseSFT on most categories. On IFEval, instruction-level accuracy on length constraints rises by +7.30 [4.43, 10.17] points over fixed noise (App. L), in line with the drop in constraint violations. The gain does not come from longer answers: mean AlpacaEval response length is 160 tokens for AdaNoiseSFT against 168 for fixed noise and 162 for the linear taper (Table 12), and AE-2 is length-controlled.

Revision: new App. L (error analysis and the IFEval length-constraint result). Sec. 5.2 summarizes the main finding in one sentence.

Minor: line 315.

Thank you. We rewrote it as two sentences without nested parentheses: "Four baselines keep the embedding noise at a fixed level: NEFTune (Jain et al., 2024), SymNoise (Yadav and Singh, 2023), and fixed Gaussian noise at a default and a tuned scale. The other five are the linear taper, R-Drop (Liang et al., 2021), balancedness-aware regularization for LoRA, BAR (Li et al., 2024a), and two recent methods, NoiseFiT (Khadangi et al., 2025) and GIFT-SW (Zhelnin et al., 2025), the last two adapted to one forward pass (Appendix D)."

Q. Do the baselines use the same peak noise as the method?

No. Each method selects its own peak by clean loss on a held-out 5% selection split, before any benchmark evaluation. The selected peaks are α = 10 for NEFTune and SymNoise, σ = 5 for tuned fixed Gaussian noise, and σmax⁡\sigma_{\max} = 10 for the linear taper and for AdaNoiseSFT (Table 5). Fixed noise selects a smaller peak, which is consistent with Sec. 4: noise that persists to the end of training keeps its bias. As the reviewer suggests, this is now in the main text. The new paragraph in Sec. 5.1 reads:

"Hyperparameter selection. Every method selects its hyperparameters by clean loss on the selection split before any benchmark evaluation (Table 4), and Table 5 lists the selected values. Each time taper selects its own peak (Table 35). The row 'Fixed Gaussian noise (σ = 10)' in Table 1 uses the AdaNoiseSFT peak, so that the two schedules differ only in shape. On all four backbones, fixed noise selects a smaller peak than the decaying schedules (Table 50)."

Proofs.

Thank you. As the reviewer notes, the lemmas use standard smoothing and biased-gradient arguments. To make checking easier, App. B now opens with a roadmap that marks what is standard and what is new:

  • Lemma 1 (second-order expansion of the bias): standard Gaussian smoothing.
  • Proposition 1 (persistence of the bias under fixed noise): new, uses tail nondegeneracy.
  • Theorem 1 and Corollary 1 (stationarity bounds): follow existing analyses (Défossez et al., 2022, and Ajalloeian and Stich, 2020).
  • Proposition 2 (distance to the optimum under fixed noise): new.
  • Proposition 3 (the LR shape is the largest schedule within the step bound): new and elementary.

We thank the reviewer again. We hope the error analysis and the example outputs, together with the clarified contribution, address both weaknesses, and we would be grateful if the reviewer would consider raising the score.

Dear Reviewer,

As the discussion period closes today, we wanted to check whether our responses and the revised PDF address your concerns. We would be glad to clarify anything that remains unclear.

Thank you again for your time.

Comments on the paper

1 comment

We thank all reviewers for their time and constructive feedback. We summarize the main concerns, our clarifications and the new results. Details are in the individual responses, and all changes in the revised PDF are marked in blue.

Contribution. Two reviewers note that decaying injected noise is known (upHa, jXTr). We agree, and the introduction now lists input-embedding noise among the earlier decaying schedules. We re-read the related work in full: SGLD (Welling and Teh, 2011) ties zero-mean parameter noise to the step size in order to sample the posterior. Neelakantan et al. (2015) decay zero-mean gradient noise with the step count. Wang et al. (2019) mention input-embedding noise that decreases over the run as a training detail. None of them analyses the gradient bias of input-space noise or compares a time-based with an LR-based schedule. Our contribution has three levels:

  1. Why fixed embedding noise hurts late in training. It leaves a gradient bias that does not vanish as the LR decays (Sec. 4). Zero-mean noise added to the gradient or to the parameter update does not create such a bias.

  2. Decay in LLM SFT. None of the current embedding-noise methods for SFT decays the noise. In Table 1, decaying it with elapsed time (the linear taper) already gains 2.3–2.5 AE-2 over tuned fixed Gaussian noise on three model families, while AdaNoiseSFT gains 3.1–3.9. Following the LR therefore adds 0.8–1.4 AE-2 on top of the decay itself.

  3. The decay should follow the optimizer's LR, not elapsed time. We identify this under four LR schedules against independently tuned time tapers: the matching schedule wins by 0.91–1.50 AE-2 over the best taper (ten seeds, all CIs above zero), replicated on a second complete 4×4 matrix and in further settings.

New results (details in the individual responses and the revised appendix):

  • Paired 95% CIs against the linear taper on MT-Bench and Inverse-IFEval with ten seeds, and against NEFTune on AE-2 (NBhM). With ten seeds, the gain over the linear taper is significant on MT-Bench only for Llama-3.1-8B and on Inverse-IFEval only for Mistral-7B-v0.3. All eight point estimates are positive but small, and Sec. 5.2 no longer claims a gain on these two benchmarks.

  • A blind error analysis of 256 responses per method, IFEval accuracy on length constraints, and side-by-side outputs on five random AlpacaEval prompts (jXTr). AdaNoiseSFT mainly reduces instruction-following errors: relative to fixed noise, format or constraint violations fall by 12.1 points (95% CI 7.4–16.8), and IFEval accuracy on length constraints rises by +7.30 [4.43, 10.17] points.

  • SFT on mathematics, code, medicine and tool use under WSD LR, ten seeds (upHa, NBhM). LR coupling beats the best time taper on GSM8K, HumanEval+, MBPP+ and MedQA (+0.60 to +1.00 points), is within noise on MATH-500, MedMCQA, PubMedQA and BFCL, and is significantly worse on none. On PubMedQA and BFCL, embedding noise itself brings no clear gain over vanilla SFT, which we add to the Limitations.

  • Full fine-tuning of Llama-3.1-8B and Qwen3-14B-Base, ten seeds each (upHa): AdaNoiseSFT − linear taper is +1.14 [0.70, 1.58] and +0.85 [0.41, 1.29] AE-2.

  • LR coupling for noise in hidden states, soft prompts, gradients and parameter updates, ten seeds (upHa): it beats the best time taper for hidden states (+0.85 [0.06, 1.65] AE-2) and has no significant effect for the other three.

  • A direct check of the tail-nondegeneracy assumption: on the fixed-noise trajectory, the bias per unit variance stays between 0.88 and 1.06 of its early value until the end of training.

Revisions to the paper: we cite Wang et al. (2019) in Sec. 1 and 2, state in Sec. 4 that zero-mean noise on the gradient or the parameter update does not shift the expected gradient, add a hyperparameter-selection paragraph to Sec. 5.1, make the claims in Sec. 5.2 more careful with the new intervals, report in Sec. 5.3 how far the weakest mismatched taper trails LR coupling, add a proof roadmap and a notation table to App. B, and add App. K (domains, full fine-tuning, other noise locations), App. L (error analysis and example outputs) and a Limitations paragraph on reasoning, tool-use and domain SFT.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.