- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes AdaNoiseSFT, which injects Gaussian noise into input embeddings during instruction fine-tuning of large language models. The noise variance follows the learning-rate schedule and gradually decreases toward the end of training. Experiments across multiple models and instruction datasets show improvements in instruction following and conversational quality over fixed noise and several noise-decay baselines.
Critical strengths
The ablation studies are thorough, including detailed ablation studies on noise schedules and the effects of noise near the end of training.
Critical weaknesses
- Adding noise during training and gradually decaying it is already a well-established practice [1–2]. More specifically, injecting noise into embeddings and annealing it was explored as early as 2019 [3]. Therefore, I view the main contribution of this work as proposing that the noise added to the input embeddings should decay with the learning rate, which is a relatively minor contribution. [1] Bayesian Learning via Stochastic Gradient Langevin Dynamics (2011) [2] Adding Gradient Noise Improves Learning for Very Deep Networks (2015) [3] Improving Neural Language Modeling via Adversarial Training (2019)
- The training data cover too few domains, focusing only on instruction tuning. The method should also be evaluated on other training data, such as mathematics, code, or more specialized domain tasks. Otherwise, its practical value remains unclear.
- The models are relatively small, mainly in the 7B–14B range, and training primarily uses LoRA. Full fine-tuning is evaluated only on a 1.7B model. At a minimum, the experiments should include full fine-tuning of 7B–14B models, and preferably larger models as well.
Questions for authors
Have the authors considered testing noise decay tied to the learning rate during LLM fine-tuning when noise is added to hidden states [1], soft prompts [2], gradients [3], or parameter updates [4]? These experiments could help assess the broader applicability of the proposed principle and strengthen the paper’s contribution.
- [1] Noise Augmented Fine Tuning for Mitigating Hallucinations in Large Language Models (2025)
- [2] From Points to Clouds: Learning Robust Semantic Distributions for Multi-modal Prompts (2025)
- [3] Adding Gradient Noise Improves Learning for Very Deep Networks (2015)
- [4] Bayesian Learning via Stochastic Gradient Langevin Dynamics (2011)
5 comments
W3. Scale and full fine-tuning
We ran full fine-tuning (all parameters, including the input embeddings) of Llama-3.1-8B and Qwen3-14B-Base, ten seeds each, under cosine LR with the LR grid re-selected:
| Model | Metric | Fixed | Linear taper | AdaNoiseSFT | Δ vs taper (95% CI) |
|---|---|---|---|---|---|
| Llama-3.1-8B | AE-2 | 17.02 | 18.51 | 19.65 | +1.14 [0.70, 1.58] |
| Llama-3.1-8B | IFEval | 50.74 | 51.79 | 52.59 | +0.79 [0.18, 1.41] |
| Llama-3.1-8B | Arena-Hard | 18.57 | 19.53 | 20.15 | +0.62 [0.39, 0.85] |
| Qwen3-14B-Base | AE-2 | 22.66 | 23.67 | 24.52 | +0.85 [0.41, 1.29] |
| Qwen3-14B-Base | IFEval | 57.78 | 58.48 | 59.26 | +0.78 [0.15, 1.40] |
| Qwen3-14B-Base | Arena-Hard | 25.44 | 26.54 | 26.83 | +0.29 [−0.10, 0.67] |
LR coupling beats the linear taper in five of six comparisons. The exception (Arena-Hard, Qwen3-14B-Base) is within noise. Our theory covers LoRA with frozen embeddings, so this is empirical evidence beyond its scope. For a larger model, Table 14 reports Qwen3-32B (LoRA): 25.0 vs 22.7 AE-2 over fixed noise. We now mention it in Sec. 5.5. Revision: App. K.2 and Sec. 5.5.
Q. Hidden states, soft prompts, gradients and parameter updates
Llama-3.1-8B + Alpaca-cleaned, WSD LR, ten seeds. Each arm selects its own peak. AE-2:
| Noise location | Fixed | Best taper | LR-coupled | Δ vs taper (95% CI) | Δ vs fixed (95% CI) |
|---|---|---|---|---|---|
| Hidden states | 14.29 | 15.63 | 16.49 | +0.85 [0.06, 1.65] | +2.19 [1.56, 2.83] |
| Soft prompts | 13.56 | 13.82 | 13.72 | −0.10 [−0.62, 0.42] | +0.16 [−0.39, 0.72] |
| Gradients | 11.81 | 11.90 | 12.30 | +0.41 [−0.07, 0.88] | +0.49 [−0.07, 1.05] |
| Parameter updates | 12.42 | 12.74 | 12.49 | −0.24 [−0.83, 0.35] | +0.07 [−0.43, 0.57] |
| Input embeddings (paper, Table 2) | — | 15.73 | 16.64 | +0.91 [0.62, 1.20] | — |
Hidden-state noise behaves like embedding noise: LR coupling beats the best taper by +0.85 and fixed noise by +2.19 AE-2. For gradients and parameter updates no schedule differs significantly, as expected for zero-mean noise, which does not bias the gradient. For soft prompts the three arms lie within 0.26 AE-2. These vectors are themselves trained, which Lemma 1 does not cover. Together with LoRA dropout (App. J.4: +0.81–0.92 AE-2 over fixed dropout), the rule thus transfers to perturbations of activations, not to zero-mean noise on gradients or updates. Revision: App. K.3. The conclusion states this scope.
Soundness and presentation
If there is a specific step the reviewer found unconvincing, we would be glad to address it directly. To make the evidence easy to check: (i) all peaks and comparators are selected on a held-out split before test evaluation (App. D), (ii) the central comparisons use ten matched seeds with paired 95% CIs, and settings whose intervals cross zero are reported (App. G), (iii) we now check the key assumption directly: on the fixed-noise runs (seeds 11–55), the bias per unit variance stays between 0.88 and 1.06 of its value at until the end of training (SE ≤ 0.018, App. I.2). For presentation, we added a notation table and a proof roadmap to App. B and a hyperparameter-selection paragraph to Sec. 5.1.
We hope these experiments and clarifications address the reviewer's concerns, and we would be grateful if the reviewer would reconsider the score.
Dear Reviewer,
As the discussion period closes today, we wanted to check whether our responses and the revised PDF address your concerns. We would be glad to clarify anything that remains unclear.
Thank you again for your time.
We thank the reviewer for the careful reading and the concrete list of experiments. We ran all of them except fine-tuning a model larger than 14B, for which we point to the existing Qwen3-32B result.
W1. Novelty
We agree that decaying injected noise is not new, and the paper does not claim it: the introduction presents decay as a known fix, calls this family time tapers, and uses independently tuned time tapers as the main baselines. We thank the reviewer for pointing us to Wang et al. (2019), which we had missed: as a training detail, they add Gaussian noise to the input embeddings whose level decreases from 0.2 to 0 over the run. We now cite it as an early time taper on input embeddings.
None of the three works analyses the bias of input-space noise or compares a time-based with an LR-based schedule. SGLD (Welling and Teh, 2011) adds zero-mean noise of variance ϵt to the parameter update in order to sample the posterior, the opposite goal. Neelakantan et al. (2015) decay the variance of zero-mean gradient noise as c/(1+t)0.55 with the step count t, and in their MNIST experiments the LR stays fixed. Wang et al. (2019) give only the endpoints of their decay and no ablation. Zero-mean noise added to the gradient or to the parameter update does not bias the gradient, whereas noise at the input of a nonlinear model does.
Our contribution is (i) the observation that noise at the input of a nonlinear model shifts the expected gradient by stHt (Lemma 1), a bias that persists under fixed noise as the LR decays (Propositions 1–2), (ii) the rule that the variance should follow the optimizer's LR rather than elapsed time, which keeps this bias at O(ηt2) per update (Proposition 3), and (iii) the identification of this rule. Under cosine LR the two schedules coincide, so we tested linear, step and WSD LR, with every taper's peak and the comparator selected on a separate split before test evaluation. Under all four LR schedules the schedule that matches the LR wins, by 0.91–1.50 AE-2 over the best time taper (ten seeds, all 95% CIs above zero, Table 2, Fig. 3). The weakest mismatched taper trails LR coupling by 1.30–2.10 AE-2 (Table 37). On a second complete 4×4 matrix (Mistral-7B), the matching schedule beats every mismatched taper by 0.74–1.61 AE-2 (App. J.1), and the step/WSD result replicates on Qwen3-14B-Base and UltraChat-50k (Table 39). In practice, no noise-injection method used for SFT (NEFTune, SymNoise, NoiseFiT, GIFT-SW) decays the noise. The change is one line, adds no model pass, and gains 3.1–3.9 AE-2 over tuned fixed noise (Table 1).
Revision: Wang et al. (2019) in Sec. 1 and 2, and a sentence in Sec. 4 on why zero-mean noise differs from input noise.
W2. Other domains
We applied the paper's protocol to three more domains with Llama-3.1-8B (LoRA, WSD LR, ten seeds): mathematics (20k examples from Tulu-3 Persona MATH/GSM and NuminaMath-TIR, evaluated on GSM8K and MATH-500 with chain of thought), code (20k from Tulu-3 Persona Python and Evol CodeAlpaca, evaluated on HumanEval+ and MBPP+) and medicine (20k from the MedMCQA, MedQA-USMLE and PubMedQA training splits, with MedMCQA evaluated on its official validation split and no train/test overlap). Each corpus re-selects the noise peak, and the comparator taper is fixed by the lowest selection loss before test evaluation. Δ = AdaNoiseSFT − time taper, paired 95% CI:
LR coupling beats the best taper on GSM8K, HumanEval+, MBPP+ and MedQA, is within noise on MATH-500, MedMCQA and PubMedQA, and is significantly worse on none. The value of embedding noise itself depends on the task: on PubMedQA no noise schedule beats vanilla SFT, which we add to the Limitations. On Alpaca-cleaned, AdaNoiseSFT leaves GSM8K, BBH, MMLU and TruthfulQA unchanged relative to fixed noise on all three models (all paired CIs include zero, Tables 43–45).
Revision: new App. K.1, a summary in Sec. 5.5, and the Limitations.