acceptodds
Under review as a conference paper at ICLR 2027

RAG-QAF: Robust Adaptation and Generalization for Quantization-Aware Fine-Tuning

Abstract

Quantization-aware fine-tuning (QAF) is crucial for adapting pretrained large language models (LLMs) to specific downstream tasks on resource-constrained devices. However, existing QAF methods tend to prioritize in-distribution (ID) performance on target adaptation tasks, while paying limited attention to out-of-distribution (OOD) generalization. Although random parameter perturbations can improve generalization in full-precision fine-tuning, directly introducing such noise into QAF may degrade both ID and OOD performance. In this work, our multi-scale analysis reveals that random noise lacks explicit optimization guidance and overlooks local differences in quantization error. We further observe that the model’s global noise sensitivity evolves throughout training, making a fixed perturbation magnitude inadequate. Based on these findings, we propose RAG-QAF, a robust QAF framework for ID adaptation and OOD generalization. Specifically, **local error-guided adversarial optimization** uses quantization errors to guide the search for adversarial perturbations through gradient ascent. **Global adaptive curriculum learning strategy** progressively increases the overall perturbation budget to accommodate evolving noise sensitivity, enabling a transition from stable early optimization to increasingly challenging robust optimization. RAG-QAF can be integrated into existing QAF pipelines as a plug-and-play module. Extensive experiments on ID and OOD benchmarks demonstrate state-of-the-art performance with low additional training overhead.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
model compression, large language model, model quantization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_hthuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose RAG-QAF, a new technique to improve robustness of quantization-aware fine-tuning for both in-distribution and out-of-distribution scenarios. They examine the robustness challenges faced by quantized model and design a method of 1) local adversarial optimization guided by quantization-error and 2) global curriculum schedule for adaptation. The result shows promising improvement on overheads comparing other baseline approaches as well as performance on the benchmarks.

Critical strengths
  • The paper is well-written with clear figure presentations to explain the current limitations and the proposed methodology (figure 1 and 2).
  • It is interesting to see a similar idea as adversarial training but on the normalized channel space guided by quantization error, and then integrated with curriculum learning for smoother control. The authors carefully compare the proposed method with adversarial optimization alone and quantization error guidance in figure 3.
  • It is good the see that the method improves the overhead over other approaches as the computation can be done in the same forward/backward pass.
Critical weaknesses
  • A core design choice of the proposed method is the local adversarial optimization, the authors state that the adversarial noise is added to the frozen RMSNorm scaling parameter without detailed explanation and empirical testing on how and why this is determined.
  • None of the reported results show standard deviation from independent runs. Some of the improvements from RAG-QAF over other baselines (table 1) and ablation results (figure 3) are not consistent and many of the numbers are very close. It would make the claims more grounded with reproducibility and std.
Questions for authors

Thanks for submitting this work to ICLR 2027! The questions below are mainly based on the comments in the previous section. I think the proposed method is promising with good presentation and a combination of two main well-motivated techniques, I am open to further discussion if the questions below can be addressed.

  • What if applying the proposed local adversarial optimization to different locations other than the RMSNorm scaling parameters (e.g., different layers or other parameters) during fine tuning? How much does the location matter comparing to techniques from related works focusing on other parameters? Does RAG-QAF still outperform others in those cases? It would be interesting to see an ablation on the proposed technique and the applied location.
  • Given that some improvements are small comparing to the baselines or ablation components, what are the standard deviations from multiple independent runs? Are the resulting numbers overlapping with each other a lot?
Reviewer_WBmROpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

Paper studies quantization-aware fine-tuning of LLMs. Authors propose RAG-QAF that adds a learnable channel-wise vector to the frozen RMSNorm gain before the linear layers of a QLoRA model. This vector is updated by gradient ascent. The ascent direction is reweighted per channel by the relative quantization error of the corresponding weight column. The budget grows from zero to τmax\tau_{max} during training by a fixed warm-up and linear ramp schedule. Qwen3-4B, Qwen3-8B and GLM-4-9B are fine-tuned on a mathematical reasoning datasets and evaluated on AIME24, AIME25 and OlympiadBench as ID benchmarks and on RBench, SuperGPQA, IFEval and CLUEWSC as OOD benchmarks. The paper also reports results with MXFP4 and MXINT4 formats, with LoftQ and EoRA pipelines, and on code and biomedical domains

Critical strengths
  1. Paper provides broad experimental coverage, with three benchmarks for the ID split, four benchmarks for the OOD split and three models. It also includes an ablation study as well as efficiency and generalization studies.
  2. RAG-QAF is cheap. It is no more expensive than the competing methods in memory and outperforms them in training time, adding only 2.1% over SFT.
  3. RAG-QAF remains effective with other number formats and is compatible with other SFT frameworks besides QLoRA.
  4. Beyond the standard benchmarks, the method is validated on the code and biomedical domains.
Critical weaknesses
  1. It seems to me that the paper validates its results incorrectly. AIME24 is evaluated with greedy decoding, thinking mode enabled and output truncation at 8,192 tokens. In BF16, Qwen3-4B and Qwen3-8B score 16.67 and 23.33, respectively. At the same time, "Qwen3 Technical Report" (Qwen team) reports 73.8 and 76.0 for the same models on the same benchmark, with output truncation at 38K tokens. In addition, the Qwen3 model card (https://huggingface.co/Qwen/Qwen3-8B) states that greedy decoding in thinking mode leads to endless repetitions. It is therefore possible that the gain after fine-tuning comes from the model learning to reason shorter rather than better. The authors should report the fraction of truncated responses or re-evaluate the models with the budget recommended in the technical report

  2. I also have a serious concern about the RBench results. Although this is an OOD task on which no training was performed, accuracy on it increases substantially after fine-tuning for both Qwen3 models. The same truncation of the reasoning chain is applied to these tasks, so the skill of reasoning shorter, acquired during fine-tuning, may produce this effect. I see no other explanation for such a large gain on an OOD task. If this is indeed the case, the reported average OOD metrics are not valid. This is indirectly supported by the fact that the RBench score drops for GLM-4-9B, which has no thinking mode.

  3. If RBench is excluded, RAG-QAF is the best method in only 3 of the 9 remaining OOD cells, and the improvement in these cells is at the noise level. This concern calls into question the significance of the proposed methodology.

  4. The Gaussian noise baseline is in fact strong. On Qwen3-8B it is the second-best method overall (50.14 All) and exceeds SFT by 5.17 ID points (Table 1), which contradicts the authors' claim that random noise degrades both ID and OOD performance. The paper also does not state where the Gaussian noise is injected (weights, RMSNorm gain or activations), so it is unclear how this baseline relates to the proposed perturbation of the RMSNorm gain

Questions for authors
  1. What fraction of responses reaches the 8,192-token limit, and what is the average output length, for the base models, SFT and RAG-QAF on AIME, OlympiadBench, R-Bench and SuperGPQA? Do the gains persist under the evaluation protocol of the Qwen3 technical report with a budget of 38K tokens?

  2. Can the authors report results over several training seeds with confidence intervals for SFT, the strongest baseline and RAG-QAF?

  3. Can authors explain the increase in scores on RBench for Qwen3 family?

Reviewer_Dy7qOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes RAG-QAF for quantization-aware fine-tuning (QAF), aiming to preserve both in-distribution (ID) adaptation performance and out-of-distribution (OOD) generalization in low-bit LLMs. Based on observations of Qwen3-4B, the paper reports that random noise lacks optimization guidance, channel-wise quantization errors vary substantially in magnitude, and loss sensitivity to fixed-magnitude Gaussian noise changes during training. These observations motivate local adversarial optimization, local quantization-error guidance, and a global adaptive curriculum learning strategy. Across the evaluated models and benchmarks, RAG-QAF achieves the highest aggregate scores among the compared methods, with additional experiments assessing its applicability across quantization formats and fine-tuning frameworks. However, the aggregate gains do not translate into improvements on every individual benchmark, and RAG-QAF does not consistently outperform the strongest baselines in OOD performance.

Critical strengths
  1. The paper addresses a well-motivated problem and offers strong practical utility. Its primary goal is to improve the reliability of adapted quantized models under distribution shift. The method is a plug-and-play module that operates only during training, introduces no inference overhead, and can be integrated into different pipelines, such as QLoRA, LoftQ, and EoRA. With only a 2.15% increase in training time per step and comparable peak memory usage in the reported Qwen3-4B efficiency experiment, these properties make it a practically valuable option for quantized-model deployment.
  2. The paper provides solid empirical evidence through extensive experiments spanning multiple model families, quantization methods, datasets, and two additional fine-tuning frameworks. The breadth of evaluation makes it unlikely that the reported gains are artifacts of a single experimental setting.
Critical weaknesses
  1. The paper lacks comparisons with closely related methods, such as Bi-QLoRA (ICLR 2026), NPFT (arXiv:2412.06858), and GIFT-SW (arXiv:2408.15300), which use adversarial perturbations or noise injection to improve generalization or quantization robustness. However, the paper neither compares against nor analyzes these methods. Notably, the paper itself cites RWP (Li et al., 2024a), whose ARWP variant already combines statistics-guided perturbation scaling with an increasing-σ schedule, yet the paper neither distinguishes its reweighting scheme from ARWP nor compares against it.
  2. The paper’s motivation rests on the premise that random noise harms both ID and OOD performance in QAF, based on observations of Qwen3-4B in Figure 1(d). However, this observation does not extend to Qwen3-8B in Table 1: Gaussian Noise outperforms SFT on both ID and OOD and is the strongest baseline overall (All: 50.14 > VCORE: 49.87 > SFT: 46.60). Furthermore, the main results are reported without variance estimates or confidence intervals.
Questions for authors
  1. Direct comparisons with related perturbation-based methods. Bi-LoRA (arXiv:2508.19564) and GIFT-SW (arXiv:2408.15300) provide directly relevant baselines in LLM and quantization settings. Their implementations are publicly available and could be included as baselines. If these methods achieve comparable or better performance, what independent benefit does the proposed method offer over existing paradigms?
  2. Multiple seeds and uncertainty estimates. Please report the mean ± standard deviation over at least three random seeds, or bootstrap confidence intervals.
  3. Consistency of the motivation across models. Please explain why Gaussian Noise outperforms SFT on both ID and OOD for Qwen3-8B in Table 1, contrary to the paper’s motivation based on Qwen3-4B. In settings where random noise itself is a strong baseline, do RAG-QAF’s gains come from the curriculum, quantization-error guidance, or the perturbation itself? While Figure 3 provides cumulative ablations of AO, QEG, and CL, could you additionally evaluate Gaussian Noise with the same curriculum schedule, and compare it with adversarial perturbations with and without quantization-error guidance under matched perturbation budgets?
  4. Inconsistent step-size definitions. Equation (10) uses the fixed value κτmax⁡\kappa\tau_{\max}, whereas Table 10 specifies αadv(s)=κτ(s)\alpha_{\mathrm{adv}}(s)=\kappa\tau(s). These differ during the ramp-up phase. Why are they different?
  5. Asymmetry between ID and OOD gains. In Table 1, relative to the respective strongest ID and OOD baselines within each model’s NF4 block, RAG-QAF achieves ID improvements of 9.57%, 8.12%, and 27.88% on Qwen3-4B, Qwen3-8B, and GLM-4-9B, respectively, whereas the corresponding OOD changes are only +1.24%, +1.49%, and −2.06%. How do the authors explain this consistent asymmetry across the three models? Does it suggest that the method primarily improves ID adaptation rather than OOD robustness?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.