- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The manuscript shows a generalization bound for machine learning model fine-tuning. In this manuscript's setting, a base model and a problem distribution are fixed in advance. The model is then fine-tuned using samples from the distribution. Such fine-tuning can be of any form, including SFT, OPD, or RLVR. After fine-tuning, an estimate of the model's average reward in the problem distribution can be obtained by evaluating on the training set. The main result of the paper bounds the difference between the actual average reward in the problem distribution and this estimate, using the maximum difference in reward, the size of the training set, and importantly, the amount of information required to reconstruct the fine-tuned model from the base model. This generalization bound evaluates to a non-vacuous one in the authors' experiment setting. The authors fine-tune Qwen3.5-4B in verifiable tasks including math, coding, text-to-sql, and knowledge QA. They use TinyLoRA and quantization to ensure the parameter difference from the base model is small in information, and a large training set. As a result, the computed bounds for generalization are not vacuous and can meaningfully distinguish between the base model and the fine-tuned models. According to the authors, this is the first generalization bound that was able to do so when the model has billions of parameters. Moreover, under the authors' peft and quantization fine-tuning scheme, to be able to match the performance of normal LoRA fine-tuning, the authors proposed progressive RLVR, which is a training recipe that retains most of the performance of normal LoRA fine-tuning for Qwen3.5-4B in the settings that the authors consider.
Critical strengths
- As the authors claimed, this is the first generalization bound that evaluates to empirically meaningful values in settings that are relavant to modern billion-parameter language model post-training. I found it very impressive that this is possible.
- The experiments convincingly shows that the bounds really evaluate to meaningful numbers that carries information, across multiple domains.
Critical weaknesses
- This result, although already meaningful in some empirical settings, is not likely to guide how people really evaluate generalization when doing post-training in practice. For the bounds to evaluate to meaningful numbers, one has to keep the number of tunable parameters very small with low precision, which sacrifices performance notably.
Questions for authors
- Q1: Feel free to respond to the weakness raised.
- Q2: In the theorem, it feels the term can be bounded with a constant , which seems to convey the message better and simplify the setting. Is there a reason why you choose not to do so?
2 comments
I thank the authors for the detailed response. I agree that PAC-Bayes generalization bounds that the authors develop has these values. I think my point that "it s not likely to guide how people really evaluate generalization when doing post-training in practice" also still holds. Overall, I think my recommendation of this paper remains weak acceptance.
We thank the reviewer for their comments. We addressed them below and revised the paper with changes highlighted in blue.
Held-out sets or benchmarks are widely used in practice to evaluate generalization and do not sacrifice performance. However, PAC-Bayes generalization bounds provide additional properties and enable additional applications, compared to held-out testing.
Scope of the guarantee. A held-out bound only covers the hypothesis fixed before the held-out set is examined, while a PAC-Bayes bound holds simultaneously for all hypotheses.
No reserved data. A PAC-Bayes bound uses only the training data and requires nothing held out. Therefore, PAC-Bayes bounds are particularly valuable when data are scarce or expensive. For example, constructing SWE-Bench Pro V2 required 404 hours from 23 professional engineers [3], while solving each task in FrontierMath took expert mathematicians multiple hours to multiple days [4].
Bound optimization. A held-out bound becomes invalid when we adaptively use it to improve training without addressing the multiple comparisons problem [1, 2]. In contrast, we can improve training to minimize a PAC-Bayes bound computed on training data, while the bound stays valid.
Explanation of generalization: A PAC-Bayes generalization bound explains why a model generalizes and enables the developer to compare training algorithms qualitatively. In contrast, a held-out set is a black-box test that does not explain generalization.
In the revised paper, we discussed benefits of PAC-Bayes generalization bound in Section 6.
We keep the factor (1+1/n) because it is close to 1 for the n used in our experiments (1.125 for n=64). Compared to a constant of 2, this reduces the PAC-Bayes penalty by 44%. In the revision, we added a paragraph in Section 3.1 to note the simplified form for readers who prefer it.
[1] Blum, Avrim and Hardt, Moritz. “The ladder: A reliable leaderboard for machine learning competitions” ICML (2015).
[2] Cawley, Gavin and Talbot, Nicola. “On over-fitting in model selection and subsequent selection bias in performance evaluation” JMLR (2010).
[3] Dan, Soham, et al. "SWE-Bench Pro V2: A Cleaner, Harder-to-Game Coding Benchmark." arXiv:2610.24206 (2026).
[4] Glazer, Elliot, et al. "FrontierMath: A benchmark for evaluating advanced mathematical reasoning in ai." arXiv:2411.04872 (2024).