- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2(+1)
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes RBC-LoRA, a framework for LoRA fine-tuning that aims to improve the trade-off between target-task specialization and retention of pretrained capabilities. Instead of directly solving the final KL-constrained optimization problem, RBC-LoRA gradually strengthens task adaptation while relaxing the retention constraint. Experiments on Qwen2.5-3B and Llama-3.2-3B using GSM8K and MBPP show that RBC-LoRA achieves higher target-task performance than direct constrained training while better preserving the model’s original capabilities.
Critical strengths
-
The paper studies an interesting and meaningful problem. Rather than introducing another retention regularizer, it asks whether the optimization path to the same final constrained problem can improve the specialization–retention trade-off. I find this perspective novel and practically relevant.
-
The experimental protocol is relatively careful. The paper separates training, controller, and final-audit reference sets and provides targeted ablations for the retention-budget schedule, anchor, task ramp, and stage progression. The results also show a consistent improvement over direct constrained training across the tested tasks and backbones.
Critical weaknesses
-
The experimental scope is limited. The experiments are restricted to two 3B-scale models and primarily focus on math and code specialization. It is therefore unclear whether the proposed method generalizes to larger models or to a broader range of adaptation objectives. In addition, retention is evaluated only on MMLU, HellaSwag, and IFEval, which provides limited evidence that the original capabilities are broadly preserved.
-
It is still unclear how important the specific continuation path is. Although the paper compares adaptive progression with fixed increments, both follow a similar increasing trajectory toward the same final constrained problem. It would be useful to know whether the proposed path itself is important, or whether other trajectories that reach the same final problem would perform similarly.
Questions for authors
-
Does RBC-LoRA show similar gains on larger models, more diverse target tasks, and a broader set of retention benchmarks?
-
Can the paper provide more direct evidence that following the proposed continuation path is beneficial? For example, comparing different trajectories that reach the same final constrained problem, including reverse or non-monotonic schedules, would help clarify whether the specific path itself is important.
2 comments
Response to Reviewer RTcg (2/2)
W2 and Q2: Our component controls directly evaluate alternative trajectories that share the exact same constrained endpoint, demonstrating that removing any element of the continuation path degrades target accuracy by 0.71–1.52 pp. Specifically, our ablations separate which intermediate problem family is traversed from how stage progression is paced. The fixed-increment control retains RBC-LoRA's full objective and allowance schedules, isolating the effect of stage pacing, whereas the constant-budget, no-anchor, and budget-only controls alter the intermediate optimization trajectory while preserving the final objective and allowance at (see revised paper lines 430–450, pages 8–9).
With denoting the task loss and , RBC-LoRA optimizes under an expanding allowance . Constant-budget anchor annealing retains this objective deformation but fixes the allowance at throughout; the no-anchor control optimizes under ; and budget-only training optimizes under . Together, these variants isolate budget expansion, the anchored start, the task ramp, and adaptive discretization along the continuation path (see revised paper lines 1084–1092, page 21).
All controls were evaluated on Qwen2.5-3B/GSM8K at rank 8 and under the shared reference sets and 24 charged GPU-hour cap. The table below summarizes the results over the original seeds 1–3, reporting target gains relative to the pretrained model (mean ± sample SD) and three-seed mean audit KL in nats per response:
| Variant | Change relative to the full continuation family | GSM8K gain (pp) ↑ | Mean audit KL ↓ |
|---|---|---|---|
| RBC-LoRA | Full objective and allowance schedules; adaptive pacing | 6.36 ± 0.22 | 1.78 |
| Direct constrained | Final objective and allowance throughout | 5.42 ± 0.35 | 1.86 |
| Constant-budget anchor | Final allowance throughout | 5.53 ± 0.46 | 1.90 |
| No-anchor task ramp | Remove the factor anchor | 5.65 ± 0.23 | 1.90 |
| Budget-only | Remove the anchor and task ramp | 4.84 ± 0.58 | 2.07 |
| Fixed increments | Full family at six prescribed levels | 5.68 ± 0.55 | 1.73 |
| Direct + warmup | Direct training with 500-step learning-rate warmup | 5.45 ± 0.48 | 1.90 |
RBC-LoRA improves target gain by 0.83 pp over constant-budget anchor annealing and by 0.71 pp over the no-anchor control. Comparing the no-anchor task ramp with budget-only training further isolates the task ramp, showing a 0.81 pp advantage. For stage pacing, fixed increments achieve a slightly lower mean audit KL than adaptive RBC-LoRA (1.73 versus 1.78), whereas adaptive progression yields 0.69 pp higher target gain and a smaller maximum retained-benchmark drop (0.90 versus 1.16 pp). Thus, pacing exhibits a clear specialization–retention trade-off, while each modification to the intermediate problem family degrades both target accuracy and audit KL (see revised paper Table 3, page 9).
By coupling increasing task pressure with anchor decay and nested feasible-set expansion toward the shared endpoint, the full continuation path achieves superior endpoint quality compared with each tested trajectory and pacing alternative. We have updated Section 5.3 to explicitly separate the intermediate problem family from stage pacing and tie our path claims directly to these controlled comparisons (see revised paper lines 430–450, pages 8–9).
Response to Reviewer RTcg (1/2)
W1 and Q1: To broaden the experimental scope, we added instruction-following adaptation on Qwen2.5-7B and Gemma-2-9B-it, extending our original 3B GSM8K and MBPP settings. Using 16 A100 GPUs, we compared RBC-LoRA, LoRA, and direct constrained training on a 20k-example UltraChat subset across ranks 8, 16, and 32 with three seeds per configuration (54 trained adapters in total). Both constrained methods use forward sequence KL from the original policy to the adapted policy under the same final allowance of 2.0 nats per response (see revised paper lines 1298–1311, page 25).
This setup cleanly separates the target adaptation objective from retained capabilities: Evol-Instruct and UltraFeedback evaluate instruction following, while GSM8K, MATH, HumanEval, and MBPP measure retained math and code performance (complementing MMLU, HellaSwag, and IFEval in the 3B suite). On both IF benchmarks, each adapted model's responses are compared against GPT-3.5-turbo responses using DeepSeek-v4.1-flash as the judge (Xu et al., 2026).
The table below reports rank-16 results (mean ± sample SD over three seeds), where IF columns show win rates (WR, %) and Δ columns show percentage-point changes from the pretrained model (see revised paper Tables 7 and 8, pages 25–26).
At rank 16, RBC-LoRA improves mean IF win rates over direct constrained training by 1.40/1.50 pp on Qwen and 1.53/0.90 pp on Gemma, while simultaneously reducing forgetting across all four retained math and code benchmarks. For example, Qwen's MBPP drop decreases from 2.00 to 0.80 pp, and Gemma's HumanEval drop decreases from 0.80 to 0.20 pp.
Across all three ranks on both backbones, RBC-LoRA outperforms both baselines on both IF evaluations and matches or exceeds direct constrained training on every retained benchmark, with a single tie on Gemma rank-8 HumanEval (see revised paper Tables 7 and 8, pages 25–26). Together with the 3B results, these findings demonstrate consistent specialization and retention gains across 3B–9B models and across math, code, and instruction-following tasks.