- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies one-step update selection in LoRA fine-tuning through fixed-state comparisons. It restores model parameters, current gradients, and optimizer history before evaluating alternative updates, including skipping, scaling the source proposal, changing factor learning-rate ratios, and matching effective-weight displacement norms. The experiments separate candidate construction, selection evidence, and held-out evaluation. Additional task-balanced examples improve mean one-step held-out cross-entropy gain on Llama and Qwen. Retrospective reassignment controls further examine whether these gains arise from overall candidate preferences, acceptance locations, or more detailed state–choice matching.
Critical strengths
-
The paper offers a fresh perspective by studying one-step update selection in LoRA fine-tuning.
-
The experimental design controls important confounders. Restoring optimizer moments and step counts, rather than only model parameters, makes the candidate comparisons substantially cleaner. Separating gradient-generation data, selection evidence, and evaluation examples is also appropriate.
-
The reassignment controls are informative. Preserving candidate frequencies and acceptance rates while changing their assignment to states provides evidence beyond a simple comparison of average acceptance rates. The acceptance-preserving and same-step controls help qualify the interpretation.
Critical weaknesses
-
The paper appears to be primarily exploratory. After reading it in full, I am still unclear about its central takeaway, as it does not seem to reach a clear conclusion.
-
The strongest result is that additional target-relevant examples improve selection over current-batch loss. However, on Qwen, acceptance-only and validation line search already achieve gains of 0.002099 and 0.002097, versus 0.002558 for full selection; the paired advantages of full selection have intervals containing zero. Moreover, same-step cross-seed reassignment retains a gain of 0.002546, close to the original 0.002558. The paper leaves unclear how much decision complexity or state-specific information is actually necessary.
-
The authors sample states from existing training trajectories, compare alternative one-step updates, and then restore the original state. If the selector were applied repeatedly during training, subsequent parameters, optimizer states, and gradients would all change. The paper does not establish whether sustained use of this selection strategy would produce a better final model.
Questions for authors
-
The experiments focus primarily on small-scale multitask binary classification fine-tuning. Why has the evaluation not been extended to long-form text generation, reasoning training, or knowledge distillation?
-
How stable are the Llama confirmation gains across independently drawn selection panels and additional training trajectories? Can uncertainty be reported over these sources of variation?
2 comments
Thank the authors for their response. I decide to maintain the score because I think my main concerns have not been adequately addressed.
Response to Reviewer mQki
Question: Why was the evaluation not extended to generation, reasoning training, or knowledge distillation?
Response: The classification setting fixes model and optimizer state, supports exact token-level CE comparisons, and allows disjoint selection and evaluation examples without decoding variability. This directly tests the paper's controlled local question; it is not intended to exhaust all downstream settings. Long-form generation, reasoning training, and distillation use different objectives and evaluation protocols and are natural extensions. A paper need not cover every dataset, seed, parameterization, and downstream task to establish a controlled result on its stated benchmark; our claim is limited to the evaluated scope and does not make a universal generation claim.
Question: How stable are the gains across independent panels and additional trajectories?
Response: The reported uncertainty intervals have different scopes. For Llama, the original mean-gain 95% bootstrap interval is [0.000449, 0.003731], and the paired interval for Original minus within-trajectory reassignment is [0.001233, 0.002917]. These intervals are conditional on the observed tasks, source states, and selection panel; they do not quantify new trajectories or newly sampled panels. For Qwen, resampling includes three seed blocks, five selection-panel draws, and evaluation examples. The paired gaps are 0.630e-3 [0.357e-3, 0.910e-3] for all-choice reassignment, 0.073e-3 [0.011e-3, 0.158e-3] for accepted-only reassignment, and 0.013e-3 [-0.016e-3, 0.045e-3] for same-step cross-seed reassignment. The last interval contains zero, so we do not claim that finer state matching is established. The corrected full-trajectory test adds a further caution: fixed panel gives Q=1033 versus 1031 for Full, while current-batch gives 1029 under the same protocol.
Question: Do simpler policies already capture the gain?
Response: Yes, the Qwen one-step results separate the usefulness of additional evidence from the necessity of the richest candidate menu. Acceptance-only and validation line search obtain mean gains of 0.002099 and 0.002097, compared with 0.002558 for full selection, and the paired advantages of full selection have intervals containing zero. The evidence therefore supports the narrower claim that evidence beyond the current batch can be useful.