- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes DualKL, a method for LoRA training that preserves past tasks' predictions when learning a new task. The paper argues that parameter space constraints for preventing forgetting, like orthogonal updates to weights, do not necessarily preserve model predictions. DualKL constraints the KL divergence between the task's prediction right after learning with predictions while learning a new task to prevent the drift in that task.
Critical strengths
- The results on the 15-task long-sequence benchmark show DualKL as having better final accuracy and less forgetting as more tasks are learned
- Their gains hold up on different model families (llama, qwen and t5-large). The paper also reports metrics on training runtime and memory
- Appendix includes ablations and sensitivity analyses (warmup steps, anchor set size, KL budget, etc.)
Critical weaknesses
- The method requires access to past task examples, making it limited for practical use cases
- For the anchor set for the previous task that is required to be maintained when learning a new task, I am concerned that it adequately represents the task
- The method controls average KL divergence over the anchor set, which does not guarantee that every anchor satisfies the individual KL condition required by the decision-preservation guarantee.
- Table 23 shows gains from tuning the warm-up and ramp schedule. It is unclear whether the gains transfer to other baselines like fixed-KL, or whether DualKL's adaptive weighting is driving those improvements.
Questions for authors
- According to Table 1, the initial learning accuracy for DualKL is much higher than that of other methods. Final performance, including retention, could benefit from this. Table 23 shows that warm-up and ramping contribute to the gain, but I am not sure if it explains it entirely. Were other baseline (like the Fixed-KL) experiments also evaluated on different warm-up steps?
- Following on the previous, can the authors provide the breakdown for per-task accuracy? What drives the increase of ~86 compared to ~80 for other tasks?
- Can the authors comment on the anchor coverage? The anchor size ablation table compares various anchor set sizes, but have the authors tried including different subsets? Like, say, some under-represented examples
Dear Reviewer qEba
Dear Reviewer qEba,
Please carefully review the authors' rebuttal and respond to each of your concerns. If any concerns remain unresolved, clearly specify what is missing and request concrete clarifications or additional evidence.
Bests,
Your AC.