- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
ReTAP replaces fixed-answer replay with aligning the student's hidden states to those of a frozen, task-specific reference model on the student's own online-generated responses, thereby sidestepping the mismatch between the prefixes of stored answers and those the student actually generates.
Critical strengths
All reference models are drawn from the existing checkpoints of the same training run (no separately trained experts, and no added inference-time cost), and it consistently achieves the highest Avg and BWT across three backbones, four capacities, and three replay ratios, while being nearly insensitive to K and yielding robust conclusions.
Critical weaknesses
I consider this a competent and rigorous paper, but not one deserving of the highest rating. The method is not particularly novel, essentially combining hidden-state alignment with the characteristics of SDFT, and it offers no distinctive highlight. The experiments show good results mainly on LoRA, while the gains under FFT remain limited, which constrains the paper's potential contribution to the community.
Questions for authors
Core unresolved issue: The authors do not adequately justify the connection between prefix mismatch and hidden-state alignment. Existing work attributes hidden-state alignment mainly to alignment at the knowledge and reasoning levels (DPRD), which appears to contradict the prefix-mismatch claim. I went through the entire paper and found Section 3.4, yet it does not resolve my confusion; the theoretical grounding is very thin.
The experimental setup is largely reasonable, and I have no further requests there. I would be glad to raise my score if the authors can resolve my core concern. I would also suggest the authors report the method's additional overhead, particularly runtime and GPU cost, since this matters for others trying to gauge the time complexity of adopting it.
4 comments
Q3: Are the benefits largely restricted to LoRA, with limited gains under FFT?
We added complete eight-task FFT experiments on Qwen3-1.7B and Qwen3-4B, including OPR-RU as a comparator. Each cell below reports Avg / BWT, using seed 1234:
| Method | Qwen3-1.7B | Qwen3-4B | Qwen2.5-7B |
|---|---|---|---|
| Replay | 53.7150 / −4.8064 | 62.6204 / −1.0315 | 65.7609 / −2.4659 |
| OPR-RU | 55.7308 / −2.3858 | 63.7859 / −0.7613 | 65.5404 / −2.3761 |
| ReTAP: | 56.7478 / +0.4135 | 64.0296 / −0.5608 | 66.5442 / −1.2158 |
ReTAP outperforms both Replay and OPR-RU in Avg and BWT on all three evaluated models under FFT. Relative to OPR-RU, ReTAP improves by 1.02 Avg / 2.80 BWT points on Qwen3-1.7B, 0.24 Avg / 0.20 BWT points on Qwen3-4B, and 1.00 Avg / 1.16 BWT points on Qwen2.5-7B.
Compared with Replay, the corresponding improvements are 3.03 Avg / 5.22 BWT, 1.41 Avg / 0.47 BWT, and 0.78 Avg / 1.25 BWT points, respectively. These results support the applicability of ReTAP to FFT across the evaluated model sizes, with improvements in both final performance and historical retention.
Q4: What is the additional computational overhead, and how can it be reduced?
The additional computation mainly comes from response generation on historical prompts, reference-model forward passes, and reference selection in the default configuration. These operations occur during training. Deployment uses only the final student model and adds no teacher inference path.
We evaluated a simplified K1 / last1 / Task-end configuration: one sampled response per historical prompt, final-layer representation supervision, and a fixed reference saved immediately after learning each historical task. This removes validation-based reference selection.
On two A800-80GB GPUs, three paired seed/task-order settings give the following results; ± denotes sample standard deviation:
| Method | Avg ↑ | BWT ↑ | Mean algorithm time ↓ |
|---|---|---|---|
| OPR-RU | 64.5780 ± 0.9213 | −2.6324 ± 0.9816 | 7.5726 h |
| OPR-SC | 64.0813 ± 1.0441 | −3.5795 ± 0.7467 | 7.5417 h |
| ReTAP: K1 / last1 / Task-end | 66.5728 ± 0.4764 | −0.4912 ± 1.0308 | 6.8685 h |
The simplified ReTAP configuration improves Avg by 1.99 / 2.49 points over OPR-RU/SC while reducing mean algorithm time by 9.30% / 8.93%, respectively. All three paired settings show improvements in Avg, BWT, and algorithm time. Compared with the default K8 / all-layer / Selected configuration’s mean of 14.8809 hours in these experiments, the simplified configuration reduces algorithm time by approximately 53.84%.
The reported algorithm time covers training-stage computation, including model training, response generation, reference-model forward passes, and reference selection.
These explanations clarify the complementary roles of prefix coverage and representation preservation. The output-distillation comparisons, FFT extensions, and three paired runtime measurements provide concrete evidence addressing your concerns about the mechanism, contribution, and practical cost.
Thank you again for your thoughtful feedback. We hope that these clarifications and additional results address your concerns, and we would greatly appreciate your reconsideration of the score in light of this response.
Dear Reviewer,
Thank you for taking the time to read our paper and share your thoughtful comments. We considered each of your points carefully and responded to them individually.
We hope our responses address your concerns. If there is anything we have not fully addressed or explained clearly, we would be happy to discuss it further.
Thank you again for your time and helpful feedback.
Best regards,
The Authors
Dear Reviewer 98eT, thank you for pressing us on the connection between prefix mismatch and hidden-state alignment. If you have a chance before the discussion closes, we’d especially welcome your thoughts on Q1 in our response above. The example there illustrates how student-generated prefixes determine where supervision is applied and how hidden-state alignment defines the training signal at those prefixes. We’d be happy to clarify anything that remains unclear.
Thank you for recognizing the rigor of our experiments, the practicality of using historical references, and the continual-learning results. We particularly appreciate your question about the relationship between prefix mismatch and representation alignment. We address this central concern first, followed by the relationship to SDFT, applicability to full fine-tuning, and computational overhead.
Q1: How is prefix mismatch related to hidden-state alignment? Does this conflict with knowledge- or reasoning-level preservation?
The two interpretations are compatible: the prefix distribution determines where preservation constraints are applied, while hidden-state alignment determines what is preserved at those states.
Concretely, the current student generates a response to a historical-task prompt. The student and the frozen historical reference then process exactly the same response prefixes, and their hidden states are aligned at corresponding positions. The reference does not need to independently generate the same response; it provides supervision on prefixes already visited by the student.
For example, when the student answers an old-task question using wording different from the stored answer, its subsequent generation follows different prefixes. Preservation constraints on the stored answer do not directly cover these prefixes. ReTAP instead evaluates the historical reference on the student-generated prefixes and constrains the student’s representations there.
This explains how the knowledge/reasoning interpretation relates to our motivation: knowledge- or reasoning-level preservation concerns the content that representation constraints may preserve, whereas prefix coverage concerns the input states on which those constraints operate. The same hidden-state loss can be applied to either fixed answers or student-generated responses; their distinction lies in the distribution of supervised states.
To further examine the empirical value of representation preservation, we supplemented the existing reverse-KL comparison with forward-KL output distillation:
The additional forward-KL experiments tested λ ∈ {1, 10, 50}; the table reports the best tested configuration. Within the respective comparison groups, MSE improves over reverse KL by 2.64 Avg / 2.39 BWT points and over forward KL by 2.50 Avg / 1.97 BWT points.
These results support the practical value of representation preservation in our setting.
Q2: Is ReTAP simply a combination of hidden-state alignment and SDFT?
Our contribution concerns how to use capabilities retained in the history of a single continual-training run to construct preservation targets for multiple previous tasks while learning the current task. The distinctions from SDFT are concrete:
These choices have complementary roles: historical checkpoints specify the reference capabilities, student trajectories specify the states receiving preservation constraints, and the representation loss implements those constraints. Together, these components form a continual-preservation procedure that uses historical capabilities to support learning across tasks.