- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies knowledge transfer, forgetting, and error accumulation in continual test-time adaptation. Without target-domain annotations, source training data, or true domain boundaries, the authors propose Interference Recovery and Protection. The method treats the average parameters of historical domain model checkpoints as a shared component and the difference between each checkpoint and the average as a domain-specific direction. When a previous domain reappears, Recovery adaptively corrects the current adapter along the historical direction. During subsequent learning, Protection minimally adjusts optimizer updates that conflict with this direction. In addition, the method detects domain changes through reconstruction loss and identifies revisited domains through gradient signatures.
Critical strengths
The paper designs an online mechanism for identifying revisited domains based on gradient signatures. By comparing parameter adjustment directions for different data on a fixed source model, it matches the current domain to historical domains without requiring true domain labels. This design identifies the environment based on the needs of model adaptation and combines the identification results with historical knowledge recovery and update protection, providing a relatively complete approach to knowledge reuse when domain identities are unknown.
Critical weaknesses
1.Pseudo-labels may be incorrect, and image reconstruction does not guarantee correct semantic recognition. Historical checkpoints are also learned from these unlabeled objectives and may therefore already contain errors.
2.Equations (4)–(7) in Section 4.2 rely on the assumption that historical parameter residuals are approximately aligned with the negative gradient of the current domain loss. However, a historical checkpoint direction describes a displacement relative to a shared reference point and is not necessarily still a descent direction at the current parameter position. Therefore, an update opposite to the historical direction may also be correcting an excessive shift caused by cross-domain adaptation. The existing performance ablations are insufficient to validate this judgment. I suggest that the authors provide an alignment analysis between historical directions and the current true negative gradients, and directly test whether the updates modified by Protection actually harm performance on the current domain.
Questions for authors
1.Section 4.2 treats historical parameter residuals as approximate descent directions for the current domain and accordingly considers updates in the opposite direction to be harmful interference. However, after the model adapts to other domains, historical directions may no longer align with the negative gradient at the current position. How do the authors validate this assumption? Could an offline analysis compare the effects of updates before and after modification by Protection on the true task loss, demonstrating that Protection mainly blocks harmful updates rather than necessary corrections?
2.If the historical domain checkpoints themselves are of poor quality, can the method correct previous errors?
3.Could the authors provide some theoretical derivations that lead to an important conclusion?
We thank the reviewer for their time and thoughtful feedback on our submission. The comments were constructive and helped us identify several areas where the paper can be strengthened. We have decided to withdraw the paper at this time. We will carefully incorporate this feedback and improve the paper substantially for a future submission.