- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes LoTiC, a parameter-efficient approach to temporal domain generalization. It assumes that optimal parameters across time lie on a common low-dimensional embedded manifold and shows that parameter increments relative to a fixed pretrained model preserve this structure. Unlike independently trained domain-specific LoRA adapters with factors and , LoTiC uses shared, time-invariant bases and , together with a time-varying core , to represent updates as . This factorization is exact when the shared bases span the relevant column and row subspaces across domains; in practice, the method learns compact shared bases. Experiments demonstrate its performance and parameter efficiency across multiple benchmarks and backbone scales, with ablations supporting the main design choices.
Critical strengths
-
The method offers a compelling way to reduce the cost of temporal parameter modeling by concentrating temporal dynamics in a compact core. The experiments provide substantial evidence of its effectiveness and efficiency.
-
The paper is clearly written and well organized, and the derivations are easy to follow
Critical weaknesses
- Trade-off between efficiency and representational capacity. The exact factorization requires shared bases that span the relevant subspaces across time, whereas the practical method uses a compact shared rank. A rank that is too small may exclude important update directions that cannot be recovered by changing the temporal core alone; increasing the rank raises the parameter and computational costs. Although Appendix A.4 reports rank sensitivity on Rotating 2-Moons, it remains unclear how this trade-off behaves on real-world datasets. Further discussion of the practical rank-selection procedure and its applicability across datasets would strengthen the paper.
Questions for authors
-
How was the shared rank of and selected for the real-world benchmarks? Beyond the controlled rank sweep in Appendix A.4, is there evidence or practical guidance on how rank affects predictive performance and efficiency on these datasets?
-
There appears to be an extra blank space at the end of the first line in Section 5.5.
3 comments
Thank you again for your positive assessment and thoughtful feedback. In our rebuttal, we have provided additional evidence and analysis to further support our contributions. As the discussion period is nearing its end, we would greatly appreciate your consideration of these additional results and analyses.
Thank you for your experiments and clarification, which well address my concerns.
Thank you for your thoughtful review and for recognizing our work. We address your concerns below.
Existing studies have shown that the configured LoRA rank need not be fully used by single-domain adaptation. The LoRA paper’s empirical analysis shows that ranks as low as 1–2 can achieve competitive performance on selected GPT-3 tasks, supporting the low effective rank of adaptation updates.
Let r1 denote the effective rank needed for single-domain adaptation. Temporal distribution shifts may also have low intrinsic complexity, requiring only a small number r2 of additional update directions. Intuitively, the combined requirement can be viewed as r1+r2. Since r1 may be smaller than the configured rank r, the remaining capacity may already accommodate temporal variation without increasing r.
Figure 4 shows strong performance on Moons already at r=4, with stable results across a broad rank range. We additionally evaluate Yearbook and arXiv to examine this behavior on real-world visual and textual tasks.
LoTiC outperforms static LoRA at every tested rank. And increasing the rank beyond r=8 provides little additional benefit on either dataset, suggesting that a compact subspace is sufficient for these tasks.
In our main experiments, we use r=8 for both LoTiC and the baselines, ensuring that the gains of LoTiC are not attributable to a higher adapter rank. For real-world tasks with greater complexity, increasing the rank slightly may yield better results.
Thank you for pointing this out. We have traced the formatting issue to Table 4 and will correct it in the final version.
We hope these clarifications address your concerns.