- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes TACTIC, an exemplar-free domain-incremental learning method on frozen encoder with per-domain AdaLoRA experts. The multiple components are a HSIC-based objective intended to disentangle semantic and domain-related representations, Synthetic Gaussian feature replay, and a hyperspherical classifier with multiple sub-centers per class.
Critical strengths
- Exemplar-free domain-incremental learning without domain IDs at inference is relevant problem setting
- Results are reported across 4 datasets, DomainNet, Office-Home, CORe50, and Camelyon17 and are promising
Critical weaknesses
-
Some theoretical claims are not fully supported, and some proofs contain mathematical errors. Theorem 1 assumes unit-norm distribution means and contains an invalid step implying that expected maximum cosine similarity can exceed 1. The resulting bound does not mean that the multiple sub-centers prevent representation collapse
-
How does minimizing empirical HSIC show the claim of class-independent, drift-free routing.
-
Theorem 2 -Wasserstein inequality is a standard result under the assumed Lipschitz condition. However, projecting synthetic Gaussian samples onto the unit sphere does not guarantee they approximate historical feature distributions. Needs more justification
-
Ablation results are confusing. The routing backbone and domain head are frozen after the first domain, yet replay projection or EMA memory merging reduces routing accuracy from 59.2% to approximately 48%. It is unclear how these components could directly affect the frozen routing representations.
-
Does the memory footprint of less than 1.5 MB include class-level Gaussian statistics and also the growing AdaLoRA experts, per-domain GMMs, or other trainable components? Need more clarification on memory and resources
Questions for authors
- How was joint training implemented? Why does it substantially underperform TACTIC? Please clarify differences between the reproduced baselines and published results?
- How is statistical independence between semantic and domain features verified empirically? Since the LoRA-disabled backbone and domain head are frozen after the first domain, what does HSIC provide beyond parameter freezing?
- In Table 2, why does removing replay projection or EMA memory merging reduce routing accuracy from 59.2% to approximately 48%, given that the routing backbone and domain head are frozen after the first domain?
- Why is DCE classified as requiring physical exemplar storage and excluded from Camelyon17, because DCE method uses Gaussian pseudo-features and not physical exemplars.
- Can you please report the total storage and computational costs, including AdaLoRA experts and GMMs, along with standard deviation over multiple seeds?
- Authors also need clarify all the mathematical issues in theorms