acceptodds
Withdrawn from ICLR 2027

Task-Agnostic Exemplar-Free Domain-Incremental Learning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Autonomous Domain-Incremental Learning (DIL) requires models to sequentially acquire new visual domains without explicit task identifiers, exemplar storage or catastrophic forgetting. Current methodologies face a fundamental trilemma: replay-based methods compromise data privacy, regularization restricts model plasticity and retrieval-based routing degrades as latent representations drift over time. We introduce Task-Agnostic Continual Training via Independent Components (TACTIC), an exemplar-free framework that resolves this trilemma through a unified geometric principle: constraining all learned representations to a shared unit hypersphere . TACTIC operationalizes this principle via three synergistic mechanisms. First, a sub-center geometric head allocates multiple angular anchors per class, providing the intra-class flexibility necessary to mathematically prevent the semantic collapse endemic to shared classifiers. Second, a feature disentanglement objective explicitly separates domain-specific style from universal semantics. By enforcing statistical independence (HSIC) between these representations, TACTIC guarantees stable, drift-free routing at inference without relying on task metadata. Third, a hyperspherical analytic memory synthesizes per-class Gaussian statistics and projects them back onto , maintaining manifold compatibility while supplying boundary-anchoring hard negatives. Theoretically, we establish intra-class interference bounds that guarantee the geometric safety of our analytic replay. Empirically, TACTIC establishes a new state-of-the-art exemplar-free DIL performance across diverse benchmarks (DomainNet, Office-Home, CORe50 and Camelyon17). Crucially, by isolating parameters and eliminating cross-domain gradient interference, TACTIC outperforms existing methodologies, all while maintaining a minimal storage footprint of less than 1.5 MB.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Continual Learning, Domain-Incremental Learning, Task-Agnostic Inference, Representation Learning, Feature Disentanglement, Parameter-Efficient Fine-Tuning (PEFT)

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_h1E9OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes TACTIC, an exemplar-free domain-incremental learning method on frozen encoder with per-domain AdaLoRA experts. The multiple components are a HSIC-based objective intended to disentangle semantic and domain-related representations, Synthetic Gaussian feature replay, and a hyperspherical classifier with multiple sub-centers per class.

Critical strengths
  • Exemplar-free domain-incremental learning without domain IDs at inference is relevant problem setting
  • Results are reported across 4 datasets, DomainNet, Office-Home, CORe50, and Camelyon17 and are promising
Critical weaknesses
  • Some theoretical claims are not fully supported, and some proofs contain mathematical errors. Theorem 1 assumes unit-norm distribution means and contains an invalid step implying that expected maximum cosine similarity can exceed 1. The resulting bound does not mean that the multiple sub-centers prevent representation collapse

  • How does minimizing empirical HSIC show the claim of class-independent, drift-free routing.

  • Theorem 2 -Wasserstein inequality is a standard result under the assumed Lipschitz condition. However, projecting synthetic Gaussian samples onto the unit sphere does not guarantee they approximate historical feature distributions. Needs more justification

  • Ablation results are confusing. The routing backbone and domain head are frozen after the first domain, yet replay projection or EMA memory merging reduces routing accuracy from 59.2% to approximately 48%. It is unclear how these components could directly affect the frozen routing representations.

  • Does the memory footprint of less than 1.5 MB include class-level Gaussian statistics and also the growing AdaLoRA experts, per-domain GMMs, or other trainable components? Need more clarification on memory and resources

Questions for authors
  • How was joint training implemented? Why does it substantially underperform TACTIC? Please clarify differences between the reproduced baselines and published results?
  • How is statistical independence between semantic and domain features verified empirically? Since the LoRA-disabled backbone and domain head are frozen after the first domain, what does HSIC provide beyond parameter freezing?
  • In Table 2, why does removing replay projection or EMA memory merging reduce routing accuracy from 59.2% to approximately 48%, given that the routing backbone and domain head are frozen after the first domain?
  • Why is DCE classified as requiring physical exemplar storage and excluded from Camelyon17, because DCE method uses Gaussian pseudo-features and not physical exemplars.
  • Can you please report the total storage and computational costs, including AdaLoRA experts and GMMs, along with standard deviation over multiple seeds?
  • Authors also need clarify all the mathematical issues in theorms
Reviewer_pmSAOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies exemplar-free, task-agnostic domain-incremental learning (DIL), where classes are shared across a sequence of changing visual domains and domain identifiers are not available at inference. TACTIC freezes an ImageNet-1K-pretrained ViT-B/16 backbone and allocates an AdaLoRA expert to each domain. A dual-forward design extracts LoRA-on semantic features for classification, and LoRA-off style features for routing; a dependence penalty including HSIC aims to disentangle these views. For each historical domain, a BIC-selected Gaussian mixture model models the routing features, and Top-2 soft routing combines expert predictions. A multi-sub-center cosine classifier with LogSumExp aggregation accommodates multiple styles per class, while class-wise mean/variance statistics are used to synthesize L2-projected feature replay without storing raw images. On DomainNet, Office-Home, CORe50, and Camelyon17, the paper reports higher average accuracy than its cited baselines, including 36.31% versus 30.05% for SOYO on DomainNet and 85.85% versus 78.13% on Office-Home. The reported DomainNet ablations indicate that the disentanglement design and analytic replay materially affect performance.

Critical strengths

The paper presents a coherent framework that jointly addresses catastrophic forgetting, task-agnostic inference, and exemplar-free learning. By combining parameter-isolated AdaLoRA experts, HSIC-based feature disentanglement, Top-K soft routing, and hyperspherical analytic replay, the proposed method offers a practical solution to several interconnected challenges in domain-incremental learning. This integrated design is a significant strength, as it improves the applicability of continual learning methods to scenarios where historical data storage and task identifiers are unavailable.

The paper demonstrates consistent improvements across four diverse benchmarks, including DomainNet, Office-Home, CORe50, and Camelyon17. In particular, TACTIC improves average accuracy over the closely related SOYO baseline by 6.26 and 7.72 percentage points on DomainNet and Office-Home, respectively. The ablation studies further demonstrate the contributions of the proposed components, particularly the disentanglement and analytic memory mechanisms. These results provide encouraging empirical evidence for the effectiveness and generalizability of the framework, constituting an important factor in favor of acceptance.

Critical weaknesses

The theoretical analysis contains several issues that weaken the claimed guarantees. In Appendix A.1, the proof of Theorem 1 suggests that the expected maximum cosine similarity can exceed 1, contradicting its bounded range for unit vectors. In Appendix A.3, the proof of Proposition 1 applies the Data Processing Inequality without establishing the required Markov-chain relationship, while minimizing empirical HSIC does not guarantee statistical independence or semantic sufficiency. Furthermore, Theorem 2 does not establish that the projected Gaussian replay distribution adequately approximates historical feature distributions. These limitations leave the theoretical claims regarding semantic collapse prevention, label-agnostic routing, and replay stability insufficiently substantiated.

Although the method achieves promising results across multiple benchmarks, several experimental limitations remain. In Table 1, TACTIC substantially outperforms Joint Training on DomainNet and Office-Home, but the fairness of these comparisons in terms of model capacity and computational budget is not sufficiently established. The absence of results across multiple random seeds and domain orders further limits the assessment of statistical reliability. Moreover, Table 2 lacks a direct comparison between the proposed Top-K soft routing and conventional hard routing under identical conditions, making its individual contribution unclear. Finally, the reported storage footprint of less than 1.5 MB does not clearly account for the growing collection of domain-specific AdaLoRA experts and routing models, leaving the overall memory efficiency insufficiently validated.

Questions for authors

In Appendix A.1, the proof assumes unit-norm distribution means and states that the expected maximum cosine similarity can strictly exceed 1, which appears inconsistent with the properties of unit vectors. Please clarify this inconsistency. This clarification is essential for assessing the validity of the paper's central theoretical contribution.

In Appendix A.3, the proof of Proposition 1 relies on a Markov-chain relationship that does not appear to follow directly from the parallel projection architecture. Please clarify the assumptions required for this relationship. This clarification would help determine whether the claimed routing stability is theoretically justified.

Table 1 shows that TACTIC substantially outperforms Joint Training on DomainNet and Office-Home, despite Joint Training being presented as an upper bound. Please clarify whether the baselines are matched in terms of model capacity and computational budget, and provide evidence of performance consistency across multiple random seeds or domain orders. This information is important for assessing whether the reported improvements reflect a robust methodological advantage rather than differences in experimental configurations.

Reviewer_4iLzOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes TACTIC for exemplar-free domain-incremental learning without domain identifiers at inference. It combines frozen pretrained features, domain-specific AdaLoRA experts, separate semantic and routing projections, a sub-center cosine classifier, and synthetic replay from stored class statistics. A Gaussian mixture router combines predictions from selected experts. Experiments on four benchmarks report improved accuracy, while theoretical results are presented to justify resistance to collapse, semantic disentanglement, and replay stability.

Critical strengths

Freezing the routing projection after the first domain and evaluating it with adapters disabled provides a concrete mechanism for preventing parameter drift in the routing features. This is a useful architectural distinction from the evolving semantic branch.

Critical weaknesses
  1. Appendix A.1 assumes unit-norm population means for unit-norm features, which implies zero variance within each component distribution. It then places sub-centers at these means rather than analyzing their optimization. The argument therefore does not establish that training prevents center collapse or preserves classification boundaries.

  2. Proposition 1 relies on a Markov relation that Appendix A.3 attributes to the two heads sharing a backbone. That conditional independence does not follow from the architecture. Moreover, approximate label sufficiency and minimizing an empirical HSIC penalty do not establish exact independence, and label independence alone does not guarantee accurate domain routing.

  3. Theorem 2’s general Lipschitz inequality is reasonable, but the covariance-based Wasserstein rate is asserted without deriving the effect of normalization or accounting for discrepancies introduced by EMA statistics. Placing both distributions on the sphere does not by itself establish a small gradient error or the claimed replay safety.

  4. Algorithm 1 freezes historical experts but does not freeze the shared task projection or classifier. Synthetic replay enters directly at the classifier, so it does not directly constrain the task projection. Consequently, historical feature statistics may become inconsistent with features produced through the updated projection, despite the claimed elimination of interference.

  5. Although the author raises an interesting question, the manuscript in its current state has significant issues that require correction. For instance, in Figure 1 (the GMM router bank diagram), the two lines do not even start from the same point—this is simply absurd. Such flaws, along with clear signs of non-human, AI-generated text, must be addressed.

Questions for authors

Please refer to weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.