acceptodds
Under review as a conference paper at ICLR 2027

LANGUAGE-AWARE COMPLEMENTARY DECOMPOSITION FOR EXEMPLAR-FREE DOMAIN- INCREMENTAL LEARNING

Abstract

Exemplar-free domain-incremental learning (DIL) requires models to continually adapt to new domains while retaining previously acquired knowledge without storing historical images. Although parameter isolation mitigates forgetting by preserving domain-specific knowledge in separate lightweight modules, effective inference still requires reliably identifying which knowledge should be invoked when domain identities are unavailable at test time. This is particularly challenging because class semantics and domain variations are entangled in visual representations, causing the evidence useful for recognition and domain selection to interfere with each other. Motivated by the insight that language semantics provide a natural reference for organizing these two types of information, we propose LaCoD, a Language-Aware Complementary Decomposition framework for exemplar-free DIL. LaCoD constructs a semantic subspace from frozen CLIP text embeddings and decomposes visual features into a language-aligned semantic component and a complementary residual. The residual retains domain-sensitive cues with reduced class-semantic interference and supports domain routing, which in turn selects suitable domain-specific predictors. The semantic component preserves domain-agnostic class evidence to complement the routed prediction. In this way, LaCoD assigns complementary roles to the two components, enabling more reliable knowledge invocation and robust classification. Extensive experiments on four DIL benchmarks demonstrate the effectiveness of LaCoD, consistently outperforming prior methods.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
domain incremental learning;semantic subspace decomposition;domain routing

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ZxAPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LaCoD for exemplar-free domain-incremental learning. It splits CLIP visual features into a language-aligned semantic component and a complementary residual. The residual is used for domain routing, while domain-specific prompts and classifiers handle prediction. A semantic branch is also added to reduce the effect of routing errors. Experiments on four benchmarks show improvements over existing methods.

Critical strengths
  1. Strong results. LaCoD outperforms the evaluated baselines on all four benchmarks, with gains of 9.35 and 6.13 points on ImageNet-R and ImageNet-Mix.

  2. Useful semantic branch. Table 4 shows that semantic projection improves accuracy on incorrectly routed samples from 61.70% to 66.12%. This supports its role in reducing the effect of routing errors.

Critical weaknesses
  1. Unclear source of improvements. In Table 5, adding a linear classifier improves ImageNet-R accuracy from 66.94% to 78.46%. The residual features and semantic projection add only 0.36 and 0.52 points over their respective controls. Most of the gain comes from the classifier, while the benefit of the proposed decomposition is relatively small. The ablations also lack results across multiple seeds.

  2. Missing baseline comparisons. SOYO is compared only for routing accuracy, while MoP-CLIP is missing from the main results. Both are closely related to domain routing and prediction fusion. End-to-end comparisons are needed to better assess LaCoD's contribution.

  3. Storage and scalability. LaCoD stores Gaussian feature statistics and adds a prompt–classifier pair for each new domain. Although inference uses only the top two predictors, memory usage grows with the number of domains. The paper does not report the actual storage cost or how inference time changes over longer streams.

Minor typo: Sec. 4.1 refers to Appendix G for temperature sensitivity analysis, but the results are actually in Appendix F.

Questions for authors
  1. Can the authors provide multi-seed ablations to better separate the gains from the linear classifier and the proposed decomposition?

  2. Can the authors compare LaCoD with SOYO and MoP-CLIP on all four benchmarks using the same evaluation protocol?

  3. Can the authors report the memory usage and inference time as more domains are added, including the stored Gaussian statistics?

Reviewer_oJ4vOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes LaCoD, for exemplar free domain-incremental learning and consists of three modules, LFD, RDP and SP. It uses frozen CLIP features to construct semantic subspace. Visual features are decomposed into language-aligned and its orthogonal residual. Domain-specific visual prompts and classifiers preserved peviously learned predictors. Experiments are provided on DomainNet, and ImageNet variants

Critical strengths
  • Exemplar-free DIL and task-agnostic inference is a relevant problem setting.
  • LaCoD compares with many relevant method and also achieves strong performance on the evaluations
Critical weaknesses
  • The main claim that residual isolates domain-sensitive information is not clear. How does orthogonality to CLIP text subspace equal disentangling class and domain information? There are many such implicit assumptions. Results in Table2 are also mixed, the residual still contains class-discriminative information?

  • Separating class-semantic and domain-sensitive features is the main contribution? Because the ablation in Table 5 suggests that much of the improvement comes from introducing learnable classifiers (LC) and the routing mechanism?

  • The methods uses gaussian mixtures for pseudo replay. In equation 6, r is obtained from v through fixed projection. Will sampling from concatenated feature space preserve these dependencies and constraints?

  • The method has many parts - linear classifiers per domain, trained router, prompts. Need more information on these numbers and also the inference cost. Also many methods in this field (ex. DUCT) show results on Core50, Officehome and Domainnet datasets.

Questions for authors
  • What is the evidence for the language-guided feature decomposition separating semantic and domain information? The claim that residual captures domain-sensitive information is not fully convincing. There are many important assumptions made in paper without supporting evidence and this is making it difficult to see the value of the method
  • Can the contribution of the language-guided projection be isolated from learned classifiers, using matched random-subspace or alternative projection controls?
  • The final fusion uses the difference between the two largest softmax routing weights. However, this measures the router's relative confidence, not necessarily the probability that its domain prediction is correct?
  • Need more information on the inference cost and also scalability of this method. (number of prompts, classifiers, statistics).
  • How are prompts configured? Paper says it contains 16 context tokens but need more specific info
  • The flow in the figure is bit confusing and can be improved for readability
Reviewer_ZcoWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies exemplar-free domain incremental learning (DIL) and argues that the difficulty lies in routing samples to the correct domain module at test time. LaCoD uses frozen CLIP text embeddings to span a semantic subspace, decomposing visual feature v into a semantic component v_sem within the subspace and an orthogonal residual r. The residual together with v is fed into a router (RDR), which preserves old domains via class-conditional Gaussian pseudo-feature replay; when routing confidence is low, v_sem provides a domain-agnostic complementary prediction (SP). On four DIL benchmarks, LaCoD's AT and AA both exceed seven reimplemented baselines, and on ImageNet-R AT leads by 9.35 points.

Critical strengths
  1. The method in this paper brings large performance gains and leads on all four benchmarks.

  2. The implementation details of the method itself are provided relatively fully.

  3. The tables and figures are clear and easy to understand.

Critical weaknesses
  1. The main gains come from a generic per-domain linear classifier (LC). In Table 5, AT on ImageNet-R rises from 66.94 for S-Prompts to 82.42; adding LC alone reaches 78.46, already surpassing all baselines in Table 1. The decomposition itself (RDR versus RDR(v), SP versus SP(v)) adds only 0.3 to 0.5 points, and is based on a single run. The baselines are not equipped with LC, so the lead in Table 1 cannot be attributed to the decomposition. It is recommended to add LC to strong baselines for comparison and report multi-seed results.

  2. Insufficient distinction from prior work. XIL has already used "image minus class text" residuals to represent domain information, and this paper mentions it only in one sentence in the introduction; the Gaussian replay plus MLP scheme of the router in Section 3.3 is basically the same as SOYO, without noting this.

Questions for authors

See Weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.