acceptodds
Under review as a conference paper at ICLR 2027

Inter-Domain Intra-Class Knowledge Complement for Long-Tail Domain Incremental Learning

Abstract

Domain Incremental Learning (DIL) aims to learn from a continuous stream of domain-specific data. However, existing DIL approaches predominantly assume class-balanced distributions, overlooking realistic scenarios where data typically follow a long-tailed distribution. To bridge this gap, we investigate the more practical Long-Tailed Domain Incremental Learning (LDIL) problem, which introduces two major challenges: severe catastrophic forgetting of head classes and underrepresentation of tail classes. To address these issues, we propose a novel Inter-Domain Intra-Class Knowledge Complement (DC-Comp) approach that mines discriminative information from previously seen domains to alleviate distributional undersampling in tail classes of new domains and to mitigate the forgetting of head-class knowledge. Specifically, a Discriminativity Modeling Network associated with an Intra-class Balanced Discriminativity Mining mechanism is developed to capture class-specific local features in long-tailed data. In addition, an Inter-domain Discriminativity Reserving and Complementing scheme is introduced to retain these discriminative features and further leverage them to generate pseudo-samples during new domain training, effectively mitigating both distribution imbalance and catastrophic forgetting. Comprehensive experiments conducted under two practical LDIL settings, unified and random cross-domain distributions, demonstrate that DC-Comp achieves state-of-the-art performance. Our source code will be released.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Recognition: categorization, detection, retrieval

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_2KBUOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses Long-Tailed Domain Incremental Learning (LDIL), where class imbalance and domain shifts occur simultaneously. The proposed DC-Comp introduces Intra-class Balanced Discriminativity Mining to augment local features through discriminative token sampling, particularly for tail classes, and Inter-domain Discriminativity Reserving and Complementing to preserve historical features and generate feature-level pseudo-samples for training on subsequent domains. Extensive experiments across multiple benchmarks and settings demonstrate improvements over existing methods.

Critical strengths
  1. The paper addresses a practical and underexplored problem by considering long-tailed distributions in domain incremental learning.
  2. The proposed discriminativity mining and inter-domain knowledge complement mechanisms are clever and intuitive, effectively addressing the key challenges of LDIL.
Critical weaknesses
  1. Despite the extensive baseline comparisons, the experimental analysis is somewhat limited. It is unclear how DC-Comp differs from the strongest baselines and what leads to the performance gaps. Also, some baselines were originally designed for different learning settings. It would be useful to clarify how they were adapted to LDIL, including hyperparameter choices, and categorize them by their original learning settings.
  2. Although the proposed mechanisms are intuitive and interesting, the paper provides limited discussion of the insights underlying their design. The paper mainly focuses on describing the procedures and reporting empirical improvements, without sufficiently explaining the reasoning behind the design choices and their implications for learning under class imbalance and domain shifts. A more insightful discussion would strengthen the methodological contribution.
Questions for authors
  1. Are samples excluded during long-tail subsampling also excluded from evaluation? How many trials were conducted, and what are the standard deviations of the reported results?
  2. In Section 3.2, what exactly are the keys used for cosine-similarity-based prompt matching? Are they learnable parameters or class prototypes?
  3. How does performance change with different Pareto distribution parameters or imbalance ratios?
  4. Why are the improvements on ImageNet-C substantially larger than those on other datasets?
  5. Among the newly introduced components of the proposed method, which hyperparameters are predefined and which are dynamically determined during training?
5 comments
AuthorsOpenReview ↗

Response to the weaknesses

Thank you for your positive review and constructive feedback. Below is our point-by-point response.

W1: Experimental analysis

(a) Difference from baselines: (1) Our DC-Comp is built on the KA-Prompt baseline by exploiting its prompt learning mechanism. (2) Compared to KA-Prompt, we introduce three original designs: (i) Discriminativity Estimation Module (DEM) identifies class relevant patch tokens, (ii) Intra-class Balanced Discriminativity Mining (IBDM) adjusts token sampling according to class frequency for information-balanced learning, (iii) Inter-domain Discriminativity Reserving and Complementing (IDRC) then uses complementary information from the same classes in previous domains.

(b) Performance gaps: (1) Existing domain incremental learning (DIL) methods, including the KA-Prompt baseline, are primarily designed for balanced data distributions. Consequently, their learning capacity is limited when facing long-tailed data streams. (2) DIL+LTL methods integrate long-tailed learning (LTL) mechanisms into DIL frameworks to improve intra-domain learning under imbalanced data distributions. However, existing LTL methods are primarily designed for static learning scenarios and overlook the complementary information across domains, restricting knowledge acquisition in LDIL. (3) In contrast, our method explicitly mines and reuses discriminative semantics across domains to facilitate learning from long-tailed data streams. This capability is enabled by the complementary designs of DEM, IBDM, and IDRC, which jointly promote reliable semantic acquisition and cross-domain information utilization.

(c) Adaptation to LDIL: (1) All compared DIL and DIL+LTL methods are implemented using the same ViT-B backbone and evaluated on identical LDIL benchmarks under consistent training settings, ensuring a fair comparison. (2) For each approach, we conduct an independent hyperparameter grid search to identify its optimal configuration.

W2: Discussion of the insights

The three proposed components address successive sources of insufficient information in LDIL through complementary designs:

(1) DEM estimates the reliability of individual tokens, enabling the model to identify and prioritize informative features while suppressing unreliable information.

(2) Building upon the estimated token-level reliability, IBDM adaptively allocates more augmentation opportunities to underrepresented classes, thereby alleviating the insufficient intra-class information caused by long-tailed distributions.

(3) IDRC preserves selected historical tokens and generates pseudo samples by combining tokens from historical instances, enabling complementary same-class knowledge from previous domains to facilitate current-domain learning.

AuthorsOpenReview ↗

Response to the questions

Thank you for the positive assessment and for the constructive suggestions. Our responses to the questions follow, point by point.

Q1: Long-tail subsampling, trials, and standard deviations

(1) As specified in Eq.(11) and (12) of our paper, no sampling is performed during evaluation. This is because the learned attention layer can be directly generalized to the full set of visual tokens, eliminating the need for sampling at inference time.

(2) The inference process can be formulated as: $ f(x)=\theta_c(\theta_a(\theta_b(x))), $ where xx denotes the input image, and θb\theta_b, θa\theta_a, and θc\theta_c represent the prompt-conditioned backbone, attention-based token aggregation module, and classifier, respectively.

(3) Accordingly, all test images are processed using the same deterministic inference pipeline, without multiple-trial sampling or random evaluation procedures. Therefore, the evaluation results are deterministic under a fixed model and test configuration, and reporting standard deviations across repeated sampling trials is not applicable.

(4) We also conducted a supplementary token-selection experiment on ImageNet-C using the classifier saved in the available checkpoints. For each test image, we compared inference using all 197 visual tokens with inference using seven visual tokens sampled via Gumbel Top-k. We repeated the sampled inference five times. Accuracy is averaged over 15 stages, and the standard deviations reflect the five sampling trials.

SettingAll tokensSampled tokens
Unif65.06%63.44% ± 0.06%
Rand65.87%64.12% ± 0.07%
Origin87.33%86.26% ± 0.02%

The experimental results show that utilizing all tokens consistently outperforms token sampling under both long-tailed and original data settings. This is because retaining all tokens preserves more comprehensive image information, allowing the model to jointly exploit complementary semantic cues from different tokens and thereby improve inference performance.

Q2: Keys used for prompt matching

(1) The keys in Section 3.2 are learnable embeddings associated with the prompts, as in the KA-Prompt approach. They are not class prototypes. (2) Specifically, each prompt pitp_i^t is paired with a key kitk_i^t; the prompt is selected by comparing that key with the image query q(x)q(x) using cosine similarity.

Q3: Sensitivity Analysis of the Pareto parameter

(1) We reported only α=6\alpha=6 in the paper because it is a commonly used setting in long-tailed learning. (2) We also evaluated different values of α\alpha. As shown below, our method surpasses the state-of-the-art Dual-Prompt+CSA by 10.6%-14.9% across varying α\alpha.

Methodsα\alpha=2α\alpha=4α\alpha=6α\alpha=8α\alpha=10
Dual-Prompt+CSA77.874.268.556.340.2
Ours88.476.080.369.755.1

Q4: Larger gains on ImageNet-C

Thank you for your careful reading. (1) The more pronounced performance gains of our method on ImageNet-C can be attributed to the construction of its domains, where different corruptions are applied to the same source images. Consequently, object content exhibits stronger semantic consistency across domains than in other benchmarks. (2) This cross-domain semantic consistency makes discriminative knowledge preserved from historical domains particularly valuable for learning the same classes in subsequent domains, thereby explaining the greater performance gains achieved by the proposed Inter-domain Discriminativity Reserving and Complementing (IDRC) module.

Q5: Predefined and dynamically determined hyperparameters

(1) The token group size nsn_s is predefined and fixed at 7 in our experiments. (2) In contrast, the class balance ratio pxp_x is computed from the class frequencies in the current domain. The number of sampling rounds rx=1/pxr_x= 1/p_x and the number of preserved tokens nxibn_x^{ib} are then determined dynamically from that distribution, with nxibn_x^{ib} calculated according to Eq. (10).

We hope these responses clarify the design and evaluation of DC-Comp. Thank you again for your constructive suggestions.

Reviewer_2KBUOpenReview ↗

Response to Authors

Thank you for the detailed responses and additional experiments. I appreciate the authors' efforts to address my concerns.

  1. Q1: My question concerned dataset-level long-tail subsampling and variance across independent training runs, rather than token sampling during inference. Thus, the response does not directly address my original question.

  2. Cross-domain complementarity: The explanation for the larger gains on ImageNet-C is reasonable and helps clarify why the improvements are relatively smaller on other benchmarks. This suggests that the effectiveness of the proposed method may depend on the degree of semantic consistency between domains, as class-discriminative features learned in one domain are not necessarily transferable to another.

Overall, I find the proposed approach interesting and the results promising. I maintain my original rating (Weak Accept).

Reviewer_R93YOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies long-tailed domain-incremental learning, where a model sequentially learns from multiple data domains that share the same class space but differ in image distributions and class frequencies. The model needs to recognize classes with few samples in new domains while maintaining its recognition ability on previous domains. The authors propose DC-Comp to achieve this goal through DMN, IBDM, and IDRC.

The problem addressed in this paper has research value: class imbalance and domain changes may occur simultaneously, and addressing only forgetting or only the long-tailed distribution may not effectively handle their combined effects. Using discriminative information from the same classes in previous domains to help learning in the current domain is also reasonable.

Critical strengths

1.The research problem is clearly defined. The introduction identifies insufficient learning of tail classes and forgetting of historical knowledge as the main challenges. The overall design follows a clear logic.

2.The main experiments cover multiple datasets and consider different patterns of change in long-tailed distributions.

3.The appendix provides valuable controlled experiments.

Critical weaknesses

A. Section 3.4 explains that IDRC stores tokens from historical samples and combines them into pseudo-samples, which are directly fed into the attention layers and classifier. When IDRC is added in Figure 5, both historical information and additional replay training are introduced. Table 3 mainly compares how tokens are selected and does not fully isolate the contribution of ordinary feature replay.

B. The claim of low storage overhead lacks quantitative validation, and the storage budget is not sufficiently clear. Section 3.4 and Equation (10) determine how many tokens are stored for each sample based on class frequency and add them to the historical memory bank. Storing only a few tokens per image does not mean that the entire memory bank is small. The total storage still depends on the number of historical samples, the number of tokens stored per sample, the feature dimension, and the numerical precision.

C. The motivation that “long-tailed distributions aggravate forgetting of head classes” has not been sufficiently validated. The decrease in overall accuracy in Table 4 may result from poor learning of tail classes and does not directly demonstrate more severe forgetting of head classes. Table 6 shows that the proposed method reduces forgetting compared with a baseline, but it does not compare changes in head-class forgetting under balanced and long-tailed conditions.

Questions for authors

A.It remains unclear whether the complementary mechanism designed by the authors is particularly effective, or whether simply allowing the model to revisit old features could provide similar improvements. Could the authors clarify this?

B. Please report the memory bank size for each dataset at each incremental stage, including the total number of tokens, storage precision, and actual storage in MB or GB. Is there a fixed upper limit on the memory bank size? Does the proposed method retain its advantage over other methods under the same storage budget?

C.Could the authors report the accuracy of head classes immediately after they are learned, together with their subsequent forgetting trajectories, under the Balance, Unif, and Rand conditions with equal sample counts? Please also clarify the dataset, experimental settings, metric calculation, and class-grouping rules used in Table 6 to validate this core challenge.

4 comments
AuthorsOpenReview ↗

Response to the weaknesses

Thanks for the valuable feedback. We hope our responses effectively address your concerns.

W1: Comparison with direct feature replay

(a) We conduct a controlled experiment on ImageNet-C, where historical tokens are directly retained and replayed during new-domain learning:

SettingDirect ReplayOursGain
Unif56.70%78.74%+22.04%
Rand60.95%80.32%+19.37%
Origin84.73%90.45%+5.72%

(1) The results show that our method consistently outperforms direct replay under both balanced and long-tailed settings, with particularly substantial improvements under long-tailed distributions. (2) These improvements arise from the proposed semantic token mining and selective replay mechanisms, which enhance the utilization of informative historical knowledge and alleviate the information scarcity faced by underrepresented classes.

(b) We further evaluate the peak replay cache usage across all 15 domains:

SettingDirect replayOurs
Unif0.700 GiB0.136 GiB
Rand0.700 GiB0.136 GiB
Origin7.891 GiB0.902 GiB

(1) Compared with direct replay, our method reduces the peak replay cache usage by 80.6% under both Unif and Rand settings and 88.6% under the Origin setting. (2) These results demonstrate that the proposed semantic token mining and selective replay mechanisms substantially reduce the storage overhead of historical knowledge while achieving superior performance.

W2: Storage overhead of the historical token memory

(a) We calculate the token storage added at each of the 15 stages. The table reports the additional cache from each stage, percentages are relative to the 358.34 MB of parameter tensors in the main model and auxiliary classifier.

StageUnif (MB)Rand (MB)Origin (MB)
14.134 (1.15%)11.212 (3.13%)15.429 (4.31%)
27.893 (2.20%)22.439 (6.26%)34.464 (9.62%)
35.223 (1.46%)19.889 (5.55%)21.849 (6.10%)
47.044 (1.97%)9.696 (2.71%)10.477 (2.92%)
53.199 (0.89%)6.281 (1.75%)7.742 (2.16%)
62.147 (0.60%)5.020 (1.40%)6.983 (1.95%)
71.320 (0.37%)5.309 (1.48%)6.103 (1.70%)
86.457 (1.80%)17.358 (4.84%)26.734 (7.46%)
95.543 (1.55%)43.107 (12.03%)57.192 (15.96%)
102.953 (0.82%)4.829 (1.35%)5.632 (1.57%)
113.494 (0.98%)11.295 (3.15%)12.904 (3.60%)
125.183 (1.45%)9.505 (2.65%)40.929 (11.42%)
130.978 (0.27%)16.481 (4.60%)19.600 (5.47%)
146.761 (1.89%)10.941 (3.05%)14.959 (4.17%)
155.663 (1.58%)10.760 (3.00%)13.965 (3.90%)

The average additional storage per stage is 4.53 MB (1.26%) under Unif, 13.61 MB (3.80%) under Rand, and 19.66 MB (5.49%) under Origin. These figures show that the storage overhead is modest.

(b) (1) Fig. 5 of our paper provides additional results without historical token storage. Specifically, Baseline+IBDM, which excludes historical token storage and replay, achieves 70.51% Avg-ACC, outperforming Dual-Prompt+CSA (68.51%), the strongest competing method in Table 1 under the same setting, by 2.00%. These results demonstrate that the proposed discriminative information mining mechanism achieves superior performance even without storing historical tokens. (2) Furthermore, incorporating IDRC (using historical token storage) improves Avg-ACC from 70.51% to 80.32%. (3) These results highlight the effectiveness of the proposed components and demonstrate that our framework can achieve competitive performance under different historical knowledge storage constraints.

W3: A more comprehensive analysis of forgetting

(a) We compared the forgetting measure (FM) of DC-Comp under the Balanced and Unif settings:

Class groupBalanced FM(%)Unif FM(%)
Head0.4050.597
Tail0.3900.253

(1) The results show that forgetting is comparable between head and tail classes under the balanced setting. However, under the long-tailed setting, head classes exhibit 136.0% more severe forgetting than tail classes. (2) Moreover, forgetting in head classes increases by 47.4% under the long-tailed setting compared with the balanced setting. (3) These results provide empirical evidence supporting our motivation that long-tailed distributions exacerbate catastrophic forgetting in head classes.

(b) We further evaluate the head-class forgetting of DC-Comp and state-of-the-art Dual-Prompt+CSA. The results are reported below.

SettingDC-Comp Head FM (%) ↓Dual-Prompt+CSA Head FM (%) ↓
Balanced0.4050.624
Unif0.5970.615
Rand0.1280.917

As a complement to Tab. 6, these results show that DC-Comp consistently exhibits lower head-class forgetting than Dual-Prompt+CSA across all three settings, demonstrating the effectiveness of our method in mitigating head-class forgetting under both balanced and long-tailed data distributions.

AuthorsOpenReview ↗

Response to the questions

Q1: Complementary mechanism

Thank you for this insightful question. We clarify that the effectiveness of our complementary mechanism stems from selectively exploiting transferable historical knowledge, rather than simply revisiting historical features.

(1) Motivation for selective knowledge complementarity: Same-class samples across domains share semantic structures and object parts, which can complement missing information for tail classes. Thus, transferring informative historical features facilitates tail-class learning.

(2) Limitations of direct replay: Historical tokens contain both class-relevant and domain-specific information. As verified in W1, replaying all historical tokens introduces accumulated domain interference, hindering new-domain learning. In contrast, our method selectively replays semantic tokens to enhance tail-class learning while mitigating cross-domain interference.

Q2: Memory bank

(a) Memory bank size: Due to character limitations, we report the memory bank size at each incremental stage across benchmarks under Unif/Rand/Origin settings in Supplementary Results. As suggested, we provide the number of stored tokens, storage overhead, and storage-to-model size ratio. The results show that the memory bank size dynamically adapts to different training data distributions, without imposing a fixed upper limit on the number of stored tokens.

(b) Storage precision: (1) Quantitative evaluation of storage precision is challenging since discriminative tokens are adaptively learned without ground-truth annotations. (2) Nevertheless, Fig. 7 of our paper visualizes the learned discriminative scores, showing that our method effectively identifies key object regions and qualitatively validates token the effectiveness of the proposed token selection mechanism.

(c) Effectiveness without memory: As shown in Fig. 5 of our paper, Baseline+IBDM represents no historical tokens is adopted, and it achieves 70.51% Avg-ACC on ImageNet-C (Rand), surpassing the strongest competitor in Table 1 by 2.00%. Besides, Historical token replay further improves Avg-ACC to 80.32% (+9.81 points), demonstrating both the effectiveness of our method without memory and the substantial benefits of selective historical replay.

Q3: Full tendency of Head-Class Accuracy and Forgetting, and details of Table 6

(a) (1) We report the results that after each stage, head-class accuracy on the newly learned domain, mean head-class accuracy over all seen domains, and mean forgetting over earlier domains for DC-Comp and DualPrompt+CSA under Balanced, Unif, and Rand on ImageNet-C. Due to the character limitation, we display the resluts in https://anonymous.4open.science/r/DC-comp-figures-Anonymized/Q3(A)_head_trajectories.pdf.

(2) The results show that the proposed DC-Comp consistently maintains an FM of no more than 0.6% across different domains. In contrast, the state-of-the-art Dual-Prompt+CSA exhibits an FM exceeding 6% on certain domains, indicating substantially more severe forgetting.

(3) These results demonstrate that DC-Comp achieves more stable and effective resistance to catastrophic forgetting than DualPrompt+CSA. This advantage can be attributed to the selective retention and replay of informative historical tokens, which help preserve previously acquired knowledge and mitigate forgetting under domain shifts.

(b) Details of Table 6.

(1) Experimental setting: Table 6 reports the results on the ImageNet-C benchmark under the Rand long-tailed setting.

(2) Evaluation metrics: The Forgetting Measure (FM) and Backward Transfer (BT) are defined as follows:

$ \mathrm{FM}=\frac{1}{T-1}\sum_{i=1}^{T-1}\left(\max_{j\in{i,\ldots,T-1}}a_{j,i}-a_{T,i}\right), $

$ \mathrm{BT}=\frac{1}{T-1}\sum_{i=1}^{T-1}(a_{T,i}-a_{i,i}), $

where TT denotes the total number of domains, and aj,ia_{j,i} represents the evaluation accuracy on domain ii after learning domain jj.

Specifically, FM measures the extent of historical knowledge forgetting throughout the domain-incremental learning process, with lower values indicating better knowledge retention. In contrast, BT measures the influence of learning new domains on previously learned domains, where higher values indicate better backward knowledge transfer.

(3) Head and tail class definitions: Classes are ranked according to their numbers of training samples. The top one-third of classes with the largest sample counts are designated as head classes, while the bottom one-third with the smallest sample counts are designated as tail classes.

We hope these responses will enable you to better understand the logic and effectiveness of our approach. Thank you again for these suggestions.

Dear Reviewer R93Y,

Thank you for your time and effort in reviewing our work. As the rebuttal period concludes in two days, we wish to respectfully confirm whether our responses have adequately addressed your initial concerns. Any feedback would be greatly appreciated.

Sincerely,

The Authors

Dear Reviewer R93Y,

We are grateful for the time and effort you have devoted to reviewing our work. As the rebuttal period closes in one day, we would like to respectfully ask whether our responses have addressed your original concerns. Any further feedback would be sincerely appreciated.

Sincerely,

The Authors

Reviewer_hk9yOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents a novel approach for long-tailed domain incremental learning approach, DC-Comp. Several modules are designed to address both catastrophic forgetting and underrepresentation of tail class.

Critical strengths
  1. the problem of long-tailed domain incremental learning is important.
Critical weaknesses
  1. Although the issue of domain shifts is claimed to be the novelty of this paper over existing works, there is no specific mechanism to address the issue of domain shifts.

  2. Novelty and contribution of this paper are limited. It just presents a variation of existing prompt-based approach for continual learning which has been well-established in the literature.

  3. Although LTDIL is different from LTCIL, why is no comparison provided with Wang et al (2024b). LTCIL approach can be easily modified for LTDIL.

Questions for authors

address three major concerns above.

4 comments
AuthorsOpenReview ↗

Response to the weaknesses

Thank you for your constructive feedback and recognition. We hope the following responses address your concerns.

W1: Domain shifts

(a) We clarify that domain shift is an inherent challenge of the LDIL setting, rather than a separate technical novelty claimed by DC-Comp.

(b) Domain shift is primarily addressed by the proposed Inter-domain Discriminativity Reserving and Complementing (IDRC) module. (i) Specifically, IDRC preserves discriminative object semantics from historical domains and generates pseudo-samples during new-domain learning, thereby mitigating catastrophic forgetting caused by domain shifts. Moreover, since the core semantic objects are relatively invariant across domains, the preserved knowledge can also enhance the learning of new domains. (ii) As shown in Fig. 5 of our paper, incorporating IDRC yields a 9.81% improvement in Avg-ACC, verifying its effectiveness in addressing domain shifts.

W2: Novelty

(a) Baseline statement. Building upon an established framework is a common practice in continual learning and does not inherently diminish methodological novelty. For example, CODA-Prompt [Ref-1] extends the prompting paradigm introduced by L2P [Ref-2], while C²Prompt [Ref-3] builds upon CODA-Prompt. Therefore, the novelty of a work is usually evaluated based on its proposed technical contributions rather than whether it adopts an existing backbone.

(b) Limitations of existing methods. Existing prompt-based approaches, including KA-Prompt (our baseline in Tab. 1), CODA-Prompt, L2P, C-Prompt, and Dual-Prompt (our baselines in Fig. 8,9,10), are primarily designed for balanced data streams. Consequently, their ability to handle LDIL data remains limited.

(c) Our technical contributions. As shown in Fig. 3, the proposed framework introduces 3 novel mechanisms to address long-tailed data streams: (1) Discriminativity Estimation Module (DEM) employs a lightweight network to estimate the class-relevant discriminative confidence of individual tokens, enabling the identification of informative semantic features. (2) Intra-class Balanced Discriminativity Mining (IBDM) leverages the estimated token-level confidence to perform class-frequency-aware, multi-round sampling, providing underrepresented classes with more opportunities to acquire informative representations. (3) Inter-domain Discriminativity Reserving and Complementing (IDRC) selectively preserves discriminative historical tokens identified by DEM and constructs class-proportion-balanced pseudo-samples to complement current-domain learning.

Together, these mechanisms enable discriminative information identification, balanced knowledge acquisition, and selective cross-domain knowledge transfer, distinguishing DC-Comp from existing prompt-based approaches.

(d) Empirical evidence of novelty and generalizability. Under the Unif setting of the DomainNet benchmark, DC-Comp outperforms the KA-Prompt baseline by 6.40% (Tab. 1). Furthermore, integrating our proposed mechanisms into CODA-Prompt, L2P, C-Prompt, and Dual-Prompt achieves improvements of 3.9%, 3.7%, 12.1%, and 6.1%, respectively, under the same setting (Fig. 8). Similar improvements on ImageNet-R and ImageNet-C are observed in Figs. 9–10. These consistent improvements demonstrate that our contributions are generalizable to different baselines, supporting their effectiveness and broader applicability.

[Ref-1] CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learning. CVPR 2023.

[Ref-2] Learning to Prompt for Continual Learning. CVPR 2022.

[Ref-3] C²Prompt: Class-aware Client Knowledge Interaction for Federated Continual Learning. NeurIPS 2025.

W3: Comparison with Wang et al. (2024b)

(1) Reason for the omission. Wang et al. (2024b) was not included in our experimental comparisons because its source code has not been publicly released, making reliable reproduction difficult.

(2) Comparison with LTCIL methods. (i) In Tab. 1, we have already included a representative long-tailed class-incremental learning (LTCIL) method, DGR (CVPR 2024). On the DomainNet benchmark, DC-Comp outperforms DGR by 14.61% and 7.48% in Avg-ACC under the Unif and Rand settings, respectively. (ii) These improvements can be attributed to the different knowledge utilization strategies. While DGR does not explicitly exploit cross-domain shared semantic knowledge, our DC-Comp selectively mines and leverages such knowledge to facilitate both new-domain learning and historical knowledge retention. (iii) These results demonstrate the effectiveness of explicitly modeling cross-domain knowledge complementarity in the challenging LDIL setting.

We hope the above clarifications are helpful. Besides, we welcome further discussion and are grateful for the opportunity to improve our work through your insights. Thank you again for your time and consideration.

Reviewer_hk9yOpenReview ↗

Partially address my concern

Some of your clarification addresses my concerns but there are some points that remain unresolved.

  1. the proposed method doesn't sufficiently extend prior works, L2P. The query and key matching step is taken from the original L2P paper making the contribution rather incremental.

  2. the issue of domain shift remains unsolved. How does the LT-DIL differ from LT-CIL in the context of technical challenges? Is your method applicable for LT-CIL? My understanding of Section 3.4 should be applicable to LT-CIL as well and not designed specifically to handle the issue of domain shift. Why is your method not compared in the LT-CIL setting happening to be more mature than LT-DIL?

I agree to increase my score to weak reject.

Thank you again for the continued recognition of our work and for the further suggestions. Below we provide additional clarifications in response to each point.

Q1: The extension over L2P

(1) We agree that DC-Comp inherits query–key prompt matching from L2P. (2) However, retaining this retrieval rule while contributing elsewhere has precedent in continual learning. (i) CoEn [a] uses L2P’s key–query matching to select prompts, then assesses their transferability and dynamically manages the prompt pool. (ii) KAC [b] uses L2P as one of its baselines and replaces only its linear classifier, leaving the original prompt retrieval and hyperparameters unchanged. (iii) More directly, PGP [c] explicitly builds on L2P and keeps the cosine-similarity selection rule in Section 3.2, Eq. 15. (3) These examples show that retaining L2P’s retrieval rule does not, by itself, make a method’s contribution limited. In DC-Comp, our contributions lie in discriminativity estimation, intra-class balanced discriminativity mining, and inter-domain discriminativity reserving and complementing.

[a] Dynamically Managing a Prompt Pool via Self-Enhancement in Continual Learning. NeurIPS 2024

[b] KAC: Kolmogorov-Arnold Classifier for Continual Learning. CVPR 2025

[c] Prompt Gradient Projection for Continual Learning. ICLR 2024

Q2: Details under the LT-CIL setting

(a) LT-DIL vs. LT-CIL: (1) In LT-CIL, the classes introduced at different stages have relatively weak semantic connections. Methods therefore tend to learn each stage largely independently, with less emphasis on transferring knowledge between classes and relatively less forgetting caused by inter-stage interference. (2) In LT-DIL, by contrast, the same classes recur across domains, but their data distributions can differ substantially. Changes in backgrounds and object appearance introduce interference, with background variation in particular aggravating forgetting. (3) To address this cross-domain interference, our discriminative representation selection module identifies key object representations. It preserves useful historical information while filtering out distracting features, thereby improving anti-forgetting performance in LT-DIL.

(b) Adaptation to the LT-CIL Datasets: (1) Main modules of DC-Comp can also be applied to LT-CIL because they primarily learn from newly introduced data. Specifically, IBDM’s class-balanced feature sampling alleviates class imbalance among new samples. (2) We therefore used APART[d] as the LT-CIL baseline, incorporated the main DC-Comp modules into its framework, and evaluated the resulting method on an LT-CIL dataset, CIFAR100-LT, under the Ordered and Shuffled B50-5 and B50-10 settings. We report average incremental accuracy (Acc) and final accuracy (Accₜ).

MethodOrdered B50-5 (Acc/Accₜ)Ordered B50-10 (Acc/Accₜ)Shuffled B50-5 (Acc/Accₜ)Shuffled B50-10 (Acc/Accₜ)
DualPrompt74.80 / 60.7277.88 / 65.8469.00 / 63.0873.75 / 67.45
CODA-Prompt75.36 / 59.0780.44 / 67.4676.45 / 70.2979.39 / 74.40
APART84.65 / 74.3081.78 / 68.3385.52 / 81.8382.96 / 79.15
APART+DC-Comp86.92 / 78.8385.50 / 75.6385.67 / 82.9584.61 / 81.93
Improvement+2.27 / +4.53+3.72 / +7.30+0.15 / +1.12+1.65 / +2.78

(1)As shown above, incorporating DC-Comp into the APART baseline yields consistent improvements ranging from 0.15/1.12 to 3.72/7.30 in Acc/Accₜ metrics across four LT-CIL settings. (2)These results demonstrate the effectiveness and generalizability of DC-Comp in LT-CIL scenarios, highlighting its potential to enhance existing continual learning frameworks.

[d] Adaptive Adapter Routing for Long-Tailed Class-Incremental Learning. Machine Learning 2025

(c) In practical scenarios, such as autonomous driving, new-domain samples of previously known classes continue to emerge. We therefore focus on learning under domain shifts and design the Inter-domain Discriminativity Reserving and Complementing (IDRC) module specifically for this setting. Accordingly, our experiments evaluate the method in LT-DIL rather than LT-CIL.

We hope these further clarifications prove useful. Thank you again for your time and attention.

Dear Reviewer hk9y,

Thank you for your time and attention in reviewing our work, and for the deeper concerns you have raised about our paper. As the rebuttal period closes in less than a day, we wish to respectfully confirm whether our further responses have sufficiently resolved your remaining concerns. Any reply would be greatly appreciated.

Sincerely,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.