acceptodds
Under review as a conference paper at ICLR 2027

Invariant Structural Distillation via Graph Condensation for History-Aware Graph Continual Learning

Abstract

Graph Continual Learning (GCL) tackles the stability-plasticity dilemma: models must retain historical knowledge while rapidly adapting to evolving graph distributions under restricted observability. Recent graph condensation methods improve stability by compressing past task data into compact, high-fidelity memory. We show that fidelity-focused condensation can lead to Temporally Illusory Invariance: memories that overfit task-specific spurious motifs, which appear stable within a task but fail to transfer as the graph structure evolves. Meanwhile, real-world graph streams often remain structurally connected across tasks. Current-task nodes may still interact with locally observable but unlabeled historical neighbors, even when raw historical graphs and labels are no longer accessible. We formalize this realistic setting as history-aware restricted observability. Exposing historical neighbors creates a tension: it may provide additional context, but it also introduces cross-task neighborhood mixing, which can blur class boundaries and amplify the brittleness of shortcut-based decision rules, ultimately propagating bias and degrading model robustness. To address these challenges, we propose ISD-GC (Invariant Structure Distillation for Graph Condensation), which reframes the memory buffer as a transferable structural prior rather than a fidelity-optimized snapshot, following a Connect-then-Condense principle. First, ISD-GC performs memory-grounded context disambiguation through Virtual Soft Edges, which construct relative evidence between historical memory prototypes and current-task anchors. A calibrated context discriminator further identifies historical candidates and distinguishes beneficial, irrelevant, and harmful context for selective message passing. Second, ISD-GC condenses each task into structure-invariant prototypes by distilling cross-view consistent structural information across complementary graph views and stochastic structural perturbations. This approach suppresses brittle motif reliance and improves transferability under structural drift. Extensive experiments on multiple class-incremental GCL benchmarks, supported by a controlled Motif Trap diagnostic, demonstrate that ISD-GC significantly enhances long-term retention, robustness, and memory-budget efficiency under strict memory budgets, particularly in realistic history-aware settings.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Graph Continual Learning, Graph Invariant Learning, Graph Neural Network, Graph Machine Learning, Graph Condensation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_bqLTOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This manuscript presents the ISD-GC. The core idea is that small-capacity memory in continuous learning should preserve knowledge that is more robust to subsequent structural changes, rather than just trying to reproduce past graphs. To this end, they first filter the useful historical neighbors in the current graph, and then compress the relatively stable category information under multiple structural views into a small number of prototypes.

Critical strengths
  1. The authors further consider a practical factor in the field of graph continual learning: new nodes may still be connected to nodes that appeared in the past. Thus, the currently visible information includes "the new task node, the labeled training samples of the new class, and the one-hop historical neighbors without labels."

  2. This manuscript addresses the central question of "what memory should preserve" and relates historical context screening to archetypal condensation.

  3. The experiment chooses the graph with a large number of nodes, which is more meaningful in the real world scenario.

Critical weaknesses
  1. The readability of the manuscript needs to be improved. As far as I can see, this paper is not a purely theoretical approach, so I think a model figure that can explain the algorithm flow, data division, and key components is necessary, otherwise it will significantly reduce the readability and reading efficiency of the paper.

  2. The evidence for "invariance" is weak, and the invariance is more based on assumptions. Is there any real visualization or theoretical analysis that can show that the invariance is real in the current dataset?

  3. The ablation gain of some datasets is very small. For example, Reddit's full model only improves marginally compared to closing both stages, which is difficult to support the strong statement that "both stages are indispensable".

  4. The novelty of the manuscript should also be further explained. It seems to me that your approach is a combination of condensation-based methods and distillation-based methods.

Questions for authors

Refer to the weakness section.

1 comment
AuthorsOpenReview ↗

Response to Reviewer bqLT

A1: To improve the readability of the manuscript, we have added an overall framework figure that provides a visual overview of ISD-GC as shown in figure D3 in the revised manuscript.

A2: To provide more direct empirical evidence, we have extended the original Figure 1 (Figure 2 in the revised manuscript) by incorporating four diagnostic metrics for ISD-GC: DiagfocusDiag_{focus},mfocus→ctrlm_{focus\rightarrow ctrl}, c2ic2i, and i2ci2c.

The results show that ISD-GC generally exhibits intermediate diagnostic behavior between PUMA and the full-history Oracle. Compared with PUMA under the same memory budget, ISD-GC maintains higher within-focus correctness, lower leakage to control classes, and less pronounced induced–conflict error transitions. These observations provide empirical evidence that ISD-GC mitigates shortcut-induced prediction drift under structural changes, although such effects are not completely eliminated.

Importantly, we interpret structural invariance in an operational sense, as robustness to structural variations rather than a guarantee of strictly invariant representations. The additional visualization supports this interpretation by demonstrating more stable classification behavior under the controlled Motif Trap setting.

A3: We observe that the contributions of the two stages vary across datasets. As shown in Tables 3 and 8, the full model improves AvgAcc over the variant with both stages disabled by 2.312 percentage points on CoraFull and 1.017 points on ogbn-products, while the gains are more modest on ogbn-arxiv and Reddit. In particular, the variant without either stage already achieves 97.972% AvgAcc on Reddit, leaving limited room for further improvement. Nevertheless, the full model improves LastAcc from 98.042% to 99.059%, demonstrating better final-stage performance despite the nearly saturated AvgAcc. Importantly, Stage I and Stage II address complementary aspects of graph continual learning: Stage I mitigates interference from historical context, while Stage II improves the transferability of condensed memory. Their relative contributions depend on the characteristics of the underlying graph streams.

We have therefore revised the statement "both stages are indispensable" to "two complementary stages" and refined the corresponding discussion to reflect their dataset-dependent contributions.

A4: The novelty of ISD-GC stems from its problem-driven methodological design for addressing two interconnected challenges in evolving graph continual learning: cross-task interference arising from observable historical neighbors and the consolidation of task-local structural shortcuts into condensed memory. To address these challenges, we develop memory-grounded context identification and structure-invariant prototype condensation.

Importantly, the two stages interact through a memory-mediated feedback mechanism. Stage I leverages previously condensed prototypes to identify and type historical context, guiding the construction of structural views for Stage II. Stage II then incorporates these views and historical-memory-based temporal evidence to construct transferable condensed prototypes, which in turn provide historical references for subsequent tasks.

We have strengthened the presentation of these problem-driven contributions in the revised manuscript to more clearly articulate the novelty of ISD-GC and its distinction from existing condensation- and distillation-based methods.

Reviewer_ip5dOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper studies class-incremental graph continual learning with condensation-based replay, in a setting where current-task nodes can see unlabeled one-hop neighbors from earlier tasks. It argues that fidelity-driven condensation stores time-local structural shortcuts ("Temporally Illusory Invariance") and that exposed historical neighbors cause cross-task neighborhood mixing, and it illustrates both with a synthetic Motif Trap and an inter-task-edge study on CoraFull. The proposed ISD-GC first detects historical neighbors with a prototype-based teacher and a distilled discriminator and types them as beneficial, irrelevant, or harmful, then condenses each task into prototypes matched to three structural views with a variance-based per-dimension weight. Experimental results show that ISD-GC outperforms selected baselines on CoraFull, ogbn-arxiv, ogbn-products, and Reddit under both a history-aware and an isolated protocol.

Critical strengths
  1. The Motif Trap is a clean, controlled diagnostic of shortcut consolidation.
  2. The experiments cover four standard datasets under two observation protocols.
Critical weaknesses
  1. The experiments do not support the motivation of the paper

The paper's own tables contradict the cross-task interference premise, the Motif Trap results do not isolate the claimed invariance effect, and the ablations attribute little of the gain to the two proposed stages.

  1. The experiments are unfair

The baselines are outdated, and the released implementation differs from the described method and is evaluated with a test-time step that uses the ground-truth task index.

Questions for authors
  1. Lines 80 to 81 and 214 to 215 state that exposing cross-task neighbors generally degrades performance, but the main tables show the opposite: unmodified PUMA reaches 81.930 AvgAcc on ogbn-products with history (Table 1) and 73.985 without it (Table 5), and Joint-train is higher with history on all four datasets. ISD-GC's margin over PUMA is also larger under the isolated protocol on ogbn-arxiv and ogbn-products, where Stage I has nothing to filter. CGLB (Zhang et al., 2022a), which defines the same protocol, likewise finds the net effect of inter-task edges dataset-dependent. How is the premise reconciled with these results?
  2. In this protocol, which nodes are historical is given by arrival order (Appendix D.2.2), yet Stage I learns a detector for them and its accuracy is never reported. Per Algorithm 1, the typed masks only build the condensation views and the predictor is retrained on self-loop prototypes, so where does the "selective message passing" of the abstract (lines 32 to 33) happen? What are the precision and recall of the historical-candidate mask and of the beneficial/harmful typing?
  3. All Stage II views are derived from the same step-t graph (Section 3.2.1), so a cue that is stable within the task window is present in every view. How can cross-view consistency separate time-local cues from cues that transfer across tasks?
  4. If u_{c,i} were unconstrained, the gate g_{c,i} in Eq. 3 would not change the minimizer: for each dimension, the optimum is the λ-weighted mean of the view targets, whatever g is. The gate acts only through the unit-norm and encoder-range constraints, so unstable dimensions are still fitted to their cross-view mean rather than suppressed, contrary to lines 342 to 344. What does a no-gate ablation show?
  5. Table 2 reports only aggregate accuracy, and the trap still costs ISD-GC about as much as PUMA (78.984 to 63.008 for ISD-GC, 74.939 to 56.010 for PUMA, under ORI). On ogbn-products ORI, the trap even raises PUMA (70.932 to 73.781) but lowers ISD-GC (80.784 to 76.955, Table 6). What are Diag_focus, m_focus→ctrl, c2i, and i2c (Figure 1) for ISD-GC?
  6. Lines 17 to 18 claim that fidelity-focused condensation leads to Temporally Illusory Invariance, but Section 2.2 compares only PUMA with a full-history oracle, and the oracle "shows the same qualitative transition signature" (line 172). How does a sampling-based memory such as ER-GNN behave under the same budget?
  7. In Tables 3 and 8, the variant without either stage already outperforms PUMA on all four datasets (for example 75.725 vs 74.779 on ogbn-arxiv), and adding both stages moves ogbn-arxiv only to 75.846. What is this variant, how is "Stage I only" run when Stage I only produces Stage II's views, and where does its gain over PUMA come from?
  8. Appendix B defines the induced and conflict sets with CoraFull class IDs (conflict classes 48 to 60), which do not exist in ogbn-arxiv and Reddit (40 classes) or ogbn-products (46 classes, Table 4), yet Table 6 reports trap results on them. How were these settings built? The released trap generator also does not implement Appendix B: it returns the original graph before any injection, its triangle-closing edges are commented out, and it attaches leaves only to nodes of selected tasks.
  9. The paper describes no test-time procedure for the history-aware graph. The released implementation reports ISD-GC accuracy after zeroing the features of every non-training node whose first prediction falls outside the classes of the task being evaluated, which uses that task's ground-truth index; the replay baselines use a variant with extra edge pruning, and the regularization baselines use a plain forward pass. Were Tables 1 and 5 produced this way, and what are the results when every method uses the same plain forward pass?
  10. The released Stage II also differs from Eq. 3 and Appendix F.2: it adds a target built from historical-memory embeddings and a variance-of-risk penalty across views, uses 100 randomly re-initialized teacher encoders instead of the frozen f_ref (line 322), and condenses pseudo-labeled test nodes. Which version produced the reported tables, and what does each addition contribute?
  11. The newest compared methods are CaT and PUMA, both first released in 2023, and Tables 2 and 6 compare only with PUMA. Published methods for this setting are missing: DeLoMe [1], TACO [2], TEM [3], Peng et al. (2025), SSRM [4], which targets the structural shift caused by inter-task edges, TPP [5], GSIP [6], DSLR [7], and OpenGC [8], which condenses evolving graphs for temporal invariance. How does ISD-GC compare with them under the same budget?

[1] Chaoxi Niu, Guansong Pang, and Ling Chen. Graph continual learning with debiased lossless memory replay. In ECAI, 2024.

[2] Xiaoxue Han, Zhuo Feng, and Yue Ning. A topology-aware graph coarsening framework for continual graph learning. In NeurIPS, 2024.

[3] Xikun Zhang, Dongjin Song, Yixin Chen, and Dacheng Tao. Topology-aware embedding memory for continual learning on expanding networks. In KDD, 2024.

[4] Junwei Su, Difan Zou, Zijun Zhang, and Chuan Wu. Towards robust graph incremental learning on evolving graphs. In ICML, 2023.

[5] Chaoxi Niu, Guansong Pang, Ling Chen, and Bing Liu. Replay-and-forget-free graph class-incremental learning: A task profiling and prompting approach. In NeurIPS, 2024.

[6] Jialu Li, Yu Wang, Pengfei Zhu, Wanyu Lin, and Qinghua Hu. What matters in graph class incremental learning? An information preservation perspective. In NeurIPS, 2024.

[7] Seungyoon Choi, Wonjoong Kim, Sungwon Kim, Yeonjun In, Sein Kim, and Chanyoung Park. DSLR: Diversity enhancement and structure learning for rehearsal-based graph continual learning. In WWW, 2024.

[8] Xinyi Gao, Tong Chen, Wentao Zhang, Yayong Li, Xiangguo Sun, and Hongzhi Yin. Graph condensation for open-world graph learning. In KDD, 2024.

13 comments
AuthorsOpenReview ↗

Response to Reviewer Reviewer ip5d

A1: We would like to clarify that cross-task connectivity has a dual effect: it may introduce cross-class neighborhood mixing and interfere with classification, but it may also provide potentially useful historical information. Its net effect depends on how the historical context is utilized. The statements in Lines 80–81 and 214–215 refer to the potential interference caused by naively incorporating cross-task neighbors, rather than suggesting that cross-task connectivity is invariably harmful.

  1. The results in Tables 1 and 5 incorporate a lightweight two-pass test-time refinement, rather than a plain forward pass. The motivation is the distribution mismatch between prototype-based training and history-aware inference. After each task, the predictor is retrained solely on the accumulated prototype memory. However, at test time, the input graph contains newly evaluated nodes together with unlabeled historical neighbors and their inter-task edges. This mismatch may introduce additional prediction interference.

    Specifically, we first perform a forward pass to obtain initial predictions. We then zero the features of every non-training node whose first prediction falls outside the classes of the task being evaluated, apply an additional edge-pruning operation, and perform a second forward pass.

    This procedure is applied to all methods, including Joint-train. Under this test-time refinement protocol, introducing historical context improves AvgAcc for Joint-train on all four datasets and for PUMA and ISD-GC on CoraFull, ogbn-arxiv, and ogbn-products. For example, on ogbn-products, PUMA improves from 73.985 to 81.930, while ISD-GC improves from 77.607 to 83.261. Reddit is an exception for PUMA, whose AvgAcc decreases slightly from 97.995 to 97.009, while ISD-GC remains almost unchanged (98.006 vs. 98.009).

  2. To directly examine the effect of cross-task connectivity without test-time refinement, we additionally report results using a plain forward pass, without prediction-based feature masking or additional edge pruning, in Table S1. For example, on ogbn-products, introducing historical context reduces PUMA's AvgAcc from 74.422 to 65.075 and ISD-GC's from 81.047 to 69.088. This contrast demonstrates that unfiltered cross-task connectivity can introduce substantial interference, whereas appropriate test-time refinement helps reduce such interference and exploit potentially useful historical information.

  3. Regarding Joint-train's higher performance under the history-aware protocol, its results provide evidence for the beneficial side of cross-task connectivity. It is important to clarify that Joint-train is evaluated on different input graphs under the two protocols. Specifically, Table 1 uses the history-aware graph with all inter-task edges retained, whereas Table 5 uses the isolated graph with inter-task edges removed. Joint-train achieves higher AvgAcc under the history-aware protocol on all four datasets, with improvements of 1.789, 2.815, 0.925, and 0.265 percentage points on CoraFull, ogbn-arxiv, ogbn-products, and Reddit, respectively. Since Joint-train retains all historical training data and labels, it can effectively exploit potentially useful relational information introduced by inter-task connectivity. This further confirms that cross-task connectivity can provide beneficial information under appropriate supervision, consistent with our motivation that its effect depends on the balance between useful historical information and cross-task interference.

  4. Regarding ISD-GC's larger performance margin over PUMA under the isolated protocol, although Stage I has no historical candidates to filter when inter-task connectivity is removed, Stage II remains active. Its structure-invariant prototype condensation exploits complementary structural views and stochastic perturbations to preserve transferable structural information. Specifically, ISD-GC improves AvgAcc over PUMA by 4.426 percentage points on ogbn-arxiv and 3.622 percentage points on ogbn-products under the isolated protocol. This confirms that the improvement is not solely attributable to historical-context identification, but also comes from structure-invariant prototype condensation.

  5. Finally, Our observations are consistent with CGLB's finding that the net effect of inter-task edges is dataset-dependent. Accordingly, we have refined the statements in Lines 80–81 and 214–215 to clarify that the impact of cross-task connectivity depends on the balance between beneficial historical information and cross-task interference.

Table S1. Comparison of AvgAcc (%) under different test-time inference protocols.

(a) With a lightweight two-pass test-time refinement

DatasetMethodHistory-awareIsolated-control
ogbn-productsPUMA81.9373.985
ISD-GC83.26177.607
Joint86.25785.332
RedditPUMA97.00997.995
ISD-GC98.00998.006
Joint98.72298.457
ogbn-arxivPUMA74.77969.508
ISD-GC75.84673.934
Joint75.86173.046
CoraFullPUMA78.38377.722
ISD-GC81.54981.176
Joint82.91981.130

(b) With a plain forward pass

DatasetMethodHistory-awareIsolated-control
ogbn-productsPUMA65.07574.422
ISD-GC69.08881.047
Joint65.51871.493
RedditPUMA93.87598.171
ISD-GC96.68198.759
Joint97.64098.215
ogbn-arxivPUMA65.45469.607
ISD-GC66.45673.988
Joint67.67070.111
CoraFullPUMA68.53077.993
ISD-GC74.00381.047
Joint61.34481.266
AuthorsOpenReview ↗

Response to Reviewer Reviewer ip5d [Q2]

**A2: **

  1. Regarding historical-node identification, arrival order is used by the data-construction protocol to simulate the realistic observability in evolving graph streams, but is not provided to the model as node-level historical identity information. Only current-task training nodes have available labels, while the remaining observed nodes have no explicit historical identity, task-ID, or arrival-time indicators. Therefore, Stage I must identify historical candidates from observable graph information and condensed memory. Importantly, our purpose is not merely to recover historical identities, but to reduce cross-task interference while preserving historical information that benefits current-task classification.

  2. Regarding selective message passing, it refers to the selective utilization of historical context during representation extraction and prototype condensation, not message passing in the final predictor. Specifically, Stage I uses the typed masks to selectively gate node features and incident edges, constructing the Disambiguated and History-aware views. In Stage II, the teacher encoder performs message passing over these structural views to extract representations, which are subsequently distilled into learnable prototype features. Although the final predictor is retrained on independent self-loop prototypes, these prototypes have already incorporated the selectively extracted structural information during condensation. Thus, selective message passing is performed during structural representation extraction and prototype condensation, rather than during final predictor training. We have clarified this distinction in the revised manuscript Line31-34.

  3. Regarding the precision and recall of historical-candidate identification and beneficial/harmful typing, we emphasize that accurately identifying historical nodes does not necessarily guarantee performance improvement. Historical identity does not directly indicate task-specific utility: even correctly identified historical nodes may contain irrelevant or harmful information. Therefore, our objective is not simply to recover historical node identities, but to preserve beneficial historical information while suppressing harmful interference. Specifically, VSE provides relative evidence between historical memory prototypes and current-task anchors, while subsequent utility-based typing distinguishes beneficial, irrelevant, and harmful context. To provide a more informative evaluation, we report task-wise precision and recall for historical-candidate identification, together with the corresponding context-typing statistics in following table. For clarity and space efficiency, the table presents results at every other task (Tasks 2, 4, ... , 18) in the 20-task Reddit stream, while the reported average performance is computed over the full task sequence.

    Specifically, the average precision and recall are approximately 89.7% and 54.8%, respectively, indicating that Stage I prioritizes high-confidence historical candidates rather than exhaustive identification. Furthermore, approximately 17.6% of the detected historical candidates are classified as harmful by the utility-based typing mechanism. This illustrates that ISD-GC selectively suppresses potentially interfering historical information rather than indiscriminately removing historical context.

    task2task4task6task8task10task12task14task16task18
    precision0.7940.2960.9970.9990.9921.0000.9970.9990.999
    recall0.0450.0070.3610.8160.5550.7500.8030.7270.866
    harmful0.3880.2000.0370.1720.1570.2220.0560.1430.208
Reviewer_L6nSOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ISD-GC, a graph continual learning framework under history-aware restricted observability. It identifies two challenges: temporally illusory invariance, where condensed memories preserve spurious structural patterns, and cross-task neighborhood mixing caused by observable historical nodes. To address these issues, ISD-GC introduces memory-grounded context disambiguation to selectively utilize historical neighbors and structure-invariant prototype distillation to improve memory transferability. Experiments on four graph benchmarks demonstrate promising improvements in continual learning performance under limited memory budgets.

Critical strengths
  1. Interesting problem formulation and motivation. The paper investigates the limitations of fidelity-oriented graph condensation under structural shifts and introduces the Motif Trap diagnostic to illustrate how spurious structural patterns can affect continual learning.

  2. Well-motivated methodological design. The proposed framework combines historical context identification with multi-view structure-invariant prototype distillation. The two components are designed to address distinct but related challenges in history-aware graph continual learning.

  3. Comprehensive experimental evaluation. The experiments cover four graph benchmarks, including large-scale datasets, and evaluate both history-aware and task-isolated settings. The paper also includes comparisons with representative baselines, memory-budget analysis, and ablation studies.

Critical weaknesses

I am aware of the reported anonymity issue but have not investigated the individual mentioned or their background, nor do I have any prior knowledge of them. The following assessment is made independently of such information.

1. Poor readability and presentation of figures and tables. Figures 1–4 are difficult to interpret due to excessively small fonts and unclear visualization. Table 1 also lacks clear categorization of general CL methods, graph CL methods, naive baselines, and joint training as a full-data reference. Furthermore, ISD-GC results are highlighted in bold even when Joint-training performs better (e.g., AvgAcc on ogbn-products and Reddit), contradicting the caption's claim that the best results are highlighted. The authors should substantially improve the clarity, organization, and accuracy of the presentation.

2. Insufficient evidence supporting the claimed Temporally Illusory Invariance. Figure 1 compares PUMA with Oracle, but their different access to historical data makes it difficult to attribute the observed degradation specifically to graph condensation. Moreover, the experimental evidence is not clearly connected to the central motivation and technical contributions. Could the authors provide more direct evidence for the claimed failure mechanism and better justify the proposed solutions?

3. Missing computational efficiency evaluation. No empirical runtime or computational cost comparison is provided. Could the authors report the overall training time, peak memory consumption, and stage-wise computational costs, particularly compared with CaT and PUMA? Given the modest improvements on some datasets, it is important to determine whether the additional computational overhead is justified.

4. Unclear necessity and reliability of the context identification module. Stage I first estimates historical affinity using Virtual Soft Edges (VSE), then uses the resulting pseudo-labels to train an additional context discriminator at each task. However, the effectiveness of this identification process is not independently evaluated. Why is an additional discriminator necessary instead of directly using the VSE scores? How accurately can it distinguish historical nodes from newly arriving nodes, particularly when their representations overlap?

Questions for authors

See weaknesses.

1 comment
AuthorsOpenReview ↗

Response to Reviewer L6nS

A1: We sincerely appreciate the reviewer's constructive feedback. We have made corresponding revisions to improve the clarity and presentation of the figures and tables. Specifically, we have replaced Figures 1–4 with clearer versions, provided a clear categorization of the baseline methods in Appendix H.2, and clarified the role of Joint-training as an upper-bound reference baseline. We have also explicitly stated in the caption of Table 1 that bold results indicate the best performance excluding Joint-training.

A2:We have extended the original Figure 1 (Figure 2 in the revised manuscript) by adding the four diagnostic metrics for ISD-GC: DiagfocusDiag_{focus},mfocus→ctrlm_{focus\rightarrow ctrl}, c2ic2i, and i2ci2c. The results show that ISD-GC generally exhibits intermediate diagnostic behavior between PUMA and the full-history Oracle. Compared with PUMA, ISD-GC maintains higher within-focus correctness, lower leakage to control classes, and less pronounced induced-conflict error transitions. These observations suggest that structure-invariant condensation mitigates shortcut-induced information-flow drift under the same bounded-memory setting, although it does not completely eliminate such effects.

A3: We sincerely appreciate the reviewer's valuable suggestion. We have reported the overall training time, peak memory consumption, and stage-wise computational costs of CaT, PUMA, and ISD-GC in the following table.

It is shown that ISD-GC achieves a model training time of 93.73 s and a peak memory consumption of 9759.80 MiB, reducing memory usage by approximately 12.7% compared with PUMA and 13.5% compared with CaT. ISD-GC introduces additional computational overhead during Stage II (memory construction), which takes 594.61 s, compared with 64.75 s for CaT and 234.54 s for PUMA. This additional cost is primarily incurred by constructing structure-invariant prototypes across complementary structural views and optimizing their cross-view consistency. This process enables the condensed memory to suppress task-specific structural correlations while retaining class-consistent and transferable structural information. The resulting prototypes provide more reliable replay knowledge for subsequent tasks, contributing to improved final performance and reduced temporal fluctuations. Therefore, the additional computational cost is a deliberate trade-off for higher-quality memory construction and more effective knowledge transfer across tasks.

MetricCaTPUMAISD-GC
Training time (s)239.1996.2193.73
Peak memory consumption (MiB)11278.4711173.609759.80
Stage I\mathrm{Stage\ I}: Historical context discovery (s)——0.34
Stage II\mathrm{Stage\ II}: Memory construction (s)64.75234.54594.61

A4: Regarding the necessity and reliability of the context discriminator, we have further clarified the motivation outlined in Lines 264–266 of the original manuscript and expanded the corresponding explanation in the revised manuscript.

Specifically, VSE provides a relative historical-affinity score, but this score alone does not fully capture the prediction and neighborhood-consistency information relevant to historical-context identification. Therefore, we use high-confidence VSE scores as pseudo-positive supervision and labeled current-task nodes as reliable negatives to train a lightweight context discriminator. The discriminator integrates these complementary signals and employs an anchor-calibrated quantile threshold to identify historical candidates without manually tuning a fixed threshold for each task.

We further emphasize that accurate historical-node identification is an intermediate objective rather than the ultimate goal of Stage I. Historical identity alone does not determine task-specific utility: even correctly identified historical nodes may provide beneficial, irrelevant, or harmful information for current-task classification. Therefore, our framework further performs utility-based context typing to preserve beneficial historical information while suppressing potentially harmful interference.

When historical and current-task representations overlap, their identities may remain inherently ambiguous. To mitigate unreliable identification, we adopt a conservative fallback when insufficient high-confidence historical candidates are available, avoiding potentially harmful context suppression. We have clarified these mechanisms in the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.