- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies heterogeneous continual graph learning. The authors argue that standard continual learning methods are insufficient for HGNNs because historical knowledge is encoded not only in target-node representations but also in their typed neighborhood contexts and relation-specific semantic interactions. To address this, they propose HERO. The authors provide theoretical analysis decomposing replay error into target-selection and context-reconstruction terms, and evaluate HERO on four benchmarks with three HGNN backbones.
Critical strengths
S1. The paper clearly identifies why standard replay and distillation fall short for HGNNs: preserving isolated target nodes does not reconstruct the typed neighborhoods and relation-specific message passing that produced historical predictions.
S2. HERO integrates three complementary mechanisms, DiSCo, HKD, and meta-learning-based adaptatiom, into a unified pipeline. The design is systematically ablated, and the interaction between retention (replay + distillation) and plasticity (meta-adaptation) is well-structured.
S3. The experiments span four heterogeneous benchmarks and three representative HGNN backbones. The authors evaluate under both standard and reordered task sequences, report multiple metrics (AP, AF, FaG), and compare against seven strong baselines from the continual graph learning literature.
Critical weaknesses
W1. HERO is evaluated under Task Incremental Learning, where the task identity is provided at inference. This is a strong assumption that sidesteps the core challenge of distinguishing among all previously seen classes without explicit task boundaries. The more realistic and widely adopted Class-IL setting is not considered.
W2. Despite claiming competitive performance, HERO falls substantially behind the Joint Train upper bound on more complex graphs. The ablation study (Table 2) reveals that removing the gradient-based meta-learning module (G-MM) causes negligible degradation on DBLP (−1.0 AP) and Yelp (−0.3 AP), with only modest increases in forgetting.
W3. While Propositions 1 and 2 decompose replay error into selection and context terms, the analysis is largely standard: Proposition 1 is a routine smoothness bound on gradient updates, and Proposition 2 is an almost tautological observation that relation-sensitive HGNNs depend on relations. The theory serves as post-hoc motivation rather than a prescriptive foundation.
W4. HKD's semantic-level alignment relies on metapath-aware or relation-aware attention coefficients, which are intrinsic to attention-based HGNNs (e.g., HAN, HGT). For backbones without explicit attention mechanisms, such as RGCN, the paper proposes an ad-hoc surrogate attention based on node similarity (Appendix A.3), which is not part of the original architecture.
Questions for authors
(1) evaluate under Class-IL without task identifiers,
(2) justify or streamline the meta-learning component given its inconsistent benefits,
(3) strengthen the theoretical analysis with more actionable bounds,
(4) validate HKD on non-attention HGNNs or reframe it as an architecture-specific technique.
5 comments
W3/Q3. Theoretical analysis and actionable bounds. We would like to clarify that our theoretical analysis provides a mechanism-level characterization of heterogeneous replay errors, directly motivating the two-stage design of DiSCo. Beyond Propositions 1 and 2, Appendix A.1 (Corollary 1) provides an explicit upper bound on context-induced replay-gradient error:
\leq \frac{L_g L_\sigma H}{|S|} \sum_{v\in S}\sum_{r\in\mathcal R} \|W_r\|_2\,\Delta_r(v;G,\widetilde G),$$ where $\Delta_r$ captures omitted relation-specific messages and changes in aggregation weights. This bound identifies concrete sources of gradient distortion and motivates preserving relation-aware heterogeneous neighborhoods during replay. To further establish the connection between the theoretical analysis and our practical design, we additionally evaluated empirical gradient-error proxies on IMDB. Stage 1 (target-selection) reduces the target-selection error by 24.9% compared with random target selection, while Stage 2 (relation-aware expansion) reduces the context-reconstruction error by 5.1% compared with target-only replay. The complete DiSCo also achieves the lowest overall approximation error. | Replay Strategy | Rel. $\epsilon_{\mathrm{sel}}$ ↓ | Rel. $\epsilon_{\mathrm{ctx}}$ ↓ | Rel. $\epsilon_B$ ↓ | | :--- | :---: | :---: | :---: | | CM + Target-only | 0.776 ± 0.040 | 1.027 ± 0.244 | 1.455 ± 0.246 | | Random + DiSCo Expansion | 1.004 ± 0.186 | 0.982 ± 0.247 | 1.403 ± 0.160 | | **DiSCo (Ours)** | **0.754 ± 0.074** | **0.975 ± 0.161** | **1.366 ± 0.158** | *Table: Empirical gradient-error proxy comparison. Lower is better.* Overall, the theoretical bound and empirical results provide a more direct evidence. **W4/Q4. Generalizability of HKD to HGNNs without attention mechanisms.** We would like to clarify that HKD is not restricted to attention-based HGNN architectures. As introduced in **Appendix A.3**, for HGNNs without explicit attention mechanisms, semantic responses can be constructed from hidden representations and aligned between teacher and student models, enabling semantic-level distillation beyond attention-based architectures. To further validate this generalizability, we conducted additional experiments using RGCN[1], a non-attention HGNN backbone. As shown below, HERO achieves the highest AP across all three datasets, outperforming strong baselines. | Method | DBLP AP (%) ↑ | IMDB AP (%) ↑ | Freebase AP (%) ↑ | | :--- | :---: | :---: | :---: | | Finetune | 74.77 ± 1.10 | 74.64 ± 1.42 | 64.62 ± 0.65 | | EWC | 76.05 ± 1.72 | 74.77 ± 2.68 | 65.53 ± 0.27 | | MAS | 80.50 ± 1.00 | 76.54 ± 1.10 | 64.47 ± 0.56 | | TWP | 80.41 ± 2.66 | 77.17 ± 1.92 | 66.95 ± 2.39 | | ER-GNN | 80.91 ± 1.40 | 76.73 ± 1.46 | 63.80 ± 0.57 | | FTF-ER | 79.65 ± 1.42 | 72.89 ± 3.77 | 64.20 ± 0.33 | | DMSG | 71.45 ± 0.82 | 71.63 ± 8.51 | 62.79 ± 1.71 | | MetaCLGraph | 79.94 ± 0.91 | 76.53 ± 0.74 | 66.73 ± 2.96 | | **HERO** | **82.18 ± 1.25** | **77.81 ± 1.10** | **69.20 ± 1.39** | *Table: Performance comparison using RGCN as the backbone.* Together with the extension described in Appendix A.3, these results provide further evidence that HERO can generalize beyond attention-based HGNNs. [1] Modeling Relational Data with Graph Convolutional NetworksDear Reviewer,
Thank you again for your thoughtful and constructive feedback. We have carefully addressed your comments in our point-by-point rebuttal and provided additional experimental results to further support our contributions.
We would greatly appreciate it if you could take a look at our responses. We would be happy to provide further clarification or address any remaining questions.
Thanks again for your time.
We thank the reviewer for the constructive suggestions. We address all questions point by point below.
W1/Q1. Evaluation under Class-Incremental setting (Class-IL). To further evaluate HERO in a more realistic and challenging scenario, we conducted additional experiments under Class-IL where task boundaries are unavailable at inference. As shown below, HERO achieves the best AP across all four datasets, outperforming the strongest baselines up to 6.8 percentage points. These results further demonstrate HERO's effectiveness in preserving historical knowledge and adapting to new classes without task boundaries.
W2/Q2. Comparison with Joint Train and the role of G-MM. We would like to clarify that joint train runs under an offline training setting rather than the continual learning protocol followed by HERO and other baselines. Specifically, Joint Train has unrestricted access to data from all tasks and trains on them jointly, whereas continual learning methods learn tasks sequentially with limited memory budget. Therefore, as stated in Section 4(Experiment Setup), joint train serves as an approximate upper bound rather than a directly comparable cl baseline.
Regarding G-MM, we would like to clarify that G-MM plays a complementary role within HERO. While DiSCo and HKD preserve historical structural and semantic knowledge, G-MM improves adaptation to incoming tasks. Specifically, it performs look-ahead adaptation using representative nodes and evaluates the updates against the overall objective. Gains are consistent across different datasets. As shown in Table 2, incorporating G-MM consistently improves AP across all four datasets, while also reducing forgetting.