acceptodds
Under review as a conference paper at ICLR 2027

Dynamic GNN Adaptation via Cached Propagation and SVD-Guided Selective Optimization

Abstract

Graph neural networks offer a potent mechanism for learning on graph data across domains and tasks. A vast majority of these mechanisms assume that the underlying graph and the corresponding node/edge attributes are static, and do not change over time. However, graphs corresponding to most real-world phenomena evolve via the addition or deletion of nodes and edges, or changes to node/edge attributes. This paper proposes efficient solutions for maintaining the GNN over such evolving graphs without recourse to full retraining. Our solution use cached propagation for the forward passes and a spectral value decomposition based selective optimization for the backward passes. We adapt our solution to two representative GNN architectures, Graph Convolution Networks (GCNs) and Graph Sample and Aggregate (SAGE) over two downstream tasks: node classification and link prediction. Extensive experimental evaluation indicates that our solutions offer competitive accuracy and speedup over existing dynamic GNN methods while keeping the spectral drift small.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Dynamic GNNs, Cached propagation, SVD-guided parameter updates, Node classification, Link prediction, GCN, GraphSAGE

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_R2VnOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors proposed DASS-GNN framework that efficiently update an already trained GNNs (GCN or GraphSAGE) given a small batch of edge insertion and deletion operations, without retraining from scratch. The framework has two parts: 1) caching the first-hop neighbor propagation using the current model and only update for nodes related to the changed edges, and 2) a masked gradient update that trains only the top rr rows with the highest energy in the left singular subspace of each weight matrix. Experiments on 5 real-world static benchmarks, with simulated edge insertion and deletion operations, the results show proposed framework gives the performance close to full retraining, and give a good trade-off among three axes: accuracy gain, speedup, and spectral drift of the weight matrix.

Critical strengths
  • The proposed backward selective update is novel in the usage in the context of GNNs training, framework is simple and easy to implement.
  • The proposed framework is evaluated on 2 backbones, 2 tasks, and can scale to a large graph with 2.4M nodes and 61M edges, with meaningful ablation study to separate the effect of propagation caching and SVD-guided update.
  • Code is released.
Critical weaknesses
  • The novelty is limited, the source of speedup is not clearly shown:

    • Same idea of selective update using leverage score is proposed in this work [1].

    • In backward pass, the main bottleneck is computing the gradient of the weights rather than how many rows are updated, and it seems the proposed framework assumes the gradients are given for free as the input of the algorithm 2.

    • The reported time is unclear if it is per-epoch or total adaptation time, and should be reported separately for forward and backward passes for clearly showing where is the source of speedup come from, or the speedup may come from warm-start fine-tuning, which clearly requires fewer epochs to converge than training from scratch. Also different learning rates for retraining and proposed methods (with other baselines) may also affect the convergence speed.

  • The SVD-guided update does not show a clear benefit:

    • The SVD-guided update shows insignificant improvement (within std) in predictive performance compare to Cached-AX/Cached-SAGE on all datasets with slightly higher time cost for the SVD computation.

    • The authors compare D_SVD (5.1.1) but it seems favor the proposed method because freezing 90% of the rows limit weight change by construction. Also, singular values difference does not measure representation stability or directly measure the predictive power.

    • No experiment compares SVD-based row energy selection with random or gradient-norm row selection.

    • The baselines are limited, closely related work on updating pre-trained GNNs on stream of edges (ContinualGNN [2]) are not discussed, and other DGNNs are mentioned but not compared.

[1] Damirchi, Hamed, et al. "The quest for winning tickets in low-rank adapters." arXiv preprint arXiv:2512.22495 (2025).

[2] Wang, Junshan, et al. "Streaming graph neural networks via continual learning." Proceedings of the 29th ACM international conference on information & knowledge management. 2020.

Questions for authors
  1. Is the reported time per epoch or the total update time? Please report forward, backward, and SVD time separately, per epoch.
  2. Please report the "Original Model" from table 2, and a warm-start fine-tuning baseline without caching or masking, using the same learning rate as the other adaptive methods. Why does Retrain use a different learning rate?
  3. Please compare SVD-based row selection with random and gradient-norm row selection at the same rank.
  4. EvolveGCN-O results vary a lot (table 3, 50.81 ± 21.02 on G1, 28.49 ± 7.96 on G2), did these runs converge?
  5. I wonder if better (more stable) DGNN models like DySAT, TGN can match the performance of the proposed framework.
  6. Please give more details in the experiment setup and hyperparams: train/val/test splits, the test edges and negative sampling for link prediction, the number of layers, the SVD rank, the LoRA rank.
Reviewer_HhbZOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes DASS-GNN, a framework for adapting graph neural networks to batches of edge insertions and deletions. It combines cached first-hop propagation with SVD-guided selective optimization, which updates weight-matrix rows selected using scores derived from the leading left singular vectors. The framework is instantiated with GCN and GraphSAGE and evaluated on node classification and link prediction across five datasets using synthetic edge updates. The authors report reduced adaptation time relative to retraining from scratch, generally competitive predictive performance, and reduced singular-value drift in several settings.

Critical strengths
  • The paper addresses a relevant practical problem: reducing the cost of updating GNNs as graph structure changes. Reusing unaffected feature aggregations is a reasonable approach, and the reported speedups on large graphs suggest practical potential.
  • The framework separates propagation caching from selective parameter optimization, making the contributions of these components amenable to independent evaluation.
  • The experiments cover two GNN backbones, two downstream tasks, and datasets of different sizes. Including caching-only ablations is particularly useful for assessing the incremental contribution of SVD-guided optimization.
Critical weaknesses

Theorem 3 claims that selecting rows using the left singular vectors of the weight matrix preserves a corresponding fraction of gradient energy. However, weight-derived row scores do not generally determine the distribution of gradient energy. Section 4.1 constructs evolving graphs from static datasets using an initial edge ratio of 0.7 and uniformly sampled edge insertions and deletions.

Questions for authors

What additional assumptions are required for Theorems 3 and 4? Can the statements and proofs be revised to address the counterexample and the unsupported gradient-norm bound? Under a matched update budget and training schedule, does SVD-guided selection offer benefits over random row selection? What practical interpretation should be attached to the reduction in singular-value drift? Can you provide results on the dynamic dataset like Wikipedia or Reddit?

Reviewer_vBohOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the problem of efficient GNN adaptation on evolving graphs without full retraining. The authors propose DASS-GNN, a framework that combines cached graph propagation with SVD-guided selective parameter optimization to reduce computational overhead while maintaining predictive performance and spectral stability.

Critical strengths

S1. The problem of efficient GNN adaptation under dynamic graph updates is important for practical applications.

S2. The proposed framework addresses redundant computation in both forward propagation and backward optimization.

Critical weaknesses

W1. The proposed method selects parameter rows based on their SVD row energy, but it remains unclear why rows with higher energy should be prioritized for updating rather than preserved to maintain the existing model's representations. Moreover, since the selection criterion does not explicitly account for graph updates or their induced gradients, it is unclear what makes this strategy particularly suitable for dynamic GNN adaptation beyond general parameter-efficient fine-tuning.

W2. The experiments simulate graph evolution through randomly sampled edge insertions and deletions on static graphs, without preserving realistic temporal dependencies. Given the availability of established temporal graph benchmarks such as TGB and TGB-Seq, which provide real-world interaction streams and standardized future link prediction protocols, I suggest evaluating the proposed method on these benchmarks to better assess its effectiveness under realistic graph evolution and clarify the link prediction evaluation protocol.

W3. The experimental evaluation relies heavily on IncreGNN (2022) as the incremental GNN baseline and lacks comparisons with more recent advances in incremental GNN computation and efficient dynamic graph learning. I suggest the authors discuss relevant recent methods, such as [1] [2], and include additional comparisons where the task settings are compatible.

[1] RIPPLE++: An Incremental Framework for Efficient GNN Inference on Evolving Graphs

[2] CacheDyG: Decoupling Temporal Propagation for Efficient Dynamic Graph Learning

Questions for authors

The paper applies gradient masking after computing the full gradient, making it unclear how SVD-guided selective optimization reduces backpropagation costs in practice. Could the authors clarify whether a sparse optimization implementation is used and report the overhead of SVD computation? Additionally, given the different learning rates and numbers of training epochs across baselines, could the authors clarify how the efficiency comparisons are controlled to ensure a fair evaluation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.