- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors proposed DASS-GNN framework that efficiently update an already trained GNNs (GCN or GraphSAGE) given a small batch of edge insertion and deletion operations, without retraining from scratch. The framework has two parts: 1) caching the first-hop neighbor propagation using the current model and only update for nodes related to the changed edges, and 2) a masked gradient update that trains only the top rows with the highest energy in the left singular subspace of each weight matrix. Experiments on 5 real-world static benchmarks, with simulated edge insertion and deletion operations, the results show proposed framework gives the performance close to full retraining, and give a good trade-off among three axes: accuracy gain, speedup, and spectral drift of the weight matrix.
Critical strengths
- The proposed backward selective update is novel in the usage in the context of GNNs training, framework is simple and easy to implement.
- The proposed framework is evaluated on 2 backbones, 2 tasks, and can scale to a large graph with 2.4M nodes and 61M edges, with meaningful ablation study to separate the effect of propagation caching and SVD-guided update.
- Code is released.
Critical weaknesses
-
The novelty is limited, the source of speedup is not clearly shown:
-
Same idea of selective update using leverage score is proposed in this work [1].
-
In backward pass, the main bottleneck is computing the gradient of the weights rather than how many rows are updated, and it seems the proposed framework assumes the gradients are given for free as the input of the algorithm 2.
-
The reported time is unclear if it is per-epoch or total adaptation time, and should be reported separately for forward and backward passes for clearly showing where is the source of speedup come from, or the speedup may come from warm-start fine-tuning, which clearly requires fewer epochs to converge than training from scratch. Also different learning rates for retraining and proposed methods (with other baselines) may also affect the convergence speed.
-
-
The SVD-guided update does not show a clear benefit:
-
The SVD-guided update shows insignificant improvement (within std) in predictive performance compare to Cached-AX/Cached-SAGE on all datasets with slightly higher time cost for the SVD computation.
-
The authors compare D_SVD (5.1.1) but it seems favor the proposed method because freezing 90% of the rows limit weight change by construction. Also, singular values difference does not measure representation stability or directly measure the predictive power.
-
No experiment compares SVD-based row energy selection with random or gradient-norm row selection.
-
The baselines are limited, closely related work on updating pre-trained GNNs on stream of edges (ContinualGNN [2]) are not discussed, and other DGNNs are mentioned but not compared.
-
[1] Damirchi, Hamed, et al. "The quest for winning tickets in low-rank adapters." arXiv preprint arXiv:2512.22495 (2025).
[2] Wang, Junshan, et al. "Streaming graph neural networks via continual learning." Proceedings of the 29th ACM international conference on information & knowledge management. 2020.
Questions for authors
- Is the reported time per epoch or the total update time? Please report forward, backward, and SVD time separately, per epoch.
- Please report the "Original Model" from table 2, and a warm-start fine-tuning baseline without caching or masking, using the same learning rate as the other adaptive methods. Why does Retrain use a different learning rate?
- Please compare SVD-based row selection with random and gradient-norm row selection at the same rank.
- EvolveGCN-O results vary a lot (table 3, 50.81 ± 21.02 on G1, 28.49 ± 7.96 on G2), did these runs converge?
- I wonder if better (more stable) DGNN models like DySAT, TGN can match the performance of the proposed framework.
- Please give more details in the experiment setup and hyperparams: train/val/test splits, the test edges and negative sampling for link prediction, the number of layers, the SVD rank, the LoRA rank.