acceptodds
Under review as a conference paper at ICLR 2027

Forget Less, Generalize More: Unifying Temporal and Structural Adaptation for Dynamic Graphs

77 views

Abstract

Representation learning on dynamic graphs requires capturing complex dependencies that evolve across both time and structure. Existing approaches typically adopt fixed temporal decay schemes or predetermined structural propagation depths, limiting their ability to generalize across graphs with diverse interaction frequencies and topological characteristics. We propose Dual-Scale Retentive Dynamics (DSRD), a unified framework that maintains a retentive representation state encoding both temporal memory and structural context. DSRD introduces two key components: (i) a retentive state with dual-scale adaptation that jointly models temporal dynamics and structural propagation within a single recurrent formulation, and (ii) adaptive decay kernels with learnable time-sensitivity parameters that automatically balance short-term responsiveness and long-term retention based on the underlying temporal and structural interaction patterns. We provide theoretical analysis that expands the recurrent state updates into weighted aggregation over temporal walks, together with boundedness guarantees for the learned dynamics. Extensive experiments on 14 real-world benchmarks demonstrate that DSRD achieves state-of-the-art performance on both link prediction and node classification tasks, with strong generalization across transductive and inductive settings. The implementation code is available at [[Code]](https://anonymous.4open.science/r/DSRD).

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Dynamic Graph, Graph Neural Networks, Representation Learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_FL5DOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

To address the challenge in dynamic graph learning of balancing short-term changes with long-term dependencies, as well as the common issue of overlooking interaction sequence, this paper proposes the DSRD method. Its core idea is to integrate the retention of historical information with message passing between nodes, dynamically adjusting the influence of historical context based on interaction characteristics while preserving chronological sequence. This enables a better understanding of how relationships evolve over time. Experiments on multiple real-world datasets demonstrate that DSRD generally outperforms most baseline methods in link prediction and node classification tasks, all while maintaining good computational efficiency.

Critical strengths

The paper jointly models temporal and structural dependencies in dynamic graphs by integrating adaptive temporal retention and high-order structural diffusion within a shared retentive state. Learnable decay kernels and gated state updates allow the model to balance short-term responsiveness and long-term retention. The experimental evaluation is also solid, covering link prediction on multiple real-world dynamic graph datasets under both transductive and inductive settings with three negative sampling strategies, where DSRD demonstrates consistently strong performance.

Critical weaknesses

Despite the extensive experimental evaluation, the additional benefits of the core design choices over existing methods remain insufficiently established. The ablation studies support the usefulness of individual components, but more targeted comparisons are needed to isolate the gains from learnable decay kernels and from coupling temporal retention with structural diffusion within a unified retentive state.

Questions for authors

How much does DSRD benefit from learning the decay parameters compared with using well-tuned fixed values?

1 comment
AuthorsOpenReview ↗

Response to Reviewer FL5D

Thank you for the constructive review. It is encouraging that you consider the joint modeling of temporal retention and structural diffusion a strength and the experimental evaluation solid. Our point-by-point responses to the Questions (Q) follow. All changes in the revised paper are highlighted in blue.


Q1. How much does DSRD benefit from learning the decay parameters compared with using well-tuned fixed values?

RQ1: We appreciate this insightful question. To quantify the contribution of adaptive decay, we evaluated two controls that keep the architecture and decay functions unchanged. Fixed decay fixes γ(ℓ)\gamma^{(\ell)}, λ\lambda, α\alpha and δ(ℓ)\delta^{(\ell)}, while fixed γ\gamma only fixes γ(ℓ)\gamma^{(\ell)} and learns the remaining decay terms. Constants are selected by validation AP via grid search. The all-fixed search includes different time scales across layers and heads. All results below are inductive AP under random negative sampling.

DatasetFixed decayFixed γ onlyDSRD
Wikipedia98.37±0.01‾\underline{98.37{\pm}0.01}98.33±0.0499.55±0.10
Reddit98.39±0.03‾\underline{98.39{\pm}0.03}98.23±0.0399.94±0.02
MOOC83.54±0.27‾\underline{83.54{\pm}0.27}83.54±0.17‾\underline{83.54{\pm}0.17}95.75±0.66
Enron83.69±0.63‾\underline{83.69{\pm}0.63}82.53±0.9493.78±1.10
UCI94.57±0.11‾\underline{94.57{\pm}0.11}94.59±0.1792.59±0.05
US Legis.74.67±0.99‾\underline{74.67{\pm}0.99}63.77±0.8784.42±1.03
UN Trade88.33±0.9988.57±1.56‾\underline{88.57{\pm}1.56}99.23±0.07

DSRD is higher on all datasets except UCI, where the fixed controls are better. Tuning all decay constants is clearly stronger than fixing γ\gamma alone on US Legis. (+10.90%). Careful tuning can therefore help, but its accuracy is bounded by the grid resolution and its cost grows with the number of decay parameters, as our all-fixed search already requires 15 training runs per dataset. DSRD learns these parameters end-to-end in a single training run and exceeds the tuned fixed decay by more than 9% on MOOC, Enron, US Legis. and UN Trade.

Q2. More targeted comparisons are needed to isolate the gains from coupling temporal retention with structural diffusion within a unified retentive state.

RQ2: We are grateful for this thoughtful suggestion. To test the benefit of coupling directly, we added a separate state control whose temporal recurrence never reads the structurally propagated state. A second state bank stores structural propagation, and the two output embeddings are averaged. Projection and feed-forward weights are shared. We also evaluated temporal-only recurrence without diffusion. Each architecture is tested with learned decay and with the same fixed constants selected in RQ1.

DatasetTemporal learnedTemporal fixedSeparate learnedSeparate fixedDSRD (full)
Wikipedia98.34±0.00‾\underline{98.34{\pm}0.00}98.28±0.0398.34±0.02‾\underline{98.34{\pm}0.02}97.93±0.2599.55±0.10
Reddit98.23±0.04‾\underline{98.23{\pm}0.04}98.12±0.0898.22±0.0397.73±0.2699.94±0.02
MOOC83.59±0.38‾\underline{83.59{\pm}0.38}83.53±0.1583.28±0.2383.18±0.2895.75±0.66
Enron82.10±1.5583.16±0.96‾\underline{83.16{\pm}0.96}81.16±1.0481.89±0.8693.78±1.10
UCI94.74±0.2494.57±0.1294.63±0.19‾\underline{94.63{\pm}0.19}94.42±0.0992.59±0.05
US Legis.52.19±2.1649.83±3.2257.03±0.6259.80±1.97‾\underline{59.80{\pm}1.97}84.42±1.03
UN Trade79.37±2.83‾\underline{79.37{\pm}2.83}61.93±5.8575.69±4.8571.77±2.7099.23±0.07

DSRD (full) outperforms the temporal-only and separate-state variants on all datasets except UCI. Compared with the separate-state variant, the gains are largest on long-span datasets (e.g., US Legis. +27.39%, UN Trade +23.54%, Enron +12.62%, MOOC +12.47%), while the one-month Wikipedia and Reddit differ by less than 2%. Under the same fixed decay, the shared-state model (Fixed decay in RQ1) also exceeds the separate-state and temporal-only controls, by up to +16.56% and +26.40% on UN Trade. Appendix C.9 shows that DSRD learns strong long-term retention on long-span datasets, so structural context helps most when it is retained in the same state over long horizons.

In addition, separate states double the persistent-state storage (144.19 → 288.38 MiB on Wikipedia) and require an additional query projection, state readout and feed-forward pass per layer. Overall, coupling temporal retention and structural diffusion within one shared state, as in DSRD, is both more accurate and more efficient than keeping them in separate states. We have added the analyses in RQ1 and RQ2 to Section 4.4 and Appendix C.12 of the revised paper.


We appreciate your helpful comments. Please do not hesitate to let us know if you have any questions.

Reviewer_e5zDOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

In this work, the authors present a dynamic graph learning framework named Dual-Scale Retentive Dynamics (DSRD), that captures long-term as well as short-term temporal and structural context. The model is primarily based on learning retentive states (second order states) through recurrent adaptive scale update equation coupled with structural diffusion. By introducing learnable decay kernels and gated memory updates, DSRD adaptively balances short-term responsiveness and long-term retention. Through numerical experiments the authors show that these retentive states are competitively viable alternatives for existing memory-based models with limited capabilities in modeling long-term information.

Critical strengths

Good empirical performance: The paper demonstrates strong results across 14 diverse real-world benchmarks, achieving state-of-the-art performance on both dynamic link prediction and node classification tasks in both transductive and inductive settings. It obtains an impressive average rank of 1.29 across link prediction benchmarks and shows strong generalization across both discrete- and continuous-time graphs.

Critical weaknesses

Missing long range models: The paper addresses temporal trade-offs and structural conflation but omits recent state-space and continuous-time models, such as CTAN [1], CTDG-SSM [2], and GraphSSM [3]. CTAN and CTDG-SSM use continuous ODE updates to handle short-term and long-term dependencies. Omitting these baselines creates a major gap in the evaluation.

Adaptive temporal Scaling: Given the missing latest baselines, the technical novelty of learnable scale is limited. Recent models like GraphSSM and CTDG-SSM use input-dependent adaptive scaling via selective scans, where decay is modeled as exp⁡(−λΔθ(x(t)))\exp(−\lambda \Delta_\theta (x(t))) using softplus parameterization. In contrast, DSRD uses a feature-agnostic decay parameter γ(l)∈(0,1),\gamma^{(l)} \in (0,1), meaning each layer globally favors either short-term or long-term memory.

Technical issues: The paper defines instantaneous injection in Eq 2, note here that this term explicitly limits the the injection to only one-hop interaction, i.e., if there is no direct link between nodes jj and ww, then Δtw=0\Delta_t^w = 0 . Further, from Eq. 4, one cannot model the influence of all nodes that are ll hops away, but only of those that have hop path as well as direct single hop path. This limited propagation is not seen in the implementation because the propagation is replaced through single-hop aggregation of the node state SS instead of injection in Eq. 10.

Minor issues, readability: Could the authors please clarify the definition of the “instantaneous transition matrix”? Given that Et\mathcal{E}_t represents all edges in the interval [0,t][0, t], it is slightly ambiguous whether the induced transition matrix incorporates all historical edges up to tt or only those appearing at the exact timestamp. Assuming the latter is intended, it would greatly improve clarity to make this distinction explicit in the paper.

[1] Gravina, Alessio, et al. “Long range propagation on continuous-time dynamic graphs.” arXiv preprint arXiv:2406.02740 (2024). [2] Raghuvanshi, Ayushman, et al. “Learning Long Range Spatio-Temporal Representations over Continuous Time Dynamic Graphs with State Space Models.” arXiv preprint arXiv:2606.04672 (2026). [3] Li, Jintang, et al. “State space models on temporal graphs: A first-principles study.”Advances in Neural Information Processing Systems 37 (2024): 127030-127058.

Questions for authors

address weakness

3 comments
AuthorsOpenReview ↗

Response to Reviewer e5zD [Part 1/2]

We sincerely appreciate the detailed assessment and your recognition of DSRD's strong empirical performance across multiple real-world benchmarks. We address each of the Questions (Q) below. All changes in the revised paper are highlighted in blue.


Q1. Missing baselines CTAN, CTDG-SSM and GraphSSM.

RQ1: We added CTAN, CTDG-SSM and GraphSSM, together with CRAFT. DSRD achieves the best performance on every dataset, as shown below.

DatasetCRAFTCTANCTDG-SSMGraphSSMDSRD
Wikipedia95.71±0.2975.45±0.6996.04±1.40‾\underline{96.04{\pm}1.40}86.75±0.7199.55±0.10
Reddit97.18±0.0571.68±2.2397.56±0.18‾\underline{97.56{\pm}0.18}91.79±0.9499.94±0.02
MOOC79.30±1.80‾\underline{79.30{\pm}1.80}58.33±3.0974.97±0.9771.87±0.8395.75±0.66
Enron82.45±0.4164.49±2.1082.46±3.84‾\underline{82.46{\pm}3.84}73.95±1.1593.78±1.10
UCI91.00±0.21‾\underline{91.00{\pm}0.21}75.76±0.6679.88±7.0683.35±1.7592.59±0.05
US Legis.51.10±3.2956.34±0.2948.49±0.4762.65±1.42‾\underline{62.65{\pm}1.42}84.42±1.03
UN Trade51.38±0.6762.60±2.0464.29±0.6472.57±5.35‾\underline{72.57{\pm}5.35}99.23±0.07

These baselines make our evaluation more comprehensive, and we have incorporated them into Tables 6 and 7 of the revised paper. We are grateful for the pointers.

Q2. γ(l)\gamma^{(l)} is feature-agnostic, whereas recent models use input-dependent decay.

RQ2: We are grateful for this insightful comparison. Indeed, γ(ℓ)\gamma^{(\ell)} does not depend on the input. In DSRD, as in RetNet [1], input-dependent selection is performed by the matrix-valued retentive state, where the query–key weight in Eq. (6) decides what is written and the readout QtjStj,(ℓ)Q_t^j S_t^{j,(\ell)} decides what is retrieved. Selective-scan models such as GraphSSM and CTDG-SSM keep a vector state, so an input-dependent step size is their main means of selection, while γ\gamma in DSRD only sets how fast stored content fades. Since γ\gamma is learned separately for each head (γh,(ℓ)\gamma^{h,(\ell)}), heads in the same layer can still retain short-term and long-term memory simultaneously. We agree that the decay form is just part of the contribution of DSRD, which lies in coupling temporal retention and multi-hop structural propagation within one shared state (Eq. (10)).

Following your suggestion, we implemented an input-dependent γ\gamma in the selective-scan form described in your comment. Omitting layer and head indices, node jj uses $ \gamma_j=\gamma^{,\mathrm{softplus}(w^{\top}X_j+b)}, $ where XjX_j is the normalized layer input of node jj, and γj\gamma_j replaces γ\gamma in Eqs. (7) and (9). This leads to exp⁡(−λΔθ(Xj))\exp(-\lambda\Delta_\theta(X_j)) with λ=−log⁡γ\lambda=-\log\gamma and Δθ(Xj)=softplus(w⊤Xj+b)\Delta_\theta(X_j)=\mathrm{softplus}(w^{\top}X_j+b). We compare this variant with a tuned fixed γ\gamma under the same training protocol:

DatasetTuned fixed γInput-dependent γDSRD
Wikipedia98.33±0.0498.36±0.02‾\underline{98.36{\pm}0.02}99.55±0.10
Reddit98.23±0.0398.24±0.03‾\underline{98.24{\pm}0.03}99.94±0.02
MOOC83.54±0.1783.67±0.23‾\underline{83.67{\pm}0.23}95.75±0.66
Enron82.53±0.9482.57±1.41‾\underline{82.57{\pm}1.41}93.78±1.10
UCI94.59±0.17‾\underline{94.59{\pm}0.17}94.74±0.2392.59±0.05
US Legis.63.77±0.8767.60±12.13‾\underline{67.60{\pm}12.13}84.42±1.03
UN Trade88.57±1.5689.96±0.66‾\underline{89.96{\pm}0.66}99.23±0.07

Compared with the tuned fixed γ\gamma, the input-dependent γ\gamma changes AP by less than 0.5% on five of the seven datasets. Its gains on UN Trade and US Legis. are measured against a fixed γ\gamma and thus also include the benefit of learning γ\gamma, which DSRD already has. DSRD, with a learned but input-independent γh,(ℓ)\gamma^{h,(\ell)}, achieves the highest AP on six of the seven datasets. We have added this discussion to Section 3.1 and the results to Section 4.4 and Appendix C.12 of the revised paper.


Reference

[1] Sun, Yutao, et al. “Retentive network: A successor to transformer for large language models.” arXiv preprint arXiv:2307.08621 (2023).


(Please see Part 2 for the remaining responses.)

AuthorsOpenReview ↗

Response to Reviewer e5zD [Part 2/2]

(Continued from the previous comment.)


Q3. The one-hop injection in Eq. 2 limits multi-hop propagation in Eq. 4 and differs from Eq. 10.

RQ3: Thank you for pointing this out, and we apologize for the indexing mistake in Eq. (2). We have revised the definition of the injection in Section 2.2 to make it clear. The injection is collected locally at each node, and the walk diffusion then routes it to non-adjacent nodes. The corrected Eq. (2) is

Δtj=∑(i,j,te)∈Etω(t,(i,j,te)) (Ktei)⊤Vtei.\Delta_t^j=\sum_{(i,j,t_e)\in\mathcal E_t}\omega(t,(i,j,t_e))\,(K_{t_e}^i)^{\top}V_{t_e}^i.

In Δˉtj,(ℓ)=∑iAt(ℓ)(i,j)Δti\bar\Delta_t^{j,(\ell)}=\sum_i A_t^{(\ell)}(i,j)\Delta_t^i, each interaction collected at node ii therefore contributes to jj whenever a temporal walk of length ℓ\ell that starts after it connects them, without requiring a direct edge. In the implementation, Eq. (8) builds the local injections and Eq. (10) passes the accumulated states further. Proposition A.4 gives the walk expansion of this update.

For example, in A→t1B→t2CA\xrightarrow{t_1}B\xrightarrow{t_2}C with t1<t2t_1<t_2, BB first absorbs information from AA, and its state later carries it to CC, although AA and CC are not adjacent. With a unit signal placed at the start of such a chain, the terminal-state norm is 0.177 after two hops and 0.044 after three, and exactly zero when the order is reversed, the first edge is removed, or propagation is disabled.

Q4. Does TtT_t include all edges up to tt or only those at tt?

RQ4: We appreciate the careful reading. The transition matrix TtT_t incorporates only the edges appearing at the exact timestamp tt, not all historical edges up to tt, i.e., Tt(u,v)T_t(u,v) counts the events (u,v,t)(u,v,t). Et\mathcal E_t still denotes all events up to tt. History enters through At−(ℓ)A_{t^-}^{(\ell)}, and At(ℓ)=At−(ℓ)+At−(ℓ−1)TtA_t^{(\ell)}=A_{t^-}^{(\ell)}+A_{t^-}^{(\ell-1)}T_t extends previous walks with current events. Following your suggestion, we have made this distinction explicit in the revised definition of TtT_t in Section 2.2.


We are grateful again for your thoughtful review and would welcome any additional questions or suggestions.

Reviewer_e5zDOpenReview ↗

Thanks for the new set of experiments

I thank the authors for the clarification and the new set of experiments. Although the contributions and novelty are still limited, given the experimental value that this submission might add I will slightly increase my score.

Reviewer_x7Q1OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper focuses on representation learning on dynamic graphs. The innovative point is that the temporal decay and the structural propagation depths are not determined before model training, which enables model adaption to various graphs. The experiments demonstrate the superiority of the proposed method.

Critical strengths

S1. The motivation is reasonable that the temporal decay and the structural propagation depths should not be fixed. S2. The model design is reasonable and achieves great performance across several datasets.

Critical weaknesses

W1. The datasets included in the experiments are limited. Several challenging benchmarks are proposed in 2024-2025, such as TGB[R0] and TGB-Seq[R1]. And these benchmarks also point out that MRR is a better measurement for link prediction than AP and AUROC. W2. Several strong baselines are not evaluated, such as CRAFT [R2].

Additional results on these benchmarks and against these baselines would strengthen the empirical evidence, particularly if the proposed method maintains its advantages under these settings. If not, I encourage the authors to provide a more detailed analysis of which dataset characteristics favor the proposed method, and under what conditions the method may be less effective.

[R0] Temporal Graph Benchmark for Machine Learning on Temporal Graphs [R1] TGB-Seq Benchmark: Challenging Temporal GNNs with Complex Sequential Dynamics [R2] Future Link Prediction Without Memory or Aggregation

Questions for authors

Table 3 indicates that none of the 14 datasets provides node features. How are the input node features xj(t)x_j(t) in Eq. (1) constructed and initialized?

1 comment
AuthorsOpenReview ↗

Response to Reviewer x7Q1

We are grateful for your careful review and glad that you found our motivation reasonable, the model design sound and its performance strong. Below, we respond to each of the Questions (Q) point by point. All changes in the revised paper are highlighted in blue.


Q1. The datasets included in the experiments are limited. Several challenging benchmarks are proposed in 2024-2025, such as TGB[R0] and TGB-Seq[R1]. And these benchmarks also point out that MRR is a better measurement for link prediction than AP and AUROC.

RQ1: Thank you for this valuable suggestion. We agree that MRR on TGB and TGB-Seq gives a more informative evaluation of link prediction, and we considered it when designing our experiments. Unfortunately, our computing resources are limited, and MRR evaluation is computationally demanding, since it ranks every positive edge against a large set of negative candidates and most TGB and TGB-Seq datasets are large. We could therefore evaluate MRR only to a limited extent, and adopted the widely used DyGLib benchmark for our main experiments. Following your suggestion and within this budget, we evaluated DSRD and four representative baselines on two small TGB datasets, tgbl-review and tgbl-wiki, and DSRD achieves the best test MRR on both:

DatasetTGNPINTDyGFormerTPNetDSRD
tgbl-review34.91±2.1123.56±2.0418.71±1.0339.19±1.52‾\underline{39.19{\pm}1.52}42.83±0.92
tgbl-wiki38.59±0.6775.34±0.1975.70±0.4280.38±0.47‾\underline{80.38{\pm}0.47}82.15±0.37

Q2. Several strong baselines are not evaluated, such as CRAFT [R2]. Additional results against these baselines would strengthen the empirical evidence, particularly if the proposed method maintains its advantages. If not, I encourage the authors to provide a more detailed analysis of which dataset characteristics favor the proposed method, and under what conditions the method may be less effective.

RQ2: We appreciate this suggestion. To make the comparison more complete, we add CRAFT, together with CTAN, CTDG-SSM and GraphSSM suggested by Reviewer e5zD. With these strong baselines added, DSRD still achieves the best performance:

DatasetCRAFTCTANCTDG-SSMGraphSSMDSRD
Wikipedia95.71±0.2975.45±0.6996.04±1.40‾\underline{96.04{\pm}1.40}86.75±0.7199.55±0.10
Reddit97.18±0.0571.68±2.2397.56±0.18‾\underline{97.56{\pm}0.18}91.79±0.9499.94±0.02
MOOC79.30±1.80‾\underline{79.30{\pm}1.80}58.33±3.0974.97±0.9771.87±0.8395.75±0.66
Enron82.45±0.4164.49±2.1082.46±3.84‾\underline{82.46{\pm}3.84}73.95±1.1593.78±1.10
UCI91.00±0.21‾\underline{91.00{\pm}0.21}75.76±0.6679.88±7.0683.35±1.7592.59±0.05
US Legis.51.10±3.2956.34±0.2948.49±0.4762.65±1.42‾\underline{62.65{\pm}1.42}84.42±1.03
UN Trade51.38±0.6762.60±2.0464.29±0.6472.57±5.35‾\underline{72.57{\pm}5.35}99.23±0.07

Section 4.3 and Appendix C.9 further relate the learned decay patterns to dataset characteristics such as time granularity, density and interaction scale. We have added these baselines and their results to Tables 6 and 7 of the revised paper.

Q3. Table 3 indicates that none of the 14 datasets provides node features. How are the input node features (xj(t)x_j(t)) in Eq. (1) constructed and initialized?

RQ3: Following DyGLib, the raw node features are zero vectors of dimension 172. Before the projections in Eq. (1), they pass through a linear layer with a trainable bias (standard PyTorch initialization), so the first-layer input xj(t)x_j(t) is the same learned bias vector for every node. Edge attributes, elapsed-time encodings and retained interaction states supply event-specific information. We have added this description to the implementation details in Appendix C.3.


Thank you again for your time and insightful feedback. We would be glad to address any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.