- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper introduces a new temporal representation learning framework leveraging sheaf neural networks particularly dynamic link prediction task. Specifically, they introduce TSNN unified memory architecture for a homeogenous, heterogenous, KG based datasets. TSNN represent the node memory through a evolving frames that are othogonal. The update writes event information into the frame state, the recurrent update through sheaf diffusion. TSNN is evaluated on wide range of benchmarks with downstream task as temporal link prediction.
Critical strengths
Strengths:
1: A unified temporal memory architecture applicable to homogeneous graphs, heterogeneous graphs, and knowledge graphs.
2: The frame-based memory formulation explicitly separates coordinate changes from content updates, preserving the represented memory through exact carry-over.
3: The empirical evaluation covers a broad range of datasets across multiple temporal graph benchmarks.
Critical weaknesses
Weakness:
W1: Advantage from evolving frames cannot be established clearly from the theoretical analysis provided. Specifically, Proposition 4 shows that the node-factored sheaf Laplacian is orthogonally conjugate to a lifted ordinary graph Laplacian $ L_{\mathcal F}=S^\top(L_G\otimes I_d)S. $ Consequently, the construction has trivial spatial holonomy and does not introduce additional spectral expressivity through its diffusion operator. Moreover, Proposition 12 establishes that the frames disappear from the ambient dynamics under equivariant content updates and scalar gains. Hence it doesn't shed enough focus on why this particular parameterization is preferable to simpler recurrent state or more recent SSM models for CTDGs.
W2: The expressivity result relies on an overly restricted comparator. Although valid under its assumptions, it does not establish an advantage over standard recurrent temporal graph models or auxiliary-state architectures with comparable resources. More importantly, the paper does not characterize the expressive power of its representations relative to the Temporal Weisfeiler–Lehman (TWL) test. Such an analysis would clarify which temporal structures evolving frames can distinguish beyond existing models.
W3: The empirical gains do not establish a favorable accuracy-cost trade-off. The framed model stores ((k+1)d) state scalars per node, compared with (d) memory scalars in a frame-free counterpart, and adds frame generation, transport, and carry-over operations. The reported frame-removal ablations show meaningful improvements, but the auxiliary-state controls are not matched in parameters or compute.
W4: The scoring and normalization choices require clearer justification. Equations (2)–(4) combine manually selected scalar features, logarithmic transforms, hard clipping of the distance-scale parameter, clipping of the residual score, and row normalization of frame parameters. These choices are mathematically permissible, but their numerical constants are not derived from the theory. Hard clipping can suppress gradients outside the permitted ranges, while row normalization affects the dynamics of subsequent additive frame updates. Sensitivity analyses and a clear account of how these choices were selected are needed, particularly because the residual scorer is crucial to performance. This choices make me hard to believe the advantages of frames and also mainly this choices kills the expressivity of the model.
W5 : Experiments : Introduction motivates persistent memory and retention of useful history, but the experiments do not directly establish improved long-range dependency modeling. Evaluation on datasets with long histories does not isolate this capability, especially given the strong contribution of pair-history features. Performance stratified by temporal dependency distance and controlled history-truncation experiments would clarify whether evolving frames improve memory retention. Furthermore, evaluation is restricted to link prediction; the inductive results concern links involving new nodes and do not demonstrate applicability to node-level prediction tasks.
Experiements on long range tasks (Like sequence classification typically used in temporal graph community) will be useful.
W6 : Importantly I want to emphasize the paper in current form is not written well.
For instance, The event-indexed state , the pre-wave snapshot , and the node-field notation are introduced densely. Check the notations on degree matrices with and without parametrizations. Similarly, (\dot F) denotes a discrete proposal rather than a derivative, and (\ell) denotes both a logarithmic feature transform and loss quantities.
More importantly, many sentences are incomplete for example as in page 3 (line 155) Only then does it read the new events, TSNN architecture on 34 settings (it should be datasets).
Several dimensions (d_{ctx}) needs to be justified and several symbols needs to be introduced before usage.
I believe the paper requires a complete rewriting for a better readability.
Questions for authors
Please address my weakness.
4 comments
Response => temporal WL and cost (W2, W3)
W2. Restricted comparator; temporal WL.
Theorem 19 demonstrates a mechanism within its stated comparator class: a frame turn moves the carried coordinates off the rest point of a gate that stays closed in shared coordinates, so later states differ. The paper does not claim an expressivity ordering over standard recurrent models (Table 6), and the revision says so next to the theorem.
Relative to temporal WL (Souza et al., 2022) we can state one exact result, which the revision adds. Take the model with a common initial memory and frame for every node, and with pair recency and optional inputs off, on six nodes. Stream A is one wave with arcs i → i+1 (mod 6). Stream B adds the arcs i → i+2. Initial memories, frames and event metadata are identical, and messages and frame proposals are aggregated by mean, so one source and one destination message average to the same value as two of each. Every node receives the same update in both streams. Both retained graphs are regular (the history cap holds every event), so diffusion leaves the uniform state unchanged, and every query receives the same score. Temporal WL separates the two streams in one refinement, since a node has two incident interactions in A and four in B. This is the known limit of mean aggregation (Xu et al., 2019), and it holds only in this anonymous setting. The reference model is outside it: it reads pair recency, and its seeded per-node initial state is fixed for each node and acts as a node identifier (cf. random node initialisation, Abboud et al., 2021), whereas the temporal WL comparison concerns models that do not use node identifiers (Souza et al., 2022). The revised conclusion lists a full temporal WL characterisation of the reference model as an open direction.
W3. Accuracy–cost trade-off; controls not matched in parameters or compute.
Cost on tgbl-wiki in the ablation configuration (d = 128, k = 2):
| Model | Trainable parameters | Stored state per node | Run time | Test MRR |
|---|---|---|---|---|
| Full TSNN | 571,566 | 1,536 B | 1.47 h | 0.8229 |
| Frame-free, same diagonal-gain diffusion | 386,734 | 512 B | 1.24 h | 0.8094 |
| Frame-free, no diffusion | 386,734 | 512 B | 0.27 h | 0.8214 |
- At the same diffusion setting, frames add 48% parameters and 19% run time (0.23 h), and store (k+1)d instead of d scalars per node, for +0.0135 test MRR here. Most of the run time is the diffusion step.
- The cheapest variant drops diffusion. On tgbl-wiki it is within 0.0015 of the full model at 0.27 h. On YouTube it is 0.19 MRR lower (reply to K48B). The trade-off depends on the dataset.
- Widening the full model from d = 128 to d = 192 or 256 does not raise MRR on any of the four datasets tested (Table 19). A frame-free model at width 3d is not among the controls.
- Matched control: a new side-state arm with the same stored state as the full model and matched parameters (574,638) did not train to a competitive level (test MRR between 0.0155 and 0.4692 on the five datasets). We draw no conclusion from it, or from the side-state arm of Table 3, which behaves the same way.
The revision adds the cost table and writes "same-configuration" for controls that share a configuration but not a parameter count.
Response => long-range memory (W5)
W5. Long-range dependency modelling; link prediction only.
We ran the three analyses you ask for.
History truncation (tgbl-wiki, test MRR, complete model). The model's state is rebuilt by replaying only the last N timestamps of history before the test queries.
| History replayed | All | Last 128 timestamps | Last 32 timestamps |
|---|---|---|---|
| Test MRR | 0.8232 | 0.5681 | 0.4749 |
History older than 128 timestamps contributes 0.255 MRR.
Dependency distance (same model), for pairs seen before, by time since their last interaction:
| Time since last interaction | Most recent quarter | Middle half | Oldest quarter |
|---|---|---|---|
| Test MRR | 0.97 | 0.94 | 0.84 |
These two analyses have no frame-free arm and do not separate node memory from the pair-recency table. They show that long history matters. They do not show that frames improve retention.
Sequence classification. We follow the task of Gravina et al. (2024): the model predicts the first node's signal (+1 or −1) from the last node of a temporal path of n nodes, so the signal has to be relayed through node memory along the path. The task has no pair history. Accuracy in % of the best run at each length (chance is 50%):
| n | 3 | 5 | 7 | 9 | 11 | 13 | 15 | 20 |
|---|---|---|---|---|---|---|---|---|
| Full TSNN | 100 | 100 | 100 | 100 | 98.15 | 96.97 | 66.0 | 57.9 |
| Frame-free | 100 | 100 | 100 | 100 | 96.5 | 91.7 | 62.2 | 51.9 |
| Without node memory | 50.0 | 50.0 | 50.0 | 50.0 | 50.0 | 50.0 | 50.0 | 50.0 |
- The task is solved reliably at 3 and 5 nodes: every run reaches 100%.
- From 7 nodes on, runs differ. The best runs hold up to 13 nodes, and other runs fall to chance.
- At 15 and 20 nodes the model is near chance.
- Frames make no difference: the framed and frame-free models behave alike at every length.
- Without node memory the model is at chance at every length, so the signal is carried by the persistent node memory.
- A framed variant with an equivariant update and a scalar gain reproduces the frame-free accuracies exactly (difference 0.0 at every length), as Proposition 12 predicts.
The frame claim rests on the controls of parts 1 and 2. All 34 benchmark settings are temporal link prediction, as the title states. Node-level prediction is not evaluated, and the revised conclusion states this scope.
Response => constants and presentation (W4, W6)
W4. Scoring and normalisation choices.
Sensitivity on tgbl-wiki (test MRR, single runs):
| Variant | MRR |
|---|---|
| Reference (constants as in the paper) | 0.8231 |
| Residual score: smooth bound instead of hard clip | 0.8229 |
| Distance scale: smooth bound instead of hard clip | 0.8221 |
| Row normalisation off | 0.8215 |
All three variants are within 0.0016 of the reference, about one seed standard deviation (0.0014 in Table 3). The smooth bounds have a non-zero gradient everywhere, and with them MRR changes by 0.0002 and 0.0010, so the hard clips do not suppress useful gradients.
Expressivity. Away from the safeguard branch of Eq. (1), each reflector depends only on the direction of its row. Row normalisation keeps that direction, so it does not change which frames can be represented. It fixes the scale against which later additive increments act. The two clips bound ranges (distance scale between 0.05 and about 20; residual score in [−10, 10]) and leave the function class inside them unchanged. Every arm of the frame comparison uses the same scorer and the same constants.
Why these inputs. Each geometric input of the residual scorer is a function of the ambient vectors alone, which keeps the base scorer frame-invariant (Propositions 9 and 16). The inner product and the squared norms of and are the Gram entries of the ambient pair, and they determine every function of the pair that is invariant to a common orthogonal map (Villar et al., 2021). The distance is one such function. The other inputs are elapsed times and relation and type embeddings, and log(1+x) compresses the wide range of the elapsed times. The clips and the floor are numerical safeguards. The same constants are used on every dataset, and none is tuned per dataset.
The revision adds the sensitivity table and one sentence on each constant.
W6. Presentation.
The revision rewrites the notation of Sections 2–3 along your list, so that each symbol is introduced before it is used:
- State notation. The event-indexed state , the pre-wave snapshot and the node-field notation are introduced one per sentence. Blocks C–E use − and + superscripts for pre- and post-wave state throughout.
- Degree matrices. One sentence states which degree each matrix uses ( and the safeguarded degree, the true degree) and that the gain is not a degree matrix.
- The frame proposal becomes an increment , since it is a discrete update and not a derivative.
- The symbol . The feature transform is written log1p(x), is kept for losses, and the diffusion round gets its own index.
- The dimension is explained term by term: two memories (2d), two half-width time encodings (), event features (), the relation embedding () and the two-component role indicator (2), plus two type embeddings on heterogeneous graphs.
- Line 155. The sentence ran on after a colon and was hard to parse. It is now two sentences.
- "34 settings". We mean dataset–protocol pairs: DGB Wikipedia and tgbl-wiki share the same underlying data under different protocols. A footnote defines the term.
- Tables. Decimals and markers are made consistent.
The bibliography gains Abboud et al. (2021), Villar et al. (2021) and Raghuvanshi et al. (2026).
Response => what frames add (W1)
Thank you for the detailed review. It led to new experiments and to a rewrite of Sections 2–3.
We agree with your reading of both propositions. Proposition 4 shows that frames do not act through the spectrum of the diffusion operator. The flat, node-factored construction is inherited from Bundle Neural Networks (Bamberger et al., 2025), as Table 4 states. What TSNN adds is temporal: persistent frames, exact carry-over, coordinate-dependent updates and a frame-dependent diagonal gain.
By Proposition 12, frames cancel when the content update is equivariant and the gain is scalar. A framed variant that meets both conditions reproduces the frame-free accuracies exactly on the sequence task of part 3. TSNN meets neither: its update is not equivariant and its default gain is diagonal, so diffusion itself can depend on the frames (Proposition 13). In that setting the full model exceeds the frame-free model with the same gain on all five datasets and on every one of three new seeds (test MRR, full minus frame-free):
Against the other frame-free variants it has the higher mean on every dataset where they were run (table in our reply to K48B). Removing exact carry-over costs 0.003–0.020 MRR on four of the five datasets (Table 3).
Simpler recurrent state. We compare with coordinate-free controls that add persistent state.
On this single seed the full model is ahead on all four datasets. The Wiki and Smallpedia margins (0.0091 and 0.0069) exceed seed variation. The YouTube and Polecat margins do not.
State-space models. Frames are a coordinate layer around the content update. The frame update, exact carry and the strict-history protocol do not depend on the GRU, so a state-space update can replace it. The related-work section will position TSNN against DyG-Mamba (Li et al., 2025), DyGMamba (Ding et al., 2025) and CTDG-SSM (Raghuvanshi et al., 2026) in these terms. Published numbers allow a comparison under the protocol of Table 13 (transductive AP, random negatives, 70/15/15 split; the shared baseline columns agree). On the 12 DGB datasets that DyG-Mamba reports (Li et al., 2025, Table 2), TSNN's AP is higher on all 12, within noise on Can. Parl. and by up to 23 points elsewhere:
The TSNN runs are those of Table 13, with the inputs of Table 14. The revision adds these numbers to Table 13.
The revision states the temporal nature of the contribution directly after Proposition 4, and the revised introduction says that TSNN uses node-factored maps.
Single-run tables in parts 1–4 come from separate runs, so the full-model value on tgbl-wiki differs slightly between them (0.8225–0.8232; three-seed standard deviation 0.0014).