acceptodds
Under review as a conference paper at ICLR 2027

Temporal Sheaf Diffusion: Memory under Changing Geometry

Abstract

When relational geometry changes, should temporal memory preserve past comparisons or reinterpret history using current ones? For a prescribed evolving sheaf, we characterize the exact finite-polynomial historical projection through signal moments and a positive-definite Gram state, deriving continuous dynamics, exact held-history updates, and a compressed representation using operator moments for temporal modes. A three-node graph example separates historical regularization, current-geometry filtering, and compression of filtered inputs, showing why time-varying pointwise filtering can leave the polynomial space. Synthetic reconstruction shows that the appropriate geometry depends on the meaning of the observed changes, with little benefit from the coupled solve over filter-then-compress, while a compact Temporal Sheaf Diffusion predictor is competitive on temporal knowledge graphs.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Temporal graph learning, cellular sheaves, sheaf diffusion, state-space models, dynamic graph neural networks

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_b6V8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies historical signal compression and prediction on time-varying graphs. It proposes finite-polynomial projection with historical geometric regularization, derives a joint Gram solution and online moment recursions, and designs a TSD predictor that combines sheaf propagation, a selective state-space model, and memory feedback. The paper evaluates these designs through signal reconstruction and dynamic graph prediction experiments.

Critical strengths

1.The treatment of the finite-polynomial constraint has clear mathematical value. The paper points out that polynomial compression followed by geometric filtering at each time point may no longer satisfy the original finite-dimensional function space constraint. It provides a joint Gram solution and sufficient-statistic recursions that preserve this constraint. This result clarifies the difference between pointwise optimization and finite-dimensional joint optimization, provides a more rigorous mathematical basis for related historical compression methods, and is the paper's most convincing contribution.

2.The experiments and appendices provide valuable checks of how the mechanisms work. The paper reports prediction performance and also compares historical geometry with current geometry, exact solving with filter-then-compress, and simple architectures under a shared prediction head. These controls help distinguish the roles of the objective function, the geometric interpretation, and the prediction modules. The authors also report differences between theory and implementation, protocol limitations in some tasks, and results that do not support the expected advantages. This makes the conclusions easier to check and provides a basis for judging the limits of the method's use.

Critical weaknesses
  1. The organization is confusing, making the core contributions and the limits of their supporting evidence hard to identify. The paper discusses exact historical projection, a closed-loop continuous system, and a practical neural predictor, but the three use different states, timing mechanisms, and update rules. The main text switches between them, while key limitations are scattered across the appendices. Even after reading the full paper, I still find it hard to clearly judge which problem the paper centers on, which theorems apply to the practical model, and which experiments validate the main contribution. This already affects the assessment of novelty and the weight of the contributions, rather than merely making the text harder to read.

  2. A key link between the theory and the evaluated model is missing, and the need for the added mechanisms is also unsupported. The exact projection derivation applies to general time-varying positive-semidefinite operators and does not depend on the specific structure of sheaves. The practical predictor does not maintain the historical statistics required by the theory or solve the Gram system, and there is no guarantee of an approximation relationship between the two. Meanwhile, ordinary propagation or GRU controls with a shared prediction head achieve similar or higher average performance on multiple datasets, and the exact solution also shows no clear advantage in the core reconstruction experiments. Therefore, the current theory and experiments do not yet jointly support the contribution of TSD's specific design.

Questions for authors
  1. Is the main new contribution of this paper the finite-polynomial projection algorithm or the TSD prediction architecture? If it is the former, what unique results does the sheaf structure add compared with general time-varying positive-semidefinite operators? If it is the latter, which theoretical result actually applies to the predictor used in the experiments?

  2. Is there a verifiable mathematical relationship or a direct experiment showing that the learned states of the practical predictor approximately implement the paper's exact projection? If not, what support does the exact projection theory provide for this predictor beyond design inspiration?

  3. Given that simple controls achieve similar or higher performance and the exact solution shows no clear reconstruction advantage, which evidence in the current results can identify the unique benefits of the paper's added mechanisms? This asks for an explanation of the existing evidence and the scope of its conclusions, rather than a plan for future experiments.

Reviewer_dqTwOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how temporal memory should handle changing graph geometry. It proposes an exact polynomial memory formulation based on historical sheaf regularization, along with continuous and discrete updates. It also compares three ways of combining geometry and temporal compression. Finally, the authors introduce a practical Temporal Sheaf Diffusion (TSD) model and evaluate it on synthetic reconstruction tasks and temporal graph benchmarks.

Critical strengths
  1. The theoretical contribution is interesting and well motivated. The distinction between historical and current geometry is clear. The Gram-based formulation and exact updates provide a useful way to understand memory under changing geometry.

  2. The paper is generally well organized and technically careful. The authors provide detailed derivations, numerical checks, and extensive ablations. I also appreciate their transparent discussion of negative results and limitations.

Critical weaknesses
  1. The connection between theory and practice is not strong enough. The proposed TSD model does not use the exact Gram-based memory or provide an approximation guarantee. Moreover, filter-then-compress performs almost identically to the exact historical solution in the main reconstruction experiments (Table 15). This raises questions about the practical value of the more complex formulation.

  2. The experimental evidence for the proposed architecture is limited. TSD achieves competitive results on several datasets, but simpler models often perform equally well or better. For example, identity maps and GRU-based models match or outperform TSD on Wiki, Forum, and ICEWS (Tables 7 and 9). The benefit of memory-conditioned sheaf geometry therefore remains unclear.

  3. Limited visualization and interpretability. The paper relies heavily on mathematical derivations and tables, with relatively few visualizations. In particular, it would be helpful to show how the learned sheaf geometry evolves over time. More visual analysis could make the proposed mechanism easier to understand and provide better insight into what the model actually learns.

Questions for authors
  1. Can the authors identify settings where the exact historical Gram solution clearly outperforms filter-then-compress?

  2. Is there evidence that the learned sheaf maps capture meaningful changes in relational geometry?

  3. Which theoretical results are specific to sheaf geometry, rather than general time-varying positive-semidefinite operators?

Reviewer_Hci9OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

While at the outset this submission lacks clarity to a great extent, in my view, this authors aim to learn representation over temporal cellular sheaves. To do so, they rely on recent developments in the temporal graph learning, namely, GraphSSM and CTDG-SSM, which are extensions of HiPPO to graphs.

Critical strengths

The idea of temporal sheaves and the advantages it might offer is a nice idea that is worthwhile investigating. Although these have been discussed in ST-Sheaf GNNs and TSNN, the extensions of studying advantages of maintaining long range dependencies using SSMs and limitations TSNNs is an interesting topic of study. However, the submission does not do proper justice beyond the ideation.

Critical weaknesses
  • The submission is very immature in terms of writing (not grammar or language) making it extremely difficult to understand even the main storyline and contributions.
  • At many places (I will not list all), sentences are vague or meaningless. For example, first line in the abstract "should temporal memory preserve past comparisons or reinterpret history using current ones" is technically meaningless. Many sentences in the submissions, reads well as an English sentence, but doesn't mean much.

Now on some more important questions, which are not touched in the submission:

  • what is the problem with TSNNs? Why is the formulation proposed in the submission even needed?
  • What is the difference between GraphSSM and CTDG-SSM formulations, and more importantly, what changes when one has a sheaf instead of a graph? To the reviewer, it looks like exact copy of the development, except the \cal{L} is now a sheaf instead of a graph.

Experiments are very limited. What is the use case of temporal sheaves? What is rationale behind the choice of the sensor array experiment? Why is sheaf needed there?

In sum, the submission lacks clarity, limited/straightforward contributions, and limited experiments.

Questions for authors

See weakness section.

Reviewer_S6EvOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces a sheaf-based geometric approach that incorporates temporal memory, using time-varying sheaf Laplacians to represent evolving relationships among node features. Its main theoretical contribution is an exact characterization of finite-polynomial memory under historical geometric regularization. The authors derive a positive-definite Gram formulation, continuous dynamics, exact recurrences for piecewise-constant histories, and a compressed representation based on 2N−12N-1 operator moments. The paper proposes Temporal Sheaf Diffusion (TSD), an architecture combining selective state-space memory, memory-conditioned orthogonal restriction maps, and sheaf-based spatial propagation. The experiments on temporal graph benchmarks show competitive performance on several temporal knowledge graph tasks, although the component comparisons do not establish a consistent advantage for memory-conditioned sheaf maps.

Critical strengths

1. A theoretical contribution to memory under changing geometry. I think the strongest part of the paper is the theoretical analysis in Section 3. The authors derive an exact solution to the historical regularization problem (Eq. (3), Theorem 3.1) and show how the corresponding memory can be updated over time. An interesting aspect is that the full Gram matrix does not need to be stored, since the relevant information can be represented using only 2N−12N-1 operator moments. I also appreciate the discussion of GraphSSM, where the authors point out that filtering with a time-varying operator does not necessarily preserve the finite-polynomial space. This seems to be a useful clarification of the existing formulation.

2. A clear motivation for distinguishing different ways of handling changing geometry. Another aspect I liked is the comparison between historical regularization, current-geometry filtering, and filtering before compression. The paper does a good job of explaining why these approaches are not equivalent and, more importantly, when one might be preferable to another. The sensor reconstruction experiments in Section 5 and Appendix F are particularly helpful here. They show that historical geometry works better when sensors actually change orientation, whereas current geometry is more appropriate when the change represents a retrospective calibration correction.

Critical weaknesses

1. The connection between the theoretical results and the actual model is not entirely clear to me. My main concern is that the theoretical part and the proposed architecture seem somewhat disconnected. In Section 3, the authors derive an exact historical projection that relies on keeping track of both signal moments and the history of the operators, and then solving a coupled Gram system. However, the model evaluated in Section 4.2 uses a learned, fixed-size state-space memory and does not actually implement this projection. The authors acknowledge this, but I would have liked to see a clearer explanation of how the theoretical results motivate or justify the architecture beyond the general idea of combining temporal memory with changing geometry.

Related to this, I am not fully convinced that the exact historical solution brings a significant practical advantage. In Section 5 and Table 15, the results are almost identical to those obtained by simply filtering the inputs before compression, which is a considerably simpler approach. To me, the experiments show quite convincingly that choosing the right geometry matters, but it is less clear why the more involved Gram formulation is needed in practice. I think the theoretical result is interesting on its own, but its practical significance could be better established.

2. It is difficult to understand how much of the predictive performance actually comes from the learned sheaf geometry. The model achieves competitive results on several temporal knowledge graph benchmarks, which is encouraging. However, looking at the ablations, I am not convinced that the memory-conditioned sheaf maps are responsible for these results. In Table 7, simpler alternatives using identity maps or standard recurrent models perform similarly on Wiki and actually achieve better results on Forum. A similar observation can be made for ICEWS in Table 9, where the identity-map and GRU variants slightly outperform the full TSD model.

I also found the synthetic experiments in Appendices E.5–E.6 particularly relevant, since they seem designed to test situations where temporal memory and changing geometry should be useful. However, even in these settings, the proposed model does not show a clear advantage over the simpler alternatives.

Of course, this does not mean that learned sheaf geometry cannot be useful. My concern is more that the current experiments do not really demonstrate when or why the additional geometric machinery is beneficial. Since the interaction between memory and sheaf geometry seems to be one of the main motivations behind the architecture, I would have expected stronger evidence supporting this choice.

Another aspect that makes the results somewhat difficult to interpret is the recurrence-aware scoring head, which already uses information about past interactions, including their frequency and recency. I appreciate that the authors keep this component consistent across the ablations, but it still makes me cautious about attributing the full model's performance to the proposed geometric mechanism.

Questions for authors
  1. What properties of the exact historical Gram projection, beyond the use of a HiPPO-inspired temporal initialization, are preserved by the evaluated learned recurrence? Is the intended relationship primarily conceptual, or can the authors identify a more precise connection between these constructions?

  2. The reconstruction experiments suggest that filter-then-compress achieves nearly identical performance to the exact historical Gram solution. Under what conditions does the coupled solution provide a meaningful advantage? Can this distinction be demonstrated or quantified within the existing experimental framework?

  3. Do the trained restriction maps exhibit meaningful nontrivial transport structure? For example, have the authors examined learned cycle holonomy or alignment with the planted geometric transformations in Appendix E.6? Such an analysis could help determine whether the additional geometric expressivity is actually exploited.

  4. How do the authors interpret the absence of a consistent improvement over identity-map and GRU-based controls, particularly on Forum and ICEWS? Is there evidence from the existing experiments that identifies a regime in which memory-conditioned sheaf transport provides a measurable advantage sufficient to justify its additional computational complexity?

3 comments

Thank you for your continued engagement with the paper. The new controlled study, GT-DC, shows that TSD learns planted nonflat geometry, uses it for prediction, and infers it from history. Part 2 addresses reconstruction, E3, causal Wiki, and theoretical scope.

1. Learned geometry: a controlled prediction test. Each node observes a vector and predicts the average of its neighbors' vectors after hidden rotations align them. Changing hidden two-dimensional node states determine the rotations. Each node observes its own state directly at every step (H0), or noisily at 25% of node-steps (H2). All arms also receive the previous observed output.

GT-DC trains the paper's recurrent/diffusion core and two-dimensional decoder on new geometry and observation channels. It uses the original transport architecture; the planted geometry lies within its decoder class by design.

Transport is flat when carrying a vector around any cycle returns it unchanged. Its net rotation around the cycle is the holonomy; a nonzero cycle rotation measures curvature that cannot be removed by changing node frames. We cross flat/curved geometry (κ=0/1) with H0/H2.

The controls share the recurrent core, diffusion input, head, data, and fixed budget, with no per-arm tuning:

  • Node frames: maps depend on memory but have flat cycle transport.
  • Current-only maps: maps can be curved but depend on current input; memory still enters diffusion.

There is no recurrence-aware scoring head (REC) in this task.

Mean test MSE per output component, over five training seeds (lower is better):

Geometry and observationsTSDNode framesCurrent-only
Flat, states observed0.01150.01160.0115
Curved, states observed0.01170.06210.0116
Flat, sparse observations0.04270.04330.1210
Curved, sparse observations0.06400.08520.1538

Thus node frames are close to TSD when the geometry is flat, and current-only maps are close when states are directly observed. TSD outperforms both under curved geometry with sparse observations.

Does curvature improve prediction (P1)? We compare the node-frame–TSD error gap under curved versus flat geometry in H2. This difference is 0.0206 MSE (registered minimum 0.012; p=4.8×10⁻⁴). It measures the change in relative performance when curvature is introduced. With states observed, the corresponding contrast is 0.050.

Do predictions use the learned curvature (P2)? We reverse the learned cycle rotation while retaining the node-frame component. The resulting one-step error increase is 0.0708 MSE larger under curved than flat geometry (p=1.3×10⁻⁸). Replacing its maps by a fitted flat connection gives a corresponding contrast of 0.0205. These interventions test predictive use, beyond merely observing nonzero holonomy.

Is that curvature inferred from history (P3)? When none of a triangle's nodes receives a fresh state observation, learned and planted holonomy have circular correlation 0.738 (one validation-fixed global sign per trained model), versus 0.001 for current-only maps (difference 0.737; p=4.0×10⁻⁹). A Kalman filter on the same observation channel gives 0.743 on these test episodes. This is a reference, not an optimality bound: the networks also receive previous observed outputs.

Does recurrent memory help the maps (P4)? With curved geometry, we compare the current-only–TSD error gap between sparse and directly observed states. This increase is 0.0900 MSE (minimum 0.040; p=3.5×10⁻⁶). It demonstrates history use; the analogous flat-geometry contrast is 0.078, computed post hoc. P1 and P3 supply the curvature-specific evidence.

The pilot-informed protocol was registered before production training and frozen before test evaluation. All four criteria pass, with expected directions in all five seeds; no prespecified adverse rule fires. The one-sided paired t-tests reflect five training seeds on one fixed 64-episode test set.

Control scope. Node-frame potentials are confined to a half-circle. Subtracting its registered estimated cost of 0.006 leaves P1 positive at 0.0146 (p=1.8×10⁻³). Its complete nonlinear predictor also exhibits some cycle-dependent input–output behavior despite flat edge maps. We disclose both qualifications; this is evidence against these tested controls, rather than every possible flat predictor. A separate one-seed identity reference scores 0.1488 in curved H2; this comparison is descriptive. Kalman-derived maps reach TSD's accuracy range in one reference seed: learning is not the only way to implement the mechanism.

2. The reconstruction gain survives the wider search. We extended every regularized method's search through strength 10⁶ and included analytical infinite-strength limits. Each method was selected on validation; evaluation then used 100 fresh paired draws.

In the primary setting, validation still selects 10 for Gram and 100 for filter-then-compress (FTC). Mean clean-reconstruction error is 0.10361 versus 0.10856: a 4.56% reduction (95% paired interval 4.03–5.10%; Gram lower in 97/100 draws). The gain survives the wider search, while remaining below the original 5% target.

At 16 polynomial modes, both selections change: Gram chooses 3000 and FTC 300. Gram's advantage is 30.08%, versus the previously reported 20.18% (95% interval 29.46–30.71%; lower in all 100 draws). No setting selects 10⁶ or infinity.

Gram's error is 2.99% higher than FTC's under a doubled disturbance, and 0.47% higher at 10 dB SNR. Without the disturbance, Gram beats FTC by 28.14%, but mean-operator and diagonal-mode regularizers beat Gram by 0.26% and 0.21%. We will report these together. Exact minimization of the historical objective and improvement against a clean signal are distinct results.

3. E3 and the scope of the mechanism evidence. New runs find nonflat transport in all three TSD seeds, with weak mean holonomy in two, and the learned cycle holonomies do not reproduce the planted ones.

Flattening increases rollout error beyond a matched perturbation control by 4.39% in seed 48, 0.10% in seed 47, and no detectable amount in seed 46. The TSD–node-frame accuracy comparison is also inconsistent across seeds. We therefore retain E3 as mixed evidence. GT-DC addresses a more specific question: whether the same decoder can learn and use moving curvature when that curvature lies within its representable class and must be estimated from history.

GT-DC also shows a response to the data: in H2, learned holonomy falls from 0.74–0.86 radians under curved geometry to 0.05–0.07 under flat geometry, in every seed. Within-episode tracking correlation is 0.66, versus 0.12 for current-only maps.

GT-DC establishes the mechanism within its decoder class. It does not establish recovery of arbitrary edge rotations or curvature's necessity on benchmarks; the existing negative ablations remain.

4. Wiki under causal training context. We corrected the fixed-context construction so an edge becomes available only after its observation is ingested. This removes future support within the training prefix at the existing 600-second snapshot boundaries. All 53 pre-training protocol checks pass; all 153 resets start with empty context. Later GPU replay parity checks failed for four checkpoints, but arm means moved by at most 0.00003 MRR and comparison signs were unchanged.

Across three seeds, test MRR is 0.7660 for TSD, 0.7647 for identity maps, and 0.7614 for current-only maps. The Table 1 static-context result is 0.762±0.003 over five seeds; the differing seed sets prevent attributing the change to context alone. TSD exceeds node frames by 0.0168 on average, with positive differences in every seed. Its advantage over current-only maps is 0.0046 on average, with mixed seed signs; these maps are time-invariant on Wiki's fixed input features. This is an architecture comparison, not an isolated memory ablation.

Wiki supports robustness under causal context, while the benefit over identity remains unresolved. GT-DC supplies the mechanism evidence, with shared recurrent cores and no REC.

5. Theory and computational scope. Theorem 3.1 solves the historical finite-polynomial problem for prescribed suitable positive-semidefinite paths. Its contribution is the exact constrained solution: changing weights couple temporal modes, and the Gram system enforces the polynomial constraint. Sheaves specify the local endpoint comparisons; GT-DC separately demonstrates use of their additional transport freedom.

We do not claim that the learned recurrence implements or provably approximates that projection. Well-posed memory–geometry feedback does not establish historical-projection optimality. The exact solver's dense operator statistics and coupled solve have substantive costs; the gate's factorized timings cover a special case. For the profiled Wiki seed, map decoding takes 2.62–3.23 times the node-frame decoder time; end-to-end overhead remains unresolved. We claim conditional benefits, not a universal accuracy–cost advantage.

Revision. We will add the expanded search, causal Wiki results, and GT-DC protocol/results with control qualifications and operational disclosures. The main text will distinguish the exact projection, learned mechanism, and benchmark conclusions.

Dear Reviewer,

As the rebuttal period is coming to an end, I would like to kindly ask you to read the authors' response and reply as soon as possible.

Thank you for your prompt attention.

Best regards,

Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.