acceptodds
Withdrawn from ICLR 2027

Hamiltonian-Structured Interaction Fields for Joint Cross-Mode Representation Dynamics

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Tensor factorization provides a compact framework for modeling high-order data by learning low-dimensional mode-wise representations and composing them for individual tensor elements. Neural tensor factorization further enriches this paradigm through nonlinear cross-mode interaction, yet such flexible fusion does not explicitly organize how changes in one mode influence the refinement of others. To address this limitation, we propose the Hamiltonian-Structured Neural Tensor Factorization (HSNTF), which introduces a Hamiltonian-structured interaction field for context-specific latent representation refinement. For each tensor element, HSNTF assembles the corresponding mode-wise embeddings into a joint latent representation. A scalar generator induces a Hamiltonian vector field that governs the evolution of the joint representation. Its Jacobian maps second-order curvature, including mixed derivatives, into explicit local cross-mode influence operators. The behavior of these operators is structurally coupled during representation refinement. Short-horizon multi-step evolution repeatedly applies the field to the updated latent state to produce context-adapted representations. Experiments on five real-world tensor datasets demonstrate consistent predictive gains. Structural ablations, perturbation analyses, and multi-step interventions show that the learned field captures local cross-mode responses and enables mediated cross-mode information transfer through multi-step integration. Controlled experiments further show that shared directional structure is particularly useful under scarce supervision, with its benefit depending on structural compatibility, sample size, and noise. These results establish Hamiltonian structure as a unified mechanism for organizing coupled cross-mode representation dynamics, without assuming physical dynamics in the observed tensor data.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Tensor representation learning, Cross-mode interaction, Hamiltonian-structured dynamics

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_awrrOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes HSNTF for neural tensor factorization, whose core module, the Hamiltonian Singular Field (HSF), refines each tensor element by assembling per-mode embeddings into a joint state z and generating all mode updates from a single shared scalar Hamiltonian, so that the mixed curvature blocks of the field Jacobian encode how a perturbation in one mode first-order affects another under a unified latent geometry, yielding context-adaptive representations read out through mode projections plus a Tucker core. Contributions include context-conditioned dynamic refinement, this geometric organization of cross-mode dynamics, and an inductive-bias analysis; validation on five real tensors shows HSNTF ranking first in 20/21 dataset, alongside structural ablations, mechanism tests, and applicability-boundary studies.

Critical strengths
  1. The paper formulates the refinement stage as a Hamiltonian dynamical system and introduces a single auxiliary state together with an ordered partition update. This is consequential because it provides a structurally grounded alternative to conventional low-rank regularization for modeling tensor mode coupling.
  2. The paper goes beyond reporting reconstruction errors by checking the consistency between the mixed curvature operator and numerical differentiation, and by evaluating the method under controlled synthetic conditions such as varying noise, supervision levels, and structural mismatch.
Critical weaknesses
  1. The paper formulates the refinement process as Hamiltonian structured refinement, introducing a single auxiliary state and an ordered partition update to enable cross-mode information propagation. While this formulation provides an interesting structural interpretation, the necessity of the Hamiltonian structure itself has not been convincingly demonstrated. Specifically, the current ablation in Fig. 3 mainly shows that the auxiliary state participates in information propagation, but it does not rule out the possibility that a non-Hamiltonian refinement module with comparable capacity could achieve similar or even better performance. Without comparing against non-Hamiltonian alternatives—such as a standard neural ODE, a residual MLP-based refinement module, or a volume-preserving but non-Hamiltonian dynamics—it remains unclear whether the Hamiltonian formulation provides unique advantages beyond serving as a structural packaging.
  2. The specific value of K refinement steps used in the final model design should be clarified. Additionally, ablation studies with different values of K are needed to verify the setting at which the model achieves optimal performance or stabilizes. A sensitivity analysis for K would help justify the chosen hyperparameter.
Questions for authors
  1. Could the authors clarify the specific value of K refinement steps used in the final model design? Additionally, have ablation studies been conducted with different values of K to verify at which setting the model achieves optimal performance or stabilizes?

  2. In Figure 6 of the supplementary material, does the solid blue line represent the proposed model?

Reviewer_bLfcOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Hamiltonian-Structured Neural Tensor Factorization (HSNTF) method for a general tensor decomposition. For each queried tensor entry, it concatenates the relevant mode embeddings into a joint representation (state zz) of a 'momentum' and a shared learnable auxiliary 'position'. This state is then evolved, following scalar Hamiltonian HθH_{\theta} dynamics z˙=J∇Hθ\dot{z}=J\nabla H_{\theta} with JJ being a canonical structure anti-symmetric matrix, and refines the joint state using derivatives of a scalar Hamiltonian. The refined representation is projected into mode-specific factors and passed to a Tucker readout. The paper further provides ablation and performance evaluation of the proposed method on five real world tensors and analyzes cross-mode Jacobian relationships and evaluates tensor completion, local perturbations, mediated propagation, and synthetic structural mismatch.

Critical strengths

Paper exposition is clear and easy to follow, despite quite a bit pointers to appendices throughout the paper that may hinder clarity for a wider audience. The core contribution is an aforementioned dynamic latent representation refinement, that makes the proposed method different from standard low rank or neural tensor decompositions.

The architecture is motivated as a latent inductive bias, without assuming that the observed data follow physical Hamiltonian dynamics and thus provide a rather generic framework (subject to 'Limitations and future work' paragraph). The physics inspired identities appear correct and the authors also appropriately distinguish their general discrete update from an exactly symplectic or energy-preserving integrator.

Experimental evidence is solid and based on real world datasets. The main comparison spans five datasets and seven evaluation settings, with five training seeds and training-only preprocessing. The reported best means and average percentage improvements are arithmetically correct (for caveats see weakness 2). Obvious structural bias induced by scalar generator of the mode interactions is shortly discussed in Limitations and future work paragraph.

Critical weaknesses
  1. The real-data ablations, e.g., Table 2, do not isolate the value of Hamiltonian structure. Static Tucker, Separable HSF, and Full HSF support the benefit of joint refinement relative to those alternatives, as stated. Yet, they do not establish that Hamiltonian parameterization and its induced mode dependency modeling is preferable to generic joint nonlinear refinement using a standard neural architecture or even (denser) tensor decomposition models. A controlled comparison isolating the Hamiltonian constraint remains missing. To fill this gap, I recommend adding an independently trained, capacity- and computation-matched joint residual network or unrestricted vector field on the same tensor-completion tasks to support the claimed benefit of Hamiltonian structure. Q-only (Section 4.3.) does not supply this comparison: it modifies an existing Full HSF checkpoint without retraining. The unrestricted neural-field comparison is conducted on synthetic directional supervision, not real tensor completion.

  2. The statistical interpretation should distinguish substantial improvements from nominal wins. For instance, in Table 1, on Network, RMSE is (30.428±0.06330.428 \pm 0.063) for HSNTF and (30.430±0.14230.430 \pm 0.142) for DEMOTE. The difference is only (0.0020.002). Several other gains are also small relative to the reported run variability. Although the table explicitly reports best means, these counts should not be interpreted as counts of statistically established improvements. The authors should consider reporting paired seed-level differences and their uncertainty alongside these counts. Additionally, Table 2 should report variability.

Overall, the model has promising predictive results, particularly on the PCTempPCTemp dataset, but the evidence currently supports joint nonlinear refinement more strongly than the distinctive Hamiltonian explanation of the proposal. The ablation study in Table 2 shows that independent mode-wise dynamics alone do not consistently produce performance gains, but does not isolate the benefit of Hamiltonian structure. This missing comparison is crucial to establishing the extent to which Hamiltonian structure accounts for the real-data performance gains remains unclear.

And this is despite Section 4.4 provides a controlled evidence for the Hamiltonian inductive bias, including a parameter-matched comparison with an unrestricted MLP and an analysis of structural mismatch. These experiments supervise synthetic directional responses or fields rather than tensor entries. A matched unrestricted joint-refinement baseline on the real tensor-completion tasks would clarify whether Hamiltonian structure accounts for the observed predictive gains.

Questions for authors

Please see to the critical weaknesses 1 and 2 described above.

Additionally,

  1. Why is physics (momentum, position, Hamiltonian dynamics) formulation important for the paper? Does it contribute anything or could the exposition avoid such an embedding and simplify the paper? It is stated on lines 123−125123-125, that a role of dynamics in the paper, i.e., to introduce an inductive bias on mode interactions, is different from their common meaning in physics. It seems like a symplectic transformation (area preservation) is perhaps important from regularization point of view, but it is actively neglected on lines 215−216215-216 ... I suggest authors clarify these aspects and explain/simplify to increase paper's generality and accessibility.

  2. Could authors extend discussion in the Limitation paragraph to include an example, where proposed HSNTF method would not work well, exactly because the structural limitations imposed by Hamiltonian scalar field?

  3. How is number of refinement steps KK guiding the Hamiltonian dynamics determined? More over, could you elaborate why is dynamical refinement on fixed embedding needed and different from end-to-end optimization and put that into perspective of Table 2 results (lines 312-316), suggesting that latent dynamics alone may not be necessary?

Reviewer_zF3XOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper propose Hamiltonian-Structured Neural Tensor Factorization (HSNTF). It refines mode-wise embeddings in a context-specific manner through a Hamiltonian Singular Field. HSNTF integrates the field over a short horizon of several steps, and uses the resulting context-adapted representations for tensor completion task. Experiments on tensor completion tasks show that HSNTF achieves competitive performance. Further analysis indicates that the Hamiltonian structure serves as a unified inductive bias for coupled cross-mode representation dynamics, rather than as a physical model of the tensor data.

Critical strengths
  1. This paper provides some mathematical theorems of the proposed method, helping explain how the Hamiltonian structure constrains different tensor modes.
  2. This paper investigates the limitations of the Hamiltonian structure, helping identify the conditions under which the proposed approach is beneficial.
Critical weaknesses

Major:

  1. The proposed method is not sufficiently novel. It combines the existing Hamiltonian Neural Network with Tucker decomposition for the tensor completion task. This paper should sufficiently claim that why these two methods should be combined, why the Hamiltonian dynamics are suitable for Tucker, and why it is necessary? Even though the experimental results show performance improvements, I am still not convinced.
  2. Lacking crucial comparison. Neural ODE is also using MLP to output p˙\dot{p} directly, and also using Tucker decomposition to readout. This paper should compare with it to support that the Hamiltonian is useful.

Minor:

  1. Line 57-64: The repeated use of “Together” could be reduced.
  2. Line 98: Two repeated citations.
  3. Line 306: “We” should be “we”.
Questions for authors
  1. Why is the Tucker decomposition not in Table 1? Even in the table 2 has the “Static”, but it still is not the original Tucker decomposition.
  2. Does the proposed method have the same mode-n rank as the Tucker decomposition?
Reviewer_VS7qOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Hamiltonian-Structured Neural Tensor Factorization (HSNTF), which introduces a Hamiltonian Singular Field (HSF) for context-dependent tensor representation refinement. The method constructs a joint latent state from mode-specific embeddings and uses a shared scalar Hamiltonian to generate coupled representation updates. The mixed Hessian of the Hamiltonian characterizes local cross-mode interactions, while multi-step integration refines the representations before Tucker reconstruction. Experiments on five real-world tensor datasets show competitive completion performance, and additional synthetic experiments investigate the properties and limitations of the proposed Hamiltonian structure.

Critical strengths

Using a Hamiltonian field to organize the joint evolution of mode-specific representations is an interesting idea and provides a different perspective on modeling cross-mode interactions. I appreciate the attempt to introduce a geometric inductive bias into tensor factorization, as well as the additional analysis of the interaction mechanism and its behavior under different supervision and structural mismatch settings.

Critical weaknesses
  • The motivation and necessity of the Hamiltonian inductive bias are not sufficiently justified.

My main concern is that the paper does not clearly explain what properties of tensor data would make Hamiltonian-structured interactions particularly suitable or necessary. While the mathematical formulation establishes certain constraints on reciprocal cross-mode responses, it remains unclear why these constraints should hold, or be beneficial, for the real-world datasets considered in this paper. I understand that the authors do not assume the data follow physical Hamiltonian dynamics, but this makes it even more important to explain the statistical or structural motivation behind the choice. The controlled synthetic experiments provide some insights into when Hamiltonian constraints can help, but they do not sufficiently connect these findings to the actual tensor completion tasks. As a result, I am not fully convinced why Hamiltonian dynamics should be preferred over more general joint representation refinement approaches.

  • Insufficient discussion of recent tensor decomposition methods and limited baseline comparisons.

Cross-mode interaction and dynamic representation learning have already been extensively studied in tensor decomposition. For example, Undirected Probabilistic Model for Tensor Decomposition (NeurIPS 2023) explores flexible nonlinear interactions among tensor factors, while more recent functional and temporal tensor decomposition methods, such as FunBaT (ICLR 2024) and CATTE (NeurIPS 2025), provide alternative ways to model interactions involving temporal and other tensor modes. Although these methods do not share exactly the same setting as HSNTF, they are highly relevant to understanding the actual novelty and positioning of this work, particularly since most of the evaluated datasets contain a temporal mode. Moreover, the existing ablations mainly demonstrate the benefit of joint cross-mode refinement over independent updates, rather than the specific advantage of Hamiltonian structure. A comparison with an unconstrained joint neural refinement model would be especially useful, and including more representative modern tensor decomposition baselines where applicable would make the empirical conclusions more convincing.

Ref:

  • Undirected Probabilistic Model for Tensor Decomposition. NeurIPS 2023.

  • Functional Bayesian Tucker Decomposition for Continuous-Indexed Tensor Data . ICLR 2024.

  • Functional Complexity-adaptive Temporal Tensor Decomposition. NeurIPS 2025

Questions for authors

What characteristics of real-world tensor data would make the Hamiltonian constraint particularly useful? Can the authors provide a clearer explanation of the expected structural properties, and how these properties relate to the different performance gains observed across the datasets?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.