- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes LASS-ODE, a framework that jointly refines physical representations and estimates dynamics through a Fixed-Point Transformer. It combines physics-aware feedback, cross-system knowledge sharing, and structured local ODE operators with adaptive stability. Theoretical analyses establish convergence and stability conditions, while extensive experiments demonstrate strong predictive accuracy, zero-shot generalization, and stable long-horizon rollout.
Critical strengths
-
The paper introduces an interesting Estimator–Refiner framework that iteratively incorporates inferred dynamics into latent representations. Unlike conventional physics-informed approaches that primarily use physical knowledge as conditioning or regularization, this framework explicitly couples representation learning with dynamics estimation. This is a meaningful contribution to transferable modeling across heterogeneous dynamical systems.
-
The extensive experiments demonstrate competitive performance in in-domain prediction, zero-shot generalization, and long-horizon rollout. The structured piecewise-linear ODE operators provide an effective balance between computational efficiency and predictive accuracy. The ablation studies and real-world evaluations further support the practical effectiveness of the proposed framework.
Critical weaknesses
-
Although the paper establishes convergence of the Estimator–Refiner iterations, the theoretical guarantee primarily concerns convergence to a self-consistent representation rather than recovery of physically correct dynamics. A converged representation may still correspond to inaccurate physical estimates. This distinction weakens the theoretical justification for the claimed optimal physical representation.
-
The proposed framework combines several components, including recurrent refinement, physics-aware encoding, cross-system attention, and structured ODE operators. Although ablation studies are provided, it remains unclear whether the improvements primarily arise from explicit physical feedback or from increased model capacity and iterative computation. More controlled comparisons would strengthen the central methodological claim.
The paper presents a well-motivated framework for jointly learning physical representations and dynamical models through fixed-point refinement. The empirical results are promising, and the structured local dynamics offer practical computational advantages. However, the theoretical connection between fixed-point convergence and physical correctness remains insufficiently established. Furthermore, the current experiments do not adequately isolate the benefits of physics-grounded reasoning from those of recurrent computation and architectural complexity. These limitations weaken the central claims of the paper.
Questions for authors
-
Does convergence to the proposed fixed point imply any guarantee regarding the accuracy of the inferred dynamics? Can the authors clarify the relationship between representation self-consistency and physical correctness?
-
How does the proposed method compare against a recurrent Transformer with matched parameters and computational budget but without physics-aware feedback? Such a comparison would help establish whether the improvement originates from physical reasoning rather than recurrent computation alone.
-
How are the contraction conditions enforced during training, and how sensitive are predictive accuracy and convergence speed to the anchoring coefficient and contraction constraints?
11 comments
Q1. Connection to Physical Correctness. (Continued)
(2) Why fixed-point reasoning benefits representation learning. The central motivation is to enrich internal representations with two complementary sources: intrinsic physical information inferred within each ODE system and extrinsic dynamical information learned across heterogeneous ODE systems. First, intrinsic physical information is obtained from inferred local dynamical operators, including , , and , which characterize local rotational behavior, directional expansion or contraction, and growth or decay rates. Through physics-aware encoding (Eq. (8)), these properties enrich the physical context of internal tokens and inform subsequent representation refinement. Second, extrinsic cross-system information is retrieved through the Common Structure Hub (CSH), which captures recurring dynamical patterns across heterogeneous ODE systems (Eqs. (9)--(10)). Together, these two sources enable the model to refine internal representations using both system-specific dynamical information and transferable structures learned across systems.
(3) Empirical evidence supporting the proposed mechanism. Beyond the theoretical convergence guarantees, we provide extensive empirical evidence to evaluate five key questions regarding the effectiveness of our framework.
(i) Does fixed-point reasoning improve internal representations and inferred dynamics? Fig. 4(a,b) and Appendix H show that internal representations progressively approach clusters associated with recurring dynamical patterns, including oscillation, dissipation, and fast--slow dynamics. More importantly, Appendix M (Fig. 19) demonstrates that intermediate trajectory predictions progressively recover system-specific dynamical behaviors across reasoning iterations. These results provide empirical evidence that our representation enrichment mechanism not only organizes internal representations around meaningful dynamical structures but also improves their ability to capture physical dynamics.
(ii) Does the reasoning process converge in practice? Fig. 4(c) shows that the normalized representation-update magnitude decreases rapidly across reasoning iterations, indicating that the Estimator--Refiner process approaches a stable equilibrium. This empirical observation is consistent with the convergence guarantee established in Theorem 1.
(iii) Does the framework learn transferable dynamical reasoning capabilities across heterogeneous ODE systems? Our evaluation covers approximately 40 GB of heterogeneous dynamical-system data, including 38 ODE systems for training and 14 unseen ODE systems for zero-shot evaluation. As shown in Fig. 2, LASS-ODE achieves approximately 90% and 85% average MSE reductions relative to the best competing model for each representative in-domain and zero-shot system, respectively. These reductions are averaged across three representative systems in each setting, with each system evaluated at three observation ratios. Importantly, the strong zero-shot performance provides empirical evidence that LASS-ODE learns a transferable capability to infer and refine local affine dynamical parameters in the latent space, enabling it to capture the dynamics of previously unseen ODE systems without access to their ground-truth governing equations or physical parameters. Complete numerical comparisons are provided in Appendices I and J.
(iv) Can the framework accurately predict realistic physical dynamics? We evaluate power-system frequency dynamics following disturbances and AI data-center voltage transients. As shown in Table 1, LASS-ODE reduces prediction MSE by approximately 85% and 90%, respectively, compared with the second-best methods. Fig. 5 further demonstrates accurate reconstruction of post-disturbance frequency trajectories across different buses and disturbance conditions. These results support the effectiveness of our framework beyond classical ODE benchmarks.
(v) How important is the Fixed-Point Transformer to the overall predictive performance? Fig. 7(a) shows that removing the Fixed-Point Transformer leads to the largest performance degradation among the architectural ablations, increasing prediction MSE by approximately 2.6--2.7 times. This provides empirical evidence for the importance of the overall fixed-point reasoning module in dynamical prediction. To further isolate the contributions of physics-aware feedback and recurrent refinement from model capacity and computation, we have conducted additional controlled ablations, which we discuss in our subsequent responses to your related questions.
Q2. Additional ablation study.
We thank the reviewer for this constructive suggestion. To isolate the contributions of explicit physical feedback from model capacity and iterative computation, we conduct additional controlled ablations. Importantly, we control model capacity at the architectural level by preserving the same hidden dimensions, FFN dimensions, attention configurations, CSH, MoE architecture, prediction head, and number of refinement layers/steps across the variants. In particular, all variants retain the same attention and MoE configurations, including the 8-expert architecture with top-2 routing. Thus, no variant is given additional network width, refinement depth, or architectural components. We also maintain comparable overall model sizes and computational budgets.
Specifically, we compare LASS-ODE against three variants: (i) Looped (No Phys.), which retains the same recurrent, weight-shared Transformer blocks and contraction constraints but removes only the physical feedback ; (ii) Untied (Phys.), which replaces the recurrent Fixed-Point Transformer with a stack of independently parameterized Transformer blocks while retaining physical feedback; and (iii) Untied (No Phys.), which further removes physical feedback from the untied architecture. Importantly, the Physics/No-Physics variants within each architecture retain identical model configurations and parameter counts, differing only in whether is incorporated. For the recurrent/untied comparisons, we preserve the same refinement depth and per-block dimensions, while changing only whether Transformer parameters are shared across refinement steps.
LASS-ODE achieves the lowest MSE of , reducing prediction errors by 37.2%, 20.2%, and approximately 45.8% compared with Looped (No Phys.), Untied (Phys.), and Untied (No Phys.), respectively. In particular, the comparison between LASS-ODE and Looped (No Phys.) isolates the contribution of physical feedback under the same recurrent architecture. Similarly, Untied (Phys.) reduces MSE by 19.6% compared with Untied (No Phys.), independently demonstrating the benefit of physical feedback without recurrent weight sharing. Furthermore, LASS-ODE outperforms Untied (Phys.), demonstrating the benefit of recurrent weight sharing when physical feedback is retained. Likewise, Looped (No Phys.) reduces MSE by 11.9% compared with Untied (No Phys.), demonstrating the contribution of recurrent weight sharing without physical feedback.
Together, these controlled comparisons provide evidence that the predictive improvements arise from explicit physical feedback and recurrent weight sharing, rather than simply increased model capacity or additional iterative computation. In particular, the consistent improvements from physical feedback under both recurrent and untied architectures support its independent contribution to predictive performance. We will include these additional ablation results and implementation details in the revised manuscript.
Q3. Fixed point and dynamics.
We thank the reviewer for this question. As clarified in our response to Question 1, convergence to the proposed fixed point guarantees a unique self-consistent equilibrium under the conditions of Theorem 1, but does not necessarily guarantee the accuracy or identifiability of the true underlying dynamics. Without known governing equations, ground-truth physical parameters, or additional assumptions, such guarantees are generally difficult to establish.
Our framework instead combines representation enrichment with intrinsic and cross-system dynamical information, fixed-point convergence, and trajectory prediction loss. The prediction loss encourages the learned self-consistent representations to support accurate estimation of local affine dynamical parameters in the latent space, enabling prediction of heterogeneous ODE dynamics from observed trajectories. Importantly, the effectiveness of this mechanism is supported by extensive empirical results, particularly strong zero-shot performance on previously unseen ODE systems without access to their ground-truth governing equations or physical parameters. Please see our response to Question 1 for the detailed theoretical formulation, the distinction between self-consistency and physical correctness, and the supporting experimental evidence.
Q1. Connection to Physical Correctness.
We thank the reviewer for this important comment. We first clarify that our objective is a "self-consistent optimal physical representation," as defined in Definition 1, rather than a globally optimal representation that guarantees recovery of the true underlying physics. We will add the qualifier "self-consistent" throughout the manuscript to eliminate this ambiguity. We use this term because, after training, (1) the learned representations are largely enriched with intrinsic physical information and extrinsic cross-system dynamical information; (2) the fixed-point iterations are guaranteed to converge to a unique self-consistent equilibrium under the conditions of Theorem 1; and (3) the trajectory prediction loss encourages the learned self-consistent optimal representation to support accurate estimation of local affine dynamic parameters in the latent space, enabling the model to capture the dynamics of heterogeneous ODE systems. These properties define the scope of our self-consistent optimality, without implying guaranteed recovery of the true governing dynamics. The effectiveness of the overall framework is further supported by extensive experiments, particularly its strong zero-shot performance on previously unseen ODE systems, providing empirical evidence that LASS-ODE learns transferable capabilities to infer and refine dynamical parameters in the latent space without any access to the ground-truth governing equations or physical parameters.
(1) Theoretical formulation and guarantees. Our work addresses data-driven representation learning across heterogeneous ODE systems without assuming known governing equations, ground-truth dynamical parameters, or system-specific physical priors. Under these general conditions, strict guarantees on prediction accuracy, physical parameter identification errors, or identifiability cannot be established without additional assumptions on the underlying dynamics and available observations. Our theoretical contribution instead concerns the self-consistency of the representation learning process.
We define a self-consistent optimal physical representation in Definition 1 by considering three essential variables in a triplet: the learned internal representation E⋆, the predicted state trajectory x^⋆, and the inferred dynamical parameters θ^⋆. We establish three pairwise consistency requirements among these variables. The first two established concerns in physics-informed learning are captured by coordinate consistency and governing consistency. Coordinate consistency addresses potential spatiotemporal coordinate mismatches among the physical system inputs, learned representations, and predicted outputs. Governing consistency, closely related to the physical residual constraints commonly employed in PINNs, requires the predicted trajectories to satisfy the governing dynamics described by the inferred physical parameters.
Beyond these established considerations, we introduce learn-estimate consistency to address a fundamental problem in internal representation learning: how can inferred physical information directly improve learned representations, and how can the refined representations subsequently support more accurate physical inference? Specifically, the Estimator infers dynamical parameters Est(E) from the current representation E, while the Refiner incorporates these inferred properties to enrich the representation through Ref(E,Est(E)). We characterize the inconsistency between physical inference and representation refinement as ∣∣E−Ref(E,Est(E))∣∣F2.
Our fixed-point reasoning framework aims to minimize this learn-estimate inconsistency through iterative Estimator-Refiner updates. Theorem 1 establishes sufficient conditions under which these iterations converge to a unique equilibrium E⋆, satisfying $\hat{\boldsymbol{\theta}}^\star=\mathrm{Est}(E^\star), E^\star=\mathrm{Ref}(E^\star,\hat{\boldsymbol{\theta}}^\star). $ At this equilibrium, further refinement using the inferred physical parameters leaves the representation unchanged, providing a principled criterion for self-consistency and termination of the reasoning process. This guarantee concerns the convergence and mutual consistency of physical inference and representation refinement, rather than recovery of the unknown true physical dynamics.