acceptodds
Withdrawn from ICLR 2027

Adaptive Causal Physics-Informed Neural Networks for Long-Horizon Dynamical System Prediction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Physics-informed neural networks (PINNs) face challenges in long-horizon temporal prediction due to the difficulty of effectively enforcing temporal causality during training. We develop a causal physics-informed framework with three complementary components. First, we introduce an inverse power-law causal weighting scheme that provides flexible control over temporal loss weighting through a single exponent. Second, we develop a learnable exponent network that dynamically adapts this exponent using training loss statistics, eliminating manual tuning. Third, we incorporate Mamba state space models as temporal encoders and develop a physics-informed fusion architecture that combines MLP and Mamba representations through causal residual training. We evaluate the proposed approaches on the Allen–Cahn equation, the chaotic Lorenz system, and two-dimensional incompressible Navier–Stokes flow. The learnable causal exponent reduces the relative error by 97.6 % on Allen–Cahn, while the Mamba architecture achieves a 96.8 % reduction. For the Lorenz system, Mamba attains a minimum physics loss of at Window 4, with component-wise errors below throughout all 5 windows. On Navier–Stokes, Path 1 (LA v2) is the stronger method at equivalent compute; the physics-informed fusion PINN provides a further 6.3% error reduction over the standalone Mamba prediction at Window 3. These results demonstrate the effectiveness of adaptive causality and heterogeneous temporal representations for accurate long-horizon physics-informed learning.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Causual ML, PINNs, State space models, Power law

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_23QwOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The authors propose three techniques to enforce causality in PINNs training: a power-law causal weighting scheme (in contrast with the usual exponencial scheme), a learnable exponent, and a novel architecture combining Mamba state space models and standard MLPs.

These strategies are evaluated on three benchmark problems: an Allen-Cahn equation, a chaotic Lorenz system and a 2D incompressible Navier-Stokes flow.

Critical strengths

The authors present a good reasoning for adopting a power-law weighting scheme.

Critical weaknesses

(1) Although there is a good mathematical explanation about the power-law weighting scheme, there is no much scientific insight for the two other techniques. To some extent, it looks more like a technical report.

(2) The experimental evaluation is limited. There is no explicit explanation on what the baselines are (I assume they are taken from Wang et al., 2024b). There is no mention to training time. There is also no comparison between number of parameters. The comparison per window is useful to get insight on how the proposed strategies work (and what are their limitations), but they seem to be presented as quantitative comparison metrics.

(3) The benchmark problems are not described in the paper. It is not explicitly stated that they are precisely the same as in Wang et al., 2024b, although it seems to be the case. This is particularly problematic because the title mentions "long-horizon dynamical systems", and there is no mention to the size of the temporal domains of the problems.

Questions for authors

I'm not sure I understand Figure 6. The CausalPINNMamba prediction seems to be quite wrong, qualitatively. Is that correct?

Reviewer_tq3rOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

Replaces exponential causal weighting with inverse power-law weighting and learns its exponent through a small neural network. Also investigates a Mamba-enhanced architecture and a physics-informed fusion of two predictors. Evaluates on Allen–Cahn, Lorenz, and Navier–Stokes systems, with different weighting strategies performing best on different problems.

Critical strengths

= The power-law weighting directly addresses the excessive suppression of late-time residuals under exponential weighting.

  • The approximately 209-parameter exponent network offers a lightweight modification to existing PINNs.
Critical weaknesses
  • The training formulation contains a central inconsistency. Applying sg(wi)sg(w_i) as written in Eq. (5) also blocks gradients to the exponent network, whereas the text and Algorithm 1 claim joint optimization.
  • The claim that fusion “cannot degrade below Path 1” is unsupported. Small residual initialization does not guarantee this after training. ㅆhe reported NS Window 3 fusion error is 0.7425, versus 0.04118 for Path 1.
  • Different training budgets and, for Lorenz, different evaluation metrics make it difficult to isolate architectural benefits and computational efficiency.
Questions for authors
  • Does the implementation detach the entire causal weight or only the cumulative residual? Please explicitly derive the gradient reaching the exponent-network parameters.
  • If gradients flow through α\alpha increasing α\alpha reduces the weighted residual objective for fixed residuals. What prevents a degenerate solution that simply pushes α toward its upper bound?
  • How can the reported fusion result be reconciled with the claimed non-degradation guarantee? Are the numbers obtained under identical evaluation conditions?
  • The abstract contains apparent formatting errors, including unintended strikethrough text and unrendered LaTeX commands, which obscure parts of the reported results and should be corrected.
Reviewer_SziaOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes several extensions of causal PINNs for long-horizon dynamical systems: inverse power-law causal weighting, a learnable causal exponent (LA v2), a Mamba-based temporal encoder, and a residual fusion model. The methods are evaluated on Allen-Cahn, Lorenz, and 2D Navier-Stokes. The strongest reported result is on Allen-Cahn, where the learnable-exponent method substantially improves over the paper's exponential and fixed-power-law baselines.

Critical strengths

1. The paper addresses an important practical failure mode of PINNs.

Temporal causality is a meaningful difficulty for stiff and long-horizon dynamical systems, and replacing rapidly collapsing exponential causal weights with a heavier-tailed alternative is intuitively well motivated.

2. The empirical results suggest potentially interesting regime-dependent behavior.

The large Allen-Cahn improvements are encouraging, while the fact that a fixed exponent outperforms the learned exponent on Lorenz suggests that the appropriate causal weighting mechanism may depend meaningfully on the temporal structure of the problem.

Critical weaknesses

1. The definition and justification of the central learnable-exponent method are internally inconsistent.

This is my primary concern. Eq. (4) defines

= \left(1+\epsilon L_{<i}(\theta)\right)^{-\alpha}, \qquad \alpha=\phi_\psi(\cdot),$$ while Eq. (5) defines $$\mathcal L(\theta,\psi) = \sum_i \operatorname{sg}(w_i)L_i(\theta) + \lambda_0L_{\rm IC}(\theta).$$ Under the standard meaning of stop-gradient, $$\frac{\partial\\,\operatorname{sg}(w_i)} {\partial w_i}=0,$$ and since $\psi$ affects the stated objective only through $w_i$, this gives $$\nabla_\psi\mathcal L=0.$$ This directly contradicts the text immediately following Eq. (5), which states that gradients reach $\psi$ through the dependence of $\alpha$ on $\psi$, as well as Algorithm 1, which states that $\theta$ and $\psi$ are jointly updated. This is consequential rather than merely notational. If the implementation literally applies stop-gradient to $w_i$, the learnable exponent network cannot be trained by the stated objective. If instead the intended implementation detaches only the residual statistics entering $w_i$ with respect to $\theta$, while preserving the $\psi \to \alpha \to w_i$ gradient path, then the mathematical formulation needs to be corrected. The same issue also makes the Appendix A.1 "gradient informativeness" motivation difficult to reconcile with the stated objective. The paper motivates power-law weighting partly through the slower decay of the derivative of the causal weight, although differentiating through $w_i$ with respect to the accumulated residual appears to be precisely what the stop-gradient removes. There is also a smaller but concrete inconsistency in the fusion discussion. The paper states that the P1-anchored residual design "ensures the fusion cannot degrade below Path 1." Near-zero initialization only ensures that fusion starts close to Path 1; it does not provide such a performance guarantee. Indeed, on Navier-Stokes W3, Path 1 has relative error $4.118\times10^{-2}$, whereas the reported fusion error is $7.425\times10^{-1}$. **2. The experiments do not yet isolate the claimed benefit of learning the causal exponent.** The main Allen-Cahn interpretation is that adaptivity of $\alpha$, rather than the power-law form alone, is the primary driver of the improvement. However, the relevant comparison is only against one fixed choice, $\alpha=2$: $$2.829\times10^{-3} \quad\text{(fixed $\alpha=2$)}, \qquad 8.069\times10^{-5} \quad\text{(LA v2)}.$$ This does not distinguish the benefit of *learning* $\alpha$ from the simpler possibility that another fixed value of $\alpha$ performs substantially better. This concern is particularly relevant because the learned exponent itself takes values different from 2, and because on Lorenz a fixed $\alpha=3$ outperforms LA v2. A small fixed-$\alpha$ comparison spanning the range used by the learned exponent would therefore materially affect the central empirical conclusion. If no reasonable fixed exponent approaches LA v2, the claim for adaptivity would become substantially stronger; if one does, the interpretation of the contribution changes. The evidence for the Mamba component is also difficult to interpret independently. In particular, Path 1 and Path 2 use substantially different collocation and training budgets on Navier-Stokes, so the claim that the observed gap is "compute-limited rather than architecture-limited" is not established by the current experiments. I do not view a large matched-compute rerun as necessary for this review, but the corresponding conclusion should be weakened unless additional evidence is already available. Similarly, the absence of a direct comparison with the closely related PINNMamba work limits how strongly the architectural contribution can be assessed.
Questions for authors

1. What exactly is stop-gradient applied to in the actual implementation of LA v2?

As written,

= \sum_i\operatorname{sg}(w_i)L_i + \lambda_0L_{\rm IC}$$ implies $\nabla_\psi\mathcal L=0$, while the text states that gradients reach $\psi$ through $\alpha$ and $w_i$. Please specify the actual computational graph or corrected objective. In particular, is $w_i$ itself detached, or are only the loss statistics $L_{<i}$ detached with respect to $\theta$ while the $\psi \to \alpha \to w_i$ path remains active? This clarification is essential to my assessment. If the implementation follows Eq. (5) literally, I do not understand how the reported learned-$\alpha$ results could arise. If this is only a notation error and the implementation preserves the gradient to $\psi$, that would substantially alleviate my main soundness concern. **2. On Allen-Cahn, does LA v2 outperform reasonable fixed choices of $\alpha$, rather than only the reported $\alpha=2$ baseline?** The paper attributes the large improvement primarily to exponent adaptivity, but only one fixed power-law exponent is reported. A small comparison using a few fixed values spanning the range actually taken by the learned exponent would be sufficient to distinguish "learning $\alpha$" from simply selecting a better constant $\alpha$. I view this as the only additional experiment that is important for assessing the paper's central empirical claim. **3. What is meant by the statement that the fusion architecture "cannot degrade below Path 1"?** The reported Navier-Stokes W3 errors are $4.118\times10^{-2}$ for Path 1 and $7.425\times10^{-1}$ for fusion, so the statement does not hold for predictive relative $\ell_2$ error. If the intended statement concerns only initialization, please revise the claim accordingly; otherwise, please clarify the precise quantity for which a non-degradation guarantee is intended.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.