- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies a data-assisted solution reconstruction and statistical learning problem for time-dependent PDEs, where partial observations of the solution and noisy measurements of the forcing term are available. The main methodological contribution is an energy-based loss function. Inspired by the weighted energy-dissipation framework, the authors use elliptic regularization to construct an approximate variational formulation for the heat equation, and incorporate this energy structure into a supervised learning framework.
The proposed loss function consists of three components: an energy regularization term that incorporates the PDE structure and noisy forcing information, a Sobolev data-fitting term, and a debiasing correction term that compensates for the approximation introduced by elliptic regularization.
The paper is primarily theory-driven. The authors show, under suitable assumptions, that the true PDE solution is a minimizer of the proposed population loss function. They further derive estimation error bounds, minimax lower bounds, and generalization bounds, characterizing the conditions under which energy regularization can improve statistical convergence rates. The theoretical analysis focuses on the heat equation, while numerical experiments explore extensions to other parabolic and hyperbolic PDEs. The experiments demonstrate improved accuracy compared with several PINN-based methods and investigate the additional benefits of energy regularization over purely Sobolev-based regression.
Critical strengths
-
For problem setting. The paper studies a meaningful solution reconstruction problem for time-dependent PDEs, combining partial solution observations with noisy forcing information.
-
Loss design. The proposed energy-guided loss is mathematically motivated by the weighted energy-dissipation framework, integrating PDE structure, Sobolev data fitting, and a debiasing correction.
-
Theoritical analysis. The paper investigates estimation and generalization bounds, particularly how spatial-temporal regularity and energy regularization affect statistical convergence rates.
-
Experiments: The paper evaluates the method on several parabolic and hyperbolic PDEs, including high-dimensional examples, and provide useful ablation studies demonstrating the benefits of energy regularization over pure Sobolev regression.
Critical weaknesses
Methodogy
The method relies on spatial and temporal derivatives of the true solution, which may be difficult to obtain in practice.
Theory Part
-
Incorrect approximation bound. Lemma E.4 (Eqs. E.138–E.139) uses a multiplicative bound for spatial and temporal truncation errors, which is generally incorrect. Since this lemma is essential to Theorem 3.1, the claimed convergence rates are not rigorously established.
-
The Fano argument in Appendix E.6 considers only noisy forcing observations while ignoring the noiseless solution labels in the stated observation model. This invalidates the KL-divergence argument used to establish the minimax lower bound in Theorem 3.3.
-
Assumption. The space-time samples share temporal coordinates and are therefore not fully independent. However, the empirical-process analysis in Appendix E.2 treats all observations as independent, raising concerns about the statistical error bounds.
-
Approximation space. The truncated wavelet approximation does not necessarily preserve the prescribed initial and boundary conditions. Consequently, its membership in the admissible function space , assumed in Appendix E.4, is not justified.
Experiments
Most experiments involve relatively simple linear PDEs or highly structured analytical solutions, which are insufficient to demonstrate the method's effectiveness on more challenging time-dependent PDEs.
Questions for authors
Questions for the Authors
-
Could the authors revisit the approximation theory and regularity assumptions, particularly Lemma E.4?
-
Does the derivative-based Sobolev loss introduce additional training difficulties or optimization instability?
-
Could the authors consider more challenging benchmarks, such as a nonlinear 3D FK equation?
-
Can the proposed method achieve comparable accuracy with fewer observation points than PINNs, accounting for the additional derivative information?
-
How robust is the method to higher noise levels, especially when derivatives are estimated from noisy data?
-
How important is the debiasing correction term in practice? Could the authors provide an ablation study?
4 comments
i.i.d. space-time sampling (W3). We apologize for the inconsistent sampling description. Please also find our response to Reviewer 4N5E on sampling model. In both the theoretical analysis and numerical experiments, the full space-time points are sampled i.i.d.:
Unif(\Omega)\otimes Unif([0,T]), \qquad k=1,\ldots,N.$$ Each observation has an independently sampled temporal coordinate. The main experiments use $N=4,000$ such points. The earlier description of several spatial observations at a shared time will be corrected throughout the manuscript. Under this design, the independent-sample symmetrization step applies to all $n$ observations. Thus the shared-time dependence identified by the reviewer arises from the incorrect description. The space--time resolution relation $\tau\asymp h^2$ in the approximation argument expresses parabolic regularity; it does not require grouping observations at common time coordinates. **Boundary and initial conditions in wavelet truncation (W4).** The reviewer is also correct that extension followed by unrestricted wavelet truncation does not automatically preserve the boundary and initial conditions in A3. There are a few technical caveats required to address this issue. These technical constructions are independent of the scope of our paper and do not affect our assumptions. We will add a remark after the definition of $\mathcal U_J$ explaining how these conditions are addressed in the literature on wavelet approximations for PDEs. In general, one needs to introduce a lifting function $\ell$, with $$\ell|_{\partial\Omega\times(0,T)}=0,\qquad \ell(\cdot,0)=u_0^*,$$ and truncate only $u^*-\ell$ using boundary-adapted wavelets. In our paper, we can choose $\phi_\xi|_{\partial\Omega}=0$ and $\phi_\tau(0)=0$ and denote their truncated span by $V_{J,J'}^0$. Then $$\bar u_J(x,t)=\ell(x,t)+ \sum_{|\xi|_\infty\leq J,\\;\tau\leq J^2} a_{\xi,\tau}\phi_\xi(x)\phi_\tau(t)$$ satisfies, at every finite truncation level, $$\bar u_J|_{\partial\Omega\times(0,T)}=0, \qquad \bar u_J(\cdot,0)=u_0^*.$$ There are several papers supporting such constructions. Dahmen and Schneider (1998) construct wavelet Riesz bases with homogeneous Dirichlet conditions on selected facets of a cube; selecting the spatial boundary facets provides the required $\phi_\xi$ on cubical domains. Han and Michelle (2021, Sections 5–6 and Example 7.2) give interval constructions with prescribed homogeneous boundary conditions. **Does the derivative-based Sobolev loss introduce additional training difficulties or optimization instability? (Q2).** Although optimization is out of scope in our paper, we've provided an analysis on our hardest example Sine-Gordon equations on different learning rates and batch-size. Optimization instability is observed on small batch size and bigger learning rate. It is worth to investigate the causes of these problems in future work. **Could the authors consider more challenging benchmarks? (Q3).** Our experiments include nonlinear viscous Hamilton--Jacobi equations in dimensions $1,3,10,20,50$ and nonlinear Sine--Gordon equations in one and three dimensions. These are computational extensions beyond linear parabolic theories. We agree that their structured reference solutions leave room for more challenging theoretical analysis. \\ For HJB at $d=50$, the mean relative $L^2$ error is $0.964\%$, compared with $2.959\%$ for SA-PINN, over ten seeds. On three-dimensional Sine--Gordon, the reported MSE is $7.39\times10^{-6}$, compared with $1.39\times10^{-4}$ for SA-PINN; the relative errors are $8.89\times10^{-4}$ and $3.85\times10^{-3}$, respectively. Due to time constraint, we will not provide more nonlinear PDEs examples here. **Achieve comparable accuracy using fewer points in PINNs using additional derivatives labels (Q4).**. Due to time constraint, we are able to run 3-experiments on LP1-3 on which PINNs reduce 3/4 collocation points if supervison labels are also provided to achieve similar comparable accuracy. | PDE | PINN+$u$ | PINN+$u$+derivatives | Reduction | |:---|---:|---:|---:| | LP1 | 8,192 | 2,048 | 75.00% | | LP2 | 8,192 | 2,048 | 75.00% | | LP3 | 8,192 | 2,048 | 75.00% |Robustness to higher noise level (Q5). We have further tested on additional noise level on both and . We obtain the following results. In our experiments with additional 20% noise, energy regularization reduced the mean MSE relative to pure Sobolev fitting by 35.2%, 72.5%, 63.4%, and 40.5% for Heat and HJBs respectively. These improvements were smaller than the original setting.
| PDE | Original: Pure | Original: Regularized | Additional 20% noise: Pure | Additional 20% noise: Regularized |
|---|---|---|---|---|
| Heat | ||||
| LP1 | ||||
| LP2 | ||||
| LP3 |
Role of debiasing term (Q6). We have already conducted a separate ablation comparing with on a manufactured forced-heat problem, using mixed Sobolev fitting. The two groups use the same sampled interior data, validation data, network architecture, learning rate , and 1,000 Adam steps. We fix , , and report all ten paired seeds.
This supplementary ablation uses interior observations organized as 32 sampled times with 32 spatial points per time. The resulting test errors, reported as means sample standard deviations, are:
| Metric | Without debiasing | With debiasing |
|---|---|---|
| MSE | ||
| Relative (%) | ||
| Relative (%) |
Using the unrounded group means, adding the correction changes mean MSE by -0.375%, mean relative error by -0.096%, and mean relative error by -0.024%.
As we state in our paper, the correction term here is negligible in practical training. The correction is linear in the candidate : it changes the first variation but has zero second variation. Thus, in function space, it shifts the minimizer without changing the curvature of the objective. Its implementation also introduces no higher-order derivatives of the candidate network; and the terminal enter as fixed labels multiplying network values.
More precisely, let and write the operator , which is independent of . If the population functional is -strongly convex in on the common convex admissible set, its minimizers with and without the correction satisfy
\leq \frac{\beta}{\mu} ||m_\epsilon||_{{\mathcal H}^*}.$$ As $\beta$ is a small constant between 0 and 1 and the dual operator norm is bounded, this leads to a negligible difference, consistent with our experimental results. ### References R. Stevenson and J. Storn. "Interpolation Operators for Parabolic Problems." *Numerische Mathematik*, **155**, 211–238, 2023. [DOI: 10.1007/s00211-023-01373-9](https://doi.org/10.1007/s00211-023-01373-9). [arXiv:2212.04134v1](https://arxiv.org/abs/2212.04134). W. Dahmen and R. Schneider. "Wavelets with Complementary Boundary Conditions—Function Spaces on the Cube." *Results in Mathematics*, **34**, 255–293, 1998. [DOI: 10.1007/BF03322055](https://doi.org/10.1007/BF03322055). B. Han and M. Michelle. "Wavelets on Intervals Derived from Arbitrary Compactly Supported Biorthogonal Multiwavelets." *Applied and Computational Harmonic Analysis*, **53**, 270–331, 2021. [arXiv:2012.05371v2](https://arxiv.org/abs/2012.05371). H. Harbrecht, C. Schwab, and M. Zank. "Wavelet Compressed, Modified Hilbert Transform in the Space–Time Discretization of the Heat Equation." *IMA Journal of Numerical Analysis*, **46**(4), 2139–2175, 2026. [DOI: 10.1093/imanum/draf061](https://doi.org/10.1093/imanum/draf061). [arXiv:2402.10346v2](https://arxiv.org/abs/2402.10346). W. Dahmen, A. Kunoth, and K. Urban. "Biorthogonal Spline Wavelets on the Interval—Stability and Moment Conditions." *Applied and Computational Harmonic Analysis*, **6**(2), 132–196, 1999. [DOI: 10.1006/acha.1998.0247](https://doi.org/10.1006/acha.1998.0247). Open preprint: WIAS Preprint No. 265 (1996; result numbering used above).Thank you for the detailed response and clarifications. I appreciate the authors' efforts to address my concerns.
-
Approximation theorem. I find the proposed correction using tensor-product approximation reasonable and appreciate the authors' clarification.
-
Minimax lower bound (Theorem 3.3). I still have one concern regarding the Fano argument. Under the stated observation model with noiseless solution labels, it remains unclear how the KL divergence between the full observation distributions can be controlled. Considering only noisy forcing observations does not appear sufficient. One possible direction is to incorporate noise into the solution observations, which may also better reflect practical data-driven PDE reconstruction settings and provide a more natural framework for the minimax analysis.
-
Numerical experiments. I acknowledge that I overlooked some of the experiments presented in the appendix during my initial review. These experiments provide additional evidence of the method's effectiveness. I would encourage the authors to consider moving some of the more challenging PDE benchmarks from the appendix to the main text, as they would help strengthen the experimental presentation.
Overall, I appreciate the authors' constructive response. My main remaining concern is the validity of the minimax lower-bound argument under the current observation model.
We thank the reviewer for the constructive feedback.
Approximation Error (W1, Q1). We agree with the reviewer that the general statement of Lemma E.4 requires correction. We have revisit approximation argument in our paper and generalized version of similar approximation argument will still give same approximation result.
Our current approximation analysis is special case because of the linear parabolic setting where space and time basis are multiplicative. For general approximation error, we need to write it in term of space-time tensor product. In general form, we can generalize the approximation argument using the tensor-product interpolation strategy of Stevenson and Storn (2023, Theorem 12, equations (22)–(23)): they use general projection operator to analyze space-time approximation. Their operators Ix,It play the roles of our QJx,QJ′t, with hx≍J−1 and ht≍(J′)−1. Their proof also uses the commutation of the spatial operator with ∂t, which allows us to apply the spatial Jackson estimate to both w and wt in our H-norm.
Using this general formulation, we can write the pointwise product of spatial and temporal basis functions in tensor product form:
=\phi_\xi(x)\phi_\tau(t) =(\phi_\xi\otimes\phi_\tau)(x,t).$$ We will present the proof idea of generalized approximation theorem: Let $\mathcal H=H^1(0,T;H^1(\Omega))$ and $\ell$ be a lifting of the prescribed traces and write $w=u^*-\ell$. For an integer $s\geq1$, the general approximation result applies to functions with $$w\in H^1_tH^{2s}_x\cap H^{s+1}_tL^2_x.$$ Let $Q_J^x$ and $Q_{J'}^t$ be directional projectors satisfying spatial $L^2$ stability, the spatial Jackson estimate of order $J^{-(2s-1)}$ in $H^1_x$, the spatial inverse bound $\lVert v_J\rVert_{H^1_x}\lesssim J\lVert v_J\rVert_{L^2_x}$, and the temporal Jackson estimate of order $(J')^{-s}$ in $H^1_tL^2_x$. The identity $$w-Q_J^xQ_{J'}^tw=(I-Q_J^x)w+Q_J^x(I-Q_{J'}^t)w$$ then gives the additive estimate $$||w-Q_J^xQ_{J'}^tw||_{\mathcal H}^{2} \lesssim J^{-(4s-2)}||w||_{H^1_tH^{2s}_x}^{2} +J^2(J')^{-2s}||\partial_t^{s+1}w||_{L^2_{t,x}}^{2}.$$ Taking $J'=J^2$ and $\bar u_J=\ell+Q_J^xQ_{J^2}^t(u^*-\ell)$ yields $$||u^*-\bar u_J||_{\mathcal H}^{2} \lesssim J^{-(4s-2)} \left( ||w||_{H^1_tH^{2s}_x}^{2} +||\partial_t^{s+1}w||_{L^2_{t,x}}^{2} \right).$$ This result also holds for general functions with the stated regularity. Hence the same approximation exponent is available in that general class; it is not a consequence of an arbitrary single mixed-Sobolev assumption. Under the assumptions in our paper, A3 supplies $u^*\in H^1_tH^{2s}_x$ and $\partial_t^s u^*\in L^2_tH^2_x$, while A1 supplies $\partial_t^s f\in L^2_{t,x}$. Differentiating the PDE in distributions therefore gives the additional temporal regularity: $$\partial_t^{s+1}u^* =\Delta\partial_t^s u^*+\partial_t^s f\in L^2_{t,x}.$$ Under the relation $2p+q=2s+2$, its approximation rate is $$J^{-(4s-2)}=J^{-2(2p+q-3)},$$ which remains the same as the original approximation result. **Solution label in lower bound (W2).** We want to clarify the misunderstanding here. First, information of the noiseless solution label $u$ is used through the source function $f$ via the inverse differential operator in the heat equation. In particular for Fano's argument, KL divergence is built through making $f$ indistinguishable first and then we apply parabolic estimate to use $f$ estimate to control $u$ estimate and therefore obtain KL divergence on $u$. It is more natural that we operate hypothesis reduction on the source space, because if one were to discretize on the solution space, then there would be no guarantee that the $u_j$ entries in the maximal finite subset to construct the packing number satisfy the PDE constraint.