acceptodds
Under review as a conference paper at ICLR 2027

Energy-Guided Learning for Heat Equation with Noisy Source: Optimal Estimation and Generalization Bounds

Abstract

Data and the governing partial differential equations (PDEs) of continuous-time dynamical systems provide rich structure for statistical learning, yet how to integrate data with PDE constraints in a principled manner remains a central challenge. We characterize the statistical estimation and generalization properties of learning weak and nonsmooth solutions to parabolic PDEs from finite, randomly sampled data. We introduce a variational formulation for a prototypical class of time-dependent heat equations with unknown forcing terms, in which the solution is characterized as the minimizer of an energy-regularized, ellipticity-debiased Sobolev regression objective. Moving beyond standard supervised learning and regression approaches, this formulation yields a learning framework that leverages auxiliary data—samples of the source function—to construct a physics-informed energy regularization. Our analysis reveals that learning in parabolic PDEs exhibits intrinsic anisotropy between spatial and temporal regularity, in sharp contrast to static elliptic problems, leading to fundamentally different statistical behaviors. We establish non-asymptotically rate-optimal estimation and generalization error bounds that explicitly capture the interplay among temporal sampling, spatial resolution, and solution smoothness. Our numerical experiments demonstrate substantial orders of magnitude of reduction in testing errors against various physics-informed neural network methods on various parabolic and hyperbolic PDEs.

Primary area:
learning on time series and dynamical systems
Keywords:
Learning PDE solution, energy regularization, minimax optimality, generalization bound

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_5aiAOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies a data-assisted solution reconstruction and statistical learning problem for time-dependent PDEs, where partial observations of the solution u(x,t)u(x,t) and noisy measurements of the forcing term are available. The main methodological contribution is an energy-based loss function. Inspired by the weighted energy-dissipation framework, the authors use elliptic regularization to construct an approximate variational formulation for the heat equation, and incorporate this energy structure into a supervised learning framework.

The proposed loss function consists of three components: an energy regularization term that incorporates the PDE structure and noisy forcing information, a Sobolev data-fitting term, and a debiasing correction term that compensates for the approximation introduced by elliptic regularization.

The paper is primarily theory-driven. The authors show, under suitable assumptions, that the true PDE solution u∗u^* is a minimizer of the proposed population loss function. They further derive estimation error bounds, minimax lower bounds, and generalization bounds, characterizing the conditions under which energy regularization can improve statistical convergence rates. The theoretical analysis focuses on the heat equation, while numerical experiments explore extensions to other parabolic and hyperbolic PDEs. The experiments demonstrate improved accuracy compared with several PINN-based methods and investigate the additional benefits of energy regularization over purely Sobolev-based regression.

Critical strengths
  1. For problem setting. The paper studies a meaningful solution reconstruction problem for time-dependent PDEs, combining partial solution observations with noisy forcing information.

  2. Loss design. The proposed energy-guided loss is mathematically motivated by the weighted energy-dissipation framework, integrating PDE structure, Sobolev data fitting, and a debiasing correction.

  3. Theoritical analysis. The paper investigates estimation and generalization bounds, particularly how spatial-temporal regularity and energy regularization affect statistical convergence rates.

  4. Experiments: The paper evaluates the method on several parabolic and hyperbolic PDEs, including high-dimensional examples, and provide useful ablation studies demonstrating the benefits of energy regularization over pure Sobolev regression.

Critical weaknesses

Methodogy

The method relies on spatial and temporal derivatives of the true solution, which may be difficult to obtain in practice.

Theory Part

  1. Incorrect approximation bound. Lemma E.4 (Eqs. E.138–E.139) uses a multiplicative bound for spatial and temporal truncation errors, which is generally incorrect. Since this lemma is essential to Theorem 3.1, the claimed convergence rates are not rigorously established.

  2. The Fano argument in Appendix E.6 considers only noisy forcing observations while ignoring the noiseless solution labels in the stated observation model. This invalidates the KL-divergence argument used to establish the minimax lower bound in Theorem 3.3.

  3. Assumption. The space-time samples share temporal coordinates and are therefore not fully independent. However, the empirical-process analysis in Appendix E.2 treats all nmnm observations as independent, raising concerns about the statistical error bounds.

  4. Approximation space. The truncated wavelet approximation does not necessarily preserve the prescribed initial and boundary conditions. Consequently, its membership in the admissible function space UJU_J, assumed in Appendix E.4, is not justified.

Experiments

Most experiments involve relatively simple linear PDEs or highly structured analytical solutions, which are insufficient to demonstrate the method's effectiveness on more challenging time-dependent PDEs.

Questions for authors

Questions for the Authors

  1. Could the authors revisit the approximation theory and regularity assumptions, particularly Lemma E.4?

  2. Does the derivative-based Sobolev loss introduce additional training difficulties or optimization instability?

  3. Could the authors consider more challenging benchmarks, such as a nonlinear 3D FK equation?

  4. Can the proposed method achieve comparable accuracy with fewer observation points than PINNs, accounting for the additional derivative information?

  5. How robust is the method to higher noise levels, especially when derivatives are estimated from noisy data?

  6. How important is the debiasing correction term in practice? Could the authors provide an ablation study?

4 comments

We thank the reviewer for the constructive feedback.

Approximation Error (W1, Q1). We agree with the reviewer that the general statement of Lemma E.4 requires correction. We have revisit approximation argument in our paper and generalized version of similar approximation argument will still give same approximation result.

Our current approximation analysis is special case because of the linear parabolic setting where space and time basis are multiplicative. For general approximation error, we need to write it in term of space-time tensor product. In general form, we can generalize the approximation argument using the tensor-product interpolation strategy of Stevenson and Storn (2023, Theorem 12, equations (22)–(23)): they use general projection operator to analyze space-time approximation. Their operators Ix,It\mathcal I_x,\mathcal I_t play the roles of our QJx,QJ′tQ_J^x,Q_{J'}^t, with hx≍J−1h_x\asymp J^{-1} and ht≍(J′)−1h_t\asymp(J')^{-1}. Their proof also uses the commutation of the spatial operator with ∂t\partial_t, which allows us to apply the spatial Jackson estimate to both ww and wtw_t in our H\mathcal H-norm.

Using this general formulation, we can write the pointwise product of spatial and temporal basis functions in tensor product form:

=\phi_\xi(x)\phi_\tau(t) =(\phi_\xi\otimes\phi_\tau)(x,t).$$ We will present the proof idea of generalized approximation theorem: Let $\mathcal H=H^1(0,T;H^1(\Omega))$ and $\ell$ be a lifting of the prescribed traces and write $w=u^*-\ell$. For an integer $s\geq1$, the general approximation result applies to functions with $$w\in H^1_tH^{2s}_x\cap H^{s+1}_tL^2_x.$$ Let $Q_J^x$ and $Q_{J'}^t$ be directional projectors satisfying spatial $L^2$ stability, the spatial Jackson estimate of order $J^{-(2s-1)}$ in $H^1_x$, the spatial inverse bound $\lVert v_J\rVert_{H^1_x}\lesssim J\lVert v_J\rVert_{L^2_x}$, and the temporal Jackson estimate of order $(J')^{-s}$ in $H^1_tL^2_x$. The identity $$w-Q_J^xQ_{J'}^tw=(I-Q_J^x)w+Q_J^x(I-Q_{J'}^t)w$$ then gives the additive estimate $$||w-Q_J^xQ_{J'}^tw||_{\mathcal H}^{2} \lesssim J^{-(4s-2)}||w||_{H^1_tH^{2s}_x}^{2} +J^2(J')^{-2s}||\partial_t^{s+1}w||_{L^2_{t,x}}^{2}.$$ Taking $J'=J^2$ and $\bar u_J=\ell+Q_J^xQ_{J^2}^t(u^*-\ell)$ yields $$||u^*-\bar u_J||_{\mathcal H}^{2} \lesssim J^{-(4s-2)} \left( ||w||_{H^1_tH^{2s}_x}^{2} +||\partial_t^{s+1}w||_{L^2_{t,x}}^{2} \right).$$ This result also holds for general functions with the stated regularity. Hence the same approximation exponent is available in that general class; it is not a consequence of an arbitrary single mixed-Sobolev assumption. Under the assumptions in our paper, A3 supplies $u^*\in H^1_tH^{2s}_x$ and $\partial_t^s u^*\in L^2_tH^2_x$, while A1 supplies $\partial_t^s f\in L^2_{t,x}$. Differentiating the PDE in distributions therefore gives the additional temporal regularity: $$\partial_t^{s+1}u^* =\Delta\partial_t^s u^*+\partial_t^s f\in L^2_{t,x}.$$ Under the relation $2p+q=2s+2$, its approximation rate is $$J^{-(4s-2)}=J^{-2(2p+q-3)},$$ which remains the same as the original approximation result. **Solution label in lower bound (W2).** We want to clarify the misunderstanding here. First, information of the noiseless solution label $u$ is used through the source function $f$ via the inverse differential operator in the heat equation. In particular for Fano's argument, KL divergence is built through making $f$ indistinguishable first and then we apply parabolic estimate to use $f$ estimate to control $u$ estimate and therefore obtain KL divergence on $u$. It is more natural that we operate hypothesis reduction on the source space, because if one were to discretize on the solution space, then there would be no guarantee that the $u_j$ entries in the maximal finite subset to construct the packing number satisfy the PDE constraint.

i.i.d. space-time sampling (W3). We apologize for the inconsistent sampling description. Please also find our response to Reviewer 4N5E on sampling model. In both the theoretical analysis and numerical experiments, the full space-time points are sampled i.i.d.:

Unif(\Omega)\otimes Unif([0,T]), \qquad k=1,\ldots,N.$$ Each observation has an independently sampled temporal coordinate. The main experiments use $N=4,000$ such points. The earlier description of several spatial observations at a shared time will be corrected throughout the manuscript. Under this design, the independent-sample symmetrization step applies to all $n$ observations. Thus the shared-time dependence identified by the reviewer arises from the incorrect description. The space--time resolution relation $\tau\asymp h^2$ in the approximation argument expresses parabolic regularity; it does not require grouping observations at common time coordinates. **Boundary and initial conditions in wavelet truncation (W4).** The reviewer is also correct that extension followed by unrestricted wavelet truncation does not automatically preserve the boundary and initial conditions in A3. There are a few technical caveats required to address this issue. These technical constructions are independent of the scope of our paper and do not affect our assumptions. We will add a remark after the definition of $\mathcal U_J$ explaining how these conditions are addressed in the literature on wavelet approximations for PDEs. In general, one needs to introduce a lifting function $\ell$, with $$\ell|_{\partial\Omega\times(0,T)}=0,\qquad \ell(\cdot,0)=u_0^*,$$ and truncate only $u^*-\ell$ using boundary-adapted wavelets. In our paper, we can choose $\phi_\xi|_{\partial\Omega}=0$ and $\phi_\tau(0)=0$ and denote their truncated span by $V_{J,J'}^0$. Then $$\bar u_J(x,t)=\ell(x,t)+ \sum_{|\xi|_\infty\leq J,\\;\tau\leq J^2} a_{\xi,\tau}\phi_\xi(x)\phi_\tau(t)$$ satisfies, at every finite truncation level, $$\bar u_J|_{\partial\Omega\times(0,T)}=0, \qquad \bar u_J(\cdot,0)=u_0^*.$$ There are several papers supporting such constructions. Dahmen and Schneider (1998) construct wavelet Riesz bases with homogeneous Dirichlet conditions on selected facets of a cube; selecting the spatial boundary facets provides the required $\phi_\xi$ on cubical domains. Han and Michelle (2021, Sections 5–6 and Example 7.2) give interval constructions with prescribed homogeneous boundary conditions. **Does the derivative-based Sobolev loss introduce additional training difficulties or optimization instability? (Q2).** Although optimization is out of scope in our paper, we've provided an analysis on our hardest example Sine-Gordon equations on different learning rates and batch-size. Optimization instability is observed on small batch size and bigger learning rate. It is worth to investigate the causes of these problems in future work. **Could the authors consider more challenging benchmarks? (Q3).** Our experiments include nonlinear viscous Hamilton--Jacobi equations in dimensions $1,3,10,20,50$ and nonlinear Sine--Gordon equations in one and three dimensions. These are computational extensions beyond linear parabolic theories. We agree that their structured reference solutions leave room for more challenging theoretical analysis. \\ For HJB at $d=50$, the mean relative $L^2$ error is $0.964\%$, compared with $2.959\%$ for SA-PINN, over ten seeds. On three-dimensional Sine--Gordon, the reported MSE is $7.39\times10^{-6}$, compared with $1.39\times10^{-4}$ for SA-PINN; the relative errors are $8.89\times10^{-4}$ and $3.85\times10^{-3}$, respectively. Due to time constraint, we will not provide more nonlinear PDEs examples here. **Achieve comparable accuracy using fewer points in PINNs using additional derivatives labels (Q4).**. Due to time constraint, we are able to run 3-experiments on LP1-3 on which PINNs reduce 3/4 collocation points if supervison labels are also provided to achieve similar comparable accuracy. | PDE | PINN+$u$ | PINN+$u$+derivatives | Reduction | |:---|---:|---:|---:| | LP1 | 8,192 | 2,048 | 75.00% | | LP2 | 8,192 | 2,048 | 75.00% | | LP3 | 8,192 | 2,048 | 75.00% |

Robustness to higher noise level (Q5). We have further tested on additional 20%20\% noise level on both uu and ff. We obtain the following results. In our experiments with additional 20% noise, energy regularization reduced the mean MSE relative to pure Sobolev fitting by 35.2%, 72.5%, 63.4%, and 40.5% for Heat and HJBs respectively. These improvements were smaller than the original setting.

PDEOriginal: Pure H1H^1Original: RegularizedAdditional 20% noise: Pure H1H^1Additional 20% noise: Regularized
Heat(4.437±3.651)×10−6(4.437\pm3.651)\times10^{-6}(9.448±7.284)×10−7(9.448\pm7.284)\times10^{-7}(4.387±3.578)×10−6(4.387\pm3.578)\times10^{-6}(2.842±3.069)×10−6(2.842\pm3.069)\times10^{-6}
LP1(1.891±2.088)×10−5(1.891\pm2.088)\times10^{-5}(1.790±1.217)×10−6(1.790\pm1.217)\times10^{-6}(1.900±2.123)×10−5(1.900\pm2.123)\times10^{-5}(5.216±4.525)×10−6(5.216\pm4.525)\times10^{-6}
LP2(3.206±2.972)×10−7(3.206\pm2.972)\times10^{-7}(3.317±1.926)×10−8(3.317\pm1.926)\times10^{-8}(3.191±2.957)×10−7(3.191\pm2.957)\times10^{-7}(1.169±1.069)×10−7(1.169\pm1.069)\times10^{-7}
LP3(1.401±1.133)×10−7(1.401\pm1.133)\times10^{-7}(3.210±1.720)×10−9(3.210\pm1.720)\times10^{-9}(1.401±1.126)×10−7(1.401\pm1.126)\times10^{-7}(8.332±9.050)×10−8(8.332\pm9.050)\times10^{-8}

Role of debiasing term (Q6). We have already conducted a separate ablation comparing D+βIϵD+\beta I_\epsilon with D+β(Iϵ+Mϵ)D+\beta(I_\epsilon+M_\epsilon) on a manufactured forced-heat problem, using mixed Sobolev fitting. The two groups use the same sampled interior data, validation data, network architecture, learning rate 0.0030.003, and 1,000 Adam steps. We fix ϵ=0.1\epsilon=0.1, β=0.01\beta=0.01, and report all ten paired seeds.

This supplementary ablation uses N=1,024N=1,024 interior observations organized as 32 sampled times with 32 spatial points per time. The resulting test errors, reported as means ±\pm sample standard deviations, are:

MetricWithout debiasingWith debiasing
MSE(1.3194±1.4547)×10−7(1.3194\pm1.4547)\times10^{-7}(1.3145±1.4421)×10−7(1.3145\pm1.4421)\times10^{-7}
Relative L2L^2 (%)0.06888±0.036370.06888\pm0.036370.06882±0.036170.06882\pm0.03617
Relative Ht1Hx1H_t^1H_x^1 (%)0.56815±0.137060.56815\pm0.137060.56801±0.136840.56801\pm0.13684

Using the unrounded group means, adding the correction changes mean MSE by -0.375%, mean relative L2L^2 error by -0.096%, and mean relative Ht1Hx1H_t^1H_x^1 error by -0.024%.

As we state in our paper, the correction term here is negligible in practical training. The correction is linear in the candidate uu: it changes the first variation but has zero second variation. Thus, in function space, it shifts the minimizer without changing the curvature of the objective. Its implementation also introduces no higher-order derivatives of the candidate network; u∗_ttu^*\_{tt} and the terminal u∗_tu^*\_t enter as fixed labels multiplying network values.

More precisely, let F=D+βIϵF=D+\beta I_\epsilon and write the operator mϵ[v]=δMϵ(u,u∗)[v]m_\epsilon[v]=\delta M_\epsilon(u,u^*)[v], which is independent of uu. If the population functional FF is μ\mu-strongly convex in H=H1(0,T;H1(Ω))\mathcal H=H^1(0,T;H^1(\Omega)) on the common convex admissible set, its minimizers with and without the correction satisfy

\leq \frac{\beta}{\mu} ||m_\epsilon||_{{\mathcal H}^*}.$$ As $\beta$ is a small constant between 0 and 1 and the dual operator norm is bounded, this leads to a negligible difference, consistent with our experimental results. ### References R. Stevenson and J. Storn. "Interpolation Operators for Parabolic Problems." *Numerische Mathematik*, **155**, 211–238, 2023. [DOI: 10.1007/s00211-023-01373-9](https://doi.org/10.1007/s00211-023-01373-9). [arXiv:2212.04134v1](https://arxiv.org/abs/2212.04134). W. Dahmen and R. Schneider. "Wavelets with Complementary Boundary Conditions—Function Spaces on the Cube." *Results in Mathematics*, **34**, 255–293, 1998. [DOI: 10.1007/BF03322055](https://doi.org/10.1007/BF03322055). B. Han and M. Michelle. "Wavelets on Intervals Derived from Arbitrary Compactly Supported Biorthogonal Multiwavelets." *Applied and Computational Harmonic Analysis*, **53**, 270–331, 2021. [arXiv:2012.05371v2](https://arxiv.org/abs/2012.05371). H. Harbrecht, C. Schwab, and M. Zank. "Wavelet Compressed, Modified Hilbert Transform in the Space–Time Discretization of the Heat Equation." *IMA Journal of Numerical Analysis*, **46**(4), 2139–2175, 2026. [DOI: 10.1093/imanum/draf061](https://doi.org/10.1093/imanum/draf061). [arXiv:2402.10346v2](https://arxiv.org/abs/2402.10346). W. Dahmen, A. Kunoth, and K. Urban. "Biorthogonal Spline Wavelets on the Interval—Stability and Moment Conditions." *Applied and Computational Harmonic Analysis*, **6**(2), 132–196, 1999. [DOI: 10.1006/acha.1998.0247](https://doi.org/10.1006/acha.1998.0247). Open preprint: WIAS Preprint No. 265 (1996; result numbering used above).

Thank you for the detailed response and clarifications. I appreciate the authors' efforts to address my concerns.

  1. Approximation theorem. I find the proposed correction using tensor-product approximation reasonable and appreciate the authors' clarification.

  2. Minimax lower bound (Theorem 3.3). I still have one concern regarding the Fano argument. Under the stated observation model with noiseless solution labels, it remains unclear how the KL divergence between the full observation distributions can be controlled. Considering only noisy forcing observations does not appear sufficient. One possible direction is to incorporate noise into the solution observations, which may also better reflect practical data-driven PDE reconstruction settings and provide a more natural framework for the minimax analysis.

  3. Numerical experiments. I acknowledge that I overlooked some of the experiments presented in the appendix during my initial review. These experiments provide additional evidence of the method's effectiveness. I would encourage the authors to consider moving some of the more challenging PDE benchmarks from the appendix to the main text, as they would help strengthen the experimental presentation.

Overall, I appreciate the authors' constructive response. My main remaining concern is the validity of the minimax lower-bound argument under the current observation model.

Reviewer_1REdOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work proposes a methodology for approximating weak solutions to the heat equation given a combination of labeled solutions and auxiliary data. In particular, the approach uses noisy observations of the source function to define an approximate energy function that can be minimized to learn the weak solution of the PDE. Theoretically, the authors show that the resulting estimator adapts to the smoothness of the true solution, and, with suitably tuned regularization, it can attain the minimax lower bound (up to log factors) even when the solution is not spatially smooth. Empirically, the authors provide evidence that this approach is a more stable solver than PINNs, on four linear parabolic PDEs and one nonlinear PDE, in dimensions up to d=50.

Critical strengths

The authors provide a novel variational problem to approximate the solutions of PDEs using a principled regularization term, which is theoretically proven to achieve the minimax optimal rate with appropriate tuning for solving heat equations. The experiments demonstrate a setting where classic PINNs fail to capture the solution globally in space-time due to an isolated singularity in the initial condition, unlike the proposed method. Moreover, the authors include several ablations with respect to parameter sensitivity and extensions to other PDEs that fall outside the theory.

Critical weaknesses
  • The current objective requires exact observations of the true solution and its derivatives (i.e., noiseless observations), which may not be realistic in practice. The noise ablation covers only mild noise, and the theory assumes noiseless data. How would the authors propose to adapt the loss with noisy data, especially derivative information?
  • It would be helpful to clarify if the comparisons to PINNs use exactly the same information as the proposed method so the experiments are all fair. The text says only that total sample sizes are matched roughly. In particular, do the PINNs use the same solution labels and their derivatives, e.g., by training with a Sobolev or higher-order norm?
Questions for authors
  • How would the method compare to another baseline, e.g., PINNs, using the exact same input data? How does it perform for a solution that does not have a singularity? Do you expect there to be the largest gap with PINNs when there is some irregularity in the solution? The theory suggests the energy term adds little when the solution is spatially smooth, but this is not tested against PINNs.
  • The estimation error bounds are stated in terms of the total sample size of snapshots in time and space, under uniform sampling. In practice, the method may need more solutions at certain times if there is a singularity (as in the first numerical example). Can the authors comment on how to allocate samples between different times in practice?
3 comments

We thank the reviewer for the constructive feedback.

Handling imperfect, noisy labels (W1). Although our theoretical model does not assume noisy solution label, the training labels are computed numerically and already contain discretization and numerical differentiation errors. The noise ablations introduce additional perturbations on top of these existing errors. Thus, the practical implementation already operates with imperfect labels and does not require access to exact analytical solutions.

Reusing numerical solutions provides a concrete source of supervision when simulation data are available. The energy regularizer then supplies additional physical structure to the learning problem. The comparison with pure Sobolev regression quantifies the improvement from this additional structure under shared supervision; we report the results below.

Our existing noise ablations consider both perturbations to solution and source labels and perturbations applied directly to derivative labels. In the joint solution--source noise experiment, derivative labels receive no additional perturbation. Over ten paired random seeds, adding 5%5\% noise to both uu and ff increases the median relative L2L^2 error by 1.50%1.50\%, 3.28%3.28\%, and 4.59%4.59\% on LP1, LP2, and LP3, respectively, relative to the corresponding runs without added noise. At 10%10\% joint noise, the increases are 28.17%28.17\%, 28.85%28.85\%, and 31.22%31.22\%. The absolute median relative L2L^2 errors at this higher noise level remain approximately 0.0207%0.0207\%--0.0209%0.0209\%. The nonlinear HJ example is more sensitive: its median relative L2L^2 error increases from 0.557%0.557\% without added noise to 0.608%0.608\% and 0.957%0.957\% at 5%5\% and 10%10\% joint noise, respectively. These results quantify robustness in the tested regimes and show that sensitivity depends on the equation and noise level.

We agree that differentiating noisy measurements deserves particular attention. Errors generated by differentiation can be correlated and depend on numerical resolution, so directly perturbing derivative labels does not fully reproduce this setting. Further analysis involving noisy differentiation estimates can be considered to pursue following our current setting.

Fair comparisons and performance on smooth solutions (W2). The existing comparison with pure Sobolev regression uses the same numerically generated solution and derivative labels, including their numerical errors. It measures the benefit of adding the energy and debiasing terms, which incorporate the auxiliary source observations, to supervised Sobolev fitting. For smooth example without singularity, we have toy example in appendix, that is ,the heat-equation experiment with N=16,384N=16{,}384 space-time samples and 5%5\% source noise, the mean test MSE over 100 runs decreases from 4.44×10−64.44\times10^{-6} for pure Sobolev fitting to 9.45×10−79.45\times10^{-7} for energy-regularized fitting, a reduction of 78.7%78.7\% under the same supervision.

The original PINNs comparisons follows with their unsupervised traditions but do not provide identical interior supervision. We provide additional ablation study on comparing our method with PINNs with label solutions. Compared to original PINNs results reported in the paper, the results show label information significantly improves over PINNs's performance; but they are still inferior to our method.

PDEMSE: Ours-H1H^1MSE: PINN+DataRelative L2L^2: Ours-H1H^1Relative L2L^2: PINN+Data
LP1(6.776±3.382)×10−9(6.776\pm3.382)\times10^{-9}(1.831±0.6296)×10−8(1.831\pm0.6296)\times10^{-8}(1.527±0.3922)×10−4(1.527\pm0.3922)\times10^{-4}(2.548±0.4139)×10−4(2.548\pm0.4139)\times10^{-4}
LP2(6.603±3.174)×10−9(6.603\pm3.174)\times10^{-9}(3.314±1.808)×10−8(3.314\pm1.808)\times10^{-8}(1.511±0.3734)×10−4(1.511\pm0.3734)\times10^{-4}(3.369±0.9126)×10−4(3.369\pm0.9126)\times10^{-4}
LP3(6.586±3.174)×10−9(6.586\pm3.174)\times10^{-9}(1.280±0.3038)×10−8(1.280\pm0.3038)\times10^{-8}(1.509±0.3740)×10−4(1.509\pm0.3740)\times10^{-4}(2.141±0.2492)×10−4(2.141\pm0.2492)\times10^{-4}
HJ1(8.169±2.932)×10−7(8.169\pm2.932)\times10^{-7}(3.931±0.9280)×10−6(3.931\pm0.9280)\times10^{-6}(5.960±1.081)×10−3(5.960\pm1.081)\times10^{-3}(1.317±0.1611)×10−2(1.317\pm0.1611)\times10^{-2}
HJ3(1.095±0.3091)×10−7(1.095\pm0.3091)\times10^{-7}(8.459±2.038)×10−7(8.459\pm2.038)\times10^{-7}(7.746±1.135)×10−4(7.746\pm1.135)\times10^{-4}(2.160±0.2528)×10−3(2.160\pm0.2528)\times10^{-3}
HJ10(1.657±0.3710)×10−5(1.657\pm0.3710)\times10^{-5}(8.188±1.263)×10−5(8.188\pm1.263)\times10^{-5}(2.935±0.3349)×10−3(2.935\pm0.3349)\times10^{-3}(6.543±0.5101)×10−3(6.543\pm0.5101)\times10^{-3}
HJ20(3.506±0.3286)×10−4(3.506\pm0.3286)\times10^{-4}(2.212±0.2166)×10−3(2.212\pm0.2166)\times10^{-3}(6.810±0.3110)×10−3(6.810\pm0.3110)\times10^{-3}(1.710±0.08297)×10−2(1.710\pm0.08297)\times10^{-2}
HJ50(4.361±0.3564)×10−3(4.361\pm0.3564)\times10^{-3}(2.124±0.1748)×10−1(2.124\pm0.1748)\times10^{-1}(9.639±0.4176)×10−3(9.639\pm0.4176)\times10^{-3}(6.726±0.2825)×10−2(6.726\pm0.2825)\times10^{-2}

Fair comparison with PINNs + label and regular solution experiments(Q1). As discussed in our response to Weakness 2, the reported experiments show that our method continues to outperform PINNs augmented with labeled solution data. For smooth solutions, the appendix already includes a simple heat-equation example in which our method achieves better accuracy than the PINN baselines.

Theoretically, in the spatially smooth regime, energy regularization need not improve the leading statistical convergence rate over the unregularized baseline, namely Sobolev regression. This result does not rule out finite-sample improvements or imply comparable accuracy between our method and PINNs. When sufficiently informative labeled data are available, supervised regression can itself outperform PINNs. Our observed improvements over the pure regression baseline, together with the direct comparisons against PINNs with labeled data, support our expectation that an accuracy advantage can persist even for smooth solutions. We therefore do not expect the benefits of our method to be confined to problems with singularities.

Space-time sampling and practical allocation (Q2). In the paper, we have considered an adversarial sampling technique to the region where optimization behavior becomes worse. Our experiments already include a comparison with Adversarial PINN[wang2022is], which uses gradient ascent to move collocation points toward regions with large PDE residuals. This concentrates training points where the current network most strongly violates the equation and targets a worst-case residual objective. In the LP1--LP3 comparisons, our method achieves lower mean test errors than this adaptive baseline. On LP3, it reduces mean test MSE by 27.2%27.2\% and mean relative L2L^2 error by 16.5%16.5\% over five random seeds. These results use the supervision settings described in Section~2; they compare the complete methods and do not isolate the effect of sampling alone.

[wang2022is] Chuwei Wang, Shanda Li, Di He, Liwei Wang. Is L2L^2 Physics-Informed Loss Always Suitable for Training Physics-Informed Neural Network? Advances in Neural Information Processing Systems 35 (NeurIPS 2022). arXiv:2206.02016.

Reviewer_4N5EOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies learning the weak solution of the heat equation u_t − Δu = f (zero Dirichlet boundary, known initial condition) from space–time samples. The data are noisy source values f_ij together with noiseless solution values and numerically computed solution derivatives at the same points.

The proposed loss combines an H¹_tH¹_x regression term with a weighted energy functional I_ε, whose Euler–Lagrange equation is the elliptic regularization −εu_tt + u_t − Δu = f, plus a debiasing term M_ε. With M_ε, the exact heat solution minimizes the loss and a two-sided stability estimate holds.

Experiments are conducted with MLPs that cover:

  • the heat equation with a singular initial condition (where a PINN fails);
  • Fokker–Planck;
  • variable-coefficient parabolic equations;
  • viscous Hamilton–Jacobi up to d = 50;
  • Sine–Gordon.

They report substantial reduce in test error over PINN variants and over pure Sobolev regression.

Critical strengths
  1. The energy term helps beyond pure Sobolev regression with identical labels. App. G.3 reports 78.7–97.7% MSE reductions at n = m = 128 with 5% source noise. This is the cleanest evidence that the regularizer, rather than the labels, does useful work.

  2. A simple, correct population-level construction that handles weak solutions. The debiased weighted energy plus a Sobolev fit makes the exact solution the population minimizer. The method also copes with a singular initial condition where a vanilla PINN fails (Sec. 4.1). This matters for problems with nonsmooth data, where residual-based methods struggle.

Critical weaknesses
  1. For the sampling model, the proof flattens the data into z₁ = (x₁₁, t₁), …, z_m = (x_{m1}, t₁), … (p. 32) and applies symmetrization with nm independent Rademacher variables. Yet the m points at each t_j share that time.

  2. Sec. 3 assumes i.i.d. space–time points, but the data model (1.2) and the experiments (50 time points × 80 locations) do not seem to satisfy this.

  3. In time-varying directions, concentration then scales with n rather than nm, so the N = nm rates are not established for the paper's design.

  4. The estimator needs noiseless solution values and high-accuracy derivative labels at every training point (4,000 in the main experiments). The PINN baselines (App. G.1–G.2) appear to lack these labels.

Questions for authors
  1. How are the shared time points t_j handled in the symmetrization and concentration steps (pp. 32–33)? Where is the Gaussian source noise of (1.3) controlled in App. E?
  2. Do the PINN baselines receive the same interior solution and derivative labels? Please add baselines with identical label access (e.g., a PINN with a data term, or Sobolev training). How does accuracy degrade when derivative labels are computed from noisy solution measurements?
  3. Please position the work against Brezis–Ekeland-based neural solvers (Carini et al. 2023), and physics-regularized regression with rates (e.g., Doumèche, Bach, Biau & Boyer, COLT 2024).
2 comments

Thank you for the authors’ explanation, corrections, and experiments.

The points W1-3 are centered around whether the samples i.i.d. in space-time sampling or not. I think the authors’ revisions address these concerns. Regarding W4, the authors have provided new experiments with a fair comparison (with the same solution and derivative fitting terms). Given this, I will increase my rating.

Thanks for your positive feedback and raising the score!

Comments on the paper

6 comments
Area_Chair_zj98OpenReview ↗

Please engage in the discussion

Authors: you have not answered any question raised. Please respond to reviewers' questions and engage in the discussion.

We thank for your further positive feedback. We will move the more challenging PDE benchmark from Appendix to the main paper.

Regarding minimax lower bound (Theorem 3.3) concern:

we below outline our current proof strategy and discuss the possibility of extending to noisy solution label data setting.

  • There are two independent random sampling components in data distribution: uniform sampling model on the space-time domain, and zero-mean Gaussian noise aroud the true source function. Given the space-time locations, the observed label data uij∗u^*_{ij} has no noise (i.e., non-random), and thus Fano's lemma cannot be directly appleid.

  • To obtain lower bound on the risk in the solution uu space, we follow the usual reduction argument to a testing problem with a finite number of hypotheses. However, different from standard discretization in the function space (here uu space), we need to ensure every uku_k in the hypothesis subset (with at least 2ϕn2\phi_n separated) is a valid solution function for some fkf_k (i.e., if we were direclty choosing hypotheses in the solution space, then many uu's do not corresponding to the heat equation solution).

  • The PDE restriction f↦uff\mapsto u_f in above step is guanranteed via an inverse differnetial operator and it is constructed via equation (E.185) in Appendix E, where we have parametrized ufku_{f_k} satisfying 2ϕn2\phi_n separation. Interestingly, we found that the inverse differential operator has a Gaussian convolution form, meaning that the Gaussian sampling structure (i.e., noise) of ff translate to the Gaussian structure of u∗u^* (again, uu space does not have its own random noise). Together with the KL calculation with Gaussians, one can reduce the comptuation in terms of L2L^2-norm in the uu-space and ff-space.

  • Next step is to use parabolic estimate for heat equation to bound uku_k using fkf_k (cf. Theorem 5 in Chapter 7.1, Evans (2010)):

    \lesssim \|f_k-f_0\|^2_{L^2([0,1];L^2([0,1]^d))}.$$ This can be used to lower bound the KL divergence in the $u$-space.
  • Putting together of the aboave steps into Fano's lemma, we effectively transfer the distributional structure from source ff to solution uu spaces, even when the target uu does not directly contain random noise.

Extension to noisy label setting. It is possible to extend our lower-bound analysis for solution label with Gaussian noise. Our intuition is that in such setting, ff and uu both have noise, but uu is smoother than ff because uu gains regularity through the heat equation. So balancing these two ingredients would naturally lead a conjecture that:

If one has additional Gaussian noise on the label data u∗+Gaussianu^*+\mathrm{Gaussian}, then the minimiax lower bound for estimating the solution function u∗u^* would stay the same as the noiseless setting.

In connection with current proof strategy, we believe KL divergence on noisy uu also will need control of KL divergence on ff, that is, L2L^2 norm of ff. This leads to similar argument in our lower bound.

However, rigorous analysis requires more work. Due to rebuttal time limit, we don't have a complete solution. Below, we support this conjecture with empirical findings that we have already reported in the paper:

As complementary empirical evidence, we extract the following results from the noise ablation already reported in the appendix. We consider observations

\qquad f_{\mathrm{obs}}=f+\rho_f\xi_f, \qquad \xi_u,\xi_f\stackrel{\mathrm{ind}}{\sim}\mathcal N(0,1).$$ We fix the forcing-noise standard deviation at $\rho_f=0.10$ and vary the solution-label noise level $\rho_u\in\{0,0.05,0.10\}$. This comparison isolates the effect of adding solution-label noise while retaining noisy forcing observations. All settings use the same hyperparameters, training budget, and ten paired seeds. **Table. Effect of additional solution-label noise at fixed forcing noise $\rho_f=0.10$.** | Equation | $\rho_u=0$ | $\rho_u=0.05$ | $\rho_u=0.10$ | |---|---:|---:|---:| | LP1 | $1.608\times10^{-4}$ | $1.636\times10^{-4}$ | $2.072\times10^{-4}$ | | LP2 | $1.597\times10^{-4}$ | $1.654\times10^{-4}$ | $2.067\times10^{-4}$ | | LP3 | $1.595\times10^{-4}$ | $1.663\times10^{-4}$ | $2.090\times10^{-4}$ | | HJB ($d=1$) | $5.839\times10^{-3}$ | $5.797\times10^{-3}$ | $9.574\times10^{-3}$ | With additional solution noise, the median relative errors remain below $2.1\times10^{-4}$ for LP1–LP3 and $10^{-2}$ for HJB, although HJB exhibits greater sensitivity. These results provide empirical evidence of robustness to solution-value noise under the stated observation protocol. ### Reference Evans, Lawrence C. (2022) *Partial Differential Equations*. Graduate Studies in Mathematics.

Again, we appreciate the reviewer’s feedback on minimax lower bound. We hope our response has addressed your concern on the Fano's argument. Since the rebuttal period will end today, we would appreciate it if the reviewer could share follow-up feedback to help us improve our paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.