acceptodds
Withdrawn from ICLR 2027

Multiscale Regularization of Physics-Informed Neural Networks Based on a Maximal Local Oscillation Operator for Problems with Sharp Gradients

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

This paper proposes a multiscale regularizer for physics-informed neural networks (PINNs) designed for differential equation problems with sharp spatial gradients. The regularizer is based on a maximal local oscillation operator that measures the deviation of a function from its local mean across several scales. Unlike standard penalties on high-order derivatives, the proposed approach does not require their explicit computation and naturally accounts for the multiscale structure of the solution. For practical use, we introduce a differentiable discrete implementation of the operator on a finite set of radii using one-dimensional convolutions, compatible with automatic differentiation in PyTorch. The method is integrated into the PINN loss function as an additional regularizer that suppresses local nonphysical oscillations. Using the viscous Burgers equation as an example, we compare the method against a standard PINN and several classical smoothness penalties at several viscosity values; the exact solution obtained via the Cole–Hopf transformation serves as the reference. Numerical results show that, in the front-formation regime, the proposed approach achieves the best mean approximation accuracy among the methods considered while preserving the sharpness of the front and avoiding the over-smoothing characteristic of penalties on high-order derivatives; in the extremely smooth and extremely stiff regimes the method is on par with the alternatives.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
physics-informed neural networks, PINN regularization, multiscale methods, maximal function, sharp gradients, Burgers equation, scientific machine learning

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_gzBdOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The manuscript proposes a regularizer for PINN training that penalizes oscillations. The proposed regularizer is evaluated on the 1D viscous Burgers equation for three viscosities (high, moderate, small). The manuscript does not contain theory. Based on the critical weaknesses listed below, I recommend rejection.

Critical strengths

The core question this paper assumes to be true---whether PINNs lead to oscillations around sharp gradients or shocks---is certainly interesting.

Critical weaknesses

The paper has significant problems:

  1. The core issue it aims to address---the fact that PINNs suffer from oscillations in the vicinity of sharp gradient regions (or shocks)---is not demonstrated at all, and neither is it a well established failure modes of PINNs. I agree that it might(!) be a failure mode of PINNs, but as a matter of fact, I have not observed it myself either.

  2. The experimental evaluations do not show a clear benefit of the proposed method. Three scenarios are tested: high, moderate, and small viscosity corresponding to smooth, moderate, and sharp gradients in the solution. The proposed regularizer deteriorates training for the smooth case, and performs on par with established methods in the moderate and small viscosity regime. The authors claim improved performance in the moderate regime, however confidence intervals overlap, so no conclusion should be drawn. The authors admit this themselves.

  3. The original PINN paper [PINN] (in Appendix A.1) treat this exact setup of Burgers equation, and report much lower errors (around 1e-3 to 1e-4 relative L2), and no oscillating behavior, so the baseline PINN is not tuned correctly. Moreover, the manuscript ignores recent advantages in PINN training: It is by now well-established that Gauss-Newton (or natural gradient methods in ML parlance) methods perform exceptionally well on PINNs. A result that claims improved PINN training (even through a regularizer) must adhere to that literature. See: [ENGD, DeepMind, Karniadakis].

Literature:

[ENGD] https://proceedings.mlr.press/v202/muller23b/muller23b.pdf

[DeepMind] https://arxiv.org/html/2509.14185v1

[Karniadakis] https://arxiv.org/pdf/2604.05230

[PINN] https://www.sciencedirect.com/science/article/pii/S0021999118307125

Questions for authors

I have no questions.

Reviewer_6rBWOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes a regularization loss for training PINNs that penalizes local non-physical oscillations. By measuring the means in local neighbourhoods centred at grid points with various radii and measuring the maximum deviation from the local mean across those radii, the regularizer aims to suppress oscillations that appear in PINNs due to spectral bias, stiff physical regimes, and shocks. This "oscillation operator" is implemented by convolutions and does not require spatial derivatives. The authors evaluate the proposed method on a 1D viscous Burgers' equation against other global smoothness and regularization methods.

Critical strengths
  • The proposed method is not computationally demanding and does not rely on explicitly calculated spatial derivatives. The convolution-based implementation also avoids numerical difficulties with backpropagating through higher-order derivatives.
  • The paper is well-motivated, easy to read, and reports the method's failures in the stiff regimes.
Critical weaknesses
  1. The method is evaluated only on the 1D viscous Burgers' PDE, and the baselines do not include well-known methods for mitigating spectral bias or capturing sharp gradients in PINNs.
  2. Since the authors only study the Burgers' equation and do not provide a mathematical analysis of the regularizer, it is not clear in what physical regimes the use of this regularizer is appropriate; e.g. whether this method would be beneficial for equations with high nonlinearity and high-order derivatives (such as KS and KdV equations).
  3. The regularizer does not distinguish between non-physical oscillations and physical shocks, steep slopes, or high-frequency waves. In a PDE with naturally oscillating solutions and shocks, the regularizer would smooth out the solution. This is in line with the authors' observations in Appendix A that higher λreg\lambda_{\text{reg}} results in over-smoothed solutions.
  4. The gains over the baselines are not statistically significant, and the empirical evaluations are not convincing. While the regularization does not improve the performance for the stiff regime (ν=0.005/π\nu=0.005/\pi) at all, the gains for ν=0.01/π\nu=0.01/\pi are also not qualitatively meaningful. Based on Figure 2, both regular PINNs and the oscillation regularization fail similarly without exhibiting spurious oscillations. The large errors seem to be a result of severe underfitting (only 3000 training epochs) or a poor choice of hyperparameters. Using methods like SIRENs [1], causal PINNs [2], or adaptive collocations and training until convergence would address the underfitting for all baselines and test whether the regularizer actually provides any benefit. In the current experimental setup, it is unclear what the regularizer fixes.
  5. A large portion of the paper is dedicated to basic PINNs and Burgers' formulations without any new insights. Unfortunately, I believe the work is not ready for ICLR at this stage.

[1] Sitzmann, Vincent, et al. "Implicit neural representations with periodic activation functions." Advances in neural information processing systems 33 (2020): 7462-7473.

[2] Wang, Sifan, Shyam Sankaran, and Paris Perdikaris. "Respecting causality is all you need for training physics-informed neural networks." arXiv preprint arXiv:2203.07404 (2022).

Questions for authors
  1. Can the authors train the PINNs for all methods for more epochs until convergence (at least 10K epochs) or using a hybrid Adam and L-BFGS method? The loss curves in Figure 5 show that all losses are still decreasing at 3,000 epochs.
  2. The multi-radii scale sets either over-smooth the solution or deteriorate the performance compared to a single scale (Appendix A, Table 4). Have the authors successfully applied multi-scales to other PDEs? How is the single radius for Burgers' (and other PDEs) determined in practice without prior knowledge of the front width?
Reviewer_cL7GOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a multiscale local oscillation regularizer for PINNs. Experiments on 1D Burgers with three viscosities show improvement in one setting, but not in the others.

Critical strengths

The method is simple and easy to add to PINNs. It improves the mean error in one sharp-front Burgers setting, showing some potential benefit.

Critical weaknesses
  1. The proposed regularizer penalizes local variations, including physical sharp fronts. It is unclear why it can suppress nonphysical oscillations without smoothing the true solution. The baseline results do not clearly show spurious oscillations, and the paper does not directly measure their reduction.

  2. The benefit of the multiscale design is not clear. The authors use four fixed radii without explaining why these scales are chosen. Table 4 does not show an advantage over a single scale, and the ablation uses a different regularization weight from the main experiments. Also, all experiments are on one 1D Burgers equation, and the method only improves the mean error in one viscosity setting. These results do not sufficiently support the multiscale design or its general effectiveness.

Questions for authors
  1. How does the regularizer suppress nonphysical oscillations without smoothing physical sharp fronts? Can the authors quantify the reduction of spurious oscillations?

  2. Why does Table 4 use λregλ_{reg} = 10−310^{-3} instead of 10−410^{-4} as in the main experiments? Can the authors compare single scale and multiscale regularization using the same setting?

  3. Have the authors tested the method on other PDEs with sharp gradients?

  4. How should the regularization weight and radii be selected for a new PDE without knowing the exact solution?

  5. The linked code repository is not accessible, and the supplementary material only contains LaTeX files. Can the authors provide an accessible code repository?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.