acceptodds
Under review as a conference paper at ICLR 2027

LPINNs: First-Layer Gated Localization for Physics-Informed Neural Networks

Abstract

Physics-informed neural networks (PINNs) use one shared representation over the computational domain, which can become difficult to optimize on long domains and for high-order operators. We study a minimal alternative: multiply the first hidden activation of an otherwise unchanged dense PINN by input-dependent localization functions, giving first-layer units receptive fields without partitioning the domain or adding interface losses. We screen 13 families of localization functions, in up to three parameterizations each, on a nonlinear harmonic oscillator (HO), a heat equation on a long spatial interval, and a manufactured four-dimensional (4D) fourth-order problem, with ten paired seeds throughout. Three configurations give large reductions in solution error at matched budgets: (i) Fixed Gaussian localization functions on the HO domain cut mean solution RMSE from to at 3k epochs. (ii) The inverse-quadratic family with learnable centers and widths cuts it from to on the heat domain at 10k epochs. (iii)) Fixed bump localization functions cut it from to on the 4D domain at 10k epochs. Every paired seed improves in these three comparisons. The screen also shows that the mechanism is not a free win: on HO only 2 of 13 families beat the baseline, and 10 of the remaining 11 are 9 to 23 times worse; on 4D four families are non-finite and five are more than three orders of magnitude worse than the baseline. The inverse-quadratic family is the only one that beats the baseline on all three equations. Overall, these results show that first-layer localization can provide measurable improvements to baseline PINNs on long-domain and high-order problems.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Physics-Informed Neural Networks, Localization Functions, First-Layer Gating, Long-Domain Problems, High-Order Differential Equations, Scientific Machine Learning

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_jVT8OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper studies a minimal architectural modification to standard fully-connected PINNs: multiply the first hidden activation by input-dependent localization functions as described in Equation (3). The method is called LPINN. The paper focuses purely on empirical comparison and no theoretical analysis is conducted. LPINN is evaluated on 3 model problems: harmonic oscillator, 1D heat equation, and a manufactured 4th order problem; with 13 families of localization function; against vanilla PINN as the only single baseline. The experiment is conducted across different sizes of domain, epoch budgets, and 10 seeds. The numerical evidence shows that LPINN can improves 1-2 order of magnitude in solution RMSE over standard PINN if the family of localization-functions is chosen properly.

Critical strengths
  1. Clear presentation of the proposed method and experiment settings. Equations (1)-(3) clearly define the implemented methods. The set up of the numerical experiments is documented completely in Section 4.

  2. Extensive list of families of localization functions. The list of total 13 families of localization functions is exhaustive.

Critical weaknesses
  1. Lack of motivation/controbution for the proposed specific localization approach. The authors clearly state that they choose not to compare against other localization methods in the paper. However, they do not provide a clear motivation about why to consider the specific approach in LPINN, rather than other localization methods. Given no motivation and no benchmark against other similar methods, it's not clear about the main advantage of this method. The paper only shows the validity of LPINN but does not explain why it's a method worth being used. Lack of clear contribution is the major reason for me to reject the paper.

  2. Unclear whether experimental comparison is fair. The authors emphasize that every localized model is compared against a PINN with exactly the same set-up such as network size, residual, and optimizer etc. in order to identify the clear improvement read regarding the effect of introducing the localization functions. However, I'm not sure whether it's fair that PINN and LPINN should use the same hyperparameter settings for optimizer. The introduction of localization functions may favor different set of hyperparameters such as stepsize etc.

  3. Randomness of the results is not properly reported. Even though each localized model and baseline model is run for 10 seeds and Figure 1-3 provide the trajectories across all 10 runs, the table does not include the standard deviation of Loss/RMSE/MAE across 10 runs. In terms of aggregation method, since different PINN runs have different order (from Figure 1-3), arithmetic mean is not an appropriate measure since average of 10110^1 and 10−110^{-1} will stay around 10110^1, while the run of 10110^1 should be declared as non-converged. Please either report geometric mean across 10 runs or exclude the non-converged runs.

  4. The benchmarked equations are relatively simple ones. As far as I know, 1D heat equations and harmonic oscillator are simple test instances. I would like to see the authors justify LPINN on more challenging test equations.

Questions for authors
  1. Can you aggregated the numbers across seeds by geometric mean rather than arithmetic mean? Also, please provide also the standard deviation for all tables.

  2. Can you clarify the motivation of the proposed localization method? Why not the other localization methods? This can be justified by either providing numerical results to showcase the performance of LPINN over other methods or clarifying what imitations of previous methods are resolved by LPINN.

  3. Could you please address my concern regarding fairness in Weakeness #2?

2 comments
AuthorsOpenReview ↗

Response to reviewer jVT8

We thank the reviewer for the thoughtful critique and for recognizing the method definition and breadth of the family screen. Our motivation is a minimal structural intervention: input-dependent gates localize first-layer activations while retaining one global dense PINN and avoiding domain partitions or interface losses. The paper tests whether that change helps on selected long-domain and higher-order problems; it does not claim a universal localization method or superiority to Fourier/RBF embeddings or domain-decomposition methods, which address related but different design choices.

We agree that we do not yet have a principled explanation for all family-by-equation differences or an oracle-free selector. We treat gradient locality as a hypothesis and report the unsuccessful families and the 1k-epoch reversal as part of the evidence. The comparison is designed to isolate the gate: the network size and training setup were selected for the vanilla PINN, and localization was added afterward while keeping the baseline settings, collocation scheme, and paired seeds fixed. We agree that additional optimizer tuning could change either model’s performance; we do not present this as an exhaustive hyperparameter study. The harmonic-oscillator experiment uses a 2\pi horizon, rather than the short-horizon case where PINNs can be easy.

The revised appendix now includes the requested distribution details: Table S2 gives arithmetic means, medians, population standard deviations, and geometric means for all four metrics in the three primary comparisons, and Table S3 gives the paired per-seed solution RMSEs. Table S4 adds a compute-context comparison: on the oscillator, 3k-epoch LPINN has mean RMSE 0.00883 versus 0.01485 for 10k-epoch vanilla PINN, with lower error in all 10 paired seeds; the observed times are 6.87 s and 12.52 s. These timings are descriptive because the CPU runs overlapped. The 4D fourth-order manufactured problem also extends the evaluation beyond the one-dimensional examples. We have kept the conclusions bounded to these controlled experiments; comparisons with the related feature and domain-decomposition approaches and an oracle-free family selector remain open questions.

The reviewer thanks the authors' response. I keep my original assessment.

Reviewer_PJjeOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes LPINNs, a modification to physics-informed neural networks that introduces input-dependent localization functions at the first hidden layer. The authors investigate 13 families of localization functions with different parameterizations and evaluate them on three differential equation problems: a nonlinear harmonic oscillator, a heat equation, and a manufactured four-dimensional fourth-order problem. The experiments show that certain localization functions can substantially improve solution accuracy over a standard PINN baseline, while many others perform considerably worse or even fail numerically.

Critical strengths

The proposed first-layer localization mechanism is simple and easy to implement without substantially modifying the original PINN architecture. The authors also investigate a relatively broad range of localization functions and report both successful and unsuccessful configurations, providing some empirical observations about their behavior on different differential equations.

Critical weaknesses
  1. The paper lacks a convincing explanation of why certain localization functions work while others fail. Although 13 families are investigated, their performance varies dramatically across different equations, and four families produce non-finite results on the 4D problem. The paper provides a heuristic gradient-locality argument and discusses possible causes of numerical failures, but these do not sufficiently explain the substantial performance differences between localization functions. More importantly, the best-performing functions differ across problems, yet no principled criterion is provided for selecting an appropriate function without access to the reference solution. As a result, the study remains largely an empirical comparison rather than establishing a generalizable localization strategy.

  2. The experimental comparisons raise concerns about baseline optimization and fairness. In particular, the standard PINN performs surprisingly poorly on the harmonic oscillator, a relatively simple and well-studied ODE problem with a known analytical solution in terms of elliptic functions. This makes me question whether the baseline has been adequately optimized. Although the authors use matched training settings, this does not necessarily establish a fair comparison if the shared hyperparameters are not suitable for the baseline. For the heat equation, the paper shows that extending baseline training from 3k to 10k epochs barely improves performance, but this does not rule out suboptimal learning rates or other training configurations. Moreover, Table 1 lists training budgets up to 100k epochs for the harmonic oscillator, while the detailed comparison in Table 2 only reports 1k and 3k epochs. More carefully tuned baseline experiments are needed to establish the reported advantages.

  3. The comparison with existing methods is very limited. The only baseline is a standard fully connected PINN, while several established approaches also improve PINN optimization by modifying input or early-layer representations, such as Fourier features, positional embeddings, and radial-basis feature mappings. The paper cites these methods and explains how LPINN differs, but explicitly does not compare against them experimentally. Since these methods address closely related representation and optimization problems, such comparisons are important for establishing the actual advantages of the proposed localization mechanism.

  4. The experimental results also raise concerns about optimization stability and sensitivity to training configurations. For example, the orange training-loss curves in Figure 1 still exhibit noticeable fluctuations, while some baseline seeds already recover reasonably accurate solutions. Although the proposed method achieves more consistent final solution errors across seeds, the large variation in baseline performance makes the reported mean improvements difficult to interpret without more detailed per-seed comparisons. Moreover, Table 2 shows that Gaussian localization performs about three times worse than the baseline at 1k epochs but substantially better at 3k epochs, which the authors acknowledge. It remains unclear how the appropriate training budget should be determined in practice, or whether the reported advantages would persist under better-tuned baseline settings.

Questions for authors
  1. Can the authors provide a more detailed theoretical or empirical explanation for why some localization functions improve PINN training while others perform much worse? Is there a principled criterion for choosing the localization function without knowing the exact solution?

  2. Have the baseline PINNs been sufficiently tuned? In particular, why does the standard PINN perform so poorly on the harmonic oscillator? Could different learning rates, initialization strategies, or other optimization settings substantially reduce the reported performance gap?

  3. How does LPINN compare with existing representation methods such as Fourier features or radial-basis feature networks? These appear to be particularly relevant alternatives to first-layer localization.

  4. How should the training budget be selected in practice, given the performance reversal at 1k epochs? Could the authors also provide median or per-seed results and further investigate the fluctuations observed in the training histories?

1 comment
AuthorsOpenReview ↗

Response to reviewer PJje

We thank the reviewer for the careful assessment and for recognizing the broad screening of localization families. We agree that the results do not yet provide a principled, oracle-free rule for choosing a family. We therefore treat the gradient-locality explanation as a hypothesis, not an established mechanism, and report the strong family- and equation-dependence—including failures and regressions—rather than presenting localization as a universal improvement.

On the harmonic oscillator, we agree that short-horizon instances can be easy for PINNs. Our experiment uses a 2\pi time horizon; the baseline’s performance is specific to this longer-domain setting, and we do not claim that vanilla PINNs generally fail on oscillators. The revised appendix adds Table S4: at this setting the baseline’s mean solution RMSE is 0.01485 after 10k epochs, while fixed-Gaussian LPINN reaches 0.00883 after 3k epochs, with lower error in all 10 paired seeds. The observed CPU times are 12.52 s and 6.87 s, respectively. These are descriptive measurements from overlapping runs, not a broad runtime benchmark. The baseline architecture and settings were selected for the vanilla PINN; the localization gate was then added, without a separate LPINN-specific width search. This isolates the effect of the structural change, though it does not rule out that different optimizer tuning could improve either model.

The revised appendix also addresses the seed-variation concern: Table S2 reports means, medians, population standard deviations, and geometric means for all four metrics in the three primary comparisons; Table S3 lists the paired per-seed solution RMSEs. The 1k-epoch reversal remains part of the evidence, so we do not claim a universal budget or family-selection procedure. Fourier embeddings, radial-basis features, and localized/domain-decomposition methods are relevant alternatives that modify different parts of the representation or domain; our controlled study tests first-layer gating against the vanilla PINN and does not claim direct superiority to those methods.

Reviewer_gWVWOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes LPINNs, a lightweight architectural modification that introduces input-dependent localization functions into the first hidden layer of physics-informed neural networks without requiring domain decomposition or additional interface losses. The authors evaluate 13 localization-function families with different parameterizations, demonstrating substantial accuracy improvements.

Critical strengths
  1. LPINNs introduces input-dependent localization through a multiplicative gate in the first hidden layer, without changing the subsequent network architecture or introducing additional interface constraints. This make the proposed method simple and easy to play with.
  2. The method addresses the difficulty of learning spatially localized features with globally shared representations, particularly for long-domain and high-order PDE problems.
Critical weaknesses
  1. The experimental comparison is restricted to a standard fully connected PINN, despite the existence of closely related localization and domain-decomposition methods. The paper does not experimentally compare against FBPINNs, XPINNs, radial-basis feature networks, Fourier-feature PINNs, or other modern architectures such as PirateNets, PINNsformer, and PINNMamba etc. As a result, it is unclear whether the reported improvements arise from a genuinely effective localization mechanism or simply from introducing a suitable inductive bias that existing methods already provide. Moreover, the generalization ability of the proposed methods is not studied.
  2. The proposed first-layer localization mechanism is closely related to existing localized neural representations, particularly Multiplicative Filter Networks (MFNs; Fathony et al., ICLR 2021) and Physics-Informed Radial Basis Networks (PIRBNs; Bai et al., 2023). MFNs already employ input-dependent multiplicative filters to modulate hidden representations. Furthermore, PIRBNs introduce localized radial-basis neurons for solving PDEs and provide an analysis of their local approximation properties and neural tangent kernel dynamics. Although LPINNs differs from these methods by inserting localization exclusively into the first hidden layer of an otherwise standard PINN, the paper does not sufficiently establish why this particular design offers fundamentally different representational or optimization properties.
Questions for authors

How much additional wall-clock time and memory does first-layer localization introduce, particularly for high-order PDE residuals? Do the reported improvements persist when comparing against a baseline PINN trained with the same computational budget rather than the same number of epochs?

1 comment
AuthorsOpenReview ↗

Response to reviewer gWVW

We thank the reviewer for recognizing the simplicity of the first-layer gate and for raising the need to distinguish it from related approaches. We agree that domain-decomposition methods and feature embeddings are relevant, but they address different aspects of PINN structure: domain decomposition partitions the domain and couples subnetworks, while Fourier or radial-basis embeddings change the input representation. LPINNs asks whether input-dependent gates on the first hidden activation can improve an otherwise unchanged, monolithic dense PINN. These are complementary design choices; our study does not claim superiority to those methods or demonstrate generalization.

The revised appendix adds Table S4 with a budget tradeoff on the 2π2\pi harmonic oscillator: fixed-Gaussian LPINN at 3k epochs has mean solution RMSE 0.00883 and observed wall time 6.87 s, while vanilla PINN at 10k epochs has mean RMSE 0.01485 and observed time 12.52 s. LPINN has lower RMSE in all 10 paired-seed comparisons despite using fewer epochs and less observed time. These CPU timings are descriptive because runs overlapped with other workloads; they are specific to this configuration, not a general runtime or memory study. We have not measured peak memory and make no memory advantage claim.

The main comparisons use the standard PINN architecture and settings selected for the vanilla baseline; the localization gate was then added, without a separate localization-specific architecture search. This controlled intervention is intended to isolate the effect of changing the first layer. We now report the budget context and bound the paper as an initial study; direct comparisons against domain-decomposition and feature-embedding methods remain future work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.