acceptodds
Under review as a conference paper at ICLR 2027

PE-EK-PINN: Physics Embedding with Evolving Kernel for Scalable Physics-Informed Neural Networks

28 views

Abstract

Physics-Informed Neural Networks (PINNs) embed governing equations into deep learning, but enforce them only through loss residuals, leaving highly oscillatory wave behavior to be discovered by optimization. As a result, methods that achieve relative errors below on standard manufactured Helmholtz benchmarks can fail on practical radiation problems involving singular excitations, absorbing boundaries, and wave fields spanning tens of wavelengths. Architectural physics embedding addresses this limitation by factorizing the field into analytically derived oscillatory kernels and learnable envelopes. However, the kernel dictionary must be manually constructed and scales with the number of elementary units, growing exponentially with the depth of hierarchically structured systems such as antenna arrays and metasurfaces. We propose PE-EK-PINN (Physics Embedded with Evolving Kernels), which treats physics kernels as reusable learned representations rather than fixed analytical inputs. A converged subsystem field is frozen and promoted to an evolved kernel, whose transformed copies are reused to represent higher-level configurations without deriving new governing equations. The resulting hierarchy makes the peak number of active kernels independent of system size and reduces cumulative training cost from to . Experiments on dipole arrays, composite line-source geometries, and cross arrays demonstrate the dramatic training cost reduction, while achieving a reduced or comparable relative error. One notable example is PE-EK-PINN solves a -dipole array more than 30 times faster than direct PE-PINN.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Physics informed neural network, scientific machine learning, wave field reconstruction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_4FtsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a method to address a limitation in Physics-Embedded PINNs (PE-PINNs). The new method handles source configurations built from repeated subunits, such as dipole arrays, crosses, and stars. The introduced PE-EK-PINN is a recursive 3 step process, First, train a PE-PINN on a small subsystem, second, freeze its learned field and promote it into an evolved kernel. Third, reuse translated or rotated copies of that kernel to train the next-level configuration. The method is evaluated on dipole arrays, composite line-source geometries, and hierarchical cross arrays.

Critical strengths
  1. The idea paper is solving is of good practical relevance, extending a converged subsystem solution to a reusable basis element is a natural way to use repeated structure in the source configuration.
  2. Authors are open about some limitations like, error accumulation through frozen levels, MLP extrapolation under rotation, the need for homogeneous media, and the need for a known hierarchy.
Critical weaknesses
  1. Limited methodological Novelty: This paper is just extension of PE-PINN framework, proposed novelty is primarily a hierarchical reuse/training strategy, rather than a fundamentally new PINN formulation. While this reuse strategy may offer practical computational benefits, the paper does not sufficiently establish its methodological novelty beyond straightforward compositional reuse of pretrained neural representations.
  2. Insufficient Baseline comparison with no ablations: On non-manufactured problems, the paper compares against a single external method (CoPINN, one dipole, Table 1), and all main experiments (Tables 3–5) compare only against direct PE-PINN. Other methods in Table 1 are evaluated only on the manufactured benchmark, so the claim that PE-PINN “already outperforms” them on wave benchmarks is not sufficient. It is highly possible that some of the methods in Table 1 might work better than PE-EK-PINN for other settings.
Questions for authors
  1. Since the reference solutions are constructed by linear superposition, how does directly summing frozen subsystem kernels without further training compare with PE-EK-PINN ?
  2. Beyond freezing and reusing pretrained PE-PINN predictions, what is the key methodological novelty of PE-EK-PINN ?
  3. Why is PE-PINN the only baseline evaluated on the main radiation problems? Can the authors compare against other scalable PINN methods ?
  4. Why are the reported large-scale speedups based on extrapolated rather than measured PE-PINN runtimes ?
  5. What evidence shows spectral bias as a major cause of conventional PINNs poor performance on radiation problems, rather than domain size, source singularities, or absorbing boundary conditions? Can the authors provide controlled experiments to disentangle these factors?
2 comments

We agree that the proposed recursive strategy is conceptually simple, and we view this simplicity as a strength rather than a limitation. The key contribution lies in the physical insight that a converged, physics-embedded solution can be reused as a computational kernel. This preserves learned wave physics as systems grow more complex. Unlike existing approaches that repeatedly optimize growing numbers of primitive kernels, our framework reuses and evolves learned solution representations while enforcing the governing physics at every level. We believe this mechanism for hierarchical physics-based knowledge reuse is the primary contribution of the work and the source of its improved scalability.

Regarding the baseline concern, we have now implemented all comparable PINN baselines for the same dipole case. Preliminary results support our original hypothesis that these methods struggle in practical wavefield scenarios. PE-PINN achieves an L2 error of ~0.021 within 20 minutes, whereas other PINNs, including CoPINN, remain near 0.99 after more than 40 minutes. See below for detailed results.

ModelRel. L2 ↓TimeEpochs
PINN0.9943840 min392,400
gPINN0.99867240 min130,000
AHD-PINN1.0000040 min81,400
SPINN1.0193540 min5,905,192
SPINN (m)0.9958740 min4,443,064
RoPINN0.9925240 min370,000
FPINN0.9883840 min202,700
CoPINN0.9945840 min318,637
PE-PINN0.0213619 min 47 s50,000

Although experiments are ongoing, competing methods have not approached PE-PINN's performance. We attribute this mainly to spectral bias, non-separable wave fields, singular sources, absorbing boundaries, and propagation over tens of wavelengths. Extending these baselines, other than PE-PINN, to the two complex dipole-array scenarios would be costly with limited additional insight, given their failure to converge in the simplest case. Nevertheless, we plan to include these new baseline results in the revised manuscript as an expanded comparison on practical wavefield problems, which are the primary focus of this work.

Please see below our responses to the specific questions.

AQ1: For exact isolated fields in homogeneous free space, direct summation is physically valid. However, learned frozen kernels contain approximation errors that can propagate through the hierarchy. PE-EK-PINN addresses this by optimizing new envelopes under the complete source, PDE, and boundary constraints.

AQ2: The novelty lies in uncovering the physics and the capability of embedding physics as kernel in PINN, which ultimately leads to our elegant and simple freeze and reuse methodology. The evolved kernel replaces multiple primitive carriers in the field representation, while newly trained envelopes enforce the physics of the larger configuration. This enables recursive compression of the representation and bounded active-kernel complexity.

AQ3: The additional single-dipole comparisons demonstrate that conventional PINN variants struggle substantially on the radiation benchmark.Our main experiments therefore focus on the scalability improvement over the strongest physics-embedded baseline established for this setting.

AQ4: Direct PE-PINN becomes computationally expensive as the number of primitive kernels increases, making direct training of large-scale configurations impractical under our available computational resources. For example, the measured training time already increases from ~67 mins for the 2×2 array to ~4.5 hrs for the 4×4 array. Based on this observed near-linear scaling, training a 16×16 array with 512 primitive kernels is estimated to require approximately 71.4 hours for a single run, making direct evaluation and repeated experiments prohibitively time-consuming. Therefore, we extrapolate the large-scale PE-PINN runtimes from the experimentally observed scaling trend, rather than performing computationally prohibitive baseline runs. We believe that the measured results already demonstrate substantial speedups, and the extrapolated values are explicitly distinguished from measured runtimes.

AQ5: We do not attribute the poor performance of conventional PINNs solely to spectral bias. As explicitly discussed in the Introduction, practical radiation problems involve multiple challenges, including non-separable wave fields, singular sources, absorbing boundaries, and propagation over tens of wavelengths. Spectral bias is one important challenge motivating architectural physics embedding, rather than the exclusive explanation for poor convergence. In fact, the original PE-PINN framework focuses on addressing these challenges jointly through physics-guided oscillatory kernels, source treatment, and appropriate boundary conditions. Meanwhile, the major contribution of our work is to address the scalability bottleneck of physics-embedded representations through hierarchical kernel evolution and reuse.

Reviewer_4FtsOpenReview ↗

Response to Authors rebuttal

I thank the authors for the detailed rebuttal, while some of my concerns still remain open:

  1. I agree that simplicity can be a strength. However, the rebuttal restates the contribution without addressing my concern that the architecture and training objective are unchanged from PE-PINN.
  2. The authors agree direct summation is valid but do not report results for it.

Considering rebuttal, I maintain my score.

Reviewer_Wd82OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

This paper proposes a structured solution space for a set of Helmholtz equations to accelerate PINN. I see this as a valid and valuable engineering product, but have concerns of its general value to AI for science. Specifically, can you characterize the scope of engineering/scientific applications where this PINN technique will have value? For example, could it be extended to heterogenous (composites)? Or beyond Helmholtz or even linear PDEs? Would this enable an accelerated inverse model for design? New evidence of capability beyond just solving Helmholtz would make the paper more suitable for this venue.

I have a minor issue with the presentation. Sec. 4.2 and 4.3 without visualization are hard to digest for readers outside of this specific domain. As a result, I understand the concept, but fail to appreciate the elegance of the method, or its broader impact on other PINN problems.

Critical strengths

The paper focuses on one PDE problem and presented a solution that works well.

Critical weaknesses

The scope of the paper seems to be too narrow and therefore its intellectual contribution to AI for science, or more narrowly, for accelerating PDE and engineering design, is quite limited. Nonetheless, I do think the paper would be a good candidate for a top engineering journal.

Questions for authors

n/a

2 comments

We thank the reviewer for the constructive assessment and for recognizing this work as a “valid and valuable engineering product.” We address each point below.

Scope of Applications and Practical Value: The Helmholtz equation underpins a wide range of wave-propagation problems, including electromagnetics (antenna analysis, radar, microwave design), acoustics (room acoustics, sonar, noise prediction), wireless communications (channel modeling, beamforming, imaging), and medical and seismic imaging (ultrasound, tomography, full-waveform inversion). The key advantage of PE-EK-PINN in these settings is amortized reuse. Rather than training a separate PINN for each frequency, source configuration, or parameter setting, the evolved kernel hierarchy transfers learned physics-aware representations across related Helmholtz problems. Accordingly, the framework is particularly beneficial when repeated solves are required under systematic parameter variations. Regarding heterogeneous media, we agree that further validation is needed. While the underlying PE-PINN framework has demonstrated wave phenomena such as reflection, refraction, and diffraction, extending PE-EK-PINN to heterogeneous environments requires careful treatment of material interfaces and physically consistent kernel reuse. In the past few days, we have implemented an additional benchmark scenario involving reflected wavefield, where one reflector is placed. Preliminary results show that our method achieves a similar reduction in training time while maintaining comparable L2 error relative to PE-PINN. Detailed results given below:

ExperimentTimeRel. L2 ↓# of Kernels
Plane Dipole Reflection Stage 1 (5×5, 1 dipole)19 min 43.87 s0.01772
Plane Dipole Reflection Stage 2 (5×2, add a reflector)14 min 07.67 s0.12642
PE-PINN Plane Dipole Reflection (5×2)38 min 52.53 s0.13604

We will include the reflection case evaluation results, and explicitly identify heterogeneous media as future work. Potential for Accelerated Inverse Design: We agree that inverse design is a promising application area. PE-EK-PINN can serve as a reusable, differentiable forward solver, where the kernel hierarchy is evolved once and reused while trainable components adapt to changing design variables. This reuse has the potential to reduce the cost of repeated forward solves within optimization loops. Although a dedicated inverse-design study is beyond the scope of the present work, we will add a discussion of this opportunity in the revised manuscript.

Generalization Beyond Helmholtz: PE-EK-PINN framework is not inherently restricted to the Helmholtz equation. Its central idea is the hierarchical evolution and reuse of physics-aware kernel representations that capture structural properties of the governing PDE. Similar strategies can be applicable straightforwardly to other PDEs exhibiting reusable solution structures.

Thank you for your responses.

I can see the paper being acceptable with either (1) concrete applications to material/structure designs in the domains specified by the authors, which often involve heterogeneity, and/or (2) expanded theory developments and validations beyond Helmholtz/Maxwell. I maintain my score.

Reviewer_SURdOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PE-EK-PINN (Physics Embedded with Evolving Kernels), a hierarchical extension of physics-embedded PINNs that treats physics kernels as reusable learned representations rather than fixed analytical inputs. By repeatedly learning a subsystem, freezing its learned field as a kernel, and reusing translated/rotated copies at higher levels, the method keeps the peak number of active kernels independent of system size and reduces cumulative training cost from O(N) to O(log N) for hierarchically structured source configurations.

Critical strengths
  1. The paper introduces the evolving physics kernels, reusing a converged subsystem field as a kernel is a natural extension of architectural physics embedding, and it replaces a kernel dictionary that grows with the number of sources with a fixed number of trainable branches per level.

  2. The hierarchy and learn–freeze–reuse pipeline in this paper are clearly explained.

  3. The computational cost for PE-EK-PINN is lower compared to PE-PINN. Stage training time stays nearly constant across levels, whereas direct PE-PINN time grows roughly linearly with the number of kernels.

Critical weaknesses
  1. The baselines of this paper are too limited. Even the proposed PE-EK-PINN method can be viewed as an extension based on PE-PINN, the main results are only compared with PE-PINN. And in Appendix B.1 the baselines are only compared with PE-PINN in Table 1.

  2. The benchmarks are also limited than the contributions. All experiments are 2D, scalar, single-frequency free-space problems, and the Helmholtz equation is manufactured for PE-PINN even for the 2.4 GHz Dipole Radiation case.

  3. The O(log N) claim rests on constant-epoch and linear-per-kernel cost assumptions, which is a little bit strong.

Questions for authors
  1. Are frozen kernels and their derivatives precomputed on the collocation grid, and is this time included in the stage time?

  2. In every experiment the reference is the superposition of isolated element fields. What is the error at each level if the frozen kernels are simply summed without training, or combined with a single least-squares complex coefficient?

  3. Regarding the weakness, could the authors provide more experimental details and comparisons?

  4. Can you show one experimental case in which superposition does not hold? Do evolved kernels still reduce training cost at comparable accuracy in that case?

2 comments

We selected PE-PINN as the primary baseline because other SoTA PINNs in Table 1, despite strong performance on manufactured benchmarks, have performed poorly on practical wavefield modeling tasks. The original manuscript reported only CoPINN and PE-PINN for the single-dipole case as the most recent and relevant approaches. To address this concern, we have now implemented all comparable PINN baselines for the same dipole case. Preliminary results support our original hypothesis that these methods struggle in practical wavefield scenarios. Specifically, PE-PINN reaches an L2 error of ~0.021 within 20 minutes, whereas all other PINNs, including CoPINN, remain near 0.99 after 40 minutes:

ModelRel. L2 ↓TimeEpochs
PINN0.9943840 min392,400
gPINN0.99867240 min130,000
AHD-PINN1.0000040 min81,400
SPINN1.0193540 min5,905,192
SPINN (m)0.9958740 min4,443,064
RoPINN0.9925240 min370,000
FPINN0.9883840 min202,700
CoPINN0.9945840 min318,637
PE-PINN0.0213619 min 47 s50,000

Although experiments are ongoing, competing methods have not approached PE-PINN's performance. We attribute this mainly to PINN susceptibility to spectral bias, non-separable wavefields, singular sources, absorbing boundaries, and propagation over tens of wavelengths. Extending these baselines to the two more complex dipole-array scenarios would be costly and provide limited insight, given their failure to converge in the simplest setting. Nevertheless, we plan to include these new baseline results in the revised manuscript as an expanded comparison on practical wavefield problems, which are the primary focus of this work.

The benchmark problems were intentionally chosen to be generic and straightforward so that the core contribution of our PINN implementation strategy can be clearly demonstrated in settings where existing PINN approaches already encounter substantial difficulties. We believe that extending the proposed framework to more application-specific and domain-dependent scenarios is relatively straightforward and can be readily explored by researchers in their respective scientific and engineering fields. That said, we have now implemented an additional benchmark scenario involving reflected wavefield, where one reflector is placed. Preliminary results show that our method achieves a similar reduction in training time while maintaining comparable L2 error relative to PE-PINN. Detailed results given below:

ExperimentTimeRel. L2 ↓# of Kernels
Plane Dipole Reflection Stage 1 (5×5, 1 dipole)19 min 43.87 s0.01772
Plane Dipole Reflection Stage 2 (5×2, add a reflector)14 min 07.67 s0.12642
PE-PINN Plane Dipole Reflection (5×2)38 min 52.53 s0.13604

We would be pleased to include additional benchmark applications, such as beamforming, in future revisions. However, from our perspective, such examples would primarily serve as further demonstrations of the same underlying concept rather than provide fundamentally new insights.

Regarding the O(log N) claim, our intention was primarily to characterize the empirical scaling behavior observed in several representative and relatively simple scenarios. As the problem complexity increases, PE-PINN itself typically requires more training epochs and incurs higher per-kernel computational costs due to increased nonlinear interactions. Consequently, the actual computational advantages of our framework in more complex settings may exceed the improvements suggested by the current O(log N) characterization. We will clarify this point in the revised manuscript to avoid overgeneralization of the asymptotic claim.

Please see below our responses to the specific questions.

AQ1: Frozen kernels are not included in the training time reported for the current stage. However, the time required to develop and train those kernels is fully accounted for in the preceding stage in which they were generated. Therefore, the total implementation and training cost of the complete model includes all kernel-development overhead.

AQ2: Regarding the concern about superposition, we do not simply sum the individual kernel outputs without any optimization. Instead, PE-EK-PINN is the exact neural network that we train to represent the slowly varying residual profile and absorb the approximation errors introduced during the superposition process. Therefore, the final solution is obtained through a learned correction mechanism rather than a direct linear combination of precomputed kernels.

AQ3: To provide a more comprehensive experimental comparison, we have extended our experiments to include the dipole scenario for other PINN-based models. Please refer to our response to the dipole case earlier.

AQ4: To address this concern, we added evaluations on reflection scenarios, as discussed earlier in the response.

Thank you for your response. The new baseline results demonstrate the advantages of PE-PINN over existing PINN variants on the dipole problem. For PE-EK-PINN itself, the main demonstrated benefit remains the reduction in training time relative to PE-PINN. We therefore maintain our score.

Comments on the paper

2 comments

Dear authors and reviewers,

The author-reviewer discussion period closes on November 18 (AoE). Please engage/continue to engage during this period so that we have our issues discussed thoroughly.

AC

We thank the reviewers for their constructive feedback. In response, we have expanded the experimental evaluation and further clarified the scope, novelty, and practical significance of PE-EK-PINN in the revised manuscript. Specifically, we have added:

  1. Expanded baseline comparison (Table 3). We evaluated nine existing PINN variants on a physically realistic 2.4 GHz dipole-radiation problem, the same practical setting that motivates this work. All conventional PINN variants fail to converge, with relative L₂ errors remaining near 0.99 after 80 minutes of training, whereas PE-PINN achieves a relative L₂ error of approximately 0.02 in less than 20 minutes. These results demonstrate that the scalability bottleneck targeted by PE-EK-PINN is a genuine and unresolved challenge in practical wavefield modeling rather than an artifact of a narrowly selected benchmark, and they establish PE-PINN as the only viable PINN baseline for this class of problems.

  2. Reflection experiments (Section 5.2.3, Table 6). We added new dipole-reflection benchmarks involving both planar and L-shaped reflectors, introducing discontinuous boundaries that are qualitatively different from the array geometries considered in the original submission. For the planar-reflection case, PE-EK-PINN achieves accuracy comparable to direct PE-PINN while reducing training time. For the L-shaped geometry, both kernel-reuse strategies reduce training cost relative to direct PE-PINN, although the resulting accuracy depends on the geometric compatibility of the reused kernel. We explicitly discuss this limitation and identify extension to more general boundary geometries as an important direction for future work.

We have also summarized our prior responses below:

Baseline comparisons. The expanded results confirm that existing PINNs (without embedded physics kernel) are unable to solve practical problems effectively. Therefore, extending these weaker baselines to the practical configurations would incur substantial computational cost while providing little additional insight. PE-PINN therefore remains the only competitive baseline for this problem setting. Practical impact. The Helmholtz equation underpins numerous applications in electromagnetics, acoustics, wireless communications, medical imaging, and seismic imaging. However, existing PINN variants struggle to solve fundamental Helmholtz wave-propagation problems at practical scales, albeit their positive claims in the manufactured PDE problems. The primary advantage of PE-EK-PINN is its ability to reuse learned physics-aware kernels across related configurations, eliminating the need to retrain models from scratch for every new source arrangement or parameter variation. This capability also makes the framework particularly attractive for inverse-design and optimization workflows that require large numbers of repeated forward simulations.

Novelty. The novelty of the freeze-and-reuse strategy lies in the observation that converged physics-embedded solutions can themselves serve as reusable computational kernels. PE-EK-PINN recursively evolves and reuses these kernels, preserving learned wave physics while maintaining bounded active-kernel complexity. At each level, trainable envelope networks are optimized under the full PDE, source, and boundary constraints to absorb accumulated approximation errors and enforce physical consistency. Training cost and scaling. Kernel-generation costs are fully accounted for at the stages in which the kernels are created. Direct PE-PINN training scales rapidly with problem size: measured runtime increases from approximately 67 minutes for a 2×2 array to approximately 4.5 hours for a 4×4 array, while larger configurations are estimated through extrapolation of the observed trend and are clearly identified as such. We will further clarify that the reported O(log N) behavior is an empirical characterization of the benchmark problems considered here rather than a universal asymptotic guarantee.

Generality. The proposed framework is not inherently restricted to the Helmholtz equation. More broadly, the hierarchical evolution and reuse of physics-aware kernels can potentially be extended to other PDEs that exhibit reusable solution structures, representing an important avenue for future research. As mentioned earlier, the results presented here also show that existing PINN variants struggle to solve even Helmholtz wave-propagation problems at practical scales, underscoring the significance of the proposed contribution within this domain.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.