acceptodds
Withdrawn from ICLR 2027

Fuse-PINN: Combining Finite Element Methods and Physics-Informed Neural Networks to enable massively parallel simulations of non-linear PDEs

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Research in the domain of Physics-Informed Neural Networks applied to PDE modelling has advanced significantly in recent years, providing a wide ecosystem of novel methods. However, one frequent limitation is the inability of PINNs to scale out on High-Performance Computing architectures to address more ambitious challenges, particularly non-linear physics models with turbulent dynamics in large spatial domains. Typical studies restrain demonstrations to toy models with simple physics equations, few variables, and rarely any complex non-linear dynamics evolved over long timescales. Part of the AI community has even adopted the idea that “PINNs don't scale”. This work proves otherwise. We present a mathematically robust formulation of Finite-Element-Method for Neural Networks (FEM-NN) implemented into a comprehensive, modular framework: Fuse-PINN. This formulation provides the basis for a sclable parallelisation of Physics-Informed Neural Networks on GPU-based HPC systems. Furthermore, an Energy Natural Gradient Descent optimiser is implemented to allow high-precision convergence. Fuse-PINN is demonstrated on physics models relevant to Magnetic Confinement Fusion, including non-linear convective models with fine-scale dynamics. Near-perfect scaling is achieved on multi-GPU runs with increasing problem size. The code is made fully open-source (upon publication).

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Physics Informed Neural Networks, PINN, Finite Element Methods, FEM, Finite Element Methods for Neural Networks, FEM-NN, Energy Natural Gradient Descent, Multi-GPU Parallelisation, High Performance Computing, HPC, Fusion

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_R5WCOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposed Fuse-PINN, a scientific machine learning approach integrating the FEM and the conventional PINN. Instead of using only single MLP to formulate the solution, this work first divides the computational domain into multiple triangular areas (like the meshing process in the standard FEM) and represent the solution with multiple neural surrogates. In the proposed method, the authors developed a FEM-NN representation to express the global solution, which is formulated by assigning different MLPs to the corresponding nodes. To maintain the C1 continuity of the global solution, a special weight function is developed to keep the gradient continuous between two triangular element areas. And some FEM techniques and PINN techniques are also integrated, including Crank-Nicolson scheme and ENGD optimizer.

Critical strengths
  1. This manuscript proposed a valuable framework for the scientific machine learning. Several useful techniques are proposed and developed, including fem-nn representation, ENGD optimizer, Crand-Nicoson scheme.

2.This method shows per-epoch weak scalability. The proposed framework is implemented within a distributed GPU computing system. The corresponding node-communication mechanism seems effective. And the numerical results validate the parallel efficiency of such implementation.

3.The numerical experiments include several complicated nonlinear cases, including Hasegawa–Wakatani and Reduced-MHD equations, illustration the framework potential in addressing some nonlinear PDEs describing some complex systems.

Critical weaknesses
  1. The scalability is one the key contribution stated in the paper. However, the results are not fully convincing. It is not sufficient to only report per-epoch runtime. When the system scales up, it might generally require more training iteration for model convergency. Therefore, an end-to-end scalability test is necessary.

  2. The central part of the framework is the fem-nn representation. The authors claim the importance of maintaining c1/c2 continuity and thereby introduced a complicated weight function design. However, in the standard FEM, the weak form is designed for addressing this issue. For instance, the Sobolev space H¹(Ω) does not maintain such continuity but still be extensively used. Thus, to fully illustrate the importance of c1/c2 continuity and the proposed fem-nn, it is necessary to include more concrete numerical results, such as some ablation study.

  3. One potential concern about the fem-nn representation is its generalization on some PDEs with discontinuous coefficient fields. For example, considering a darcy flow problem within piecewise constant permeability, the solution gradient will jump across different interface.

  4. Existing studies already explore the combination of conventional numerical techniques, such as domain decomposition, with PINN. And utilizing multiple MLPs to represent one PINN solution is also not new (which is also mentioned in the manuscript). However, the comparison with these existing study, such xPINN, is missed.

Questions for authors

Please refer to the concerns in the weaknesses section.

Reviewer_uHSVOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes a distributed physics-informed neural network framework for solving nonlinear PDE on multi-node, multi-GPU HPC platform. The framework combines several strategies and methods including a domain decompositon strategy with mesh neural networks, Crank-Nicolson solver, separate networks for different variables, and an ENGD optimiser. The individual methods are not new. The authors demonstrate the framework on fusion physics models. Experiments report near-linear weak scaling with communication overhead accouting for a small fraction of the computation time. However, the current implementation is three orders of magnitude less computationally efficient than conventional numerical solver. The reported scaling measurements are for per training epoch, and do not represnt the complete time-to-solution.

Critical strengths
  • This paper addresses the important challenge of scaling PINN-based PDE solvers to distributed HPC architectures.
  • The paper provides extensive implementation details and their discussions.
Critical weaknesses
  • The paper emphasises near-perfect linear scaling in parallel simulation. However, the weak scaling is measured in terms of computation time per training epoch, not the end to end time required to solve a PDE to a specified accuracy.
  • The methodological novelty is limited. The framework combines several established techniques. The paper does not establish which components constitute substantive methodological advances over existing approaches.
  • Practical computational benefits are limitted. The framework is three orders of magnitude less efficient than conventional numerical solvers.
Questions for authors
  • can the authors provide end to end scalability results rather than per-epoch weak-scaling measurements?
  • What are the specific methodological advantages of the framework over existing distributed PINN approaches?
  • How does the proposed framework compare with conventional numerical solvers under matched accuracy and computational resources?
  • What is the individial contribution of each component to accuracy, convergence and scalability?
Reviewer_JmwNOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces Fuse-PINN, a framework combining finite-element-inspired domain decomposition with PINNs. Separate neural networks are assigned to each vertex of a coarse domain decomposition mesh and to each physical variable of the PDE. Energy Natural Gradient Descent (ENGD) is used as the optimiser. The network outputs are combined using barycentric interpolation with local P1-type basis functions. The method is demonstrated on nonlinear PDEs relevant to fusion modelling, with spatial resolutions of up to 640 × 640 collocation points. A weak-scaling experiment demonstrates near-constant training time per epoch as the number of network parameters and GPUs increases.

Critical strengths

The paper addresses the computational scalability of PINNs for nonlinear, coupled PDE systems, an important challenge in scientific machine learning. The application to fusion modelling provides a potentially interesting setting for distributed PINN solvers. The weak-scaling experiments demonstrate approximately constant training time per epoch as the number of networks and GPUs increases.

Critical weaknesses

Limited novelty. The proposed architecture closely resembles existing domain-decomposed PINNs, particularly FBPINNs (Moseley et al., 2023), multilevel FBPINNs (Dolean et al., 2024), XPINNs (Jagtap & Karniadakis, 2020), and parallel PINNs (Shukla et al., 2021).

Redundancy in the representation. Each point within a triangular element in the domain is represented by the barycentrically weighted sum of 3 networks, however there is no constraint on the magnitude of each network. It is not clear the benefit that comes from representing the solution with the sum of 3 networks, with no demonstrated advantage over existing approaches.

Weak coupling across domains and PDE variables. Networks share no learned representation and interact only through PDE residuals during the optimisation. It is unclear how effectively the architecture captures coupled physical dynamics. Insufficient methodological detail. Section 2 is a bullet point list and Figure 2 do not clearly explain the network assembly, distributed training or implementation of ENGD.

No comparative experiments. There are zero direct comparisons with existing PINN approaches for either speed or accuracy, making it impossible to assess relative accuracy or efficiency.

Unsupported scalability claims. Near-constant time per epoch under weak scaling does not establish improved solution convergence, time to solution and it is not clear how the grid/collation points are scaled as the FEM-NN nodes are increased in Table 1. Implementation currently ties every FEM-NN node to its own GPU seems unnecessary at this scale.

Overstated motivation. The claim that PINNs are broadly considered unscalable overlooks substantial existing work on parallel and domain-decomposed PINNs.

Questions for authors
  1. Novelty: What is the principal methodological advantage of Fuse-PINN over FBPINNs and other domain-decomposed PINN approaches?

  2. Representation: What motivates assigning independent networks to each mesh vertex and combining their outputs through P1 basis functions? Why is this preferable to a single network per subdomain?

  3. Scalability: How do the number of collocation points and spatial resolution change across the weak-scaling experiments in Table 1? How does the number of epochs required to reach a fixed accuracy change?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.