acceptodds
Withdrawn from ICLR 2027

Physics-Informed Neural Plasticity: PDE Solvers That Reshape Themselves

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Physics-informed neural PDE solvers adapt their parameters to satisfy governing equations, yet their representational structure typically remains fixed throughout training. This rigidity is poorly matched to PDE solutions with strongly heterogeneous complexity across space and space–time, leaving capacity insufficient where the physics is difficult and redundant where it is simple. We introduce physics-informed neural plasticity, a paradigm in which the representation itself reshapes during optimization in response to unresolved physics. We instantiate this principle with Representation Capacity Adaptation for PDEs (ReCAP), a Gaussian-localized solver that dynamically redistributes capacity through local enrichment, residual-directed splitting, gate-based pruning, and function-aware merging. ReCAP uses responsibility-weighted error indicators and the geometry of residual energy to determine where and how to refine. To limit the disturbance introduced by splitting, we introduce quiet-child refinement, which initializes new components by transporting the parent representation while controlling instantaneous functional perturbation. We further establish conditional a posteriori reliability and structural-stability guarantees linking localized physics residuals to solution error and stable refinement. Across five challenging 3D and 4D PDE benchmarks against 11 physics-informed solvers, ReCAP achieves the lowest relative error on every problem, reducing error by – relative to the strongest competing result. These results suggest that physics-informed solvers need not merely learn their parameters—they can learn how their representational capacity should be organized.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
PINNs, PDEs, numerical method.

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_QL32OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The authors introduce a method for adaptively allocating representational capacity in a physics-informed solver during optimization. The representation consists of Gaussian-localized components, each equipped with a local polynomial (monomial basis), whose responsibilities form a partition of unity. Capacity is adapted by splitting, polynomial enrichment, merging, and pruning, driven by residual-derived indicators. Since inserting new components can disrupt the learned solution, the authors introduce quiet-child refinement, which initializes the child as an exact copy of the parent's local polynomial with a small gate. The theoretical results show that, under an assumed PDE stability estimate, the mass-weighted local indicators bound the solution error, and that, for linear transport, the loss increase caused by a quiet-child insertion is controlled by the child's gate. Experiments on five smooth 3D/4D benchmarks against 11 physics-informed baselines report the lowest relative L2 error on all problems, with ablations on one benchmark (KG3D).

Critical strengths

1. Conceptually interesting approach. Treating the representation of a physics-informed solver as an adaptive variable, refined where the physics is unresolved (splitting, polynomial enrichment) and coarsened where it is redundant (pruning, merging), is a relevant direction for solutions with varying complexity.

2. Link between local indicators and theoretical guarantees.​ The paper makes explicit that the mass-weighted local residual indicators decompose the global residual exactly (via the partition-of-unity property), so that under a PDE stability estimate they bound the solution error; it also bounds the loss increase caused by quiet-child insertion for linear transport. This gives the localized adaptation signals a principled interpretation.

Critical weaknesses
  1. The method is not specified well enough to be understood, reproduced, or connected to the theory.​ Parts of the method are well explained, in particular how the residual-based indicators drive splitting, merging, pruning, and enrichment. However, the model these rules operate on is never defined. Although the components are described as Gaussian-localized, the Gaussian function is not given, and the solution is only defined abstractly as uΘ=AΘ(πm,ℓm,gm)u_Θ = A_Θ({π_m, ℓ_m, g_m}), without specifying the aggregation map AΘA_Θ. The responsibilities π_m are only required to be nonnegative, yet they weight every adaptation signal, and it is not stated how they are computed or whether they include the gates. The abstract formulation may be intended for generality (Appendix A.2 states that the procedure is "agnostic to the specific responsibility and output-aggregation rules"), but the experiments necessarily use a specific choice, which is not reported, and no second instantiation demonstrates the claimed generality. Appendix A.1 further states that ΘΘ contains "the parameters of the base model and the parameters of the adaptive Gaussian representation," and Table 2 lists "hidden features," yet the base model is never described. Similarly, the method allows hard or soft enforcement of initial/boundary conditions, but the experiments do not specify which is used (only M1 in the ablation is described as hard-constrained) or how the hard constraints are constructed. As a consequence, the method cannot be reproduced, and it is not possible to judge which ablations would be needed to support the claims, since these depend on the model (e.g., whether a feature network sits on top of the Gaussians). It is also unclear whether the theoretical results apply to the implemented model, since Theorem 3.1 assumes normalized responsibilities and Theorem 3.2 a normalized gated mixture (Eq. 12, the only explicit model form in the paper). A complete model definition and a pseudo-algorithm of the training/adaptation loop are needed to assess the method.

  2. The experiments do not establish that the improvements are reliable or that they come from adaptation.​

  • (i) Reliability.​ Table 1 reports single runs without standard deviations or a stated number of seeds. The KG3D result for the full method in Table 1 also differs from those in the cumulative ablation and the threshold sweep (Fig. 3, left and right), where the method performs on par with SPINN, the strongest baseline. This might be explained by seed variance, as the standard deviation reported in the ablation exceeds the claimed margin over SPINN, or by differing configurations, which the paper does not clarify. No baseline configurations, budgets, or tuning procedures are given.
  • (ii) Attribution to adaptation.​ A comparison with and without adaptation (Fig. 3, middle) is given only on KG3D, as are all ablations, and it matches the final component count rather than parameters or compute; for the other four benchmarks, the gain from adaptation cannot be separated from the choice of base model. No parameter counts are reported, and the cost of the adaptation events themselves is not measured. It is also unclear what coarsening (pruning and merging) contributes: in the ablation, adding it lowers the mean error by less than the reported standard deviation, and the paper does not report how many components are removed or whether this reduces cost. Finally, the paper never shows the adapted representation itself (where components are split, merged, or pruned, and how their number evolves), so it is not visible whether capacity is actually allocated where the residual is high. Moreover, each equation is evaluated at a single parameter setting (e.g., one set of wavenumbers for H3D), so it is not shown whether the method is robust across parameters, or whether the benefit of adaptation grows as the solution becomes more heterogeneous, which is the paper's motivation. Depending on the implemented model (see Weakness 1), further ablations may be needed. This information is crucial for a final assessment of the method: without it, the improvements cannot be distinguished from seed variation or tuning differences, nor attributed to the adaptation rather than to capacity, adaptive sampling, or the base model.
Questions for authors

Clarifications:​

  1. Model definition.​ How exactly is u_Θ constructed in the experiments? Please give the aggregation map A_Θ, the Gaussian function, and how the responsibilities π_m are computed (including whether they contain the gates). What is the "base model" (Appendix A.1), and what do the "hidden features" in Table 2 represent? Which boundary/initial conditions are hard-constrained in the main experiments, and how? Does the implemented model satisfy the assumptions of Theorems 3.1 and 3.2?

  2. Statistical reliability and baselines.​ How many seeds were used for Table 1? Please report mean ± std for ReCAP and the strongest baselines on all benchmarks, and explain why the KG3D results differ between Table 1 and Fig. 3. How were the baselines configured and tuned, and were they given the same budget (iterations, collocation points, adaptive sampling)?

  3. Effect of adaptation.​ On all benchmarks, how does ReCAP compare with the same model without adaptation at matched parameter count and compute? Please report parameter counts and wall-clock time, including the overhead of the adaptation events.

  4. Relation to prior work.​ How does ReCAP differ from PINN Balls (Bonfanti et al., 2025, arXiv:2510.21262), which also uses a learnable, residual-driven partition-of-unity mixture of local experts? Please also discuss partition-of-unity and RBF-PU methods, which have the same form as Eq. 12.

  5. Finally, how are new components implemented: are up to M_max components preallocated and only activated (as for the polynomial coefficients), or are parameters added during training? Which optimizer is used, and is it reset after adaptation events, or how are the optimizer states of new components initialized?

Suggestions (not decisecive):​

  1. Restructure Section 3.4: move Theorem 3.1 to the appendix, keeping a pointer in the main text, and motivate the paragraph on controlling the effect of a new child (bounding the loss increase caused by inserting a component), placing it directly after the split initialization.

  2. Visualize the adapted representation: component placement and count over training (as in PIG).

  3. Evaluate each equation at several parameter settings, and on problems with localized features (e.g., an L-shaped domain).

  4. Single-factor ablations: enrichment, pruning, and merging separately; leave-one-out; quiet-child gate and coordinate translation.

  5. Quantify the perturbation caused by merging, and check whether the indicators η_m correlate with the true local error.

  6. Related work: quiet-child insertion and error-driven splitting/pruning relate to function-preserving network growth (e.g., Net2Net, Chen et al., 2016), resource-allocating RBF networks (e.g., Platt, 1991), and adaptive density control in Gaussian Splatting (e.g., Kerbl et al., 2023). Discussing these would clarify what is specific to ReCAP, e.g., the coordinate translation of local polynomials.

Reviewer_Z2d7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The proposed paper studied the neural plasticity in physics informed methods. The algorithm takes place at training time and allows the model to merge, split... some component so that it adapts itself its sizes to the difficulty of the problem. The proposed method is first introduced with each operation being described, then some theoretical justifications are proposed. Finally an experimental section evaluated the proposed model against baselines. Supplementary materials provide additional elements such as hyper parameters, visualization or dataset details.

Critical strengths
  • The idea of reshaping the solution during training is very interesting to me as it could help having well dimensioned models for each problem.
  • The proposed method is evaluated against numerous baselines of the literature
  • A theoretical analysis of the method is proposed.
Critical weaknesses
  • I found the paper a bit hard to read. It is very technical and a lot of notations are used (for instance several τ\tau that are mentioned only once in the paper - line 229). Maybe a notation table in the appendices would help clarify it, also a pseudo-code can be very informative in cases where the training procedure is specific.

Minor

  • Why on figure 1, the start point and enriched versions are similar?

Comment

I am not an expert in neural plasticity methods and literature, so I could not check the technical proofs. I reflect this with a low confidence on my review.

Questions for authors
  • Have you experimented optimization instabilities or conditioning issues during training ? It is widely studied and known in the literature that physics informed losses and gradient are ill-conditioned, making the training of physics informed method highly inefficient ([1, 2]). This was particularly studied on the Helmholtz equation that was used in the paper.

  • Can you provide a comparison of the proposed method wrt to baselines regarding training time (I saw flops in the appendices) and inference latencies ? How does the neural plasticity influence these metrics ? If Eigenvectors (line 255) are required to be computed, I would expect a high computational cost of the method. Could you comment on that ? Also, what are the architecture sizes of the baselines ?

  • Have you studied the effect of the hyper parameters MmaxM_{max} on the method ? What happens if the budget is under estimated for a given problem ? Also, how is the starting size of the model chosen ? Is there a heuristic ?

  • I found experiment fig 3. Middle very interesting as it isolated the contribution of the training method on the result. Would you have any insight on why does the training procedure used provide better results than a standard one, using from the beginning all the required weights / model size ?

  • Just being curious, have you tested if the found solution could be easily used as a starting point for a similar equation (let say modifying one parameter or changing an initial condition) ? I believe that one application could be to adapt surrogates to new PDEs that could exhibit more complex behaviors. better than with a standard training procedure that is limited by the capacity of the considered model.

[1] An operator preconditioning perspective on training in physics-informed machine learning. Tim De Ryck, Florent Bonnet, Siddhartha Mishra, Emmanuel de Bézenac, 2023.

[2] Learning a Neural Solver for Parametric PDE to Enhance Physics-Informed Methods. Lise Le Boudec, Emmanuel de Bezenac, Louis Serrano, Ramon Daniel Regueiro-Espino, Yuan Yin, Patrick Gallinari, 2024.

Reviewer_GtbSOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The authors introduce ReCAP (Representation Capacity Adaptation for PDEs), a physics-informed PDE solver that dynamically adapts its representational structure during, which the authors call "physics-informed neural plasticity." Unlike standard PINNs, which fix their architecture before training, ReCAP uses Gaussian-localized components that can undergo polynomial enrichment (p-like), residual-directed splitting (h-like), gate-based pruning, and function-aware merging during optimization. The authors provide a theoretical analysis. Experiments are conducted on five 3D/4D PDE benchmarks (FM3D, H3D, ND3D, KG3D, KG4D) against eleven competing methods.

Critical strengths

Benchmarks and baselines. The paper compares against eleven competitors across five benchmarks of varying PDE type and dimensionality (3D flow mixing, 3D Helmholtz, 3D nonlinear diffusion, 3D and 4D Klein–Gordon equations), which is more thorough than most PINN papers.

Ablation study. The progressive ablation on KG3D (M1 through M4) and the comparisons (PIG-small, PIG-large, Fixed-pmax, Split-only vs. ReCAP) is a strength of the paper.

Critical weaknesses

PINN is a very mature area by now and the bar to justify publishing yet another PINN improvement paper is high. Another problem I see is that the approach proposed is very complex and therefore requires a lot of tuning. I detail these points below.

Incremental Contribution in a Saturated Area. The literature on PINNs has produced a vast and still-accelerating catalogue of proposed improvements spanning architecture design, loss balancing, adaptive sampling, domain decomposition, spectral bias mitigation, and more. Therefore, it is no longer sufficient to demonstrate lower relative L2 error on a handful of benchmarks, because the field has accumulated dozens of papers making exactly that claim, each on a slightly different benchmark suite, against a slightly different selection of baselines, under slightly different experimental conditions, even if ReCAP has a broader experimental evaluation than most other PINN papers. The paper does not examine why adaptive representational capacity in PINNs is worth pursuing over more recent competing paradigms (e.g., operator learning, multi-operator learning, foundation models.

Hyperparameter sensitivity and tuning. ReCAP is a complex algorithm that introduces a large number of hyperparameters (Table 2) and all of them vary across benchmarks. The paper does not provide a principled, problem-agnostic strategy for setting these. For example, the splitting threshold sensitivity plot (Figure 3) shows a complex response, suggesting that poor choices can substantially degrade performance. This raises serious concerns about the practical usability of the method on new problems where ground truth is unavailable for tuning.

Questions for authors

Question 1. Achieving the best result on every single benchmark against eleven competitors is an extraordinary claim. Different PINN methods tend to have different inductive biases suited to different PDE types. Notably, some reported baseline numbers (e.g., PINN on H3D: 0.528, Scale-PINN on H3D: 0.566) appear quite poor relative to what these methods are known to achieve on similar problems, suggesting possible misconfiguration. Can the authors show that all the baselines were tuned with a comparable effort to ReCAP?

Question 2. The overhead of ReCAP relative to simpler baselines is probably considerable. Hence, one needs to see, in addition to accuracy, a comparison of wall-clock training times. Only a comparison floating-point operation counts for one of the benchmarks is provided.

Question 3. The theoretical guarantees require strong assumptions. Can the authors provide concrete examples where these assumptions are valid?

Question 4. The authors should provide confidence intervals or multiple-run statistics in all experiments. Given the stochastic nature of training, single-run comparisons are insufficient.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.