acceptodds
Under review as a conference paper at ICLR 2027

PINNForge: Execution-Grounded Evolutionary Design of Physics-Informed Neural Networks for PDE Solving via Large Language Models

Abstract

Physics-informed neural networks (PINNs) require coordinated choices across representation, architecture, sampling, constraints, loss construction, and optimization, yet effective configurations can vary substantially across partial differential equations (PDEs). Existing automated design methods search over these choices, but training outcomes are still used primarily to rank candidates rather than to inform subsequent designs. We develop PINNForge, an execution-grounded large language model (LLM)-driven evolutionary framework that treats observed PINN training behavior as a cross-generation design signal. PINNForge initializes diverse configurations from PDE-related prior knowledge, executes them, retains globally strong configurations, and feeds their execution evidence and accumulated run-level experience back to the LLM. The LLM then revises and recombines coupled design components or explores new combinations within a generate–execute–evaluate–evolve loop. Across 25 PDE benchmarks, PINNForge achieves the lowest mean MSE on 24 tasks among RoPINN, PINNsFormer, PINNsAgent, and PINNForge. Removing knowledge guidance, execution feedback, or evolutionary search increases the median task-wise MSE ratio to 3.74×, 12.10×, and 10.10×, respectively, relative to full PINNForge. Anonymous code: https://anonymous.4open.science/r/PINNForge/

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Physics-Informed Neural Networks, Partial Differential Equations, Large Language Models, Evolutionary Optimization, Automated Machine Learning, Scientific Machine Learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_z15gOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PINNForge is an LLM-driven evolutionary framework that jointly designs PINN representations, architectures, sampling, constraints, losses, and optimization settings. It draws on PDE knowledge, historical elites, training diagnostics, and accumulated memory to guide configuration refinement and recombination, using a low-budget search followed by higher-budget retraining. On 25 PDEs across 10 independent runs, it achieves the lowest mean MSE on 24 tasks compared with three baselines. Ablations on 12 PDEs, along with search trajectories and optimizer-step comparisons, further examine its components and search behavior.

Critical strengths

PINNForge is built with practical use in mind: it combines training diagnostics, historical elites, and accumulated experience to jointly tune coupled PINN components, cutting down the manual configuration effort needed across PDEs of varying training difficulty. Its value is backed by broad experiments covering 25 PDEs with 10 independent runs of the full search, achieving the lowest mean MSE on 24 tasks, plus component ablations, showing it works beyond isolated examples.

Critical weaknesses

First, there is no held-out evaluation on independent reference points: Sections 4.1 and 4.5 reuse the same reference points for search, final selection, and reporting, so repeated adaptation may inflate accuracy and gains on independent points remain unverified. Second, the methodological increment isn't well isolated: PINNsAgent already uses execution feedback and memory, and ReEvo accumulates reflections to guide evolution, yet PINNForge's feedback ablation removes both execution evidence and posterior memory, so the benefit of richer diagnostics over existing mechanisms isn't clearly separated out.

Questions for authors

Please see weakness.

1 comment

We thank the reviewer for recognizing the practical value and broad evaluation of PINNForge. We have added held-out evaluation and controlled ablations to address both concerns.

1. Held-out evaluation

We reran the complete PINNForge pipeline on all 25 PDEs, with 10 independent runs per PDE, using a 70% validation / 30% test split fixed before search. Search and model selection use the validation subset; final evaluation uses the held-out test subset.

Table R1. Validation and test MSE (mean ± standard deviation over 10 runs).

PDEValidation MSETest MSE
Burgers 1D6.47E-05 ± 1.25E-066.44E-05 ± 1.34E-05
Gray–Scott 2D3.44E-03 ± 2.12E-043.43E-03 ± 1.23E-04
Heat 2D Multiscale1.19E-05 ± 4.11E-061.21E-05 ± 5.34E-06
Heat 5D4.48E-08 ± 3.89E-084.46E-08 ± 7.43E-09
Poisson 3D Complex Geometry1.09E-03 ± 2.46E-041.14E-03 ± 5.24E-04
Poisson 5D5.85E-08 ± 2.19E-085.98E-08 ± 3.23E-08

Across these six representative PDEs, validation and test mean MSE differ by less than 5%, showing consistent accuracy on points excluded from search.

All five methods are evaluated on the same 30% test subset. PINNForge achieves the lowest mean MSE on 24 of 25 tasks, with an average Friedman rank of 1.04, and outperforms our Lang-PINN reproduction on all 25 tasks. We have included the evaluation protocol in Section 4.1 and Appendix B.4, and the complete test results in Table 1 of the main text and Appendix B.6.

2. Isolating the contributions of diagnostics and memory

Building on feedback and memory mechanisms in PINNsAgent and ReEvo, PINNForge uses parent-specific physical and optimization diagnostics to guide inheritance and coordinated revision of complete PINN configurations.

To separately evaluate detailed diagnostics and posterior memory, we added three controlled settings that retain PDE priors, scalar validation feedback, and Global Top-3 configurations and rankings:

  • Scalar Only (D=0, M=0): scalar performance feedback and parent-based evolution.
  • Diagnostics (D=1, M=0): additionally supplies PDE residuals, constraint errors, and training diagnostics, with posterior summaries disabled.
  • Memory (D=0, M=1): additionally supplies posterior summaries built exclusively from configurations and scalar performance history. Detailed diagnostics are excluded before summarization.

The three settings use Qwen3.8-27B, matched seeds and initial configurations, identical training and candidate-evaluation budgets, and the same validation/test split, with 10 independent complete runs per setting and PDE.

Table R2. Held-out test MSE (mean ± standard deviation over 10 runs). Full Reference reports the earlier held-out results.

PDEScalar Only (D=0, M=0), n=10Diagnostics (D=1, M=0), n=10Memory (D=0, M=1), n=10Full Reference (D=1, M=1), n=10
Burgers 1D7.05E-05 ± 2.78E-066.79E-05 ± 1.13E-066.67E-05 ± 2.35E-066.44E-05 ± 1.34E-05
Heat 2D Multiscale2.76E-05 ± 8.58E-062.13E-05 ± 5.78E-061.56E-05 ± 4.76E-061.21E-05 ± 5.34E-06
Navier–Stokes 2D Classic7.34E-06 ± 4.13E-066.12E-06 ± 4.37E-065.67E-06 ± 2.46E-064.68E-06 ± 1.65E-06
Poisson 3D Complex Geometry3.53E-03 ± 1.31E-042.23E-03 ± 4.22E-041.89E-03 ± 1.32E-041.14E-03 ± 5.24E-04

Relative to Scalar Only:

  • Adding diagnostics without memory reduces mean test MSE by 3.7%, 22.8%, 16.6%, and 36.8% on Burgers, Heat, Navier–Stokes, and Poisson, respectively.
  • Adding memory without diagnostics reduces mean test MSE by 5.4%, 43.5%, 22.8%, and 46.5%, respectively.

Across all four PDEs, detailed diagnostics improve mean held-out accuracy beyond scalar feedback even without posterior memory, while scalar-history memory provides a separate improvement. These comparisons directly separate the two information channels within PINNForge.

We have included the additional diagnostic–memory controls and complete results in Appendix B.7 of the revised manuscript. We have also clarified the technical distinction from PINNsAgent and ReEvo in the related-work discussion.

Reviewer_XseROpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2(+1)
Read review
Paper summary

This paper proposes an LLM-driven evolutionary framework for automated PINN design. The method initializes a diverse population of PINN configurations informed by PDE-related prior knowledge, executes them, and feeds execution evidence (PDE residuals, constraint satisfaction, loss dynamics, convergence behavior) back to an LLM, which then generates new candidates. The framework employs a two-stage multi-fidelity protocol: low-fidelity search followed by high-fidelity retraining. Experiments are conducted across 25 heterogeneous PDE benchmarks, where the proposed method is compared against RoPINN, PINNsFormer, and PINNsAgent, achieving the lowest mean MSE on 24 out of 25 tasks.

Critical strengths

Extensive evaluation and ablation study. The paper evaluates PINNForge on 25 heterogeneous PDEs spanning nonlinear transport, reaction–diffusion, multiscale dynamics, complex geometries, high-dimensional PDEs, inverse problems, and fluid systems. There is an ablation study that effectively provides insight into the relative contributions of each component.

Critical weaknesses

Limited advancement over state-of-art. This is the main issue with the paper. There is already a multitude of papers about LLM-based and agentic-based design of PINNs claiming to improve accuracy by orders of magnitude. Despite innovations such as using a prior-knowledge database and cross-generation inheritance, the methodology follows the same basic loop of generate, execute, evaluate, and evolve already proposed by many other authors.

Questions for authors

I enjoyed reading the paper and I appreciated some of its features, such as the prior-knowledge database, which itself could benefit the research community if released. However, the paper does not seem to propose a fundamentally novel approach to the problem of automatic PINN design. Some questions that could change this assessment are listed below.

  1. What do the authors consider the most technically novel contribution of PINNForge relative to related approaches in the literature, such as EoH, PINNsAgent and Lang-PINN? Could the authors provide a more precise technical differentiation from these methods?

  2. Why was Lang-PINN excluded from the comparison? Given that it also uses LLM-based multi-agent PINN design, its inclusion would improve evaluation of the claimed contribution.

  3. All experiments use Qwen3.8-27B. Have the authors tested PINNForge with any other LLM? An experiment with at least one alternative model would clarify how sensitive the reported results are to Qwen3.8-27B's code-generation and reasoning capabilities and whether they generalize.

  4. Fixed reference points are used for both search control and final reporting. Experiments with a separate held-out evaluation are necessary to avoid potential bias.

3 comments

We thank the reviewer for the constructive feedback. We have clarified the technical contribution and added Lang-PINN comparisons, backbone experiments, and held-out evaluation.

1. Held-out evaluation

We reran PINNForge on all 25 PDEs with a 70% validation / 30% test split fixed before search, using 10 independent complete runs per PDE. Validation MSE guides search and model selection; all five methods are evaluated on the same held-out test subset. All additional experiments and ablations reported below use this same 70% validation / 30% test split.

PINNForge achieves the lowest mean test MSE on 24 of 25 tasks and an average Friedman rank of 1.04. We have included the evaluation protocol in Section 4.1 and Appendix B.4, and the complete test results in Table 1 of the main text and Appendix B.6.

2. Technical contribution

PINNForge builds on EoH's evolutionary framework. Its contribution is coupling complete, controller-validated AlgorithmSpecs with parent-specific physical and optimization diagnostics. The historical Global Top-3 provides explicit parents for joint revision and multi-parent synthesis across representation, architecture, sampling, constraints, loss, and optimization, supported by run-level memory.

EoH provides the general evolutionary foundation; PINNForge specializes it to complete PINN designs and PDE-specific diagnostics. PINNsAgent uses knowledge replay, feedback-driven planning, and Memory Tree Reasoning; PINNForge explicitly evolves historical elite configurations. Lang-PINN coordinates formulation, architecture, code generation, and diagnostic refinement; PINNForge organizes population-based evolution around Global Top-3 parents and their execution evidence. We have clarified these distinctions in the introduction and related work.

The original ablations increase median task-wise MSE to 10.10× and 12.10× the full-method value without parent-driven evolution and execution feedback, respectively. We also added diagnostic–memory controls on four PDEs with 10 runs per setting. With scalar feedback and parent evolution retained, adding diagnostics without memory reduces mean test MSE by 3.7–36.8%; adding memory without diagnostics reduces it by 5.4–46.5%. These results separate the two contributions. The complete controls and results are included in Appendix B.7.

3. Lang-PINN comparison

We originally omitted Lang-PINN because we could not locate public code. We have now reproduced its published method using Qwen3.8-27B, the same backbone as PINNForge. Like the other baselines, each run trains five candidates for 20,000 steps each (100,000 total), selects by validation MSE, and reports test MSE. We conduct 10 independent complete runs per PDE.

PINNForge has lower mean test MSE than our Lang-PINN reproduction on all 25 PDEs. Table R1 shows five representative tasks. We have added the complete 25-PDE comparison to Table 1 of the main text and Appendix B.6, with the setup and budget described in Sections 4.1 and 4.3 and Appendices B.4 and B.8.

Table R1. Test MSE (mean ± standard deviation, 10 runs). Lang-PINN is our reproduction.

PDELang-PINNPINNForge
Burgers 1D7.49E-04 ± 6.02E-056.44E-05 ± 1.34E-05
Gray–Scott 2D7.23E-03 ± 4.54E-043.43E-03 ± 1.23E-04
Heat 2D Complex Geometry5.47E-04 ± 8.98E-054.73E-04 ± 9.31E-05
Heat 2D Multiscale3.25E-05 ± 7.47E-061.21E-05 ± 5.34E-06
Poisson 5D2.76E-06 ± 1.11E-065.98E-08 ± 3.23E-08

4. Alternative LLM backbones

We evaluated DeepSeek-V4.1-Flash-1017 and GLM-5.3 on six PDEs, with 10 independent complete runs per backbone and PDE under the same protocol.

Table R2. Test MSE (mean ± standard deviation, 10 runs).

PDEQwen3.8-27BDeepSeek-V4.1-Flash-1017GLM-5.3
Burgers 1D6.44E-05 ± 1.34E-056.45E-05 ± 2.56E-066.49E-05 ± 1.16E-06
Gray–Scott 2D3.43E-03 ± 1.23E-043.31E-03 ± 2.51E-043.41E-03 ± 5.46E-04
Heat 2D Multiscale1.21E-05 ± 5.34E-061.25E-05 ± 2.13E-061.38E-05 ± 3.89E-06
Heat 5D4.46E-08 ± 7.43E-094.55E-08 ± 4.68E-094.34E-08 ± 9.41E-09
Poisson 3D Complex Geometry1.14E-03 ± 5.24E-041.18E-03 ± 4.11E-041.09E-03 ± 4.53E-04
Poisson 5D5.98E-08 ± 3.23E-086.12E-08 ± 7.82E-085.85E-08 ± 8.33E-08

Relative to Qwen3.8-27B, the largest difference in mean MSE is 3.51% for DeepSeek and 14.05% for GLM. These results show comparable mean accuracy across the tested backbones. We have added the backbone comparison to Section 4.5, with complete results in Appendix B.10.

Thanks for replying to my questions in detail. I am convinced and will raise my score. If the paper is accepted, please add all new results to the camera-ready version.

Thank you for your thoughtful review and positive feedback. If the paper is accepted, we will include new results and the corresponding discussions in the camera-ready version.

Reviewer_qTnhOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes PINNForge, an LLM-driven evolutionary framework for automated design of physics-informed neural networks. PINN design is framed as an execution-grounded multi-generation search over a unified design space spanning representation, architecture, sampling, constraints, loss construction, optimization, and training control. Experiments cover 25 heterogeneous PDEs, mostly from PINNacle plus four additional equations. PINNForge achieves the lowest mean MSE on 24 of 25 tasks, with Friedman rank 1.04, and ablations show large degradations when execution feedback or evolutionary search is removed.

Critical strengths
  1. The execution-grounded search formulation is well motivated and the ablation evidence is strong. 2.The benchmark coverage is broad and the statistical reporting is appropriate.
Critical weaknesses
  1. The reuse of fixed reference points for search and reporting invalidates the headline comparison. Since candidate selection, final configuration choice, and reported MSE all use the same points, the method can overfit to those points.
  2. The baselines are not given an equivalent search or tuning budget. PINNForge uses an LLM-driven search over complete configurations, with up to 83,000 optimizer steps plus LLM inference. The baselines are trained with 100,000 steps and best-of-five selection.
Questions for authors
  1. Can you report results with a held-out set of evaluation points that is never used for candidate comparison or final selection? Or intoruce some integral quality metrics that clear good gain from your method.
  2. Were RoPINN, PINNsFormer, and PINNsAgent given a comparable hyperparameter search on a validation set?
  3. How much does the final result vary across LLM seeds or sampling temperatures?
3 comments

We thank the reviewer for the constructive comments and for recognizing the methodological motivation, extensive evaluation, and ablation evidence of PINNForge. We address the concerns below.

  1. To address the potential selection bias, we introduced a 70% Search/Validation and 30% Test split across all 25 PDEs, with 10 independent runs per PDE. The validation set is used for candidate comparison, evolutionary search, and final model selection, while the test set is strictly excluded from these procedures and used only for final evaluation.

Table R1. PINNForge MSE before and after introducing the held-out split (mean ± std over 10 runs). Original MSE is retained as a historical reference.

PDEOriginal MSEValidation MSETest MSE
Burgers 1D6.50E-05 ± 4.42E-076.47E-05 ± 1.25E-066.44E-05 ± 1.34E-05
Gray–Scott 2D3.60E-03 ± 2.65E-043.44E-03 ± 2.12E-043.43E-03 ± 1.23E-04
Heat 2D Multiscale1.19E-05 ± 2.16E-061.19E-05 ± 4.11E-061.21E-05 ± 5.34E-06
Heat 5D3.21E-08 ± 3.26E-084.48E-08 ± 3.89E-084.46E-08 ± 7.43E-09
Poisson 3D Complex Geometry1.04E-03 ± 2.34E-041.09E-03 ± 2.46E-041.14E-03 ± 5.24E-04
Poisson 5D5.64E-08 ± 3.95E-085.85E-08 ± 2.19E-085.98E-08 ± 3.23E-08

As shown in Table R1, all six representative PDEs exhibit less than 5% relative difference between validation and test mean MSE.

All five methods are evaluated on the same 30% test subset. PINNForge achieves the lowest mean MSE on 24 of 25 tasks, with an average Friedman rank of 1.04, and outperforms our Lang-PINN reproduction on all 25 tasks. We have included the complete test results in Appendix B.6 of the revised manuscript.

  1. RoPINN, PINNsFormer, and PINNsAgent underwent method-specific searches following their original methods. Each complete baseline run comprises five rounds of candidate training with 20,000 optimizer steps each, totaling 100,000 steps, including all candidate training used for search and selection. The candidate with the lowest validation MSE is selected for held-out test evaluation. Our Lang-PINN reproduction uses Qwen3.8-27B and the same 100,000-step baseline budget. PINNForge uses at most 83,000 optimizer steps per complete run.

  2. We evaluated PINNForge under three LLM sampling temperature settings on three representative PDEs, with 10 independent complete runs per setting. The lower and higher settings uniformly shift the temperatures of all candidate-generation roles and strategies by −0.1 and +0.1 relative to their defaults.

Table R2. Sensitivity to LLM sampling temperature (Test MSE, mean ± std).

PDELow (−0.1)DefaultHigh (+0.1)
Burgers 1D6.65E-05 ± 2.86E-066.44E-05 ± 1.34E-056.46E-05 ± 1.67E-06
Heat 2D Multiscale2.67E-05 ± 2.70E-061.21E-05 ± 5.34E-061.73E-05 ± 8.32E-06
Poisson 5D4.57E-08 ± 2.48E-085.98E-08 ± 3.23E-085.56E-08 ± 4.41E-08

Sensitivity is task-dependent: Burgers 1D mean MSE changes little, while Heat 2D Multiscale has mean test MSE 2.21× and 1.43× the default value at the lower and higher settings, respectively. We have included the complete temperature-sensitivity results in Appendix B.9.

Thank you very much, I am fine with your answers, I will update the score

Thank you for your positive feedback. We do appreciate your careful review and helpful suggestions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.