- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
PINNForge is an LLM-driven evolutionary framework that jointly designs PINN representations, architectures, sampling, constraints, losses, and optimization settings. It draws on PDE knowledge, historical elites, training diagnostics, and accumulated memory to guide configuration refinement and recombination, using a low-budget search followed by higher-budget retraining. On 25 PDEs across 10 independent runs, it achieves the lowest mean MSE on 24 tasks compared with three baselines. Ablations on 12 PDEs, along with search trajectories and optimizer-step comparisons, further examine its components and search behavior.
Critical strengths
PINNForge is built with practical use in mind: it combines training diagnostics, historical elites, and accumulated experience to jointly tune coupled PINN components, cutting down the manual configuration effort needed across PDEs of varying training difficulty. Its value is backed by broad experiments covering 25 PDEs with 10 independent runs of the full search, achieving the lowest mean MSE on 24 tasks, plus component ablations, showing it works beyond isolated examples.
Critical weaknesses
First, there is no held-out evaluation on independent reference points: Sections 4.1 and 4.5 reuse the same reference points for search, final selection, and reporting, so repeated adaptation may inflate accuracy and gains on independent points remain unverified. Second, the methodological increment isn't well isolated: PINNsAgent already uses execution feedback and memory, and ReEvo accumulates reflections to guide evolution, yet PINNForge's feedback ablation removes both execution evidence and posterior memory, so the benefit of richer diagnostics over existing mechanisms isn't clearly separated out.
Questions for authors
Please see weakness.
We thank the reviewer for recognizing the practical value and broad evaluation of PINNForge. We have added held-out evaluation and controlled ablations to address both concerns.
1. Held-out evaluation
We reran the complete PINNForge pipeline on all 25 PDEs, with 10 independent runs per PDE, using a 70% validation / 30% test split fixed before search. Search and model selection use the validation subset; final evaluation uses the held-out test subset.
Table R1. Validation and test MSE (mean ± standard deviation over 10 runs).
Across these six representative PDEs, validation and test mean MSE differ by less than 5%, showing consistent accuracy on points excluded from search.
All five methods are evaluated on the same 30% test subset. PINNForge achieves the lowest mean MSE on 24 of 25 tasks, with an average Friedman rank of 1.04, and outperforms our Lang-PINN reproduction on all 25 tasks. We have included the evaluation protocol in Section 4.1 and Appendix B.4, and the complete test results in Table 1 of the main text and Appendix B.6.
2. Isolating the contributions of diagnostics and memory
Building on feedback and memory mechanisms in PINNsAgent and ReEvo, PINNForge uses parent-specific physical and optimization diagnostics to guide inheritance and coordinated revision of complete PINN configurations.
To separately evaluate detailed diagnostics and posterior memory, we added three controlled settings that retain PDE priors, scalar validation feedback, and Global Top-3 configurations and rankings:
The three settings use Qwen3.8-27B, matched seeds and initial configurations, identical training and candidate-evaluation budgets, and the same validation/test split, with 10 independent complete runs per setting and PDE.
Table R2. Held-out test MSE (mean ± standard deviation over 10 runs). Full Reference reports the earlier held-out results.
Relative to Scalar Only:
Across all four PDEs, detailed diagnostics improve mean held-out accuracy beyond scalar feedback even without posterior memory, while scalar-history memory provides a separate improvement. These comparisons directly separate the two information channels within PINNForge.
We have included the additional diagnostic–memory controls and complete results in Appendix B.7 of the revised manuscript. We have also clarified the technical distinction from PINNsAgent and ReEvo in the related-work discussion.