- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes using a model-based reinforcement learning controller to dynamically adjust PINN optimization settings, including learning rate, Adam parameters, and residual loss weights. The primary experiments investigate eight experience-collection schedules that vary the amount and timing of RL training data. Evaluation is conducted on a single holographic inverse problem involving scalar potential reconstruction. The authors report substantial variation across schedules and replicates, with some promising individual results but incomplete recovery of the target potential.
Critical strengths
The paper reports replicate variability, unsuccessful runs, and non-monotonic performance across controller-training cycles rather than focusing solely on the best outcomes. Its examination of reconstructed potential geometry, beyond scalar optimization loss, can also be valuable for assessing whether numerical improvements translate into meaningful solutions.
Critical weaknesses
1. Lack of benchmarks against existing optimization methods.
The paper compares eight experience-collection schedules for essentially the same RL controller, but provides no comparison against conventional or alternative adaptive PINN optimization methods. No baselines are included. Several important baselines should be considered such as tuned Adam/L-BFGS, adaptive loss weighting (Wang et al. (2021) [1] or ReLoBRaLo (Bischof & Kraus, 2025) [2] as an example), as well as established hyperparameter optimization approaches, including Bayesian optimization for PINNs (Escapil-Inchauspé & Ruz, 2023) [3] and dynamic methods such as Population Based Training (Jaderberg et al., 2017) [4] or PB2 (Parker-Holder et al., 2020) [5].
2. Limited novelty and inconclusive experimental contribution.
The general idea of using RL to dynamically adjust optimization parameters has already been studied in Dynamic Algorithm Configuration (Biedenkapp et al., 2020) [6], while Bihlo (2024) [7] investigates learned optimization specifically for PINNs. The paper does not clearly establish a substantial methodological advance over these directions. Its main distinguishing contribution appears to be the study of experience-collection schedules, but the eight configurations vary multiple factors simultaneously, including memory allocation, exploration, stopping criteria, and training-window lengths, without matched computational budgets. The evaluation further covers only one specialized inverse problem, with three replicates for most schedules and one for some. Consequently, the experiments neither isolate the effect of experience allocation nor establish generalizable insights into RL-based PINN optimization.
References
[1] S. Wang, Y. Teng, and P. Perdikaris. "Understanding and Mitigating Gradient Flow Pathologies in Physics-Informed Neural Networks." SIAM Journal on Scientific Computing, 43(5):A3055–A3081, 2021. doi:10.1137/20M1318043.
[2] R. Bischof and M. A. Kraus. "Multi-Objective Loss Balancing for Physics-Informed Deep Learning." Computer Methods in Applied Mechanics and Engineering, 439:117914, 2025. doi:10.1016/j.cma.2025.117914.
[3] P. Escapil-Inchauspé and G. A. Ruz. "Hyper-Parameter Tuning of Physics-Informed Neural Networks: Application to Helmholtz Problems." Neurocomputing, 561:126826, 2023. doi:10.1016/j.neucom.2023.126826.
[4] M. Jaderberg et al. "Population Based Training of Neural Networks." arXiv:1711.09846, 2017.
[5] J. Parker-Holder, V. Nguyen, and S. J. Roberts. "Provably Efficient Online Hyperparameter Optimization with Population-Based Bandits." Advances in Neural Information Processing Systems 33, pp. 17200–17211, 2020.
[6] A. Biedenkapp, H. F. Bozkurt, T. Eimer, F. Hutter, and M. Lindauer. "Dynamic Algorithm Configuration: Foundation of a New Meta-Algorithmic Framework." Proceedings of ECAI 2020, pp. 427–434, 2020. doi:10.3233/FAIA200122.
[7] A. Bihlo. "Improving Physics-Informed Neural Networks with Meta-Learned Optimization." Journal of Machine Learning Research, 25(14):1–26, 2024.
Questions for authors
-
Can the authors provide comparisons against conventional PINN optimization, adaptive loss weighting, and non-RL hyperparameter optimization approaches? How much benefit does RL provide relative to its additional computational cost?
-
What is the specific methodological contribution beyond existing dynamic algorithm configuration and learned PINN optimization?
Response to reviewer aMkY
We appreciate the reviewer’s concrete suggestions and the positive assessment of our geometry-based evaluation and reporting of unsuccessful runs. We agree that tuned conventional optimizers, adaptive loss weighting, and non-RL hyperparameter-optimization methods are relevant comparisons, and that the current single-problem study cannot establish generality.
The revised paper adds a fixed-Adam vanilla PINN sweep in Table 3 and measured elapsed times in Table 4. Vanilla PINN mean best potential RMSEs are 1.83 at 1M epochs (187.6 min), 1.85 at 3M (492.5 min), and 1.86 at 10M (1532.0 min). Selected RL results include Schedule 7 at mean RMSE 1.49 and 188.9 min, and Schedules 1–3 at means 1.38–1.56 and 354.8–398.4 min. RL timings include controller training and experience collection; collection uses eight parallel workers. Since runs are unpaired and the compute setups differ, these tables provide useful context but do not establish equal-compute superiority.
We recognize the connection to learned optimizer control and algorithm configuration. The focus here is a sequential controller acting on the evolving PINN training process, with experience-collection schedules evaluated for one holographic inverse problem; it is not a claim of a general-purpose advance over dynamic algorithm configuration or learned optimization. We have not added direct experiments against L-BFGS, adaptive weighting, Bayesian optimization, PBT, or PB2, and now provide the vanilla Adam reference while keeping those broader comparisons as future work. The paper’s conclusions remain limited to a proof of concept on this equation.