- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper is a nice and clear benchmark showcasing that the model soup idea of Wortsman et al., 2022 generalizes to PINNs.
Critical strengths
The benchmarks are complete and thorough, I think it's pretty well-established from this that the model soup idea does indeed generalize to this case. And there are some caveats that are looked into, such as being outside of the saturating regions for many trained networks, which makes it so that it's not 100% obvious that it would work out.
Critical weaknesses
It's not anything new. It's more of a supplemental to Wortsman et al., 2022, it's just that method applied on PINNs. Well written and benchmarked throughly, but there isn't anything new here. It's basically as if an undergrad was shown Wortsman et al., 2022 and was told to now make a PINN version of the paper, and they handed that to Claude, and benchmarks came out that could have just been a section of that paper.
Questions for authors
I'm not sure there are questions. The text is rather clear.
Thank you for your review
We thank the reviewer for the positive remarks regarding the clarity of the text and the thoroughness of our benchmark.
However, we believe there is a fundamental misunderstanding regarding the core finding of our paper. The review states that our work shows that the model soup idea of Wortsman et al. (2022) "generalizes to PINNs". Our results show precisely the opposite: weight averaging systematically fails for PINNs, whereas output-space ensembles do offer modest gains. This conclusion is supported by several key empirical findings presented in Sec. 3.2 and expanded upon in Appendix D.
Regarding novelty, Wortsman et al. (2022) strictly focus on standard supervised classification, and our finding for PINNs is directly opposite to their conclusions. Beyond delivering a comprehensive benchmark with output-ensemble controls, the manuscript systematic characterizes the root cause of this failure through controlled experiments: it shows that the soup fails most on linear PDEs where output ensembles are exact by superposition, and it evaluates weight averaging under strict control mechanisms—including network width scaling, hard constraints, and symmetry alignment (accounting for odd activations).
Additionally, we could not find any reference to "saturating regions" in our manuscript. If the reviewer could clarify which section or concept they are referring to, we would be glad to address it. We also respectfully request that review feedback remain focused on the technical and empirical content of the work.
Acknowledgments. We thank again the reviewer for their time.