- Rating
- 4(+2)
- Confidence
- 4
- Informativeness
- 3(+1)
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes a multi-step approach to train PINNs on Parameteric ODEs. First, a Neural Operator (a DeepONet) is trained in a given domain of the parameter space, obtaining initial trunk and branch networks. Then, the trunk is frozen providing a fixed spatial base and, after selecting anchor points and calculating a polynomial interpolation of the branch from these anchors, the working domain of the branch is extended in one or more step using the polynomial extrapolation as starting base. FInally, a PINN is trained on the final parameters of interest, and the solution predicted by the NO is used to distill information about how the solution should look like. Results show improvement compared to a PINN trained on the test parameter alone, especially where there exists a trivial solution or discontinuities.
Critical strengths
- The overall paper structure is good, the pipeline is well presented in clear steps. Some additional informations can be added in the main texts, but the overall contribution is clear.
- The results are nice and presented in a clear manner, with visualizations that are helpful to understand the results.
- The topic of knowledge distillation in PINNs and physics is an interesting topic that is not yet fully-understood. The methodology is sound to provide a good candidate solution out of distribution of the initial training region of the NO.
Critical weaknesses
- Right now, the paper does not compare to other relevant methodologies for knowledge transfer in PINNs, and meta-learning, multi-head or related approaches for parametric PDEs. For example, [1] studied the problem of transferring physical information between physics-informed models, also showing that derivatives are significant, if not fundamental, quantities. [2] is a meta-learning approach on model weights, while [3] applies a multi-head approach where new heads are added when the model is going too much OOD (with a similar spirit to step 2 of the paper). Acknowledgement and discussion or comparison with these models is suggested.
- Some of the motivation is not clear: why during step 2 is the NO fully fine-tuned, and only the branch is done? Possibly, fine-tuning the trunk as well might be helpful.
- In the Allen-Cahn experiment, the PINN is not helped at all to see the lift that brings it to the correct solution, while the model pipeline does, and thus has an advantage. It would be interesting to see how this helps the PINN. This experiment also needs more details on the different possible solutions (the bifurcation).
[1] Trenta et al. "Learning and Transferring Physical Models through Derivatives", TMLR 2026.
[2] Penwarden et al., "A metalearning approach for Physics-Informed Neural Networks (PINNs): Application to parameterized PDEs", Journal of Computational physics, 2023.
[3] Zou et al., "L-Hydra: Multi-head Physics-Informed Neural Networks", 2023.
Questions for authors
- Why does the model not fine-tune the whole NO, and what is the actual effect of having the anchors?
- How are the anchors selected? Is there a strategy or by hand with enough points?
- How does the different schedulers behave on different experiments and how to know which one is appropriate? See also the weaknesses.
Minor things:
- the plots are very interesting, especially those that put together the schedule with the loss and accuracy to see how they are related. This is done especially well with Allen Cahn where the relation is clear. I would also do this for the others.
- In the text the authors mainly talk about the accuracy metric (which is also a bit strange than the common ones in literature, but is still ok as it is based on L2. However, it is not clear what Table 2 has inside as metrics and how they are related to the accuracy. Also, the figure is too large.
3 comments
y73e (2/2)
Q2.
Response: The anchors are placed on a uniform (equally spaced) grid over the training range; no special selection strategy is needed. The grids are for Burgers, for Allen–Cahn, and Re for the cavity (Tables 5–7).
Changes: Section 3.2, fourth sentence.
Q3.
Response: The schedule is not a critical design choice in our framework. What the framework requires is that the teacher's weight vanish before training ends, so that the PDE residual governs final convergence; the two schedules are alternative instantiations of this mechanism. For example, swapping the schedule on Allen–Cahn (5 seeds) gives % with the piecewise-linear schedule used in the paper and % with the physics-adaptive schedule, both recovering the correct branch in all seeds. In practice, the piecewise-linear schedule is a simple default, while the physics-adaptive schedule avoids choosing a decay length and attenuates the teacher where it is locally inaccurate.
Changes: Section 3.4, sentence before the reference to Figure 1; Appendix B, new paragraph "Schedule swap".
Minor 1.
Response: For Burgers, the schedule is shown in a dedicated panel of Figure 2, next to the loss and accuracy histories. We have now added the distillation weight to the accuracy panel of the cavity experiment (Figure 5) in the same way as Figure 3 for Allen–Cahn.
Changes: Figure 5 updated.
Minor 2.
Response: Table 2 reports the relative error , and the accuracy used in the text is (Eq. 3). We now state this explicitly in the caption of Table 2. We have also reduced the size of the figures.
Changes: caption of Table 2
I thank the authors for their responses to my questions and concerns. Practically all of them are now resolved, and I have increased my score, as I think this is a nice work and a topic worth discussing.
I also appreciate the additional experiment on the noisy teacher. One last thing that would make the paper even stronger is the training of a PINN with some knowledge of the branch or the noisy data, but this is not really necessary for my evaluation.
y73e (1/2)
W1.
Response: We now discuss these references in Section 2. They address related but different settings. DERL (Trenta et al. (2026)) distills a trained teacher's derivatives into a student to extend a physical model to new domains or parameter ranges while preserving the teacher's knowledge; in our case the teacher is an inaccurate out-of-distribution estimate whose influence is deliberately removed during training. Penwarden et al. (2023) interpolate trained PINN weights across the parameter range to initialize new tasks, which accelerates training within the training distribution. L-HYDRA (Zou & Kardianakis (2023)) learns a shared basis with task-specific heads that are refit for new tasks, a structure related to our frozen-trunk Part 2; notably, the authors report that this approach degrades for out-of-distribution tasks, which is precisely the regime we target. This motivates the continuation prior of Part 2 and the fresh PINN of Part 3. None of these methods targets basin selection.
Changes: Section 2, paragraph "Operator learning" and paragraph "Knowledge distillation", third sentence (DERL).
W2 and Q1.
Response: We keep the trunk frozen in Part 2 for three reasons. First, cost: retraining the trunk requires differentiating the full network (up to fourth order for the cavity) at every step, whereas with a frozen trunk the basis functions and their derivatives are precomputed once, and Part 2 only optimizes the branch. Second, the role of each part: Part 1 builds the set of basis functions, and Part 2 only learns how to combine them at a new parameter. Third, the continuation prior relies on it: the anchors are coefficient vectors with respect to a fixed basis, so the polynomial fitted through them is meaningful only if the basis does not change. Since Part 3 trains a fresh PINN, the limited accuracy of a frozen basis is not a concern: Part 2 only needs to identify the correct basin.
Changes: Section 3.3, last two sentences of the first paragraph.
W3.
Response: We agree that the Part 1 operator benefits from the seed lift, and that adding the lift to the PINN would also prevent collapse. However, the lift requires knowing the branch shape in advance and does not generalize; our point is that any imperfect teacher suffices. To illustrate this, we bypass Parts 1–2 and built teachers with no lift at all, from 20 synthetic measurements of the target with noise of up to 100% of the solution peak: despite relative errors up to 1.9 (farther from the reference than u≡0), all students reach the non-trivial branch with at least 99.6% accuracy, while teachers fitted to the same noise without the solution signal lead to collapse. Regarding the bifurcation: at the target, the equation admits three solutions, the trivial state u≡0 and the pair ±u⋆ that bifurcates at λ1(H) (Appendix A.3); the vanilla PINN converges to u≡0, and the distilled student to u⋆.
Changes: Section 4.2, first paragraph (the three solutions at the target) and the two sentences before the last (noisy-data teachers); new Appendix F with Table 8.