acceptodds
Under review as a conference paper at ICLR 2027

Rich Representations, Small Heads: Fast PDE Solves and Flows in Weight Space

Abstract

The study of embeddings of partial differential equations (PDEs) using Physics-Informed Neural Networks (PINNs) involves learning a latent space basis that represents a family of solutions. For linear PDEs, a sufficiently expressive learned basis yields approximate solutions for unseen equation parameters (EPs) or initial conditions (ICs) through a direct solve for the output-head weights, without retraining the shared network. This procedure is referred to as one-shot transfer learning (OTL). For nonlinear equations, however, a direct closed-form solution for the head weights is generally unavailable, and existing extensions of OTL instead use approaches that rely on linearizing the nonlinearity, such as perturbative decompositions or Chebyshev surrogates. In this work, we investigate second-order methods that adapt the head weights to the nonlinear problem directly while keeping the learned basis fixed. We train multi-head PINNs on families of linear and nonlinear PDEs with varying EPs and ICs, learning a shared basis for each equation with a separate linear head for each training instance. This low-dimensional learned basis restricts adaptation to a small space of solution coefficients, making second-order optimization tractable. With the learned basis fixed, we investigate two second-order methods for obtaining new solutions at unseen EPs or ICs. The first applies Gauss–Newton to the least-squares problem for the head weights. The second uses continuation in head-weight space, for which we derive an analytic expression, and returns the path of heads between a trained instance and the target. We compare both methods against training a new head in terms of accuracy and computational cost. On the heat, wave, Allen–Cahn and Burgers equations, both methods improve the accuracy of a trained head for the same computational cost, improving the efficiency of transfer learning in PINNs.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
PINNs, Transfer Learning, Multi-Head Architecture, Partial Differential Equations

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_NF9VOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper builds upon multihead architectures for PINNs in parametric PDEs and different initial conditions. First, the basis and heads are trained together on a set of instances of the parameters (or ICs). Then, the basis is frozen. Unlike other works, this paper adopts two second-order optimization procedures for the head on new instances, which is especially useful for nonlinear PDEs. With Hessian continuation, the authors are also able to obtain paths in the head space based on the parameters. The results are very good compared to first-order optimization based on Adam.

Critical strengths
  • The paper is well written and easy to follow, with a clear presentation of the problem. The main text reports almost all relevant information and steps to understand the methodology and the results. Overall, the paper is in very good shape.
  • The ideas and methodology are sound, and the different choices are justified.
  • The results are good, and the visualizations are also very helpful. The paper also reports some useful ablations.
  • The idea of using a VAE for the different IC is very nice
Critical weaknesses
  • I have some concerns only about some choices and possible ablations required, mainly regarding baselines and other details.
Questions for authors
  • What is the motivation behind the choice of dz=2d_z=2 for the VAE? I understand this dimension shouldn't be too high, but the specific choice might depend heavily on the problem.
  • The initial condition is calculated with a sum of sine functions. This only works for problems with null Boundary conditions. What can be done in other cases? Are there experiments for this?
  • I am unsure about the choice of using an orthonormal basis. I understand some of the motivation, but not everything, and not the concrete advantages. Could the authors provide more comments on why it is useful and what happens if you do not use it? Also, an ablation without this constraint could be nice.
  • Related to the previous point: on lines 255-256, the phrase "With varying ICs, Aj changes too and head distances measure the part of the solution carried by the head only." is a bit unclear. Please provide more details.
  • The choice of Adam as a baseline is fair, but I am also interested in seeing what happens with L-BFGS, as it has clear advantages on a lot of PINN-related problems.
  • On line 312: J+F=0  ⟺  J⊤F=0J^+F=0 \iff J^\top F=0 is not clear to me.
  • I would report some insights from the results on Adam without the same wall-clock time (Appendix B6 and Table 3) in the main text, at least to understand the full comparison.
  • In Figure 3, the known heads are already very dense, so interpolation might be easy. What happens if there is a lot of space in between or if we extrapolate far away?

On a final note, Images are very small.

In general, the paper is sound and already in good shape, and I lean towards acceptance with possible increase in the score

4 comments
AuthorsOpenReview ↗

Latent dimension of the VAE and boundary conditions

Motivation behind the choice of the latent dimension of the VAE.

The motivation is merely for visualization purposes, in the sense that one can represent/visualize the surface w(z)\mathbf w(\mathbf z) presented in Figs. 4 and 8 (Section 4.1.2, footnote 1). It is true that it should not be too high because one is looking for a low-dimensional parametrization of the ICs (a profile sampled on the spatial grid lives in Rnx\mathbb{R}^{n_x}, which becomes far too large for this purpose), but it can definitely be higher. More concretely, for a specified family of possible ICs (e.g. ones that can be produced from a finite set of KK Fourier modes as in Eq. 8), a larger dimension dzd_z of the encoding space Rdz\mathbb{R}^{d_z} translates in a lower loss (better training) of the VAE. This can improve accuracy of the TL results if a new instance of IC is sampled directly in Rnx\mathbb{R}^{n_x} and encoded into z⋆=E(IC)\mathbf z^\star=\mathcal{E}(\mathrm{IC}), to later be decoded as IC~=D(z⋆)\widetilde{\mathrm{IC}}=\mathcal{D}(\mathbf z^\star). In other words, higher dzd_z is expected to produce higher agreement between IC and IC~\widetilde{\mathrm{IC}}, and therefore a more accurate answer for the continuation from the learned latent space. In the current work and for our purposes, we sample z⋆\mathbf z^\star directly in the encoding space Rdz\mathbb{R}^{d_z} and generate new ICs from the trained decoder D(z⋆)\mathcal{D}(\mathbf z^\star), so no real effect comes from the choice dz=2d_z=2 besides the visualization of Figs. 4 and 8 and the interactive visualizations in the repository Weight_Space_Flows (https://anonymous.4open.science/r/Weight_Space_Flows).

The initial condition is a sum of sine functions which only works with null boundary conditions.

Indeed, this is a choice made for simplicity and to have a controlled experimental workspace, where we reparametrize the output of the NNs to satisfy exactly the IC and BCs. However, this reparametrization (Eq. 6, Section 4.1) is general for any time-independent boundary value problem (BVP) with IC profile A(x,0)=u0(x)A(x,0)=u_0(x), not only for the cases u0(xlo)=u0(xhi)=0u_0(x_{\rm lo})=u_0(x_{\rm hi})=0, so it is directly applicable in the general time-independent BVP; one would only have to choose a family of ICs that does not vanish at the boundary, and A(xlo,t)=uloA(x_{\rm lo},t)=u_{\rm lo}, A(xhi,t)=uhiA(x_{\rm hi},t)=u_{\rm hi}. However, it does not cover time-dependent boundary data as it is structured now.

Concretely, one can see in Eq. 6 that, since 1−e−tp1-e^{-t^{p}} vanishes at t=0t=0 and x~(1−x~)\tilde x(1-\tilde x) vanishes at xlox_{\rm lo} and xhix_{\rm hi}, the envelope gg is zero at t=0t=0, x=xlox=x_{\rm lo} and x=xhix=x_{\rm hi}, so uj(x,0)=Aj(x,0)=u0,j(x)u_j(x,0)=A_j(x,0)=u_{0,j}(x) and uj(xlo,t)=Aj(xlo,t)u_j(x_{\rm lo},t)=A_j(x_{\rm lo},t), uj(xhi,t)=Aj(xhi,t)u_j(x_{\rm hi},t)=A_j(x_{\rm hi},t) hold identically, whatever the head wj\mathbf w_j. For p=2p=2 the time factor additionally satisfies ∂t(1−e−t2)∣t=0=0\partial_t\big(1-e^{-t^{2}}\big)\big\rvert_{t=0}=0, hence gt(x,0)=0g_t(x,0)=0 and ∂tuj(x,0)=∂tAj(x,0)=0\partial_t u_j(x,0)=\partial_t A_j(x,0)=0 is exact as well, while ∂tt(1−e−t2)∣t=0=2≠0\partial_{tt}\big(1-e^{-t^{2}}\big)\big\rvert_{t=0}=2\neq 0 leaves ∂ttuj(x,0)\partial_{tt}u_j(x,0) unconstrained, as required by the wave equation. The constraints are thus met by construction and never enter the loss.

AuthorsOpenReview ↗

Orthonormal basis and its ablation

Why an orthonormal basis, and what happens without it?

The motivation is twofold: it fixes the head of each instance uniquely, and it makes the weights readable as coordinates (see the next point). We ran an ablation on the four multi-IC models: no penalty (λ⊥=0\lambda_\perp=0), the value of the paper (λ⊥=0.02\lambda_\perp=0.02) and a stronger one (λ⊥=2\lambda_\perp=2), with three training seeds for the first two. It is reported in the new Appendix H.1 (Fig. 12). Between λ⊥=0\lambda_\perp=0 and 0.020.02 neither the accuracy of the trained heads nor that of the transfer (GN, Hessian continuation) changes beyond the seed-to-seed spread, and the wall-clock time is the same at every λ⊥\lambda_\perp. This is expected: both second-order methods depend only on the span of the basis. At λ⊥=2\lambda_\perp=2 the trained-head error increases by 30–40% on Burgers and Allen–Cahn, so the penalty is not free.

What changes is the basis itself. Without the penalty, ∥ujNN∥/∥wj∥\lVert u_j^{\rm NN}\rVert/\lVert\mathbf w_j\rVert (which should be 1 for a perfectly orthonormal basis) ranges from 0.50.5 to 99 over the heads. However, at λ⊥=0.02\lambda_\perp=0.02 it is within 7% of 1 for heat and between 0.350.35 and 1.561.56 for the other equations. At λ⊥=2\lambda_\perp=2 it is within 2% of 1 for all four (Fig. 12b). The second effect is uniqueness (Fig. 12a). Let vi\mathbf v_i be the right singular vectors of J\mathbf J at a held-out head and δi=∣vi⊤(w−wval)∣/si\delta_i=\lvert\mathbf v_i^\top(\mathbf w-\mathbf w_{\rm val})\rvert/s_i, with sis_i the spread of that coordinate over the trained heads. For λ⊥=0\lambda_\perp=0, the transferred head agrees with the jointly trained one (δi<10−2\delta_i < 10^{-2}) only in the first ∼10\sim 10 (heat) and ∼15\sim 15 (wave) of the 48 directions; δi\delta_i then grows as σi\sigma_i falls and reaches order one in about 30% of the directions, while the relative L2L^2 error stays at or below that of the trained head. Those coordinates are not determined by the residual, and the value of the corresponding components of the weight vector is degenerate. With increasing λ⊥=0→0.02→2\lambda_\perp=0\to 0.02\to 2 the number of determined directions (σi>10−2σ1\sigma_i > 10^{-2}\sigma_1) grows 17→41→4817 \to 41 \to 48 for heat and 26→30→4226 \to 30 \to 42 for wave. Burgers and Allen–Cahn have no undetermined directions at this basis size, with or without the penalty.

The sentence on lines 255–256 ("With varying ICs, AjA_j changes too and head distances measure the part of the solution carried by the head only").

We agree that the sentence is unclear and have rephrased it (Section 4.1.3). With a fixed IC and different solutions coming from variations of the EP, Ai=AjA_i=A_j and

ui−uj=g (wi−wj)⊤Hθ,u_i-u_j = g\ (\mathbf w_i - \mathbf w_j)^\top \mathbf H_\theta ,

so the difference between two solutions is linear in the difference of their heads, and for an orthonormal basis ∥wi−wj∥\lVert\mathbf w_i-\mathbf w_j\rVert is the distance between the network outputs uiNNu_i^{\rm NN} and ujNNu_j^{\rm NN} (Eq. 11), later weighted by the envelope gg. In the four multi-EP models, the rank correlation of the head distance ∥wi−wj∥\lVert\mathbf w_i-\mathbf w_j\rVert with the distance between network outputs ∥uiNN−ujNN∥\lVert u_i^{\rm NN}-u_j^{\rm NN}\rVert is 1.001.00, and with the distance between solutions ∥ui−uj∥\lVert u_i-u_j\rVert (after reweighting) it is 0.990.99 (heat), 0.710.71 (wave), 0.900.90 (Burgers) and 0.900.90 (Allen–Cahn). Without the orthonormality penalty these drop to 0.670.67, 0.470.47, −0.01-0.01, 0.640.64 for ∥uiNN−ujNN∥\lVert u_i^{\rm NN}-u_j^{\rm NN}\rVert, and 0.600.60, 0.320.32, 0.100.10, 0.310.31 for ∥ui−uj∥\lVert u_i-u_j\rVert.

With varying ICs, Ai≠AjA_i\neq A_j and

ui−uj=Ai−Aj+g (wi−wj)⊤Hθ.u_i-u_j = A_i-A_j + g\ (\mathbf w_i - \mathbf w_j)^\top \mathbf H_\theta .

The heads then describe only how each solution evolves away from its own initial profile,

(ui−Ai)−(uj−Aj)=g (wi−wj)⊤Hθ,(u_i-A_i)-(u_j-A_j) = g\ (\mathbf w_i - \mathbf w_j)^\top \mathbf H_\theta ,

and two solutions can be far apart while their heads are close. In the four multi-IC models the rank correlation of the head distance is 0.910.91 to 1.001.00 with the distance between network outputs, 0.560.56 to 0.750.75 with the distance between the evolved parts uj−Aju_j-A_j, and 00 to 0.560.56 with the distance between the full solutions, which is dominated by Ai−AjA_i-A_j (rank correlation 0.670.67 to 1.01.0). These numbers are also reported in Appendix H.1.

AuthorsOpenReview ↗

L-BFGS, the condition on line 312, Adam without a time budget

L-BFGS as a baseline.

We thank the referee for this point. We have run transfer learning tests with second order optimization methods as well, mainly L-BFGS and Levenberg–Marquardt optimizers. The multi-head training of the basis with L-BFGS had been tested prior to the initial submission, but was not used moving forward because it led to overfitting of the basis to the training solutions. Also, the optimization times we are interested in are in the TL phase (which is the one that is repeated many times when looking for new solutions), in contrast to the MH training phase which is run only once per model. The comparison for the eight models of Table 1, on CPU, is reported in the new Appendix E.1 (Table 7) and summarised in Section 5.1. The table below gives the same comparison for the multi-IC models with every computation on a GPU.

Head solvers for TL to unseen ξ⋆\xi^\star in the multi-IC setting (15 targets per model, as in Table 1), all on the same frozen basis, collocation system and start head, with every computation on a GPU (NVIDIA A30): median relative L2L^2 error against the reference solution and median wall time (online) per transfer. Levenberg–Marquardt (LM) and L-BFGS stop with the rule of Gauss–Newton (GN). The Adam baseline is shown twice: run for the wall-clock time of the slower second-order method (the Hessian continuation), and run to a fixed budget of 2000 steps.

PDEGN errorGN msLM errorLM msL-BFGS errorL-BFGS msHessian + RK4 errorHessian + RK4 msAdam error (wall-clock of Hessian)Adam error (2000 steps)Adam ms (2000 steps)
heat1.6×10−31.6\times10^{-3}6.91.6×10−31.6\times10^{-3}141.6×10−31.6\times10^{-3}2021.6×10−31.6\times10^{-3}881.9×10−21.9\times10^{-2}1.8×10−31.8\times10^{-3}1647
wave7.2×10−37.2\times10^{-3}6.17.2×10−37.2\times10^{-3}147.2×10−37.2\times10^{-3}2997.2×10−37.2\times10^{-3}844.2×10−14.2\times10^{-1}4.5×10−24.5\times10^{-2}1856
Allen–Cahn2.1×10−22.1\times10^{-2}252.1×10−22.1\times10^{-2}202.1×10−22.1\times10^{-2}1645.2×10−25.2\times10^{-2}1002.8×10−12.8\times10^{-1}8.4×10−28.4\times10^{-2}1861
Burgers7.4×10−27.4\times10^{-2}207.4×10−27.4\times10^{-2}107.4×10−27.4\times10^{-2}557.4×10−27.4\times10^{-2}983.8×10−13.8\times10^{-1}1.0×10−11.0\times10^{-1}1797

On line 312, J+F=0⇔J⊤F=0\mathbf J^{+}\mathbf F=\mathbf 0 \Leftrightarrow \mathbf J^\top\mathbf F=\mathbf 0 is not clear.

We thank the reviewer for pointing this out, the statement was given without explanation. The quantity J⊤F\mathbf J^\top\mathbf F is the gradient of the loss, ∇wΦ=J⊤F\nabla_{\mathbf w}\Phi=\mathbf J^\top\mathbf F, which is Eq. (16) written in matrix form. The Gauss–Newton step of Eq. (15) is this same gradient multiplied by a matrix: since N≫BN \gg B and the columns of J\mathbf J are linearly independent, the pseudo-inverse is J+=(J⊤J)−1J⊤\mathbf J^{+}=(\mathbf J^\top\mathbf J)^{-1}\mathbf J^\top, so the step is −J+F=−(J⊤J)−1∇wΦ-\mathbf J^{+}\mathbf F=-(\mathbf J^\top\mathbf J)^{-1}\nabla_{\mathbf w}\Phi. The matrix (J⊤J)−1(\mathbf J^\top\mathbf J)^{-1} is invertible, so the step is zero exactly when the gradient is zero. In other words, Gauss–Newton stops moving only at a stationary point of the loss, which is the condition that the Hessian continuation follows along its path. The two methods therefore look for the same kind of point and differ only in how they get there: Gauss–Newton iterates directly at the target ξ⋆\xi^\star, while the continuation starts from a trained head and follows the minimizer as ξ\xi changes. This does not mean that they always end at the same point; the cases where they do not are reported in Appendix B.4. The SVD cutoff used in the Gauss–Newton solve does not change this, since no singular value is discarded in our experiments. We have rewritten that sentence to include this explanation (Section 4.2.3, below Eq. 16).

Reporting in the main text the results on Adam without the same wall-clock time (Appendix B.6 and Table 3).

We have added two columns to Table 1 with Adam run without a time budget, stopped at the iteration where its residual has improved by less than 1% over 200 steps, and a summary in Section 5.1: run to convergence, Adam matches the second-order errors in the multi-EP experiments and for the heat and wave ICs, but takes roughly 15 to 390 times longer than GN, and it remains less accurate for the Allen–Cahn and Burgers ICs. The details remain in Appendix B.6 (Table 3).

AuthorsOpenReview ↗

Figure 3, sparser trained heads and extrapolation

In Figure 3 the known heads are already very dense. What happens with a lot of space in between, or far extrapolation?

We apologize if Figure 3 may have given the wrong impression. The continuation curve is not an interpolation of the trained heads, but is an implementation of the Hessian continuation method. As described in Section 4.2.3, starting from the head at the smallest ν\nu the method first applies a Gauss–Newton solve that places this head on the minimiser of the PDE residual. It then integrates Eq. 20 with RK4 up to the largest ν\nu. Each integration step evaluates the slope dw/dνd\mathbf w/d\nu from the PDE residual at the current ν\nu, such that the other 15 trained heads are never used and are drawn only as a check. This is now stated in the caption of Figure 3 and in the new Appendix G.

We had also tested the accuracy of this procedure with respect to the starting distance from the target without reporting it. From a head up to 7.5 grid spacings away from an unseen ν\nu, Gauss–Newton and the continuation reach the same error as from the nearest head for the four equations. Adam with a fixed budget does not (wave: 4.4×10−24.4\times10^{-2} at 0.5 spacings, 0.710.71 at 7.5) (Table 13).

To address sparser training directly, we retrained the four models of Figure 3 with 12, 8 and 4 heads instead of 16, keeping the eight basis functions, and evaluated them at the same 15 unseen ν\nu (Table 14). With 8 heads the trained ν\nu are twice as far apart. The median error is 5.5×10−25.5\times10^{-2} against 3.7×10−23.7\times10^{-2} for Burgers and unchanged for the other three equations, and the continuation started from the smallest ν\nu alone still reproduces the Gauss–Newton heads over the whole range (Fig. 11a). With 4 heads, fewer than the basis size, heat and Allen–Cahn remain accurate (6.0×10−36.0\times10^{-3} and 6.9×10−36.9\times10^{-3}). The wave basis however no longer represents unseen ν\nu (0.620.62) and Burgers degrades (0.140.14) (Fig. 11b). This suggests using at least as many training heads as basis functions.

Extrapolation is reported in Appendix B.9 (Table 6), which we should have pointed to from the main text; Section 5.1 now does. For varying ν\nu, Gauss–Newton and the continuation return the same head and the error grows with the distance from the trained range. It exceeds 10% one grid spacing out for all equations but heat. We have now separated the two ends of the range and computed the best fit that the basis allows (Table 15). Above the largest trained ν\nu the error grows as a power of the distance (exponent 0.9 to 1.5) and stays within about a factor of two of that best fit, except for wave. Below the smallest trained ν\nu the degradation is mild for heat and Allen–Cahn (at most 6×10−26\times10^{-2} at eight spacings) and stronger for Burgers. The limitation is therefore the frozen basis and not the transfer method. For varying ICs, Gauss–Newton keeps its interpolation accuracy outside the training box for all equations except Burgers.

The new Appendix G reports the start-distance and sparse-head tests and repeats Figure 3 for the models trained with 8 and 4 heads (Tables 13–15, Fig. 11).

Reviewer_oej3OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a fast transfer learning method that uses a multi-head PINN to learn a shared basis representing the solution set of a PDE and optimizes only the weights of the output layer for unknown PDE parameters and initial conditions. Specifically, it introduces fast optimization using the Gauss–Newton method and the tracking of optimal solutions in the weight space via Hessian continuation. Experiments using four types of PDEs demonstrate improvements in computational efficiency and accuracy compared to optimization using the Adam optimizer.

Critical strengths
  • By fixing a shared basis, it is reasonable that low-dimensional quadratic optimization can be applied even to nonlinear PDEs.
  • Using Hessian continuation, the authors can compute the trajectories and sensitivities of the optimal output layer weights as PDE parameters and initial conditions vary.
  • The authors conducted experiments on both linear and nonlinear PDEs and demonstrated improvements in computational efficiency and accuracy compared to Adam.
Critical weaknesses
  • The work does not sufficiently differentiate itself from related research. The technical differences from existing methods, such as the meta-learning approaches for PINNs [R1, R2, R3], that use learned basis functions to rapidly solve unknown PDEs are not clearly articulated.
  • The comparison methods are limited. The experiments primarily use Adam as the baseline, and there are no quantitative comparisons with relevant PDE solvers or meta-learning methods.
  • The usefulness of Hessian continuation has not been sufficiently demonstrated. Although the trajectories in the weight space are visualized, it is unclear what useful insights regarding the solutions to the PDEs can be gained from this. Furthermore, despite having a higher computational cost than GN, the solution accuracy is comparable in many experiments.

[R1] Huang, X, et al., Meta-Auto-Decoder for Solving Parametric Partial Differential Equations, Advances in Neural Information Processing Systems, volume 35, 23426–23438, 2022.

[R2] Iwata, T, et al., Meta-learning of Physics-informed Neural Networks for Efficiently Solving Newly Given PDEs, arXiv:2310.13270, 2023.

[R3] Fan, Y, et al., Newton-PINet: A fast physics-informed neural network with Newton linearization for meta-learning nonlinear PDEs, 2026.

Questions for authors
  • How do the performance of the Gauss–Newton method and Hessian continuation change when the order of differentiation increases or nonlinearity becomes stronger?
  • How do the trajectories and sensitivities in the weight space obtained via Hessian continuation aid in understanding and analyzing solutions to PDEs? Can you provide concrete examples of applications that go beyond mere visualization? For example, is it possible to analyze how the order of differentiation and the strength of nonlinearity in a PDE affect the accuracy, computational efficiency, and stability of the proposed method?
  • If the number of shared bases is increased, what trade-off arises between the accuracy of the solution and the computational cost of the quadratic optimization?
3 comments
AuthorsOpenReview ↗

Response to Reviewer (2/3): order of differentiation, nonlinearity, and uses of the continuation

Question 1 (order of differentiation and strength of the nonlinearity).

First, for a linear PDE of any order the residual is linear in the head, so GN is a single least-squares solve and the continuation is exact: heat (first order in time) and wave (second order) take the same 4 evaluations at every ν\nu (Table 4). Second, the order of differentiation enters through the conditioning of J\mathbf J, not through the accuracy: in the multi-IC models the condition number is about 10310^3 for wave against 4×1024\times10^{2} for heat (Table 10 reports the condition number of HwwH_{ww}, which is the square of these values), with no effect on the number of evaluations. Third, non-linearity costs iterations and robustness: Burgers and Allen–Cahn need 12 to 13 evaluations on average (9 to 22 depending on the target), with 2 to 4 more at the smallest viscosities (Appendix B.4, Table 4), and in the multi-IC setting about 3 of 15 unseen instances converge to a wrong stationary point (Appendix B.1). We have not tested equations of order higher than two. We leave this point for future work.

Question 2 (uses of the trajectories and sensitivities beyond visualisation).

We thank the referee for this question, which led to two new appendices (Appendices F.1 and F.2). For Appendix F.2 we repeated the continuation of Fig. 4 for 1,100 new initial conditions per equation, recording the head w\mathbf w, its derivative ∂w/∂z\partial \mathbf w/\partial \mathbf z and the spectrum of the Hessian HwwH_{ww} along every trajectory (Table 10). Five uses go beyond visualisation.

  1. Paths between instances (Appendix F.1, Table 9). When the whole path w(ξ)\mathbf w(\xi) between a known solution and a new one is needed, and not only its endpoint, the continuation returns at each node both the head and its slope ∂w/∂ξ\partial \mathbf w/\partial \xi, so the path can be interpolated with cubic Hermite polynomials. Warm-started Gauss–Newton returns only the heads, which we interpolate with a cubic spline. On the paths to the 15 targets of Table 1, with a single interval the error of the continuation is 20–1200$\times$ smaller than that of Gauss–Newton in the multi-EP setting and 1.3–12$\times$ smaller in the multi-IC one. At matched accuracy (one tenth of the model's own error at the target) it needs half the intervals on five of the eight models and is 1.4–3.2$\times$ faster on seven, with a tie on the eighth. The gain comes from the slopes, which are also the sensitivity of the solution to the EP or IC and which Gauss–Newton does not return. Without interpolation, at the nodes alone, a warm-started Gauss–Newton chain is about as fast as the continuation.
  2. A solution map (Appendix F.2.2). With enough trajectories the surface w(z)\mathbf w(\mathbf z) can be interpolated, and the solution for any initial condition in the sampled region is then read off it without a new solve. With 1,100 trajectories the read-off error is 0.8–3.6% for heat, wave and Allen–Cahn, within a factor of four of running the continuation, and 13% for Burgers, where the continuation itself is at 11%. With the 100 trajectories of Fig. 4 it is 9–35%, so the surface in that figure is a visual guide between its marked points (Table 11).
  3. Sensitivities (Appendix F.2.3). The continuation integrates ∂w/∂z\partial \mathbf w/\partial \mathbf z, so one extra linear solve gives the sensitivity of the solution to the initial condition, ∂u/∂z\partial u/\partial \mathbf z (Eq. 22). It agrees with finite differences of the reference solver to about 1% for the linear equations. For the nonlinear ones it gets the magnitude right (rank correlation 0.94 and 0.92) but the fields differ by 39% and 17%, less where the solution itself is accurate (Fig. 9).
  4. Stability (Appendix F.2.4). For the linear equations HwwH_{ww} is positive definite everywhere and no transfer fails. For the nonlinear ones it becomes indefinite at a small fraction of the sampled points, and trajectories crossing them fail in 85% (Burgers) and 50% (Allen–Cahn) of cases, against 8% and 16% otherwise (Table 10, rows "Spectrum of HwwH_{ww}").
  5. Reliability (Appendix F.2.5). The PDE residual at the end of the trajectory flags failed transfers without a reference solution (AUC 0.90 and 0.93; Table 12, Fig. 10).

On cost, the continuation takes a fixed number of linear solves whatever the equation, while Gauss–Newton needs about three times more residual evaluations for the nonlinear equations than for the linear ones (Table 4). We also state two limitations (Appendix F.2.6): nothing we measured predicts a failure before the trajectory is computed (Table 12), and our four equations do not isolate the effect of the order of differentiation. Both appendices are referenced from Section 5.2.

AuthorsOpenReview ↗

Response to question 3: number of basis functions, accuracy and cost

Question 3 (trade-off between accuracy and cost when the number of shared basis functions increases).

We thank the referee for this suggestion. We trained 16 new models, two more basis sizes for each of the eight models of the draft: B=4B=4 and 1616 around the multi-EP B=8B=8, and B=24B=24 and 9696 around the multi-IC B=48B=48. Each uses the same recipe, data, training range and collocation as the draft's model, with only BB changed. Table 16 (new Appendix H.2) reports, for each size, the transfer and extrapolation accuracy of GN, the per-target assembly and solve times, the one-off body evaluation and the memory. All timings are on one CPU thread, on a different machine from the draft.

A larger basis improves the transfer where the basis limits it. Doubling the draft's size lowers the GN error by roughly 35–45% for wave and Burgers in the multi-EP setting and for Allen–Cahn and Burgers in the multi-IC setting, while heat barely changes. Halving the basis is too small for some models, for example wave in the multi-EP setting.

Extrapolation also improves with BB, but a larger basis does not remove the degradation. In the multi-IC setting, where the targets lie just outside the trained latents, the error falls clearly with BB for Allen–Cahn and Burgers. In the multi-EP setting, where the targets reach up to 8 anchor spacings beyond the trained range, the error falls for wave and Burgers but stays large, and it grows slightly for Allen–Cahn. The best fit of the frozen basis to the reference is also large there (new Appendix E.2, Table 8). Far from the training range, the limit is what the basis has seen in training more than its size.

The adaptation stays inexpensive. Doubling BB multiplies the per-target time by 1.3–2.5, and at the largest size a transfer still takes at most 11 ms in the multi-EP setting and 177 ms in the multi-IC one. The only other cost is a one-off forward pass of the frozen body on the collocation points, which takes seconds, is shared by all targets and stores at most 37 MB. At a fixed basis size, time and memory grow about linearly with the number of collocation points, while the GN error changes by less than 5%. The accuracy is therefore set by the basis size, not by the collocation.

AuthorsOpenReview ↗

Response to Reviewer (1/3): relation to meta-learning methods

Weakness 1 (differences from the meta-learning approaches R1–R3).

We thank the reviewer for pointing out these references. We agree that the submission did not position our work against meta-learning clearly enough. Although our multi-head training can itself be read as meta-learning in the broad sense, the difference lies in what is adapted for a new instance. In R1, it is a latent vector fed to the network as an input. The solution is nonlinear in it, so adaptation is gradient descent even for a linear PDE, and the more accurate variant also updates the network. Iwata et al. in R2 predict the solution in a forward pass from an encoding of the problem. Their model covers different governing equations, which is broader than our setting, but the residual of the new problem is not minimized unless one fine-tunes. In our case the solution is linear in the head, so the residual is polynomial in 8 to 48 unknowns with exact Jacobian and Hessian. This gives the Gauss–Newton solve and the Hessian continuation, which returns the derivative of the optimal head with respect to the instance label and a path of solutions between instances. None of the three references provides this. This comparison with R1 and R2 is now part of Section 2 (Related Work) (lines 118-123).

With regards to R3, we would like to point out that the referred work was just recently accepted at NeurIPS 2026 after our submission, and as far as we can tell only the abstract and GitHub repository of this work are publicly available at the moment. Due to the timeline, and following the ICLR 2027 guidelines on contemporaneous work, being this defined as work which has been published within the last 4 months, we believe it is fair to indeed regard it as contemporaneous work. Moreover, due to the aforementioned lack of public manuscript available at the moment, we do not think we can fairly or accurately assess the similarities and differences between the two works.

However, we would like to note that we do not claim novelty with regards to the Gauss–Newton step, as it already appears in published work cited in Appendix A (Dong & Li 2021, Huang & Wu 2025). We have added a clear indication of this fact in the revised version of the manuscript (Section 4.2.2, lines 306-307). What we add in this work is the Hessian continuation method with its sensitivities and the orthonormal head coordinates, in which distances between heads are L2L^2 distances between network outputs.

Reviewer_wfyoOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

Model reuse is an important research direction for PINNs in multi-query settings. Several approaches represent PDE solutions as linear combinations of basis functions shared across instances. This paper uses a multi-head architecture in which a shared backbone learns the basis functions jointly with instance-specific linear heads. At transfer time, the backbone is frozen, and a new head is obtained by minimizing the PDE residual. The authors investigate two optimization methods to solve for the heads: Gauss–Newton and Hessian continuation. Gauss–Newton solves for the target head, while Hessian continuation tracks a path of stationary heads as the equation parameters or initial conditions vary, also providing their sensitivities. An orthonormality penalty encourages the basis functions to be approximately orthonormal. The methods are compared against Adam in accuracy and computational cost on the Burgers, Allen–Cahn, heat, and wave equations in one spatial dimension plus time, with varying equation parameters or initial conditions.

Critical strengths

1. Efficient adaptation within a reusable learned representation. ​ Restricting adaptation to the linear head makes second-order optimization computationally practical. The same Gauss–Newton formulation handles linear and nonlinear PDEs, recovering OTL exactly in the linear case while optimizing the original nonlinear residual, without perturbative assumptions, in the nonlinear case. The experiments demonstrate millisecond-scale online head solves on a CPU. Under the longer-budget Adam schedules tested, GN retains lower median solution errors on the nonlinear multi-IC tasks, while the two solvers achieve comparable median accuracy on heat-IC and wave-IC (Table 3). This makes the approach a practical tool for many-query settings within the range covered by the trained basis.

2. Transparent reporting of the transfer procedure. ​The solver settings, numerical precision, timing protocol and baseline configuration are documented clearly (App. B). The appendix includes informative supplementary analyses (longer-budget Adam runs, cold-start Gauss–Newton, recovery of trained heads, extrapolation) and openly discloses failed targets and cases where the continuation performs worse. This makes the claims verifiable and their limitations visible.

Critical weaknesses

1. The added value of the proposed methodology is not sufficiently established. The framework combines established Gauss–Newton and implicit-function-based continuation methods, so its contribution depends on demonstrating a useful advantage from their integration. The experiments establish gains over the selected Adam baseline but do not establish an advantage over strong alternative head solvers or the nonlinear OTL methods used to motivate the work. Moreover, continuation—the component providing paths and sensitivities—is 2.9–6.4× slower than GN without improving the reported median accuracy, and no quantitative application demonstrates the value of its additional outputs. Consequently, the evidence does not establish a sufficiently compelling methodological or practical advance beyond applying established solvers to a small coefficient problem.

2. The central trade-off between representation capacity and efficient adaptation is not characterized. The approach makes second-order optimization affordable by restricting adaptation to a small frozen basis. The authors acknowledge this limitation and attribute extrapolation degradation to the basis being trained only within the original parameter range (Appendix B.9). However, they do not test this explanation through a controlled basis-size ablation or quantify the best achievable approximation error within the frozen representation. Increasing the basis size while measuring both transfer accuracy and computational cost would show whether greater capacity alleviates the degradation and whether the efficiency advantage persists. Without this analysis, the paper does not establish when a small head provides both reliable transfer and inexpensive adaptation.

Questions for authors

1. Advantage over alternative methods (Weakness 1). Does the accuracy–cost advantage persist against a strong nonlinear least-squares solver, such as Levenberg–Marquardt and/or L-BFGS, on the same frozen basis, collocation system, and initialization? Please provide error-versus-time comparisons for nonlinear EP and IC tasks. Since the introduction motivates the method through limitations of perturbative and Chebyshev-based nonlinear OTL, can the authors also compare with a representative approach in a shared validity regime, or explain why such a comparison is inappropriate?

2. Added value of continuation (Weakness 1). What quantitative task benefits from the paths or sensitivities enough to justify continuation’s additional cost? For example, can it obtain a parameter sweep more efficiently than sequential warm-started GN at matched accuracy and sampling locations? This is particularly relevant because continuation is 2.9–6.4× slower for target solves, with an Allen–Cahn-IC median error of 5.2 × 10⁻² versus 2.1 × 10⁻² for GN.

3. Basis capacity versus computational cost (Weakness 2). Does increasing the basis size improve transfer and extrapolation accuracy while retaining inexpensive adaptation? The EP and IC configurations use different basis sizes (B = 8 and B = 48), collocation counts, and training setups, so their comparison does not isolate the effect of basis size. Please provide a controlled basis-size ablation within the same PDE family and training range, with the collocation count fixed. For each basis size, report accuracy, matrix-assembly time, linear-solve time, and memory, with one-off body evaluation reported separately from per-target costs, following the distinction in Appendix B.5. Separately, how do these costs change with collocation count at a fixed basis size?

4. Basis limitation versus transfer failure (Weakness 2). For the difficult interpolation and extrapolation targets, can the authors fit the head directly to the reference solution while keeping the backbone frozen, and compare this best-fit error with the errors from GN and continuation? This would test whether the frozen basis itself cannot represent the target accurately, or whether the transfer procedure fails to obtain an accurate solution that the basis can represent.


Additional suggestions (not decisive)

Sensitivity to backbone pretraining. How sensitive are the transfer errors and relative solver performance to backbone pretraining? Results across several training seeds or checkpoints, with basis size fixed, would help establish the robustness of the reported findings.

Clarify the relationship to GPT-PINN and reduced residual-minimization methods. Comparing the same GN solver on learned and random-feature bases of equal size would also help isolate the benefit of multi-head basis learning.

Minor presentation note (not decisive): The title’s term “rich representations” is not clearly defined or quantified; please clarify its intended meaning. In the abstract, please also distinguish the Chebyshev surrogates used to approximate nonlinear terms in prior work from the learned solution basis used in this paper.

4 comments
AuthorsOpenReview ↗

Response to Reviewer (1/4): other head solvers and nonlinear OTL

Advantage over alternative methods (Weakness 1).

We thank the referee for this question. We have re-run the transfer to the 15 unseen targets of every model of Table 1 with two additional head solvers, Levenberg–Marquardt (LM) and L-BFGS, on exactly the same frozen basis, collocation system, start head (the nearest trained head) and stopping rules. The results are reported in the new Appendix E.1 (Table 7).

LM and L-BFGS reach the same median error as GN on all eight models (they converge to the same head), so the accuracy is set by the frozen system and not by the choice of second-order solver. In time, GN and LM are the fastest (2–6 ms per transfer in the multi-EP setting, 18–72 ms in the multi-IC one), L-BFGS is 1.1–22$\times$ slower than GN, and Adam stays far behind both at equal time and after 2000 steps (0.4–3.2 s), where it is still less accurate on wave, Allen–Cahn and Burgers in the multi-IC setting. We have also measured the time each method needs to come within 5% of the GN error on the same target: 0.9–1.2 ms (GN), 1.3–2.5 ms (LM) and 3.6–10 ms (L-BFGS) in the multi-EP setting, and 7–37, 22–46 and 31–82 ms in the multi-IC one, reached on every target. Adam reaches it on 10–15 of the 15 multi-EP targets but only on 2–10 of the 15 multi-IC ones.

The advantage we report is therefore that of a second-order solve of the small head problem on the frozen basis, against first-order training. GN is the simplest such solver, and LM is an equally good choice (marginally faster on the nonlinear multi-EP problems). We now state this explicitly in Section 5.1 and Appendix E.1, rather than attributing the advantage to GN in particular.

Regarding the second question, our approach is fundamentally different from perturbative and Chebyshev-based OTL. These methods replace the nonlinear problem by a sequence of linear ones. They expand the solution in a small parameter ε\varepsilon that multiplies the nonlinear term, u≈∑j=0pεjuju \approx \sum_{j=0}^{p} \varepsilon^j u_j, and solve one linear problem per order in closed form. Rao & Protopapas (2026) first replace the nonlinearity by a truncated Chebyshev series, so their Chebyshev polynomials approximate the nonlinear term, not the solution basis.

We never replace the nonlinear problem. GN and the Hessian continuation work on the full nonlinear residual, with no expansion, surrogate or small parameter. GN linearises only locally, at each iterate, and converges to the minimiser of the nonlinear least-squares problem. The Hessian continuation integrates the exact path of these minimisers, ∂w/∂ξ=−Hww−1Hwξ\partial \mathbf w/\partial \xi = -H_{ww}^{-1}H_{w\xi} (Eq. 17), starting from a trained solution of the nonlinear problem. Its accuracy is set by the integration step, not by an expansion order, and it also returns the sensitivities ∂w⋆/∂ξ\partial \mathbf w^\star/\partial \xi. The basis differs too. Their network is trained on linear problems, and a nonlinear solution is rebuilt as a sum of corrections, one per order. Ours is trained on the nonlinear family itself, so a new instance needs a single head.

For these reasons the two approaches share a validity regime only in the linear case (heat and wave). There, GN reduces exactly to the OTL solve and the methods coincide. Our nonlinear benchmarks have no small parameter. The Allen–Cahn reaction is O(1)O(1) (k=1k=1), the Burgers term uuxu u_x contains a derivative of uu, and in the multi-EP setting ν\nu changes the linear operator itself. All three fall outside the assumptions of perturbative and Chebyshev OTL, so a numerical comparison there would not be meaningful. We have revised the introduction (Section 1) to make this distinction explicit.

AuthorsOpenReview ↗

Response to Reviewer (2/4): added value of the continuation

Added value of continuation (Weakness 1).

We thank the referee for this question. We agree that GN is the method of choice for a single target. It is 2.9–6.4$\times$ faster, and on 8 Allen–Cahn and 3 Burgers multi-IC targets the continuation ends on a different stationary point (Appendix B.4). The continuation follows the branch of the start head, while GN can jump to another basin. We now state this in Section 5.2.

The continuation pays off when the path w(ξ)\mathbf w(\xi) between a known solution and a new one is needed, and not only its endpoint. Both methods compute w\mathbf w at N+1N+1 equally spaced nodes along the path and interpolate between them. GN, warm-started from node to node, gives the value of w\mathbf w at each node. We interpolate these values with a cubic spline, which for N=1N=1 is a straight line. The continuation also gives the slope ∂w/∂ξ\partial \mathbf w/\partial \xi at each node, which RK4 evaluates anyway. With the value and the slope at both ends of an interval, a cubic Hermite polynomial follows the path over longer intervals.

Table 9 (new Appendix F.1) compares both methods at matched accuracy and sampling locations, on the path from the nearest trained head to each of the 15 unseen targets of Table 1. As a reference, we solve GN at 64 equally spaced points along the path, each warm-started from the previous one. The error of a method is the largest relative difference between its interpolated path and this reference at the 64 points. Since the basis is orthonormal, this is the relative L2L^2 distance between the fields. The tolerance is one tenth of the model's own error at the target, and for each method we take the smallest NN that meets it. This test favours GN, whose nodes lie on the reference by construction.

With a single interval, the error of the continuation is 20–1200$\times$ smaller than that of GN in the multi-EP setting and 1.3–12$\times$ smaller in the multi-IC one. The continuation meets the tolerance with half the intervals on five of the eight models and with the same number on the other three. It is faster on seven models, by 1.4–3.2$\times$, and the two methods tie for wave with varying EP. The gain comes from the slopes. Giving GN the same slopes costs one extra Hessian solve per node and brings its error to that of the continuation.

Two limits should be stated. These paths are short, half the spacing between trained heads in the multi-EP setting. Over the whole trained range of ν\nu the continuation is 1.7$\times$ faster for Allen–Cahn and the two methods are within 20% of each other for the other three equations. Without interpolation, at the 4 nodes of Table 1 alone, a warm-started GN chain is about as fast as the continuation (0.5–1.3$\times$). What the continuation adds is therefore the slope ∂w⋆/∂ξ\partial \mathbf w^\star/\partial \xi at every node. It is the sensitivity of the solution to the EP or IC, and GN does not return it.

This result is summarised in Section 5.2. Further uses of the paths and of the sensitivities (the weight-space surface as a solution map, the sensitivity of the solution to the IC, the stability of the branch and the detection of failed transfers) are examined in the new Appendix F.2.

AuthorsOpenReview ↗

Response to Reviewer (3/4): basis capacity versus computational cost

Basis capacity versus computational cost (Weakness 2).

We thank the referee for this suggestion. We trained 16 new models, two more basis sizes for each of the eight models of the draft: B=4B=4 and 1616 around the multi-EP B=8B=8, and B=24B=24 and 9696 around the multi-IC B=48B=48. Each uses the same recipe, data, training range and collocation as the draft's model, with only BB changed. Table 16 (new Appendix H.2) reports, for each size, the transfer and extrapolation accuracy of GN, the per-target assembly and solve times, the one-off body evaluation and the memory. All timings are on one CPU thread, on a different machine from the draft.

A larger basis improves the transfer where the basis limits it. From the draft's size to twice that size, the GN error drops from 4.4×10−24.4\times10^{-2} to 2.9×10−22.9\times10^{-2} for wave and from 3.7×10−23.7\times10^{-2} to 2.3×10−22.3\times10^{-2} for Burgers in the multi-EP setting. In the multi-IC setting it drops from 2.1×10−22.1\times10^{-2} to 1.2×10−21.2\times10^{-2} for Allen–Cahn and from 7.4×10−27.4\times10^{-2} to 4.2×10−24.2\times10^{-2} for Burgers. Heat barely changes. Halving the basis is too small for some models, for example wave in the multi-EP setting (2.8×10−12.8\times10^{-1}).

Extrapolation also improves with BB, but a larger basis does not remove the degradation. In the multi-IC setting the targets lie just outside the trained latents. There the error falls with BB for Allen–Cahn, from 7.1×10−27.1\times10^{-2} to 1.1×10−21.1\times10^{-2}, and for Burgers, from 3.1×10−13.1\times10^{-1} to 1.5×10−11.5\times10^{-1}. In the multi-EP setting the targets reach up to 8 anchor spacings beyond the trained range. There the error falls with BB for wave and Burgers but stays large, 6.0×10−16.0\times10^{-1} and 2.5×10−12.5\times10^{-1} at B=16B=16, and it grows slightly for Allen–Cahn. The best fit of the frozen basis to the reference is also large there (see our answer to the next question, and Appendix E.2). This supports the referee's reading: far from the training range, the limit is what the basis has seen in training more than its size.

The adaptation stays inexpensive. Doubling BB multiplies the per-target time by 1.3–2.5. At the largest size a transfer takes 5–11 ms in the multi-EP setting and 53–177 ms in the multi-IC one. The least-squares solve grows faster with BB than the assembly of J\mathbf J. Before any target, a single forward pass of the frozen body on the collocation points, derivatives included, gives the basis functions and the derivatives the residual needs. This one-off step takes 0.6–3.5 s (multi-EP) and 3–38 s (multi-IC), and is shared by all targets. Each target then only combines these stored matrices with its head weights. The stored basis takes at most 37 MB and one Jacobian at most 7 MB.

At a fixed basis size, the costs grow about linearly with the number of collocation points, while the accuracy does not change. In the multi-EP setting (B=8B=8), going from 40240^2 to 1602160^2 points multiplies the per-target time by 5–8, the one-off evaluation by 14–20 and the memory by 16. The GN error changes by less than 5%. In the multi-IC setting (B=48B=48), going from 2,500 to 20,000 points multiplies the per-target time by 5–7 and the memory by 8. The error again changes by less than 5%. The accuracy is therefore set by the basis size, not by the collocation. These results are also in Appendix H.2.

AuthorsOpenReview ↗

Title: Response to Reviewer (4/4): basis limitation versus transfer failure

Basis limitation versus transfer failure (Weakness 2).

We thank the referee for this suggestion. For every target of Table 1 and for the extrapolation targets of Appendix B.9 (Table 6; extended to 8 anchor spacings in the multi-EP setting), 250 targets in total, we fitted the head directly to the numerical reference, with the body frozen. This best fit bounds the error of any head from below. To tell the two explanations apart, we also restarted GN from the best fit. If the transfer had missed an accurate solution that the basis can represent, the restart would find a lower PDE residual than GN. Table 8 (new Appendix E.2) summarises the results.

The transfer does not fail on 246 of the 250 targets. The restart finds no lower residual, and the best fit always has a higher residual than GN, by 1.03–48$\times$. The basis represents the targets 1.5–6.5$\times$ more accurately than GN in interpolation, but not with a smaller residual. This gap comes from the physics-only objective in the frozen basis, not from the solver. It also explains the hard multi-IC targets of footnote 3 (best fit 0.007–0.11, GN 0.11–0.43). There the GN head has the lowest residual found, but it lies far from the reference; the footnote now points to Appendix E.2, where this is stated. The Hessian continuation matches GN in the multi-EP setting and is 1.4–7$\times$ less accurate in multi-IC extrapolation.

Two cases differ. In multi-EP extrapolation above the trained range, the basis itself is the limit. The best-fit error reaches 1.0–2.3 at 8 anchor spacings (Table 15, new Appendix G), and GN follows it, as the referee suspected. On 4 of the 20 multi-IC Allen–Cahn extrapolation targets, the transfer does fail. The restart finds a 6–17% lower residual and a 2–3$\times$ smaller error. There the nearest trained head lies in the basin of a worse minimum.

Comments on the paper

1 comment

Dear authors, I received an email about your possible rebuttals that may have not been answered.

Unfortunately, I cannot see such responses, and I suspect the other reviewers as well. Check if you selected the correct visibility setting for your comments.

Regards,

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.