acceptodds
Under review as a conference paper at ICLR 2027

MetaColloc: Optimization-Free PDE Solving via Meta-Learned Basis Functions

Abstract

We introduce MetaColloc, which freezes a meta-learned basis dictionary and recovers each PDE solution by a single linear least-squares solve on a collocation system: no per-equation training, no solution data. Under the unregularised solve its accuracies on smooth two-dimensional benchmarks span to . We also report the two quantities that bound what those numbers can mean; both are cheap to compute before any solve is attempted. The first is capacity. The dictionary is meta-trained on Gaussian Random Field samples, so it can represent that prior no better than functions can. The prior needs 594 Karhunen-Loeve modes to capture 90% of its energy against , so the trained dictionary reproduces 16% of a fresh prior draw and fails an in-distribution manufactured problem under every backend we tried — worse than predicting zero. The second is numerical rank. The row-equilibrated collocation system has condition number and retains rank 309 of 512 at , so the solve is rank-deficient beyond double precision and its answer is decided by the cutoff the library chooses: the same checkpoint solves Poisson to under cuSOLVER and to under LAPACK. Choosing a Tikhonov level by generalized cross-validation stays within an order of magnitude of the oracle cutoff, matching it on the equation that needs regularisation, while making the result a property of the data rather than of the backend. On a band-matched prior, where the dictionary is representable, the comparison does not resolve: across three task banks the meta-trained dictionary scores , and , while a random basis from the same prior's spectrum ranges from to 1.35.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
PDE solving, meta-learning, basis functions, numerical stability, scientific computing

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_zoD6OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes MetaColloc, a PDE-solving framework that uses a meta-learned basis dictionary trained on Gaussian random fields. For new PDE instances, the frozen basis is combined with collocation-based least-squares solving, avoiding per-instance neural network training. The paper further investigates two major limitations: insufficient basis capacity and severe numerical ill-conditioning. It analyzes their effects on solution accuracy and reproducibility, and explores Tikhonov regularization with GCV as a remedy.

Critical strengths
  1. Useful diagnostic analysis. The paper provides a detailed investigation of basis capacity and numerical conditioning, including the substantial sensitivity of the results to different linear algebra backends.
Critical weaknesses
  1. Incomplete methodological contribution. The paper identifies fundamental limitations of its own approach, including insufficient basis capacity, severe numerical ill-conditioning, and the lack of a clear advantage over random basis methods. However, it does not provide a convincing new method to address these limitations. Although Tikhonov regularization with GCV improves numerical stability, it is a standard technique and does not resolve the underlying representation problem. As a result, the work feels incomplete: it identifies important problems but stops short of developing and validating an effective solution.

  2. Limited novelty and insufficient empirical validation. The proposed framework largely follows existing fixed-basis collocation and reduced-order modeling approaches, while the diagnostic tools are also well established. Moreover, the experiments mainly focus on smooth manufactured PDEs, without comparisons against standard numerical solvers. The paper does not establish a clear benefit of meta-learning or demonstrate sufficient generalizability. Overall, the current evidence is insufficient to support a complete and compelling methodological contribution for ICLR.

Questions for authors
  1. What is the main methodological contribution beyond existing fixed-basis collocation methods and standard numerical diagnostics?

  2. Can the authors demonstrate a consistent advantage of meta-learned bases over appropriately matched random or classical bases under the same regularized solving procedure?

  3. Could the authors provide results on more challenging PDEs and comparisons with standard numerical solvers to better establish the practical usefulness of the proposed method?

Reviewer_rx9UOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes MetaColloc, a meta-learning-based framework that uses a frozen neural basis dictionary to solve PDEs through least-squares collocation without per-instance optimization. The authors analyze its approximation capacity and numerical stability, and evaluate it on various 2D and 3D PDE benchmarks.

Critical strengths
  1. the proposed method enables optimization-free inference by avoiding per-instance gradient-based training, potentially reducing the cost of solving new PDEs.

  2. the author provides numerical diagnostics that identify approximation and conditioning limitations, offering insights into solver reliability.

Critical weaknesses
  1. Meta-learning shows no clear advantage over random basis functions.
  2. The theoretical contributions largely rely on established methods, with limited novelty.
  3. The overall framework of the paper is somewhat concerning. It initially presents a new PDE-solving method,MetaColloc, but much of the subsequent analysis focuses on diagnosing the limitations and failures of this method.
Questions for authors
  1. Compared with directly constructing basis functions from the training prior, e.g. RFF, what specific advantages can the proposed meta-training offer?

  2. The proposed diagnostics largely apply established KL decomposition and SVD analysis, while the regularization relies on standard GCV techniques, resulting in limited methodological novelty.

  3. Could the authors provide inference-time comparisons with similar methods, such as ELM and RFF, to demonstrate the computational efficiency of MetaColloc?

Reviewer_4w2rOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

MetaColloc learns a fixed dictionary of basis functions and solves new PDEs through collocation and least squares, rather than retraining a neural network for each equation. Nonlinear equations are handled through repeated linearized solves.

The paper’s main contribution is a diagnostic study. It examines whether the learned dictionary has sufficient capacity to represent its training prior and whether the resulting collocation systems can be solved reliably. The authors report severe numerical rank deficiency and large CPU–GPU discrepancies for the same checkpoint, then investigate Tikhonov regularization selected by generalized cross-validation (GCV).

Critical strengths

The paper examines numerical failure rather than reporting only favorable accuracy. The same-checkpoint CPU–GPU comparison, rank analysis, and collocation-count experiment show how strongly results can depend on numerical cutoffs. These observations are relevant to learned-basis PDE solvers beyond this particular method.

The baseline comparisons and negative results are candid. The authors compare against prior-matched random bases and acknowledge settings where these outperform the learned dictionary. They also distinguish manufactured nonlinear boundary-value problems from physical time-evolution simulations. A well-supported negative result of this kind could be valuable.

Critical weaknesses
  1. The central capacity diagnosis is not presented consistently, and some supporting material appears to be missing. Table 1 presents 594 modes for 90% energy alongside 39.3% energy at 512 modes as though they came from the same spectrum. The introduction indicates that the latter comes from a different grid and snapshot count. This appears to conflate measurements rather than demonstrate an error in either run, but the table needs correction and convergence checks. The supplied PDF also refers to appendices, cutoff sweeps, and a reproduction table that I could not find. These omissions matter because they support the paper’s central diagnostic claims.
  2. The proposed numerical remedy is not fully validated across backends. The paper suggests that GCV-selected regularization reduces backend dependence, but the provided results do not show matched CPU-versus-GPU GCV comparisons across all six equations or report the selected regularization levels. The table also shows a substantial accuracy trade-off on some smooth problems. Direct cross-backend evidence and controls for redundant or poorly scaled basis functions are needed to clarify what is fixed and at what cost. Additional related-work concern

The paper does not cite Meta-Auto-Decoder (MAD), including Huang et al., “Meta-Auto-Decoder for Solving Parametric Partial Differential Equations”, or the later MAD reduced-order-model paper. Both MAD and MetaColloc amortize PDE solving through an offline-learned representation and adapt it to a new equation. They are not identical: MAD generally uses a nonlinear decoder and optimizes a latent variable online, whereas MetaColloc freezes a linear dictionary and solves for coefficients by least squares. The authors should discuss these differences and clarify how their diagnostic findings relate to prior meta-learned PDE solvers.

Questions for authors

Can Table 1 be corrected so that the mode count and captured energy use the same spectrum, with convergence checks across snapshot counts and grid resolutions? Can the missing supporting appendices, reproduction table, and cutoff curves for all dictionaries be provided? For each PDE, what GCV regularization level is selected on CPU and GPU? Using the same stored system, how do solution errors, residuals, and cross-backend differences compare? Does the conditioning issue persist after orthonormalizing the basis or removing redundant directions? This would help separate basis parameterization from instability of the underlying trial space. How does MetaColloc relate to MAD in its training data, trial-space expressiveness, and online solve? Would a comparison with MAD help determine which observations are specific to a linear least-squares dictionary? The capacity bound is an average statement about a prior. Can the authors qualify the claim that a mode count above the dictionary size rules out a successful parameterization for a particular target PDE?

Comments on the paper

1 comment

Thanks for the additional experiments. However, as the authors themselves acknowledge, "The comparison shows no consistent accuracy advantage for MetaColloc." The boundary-layer results further highlight this issue, with MetaColloc achieving RMSEs of 0.453–0.459, compared to 0.00919 for prior-shaped RFF.

While I appreciate the analysis of these failure modes, GCV does not address the underlying limitations of the learned basis. Given the poor empirical performance and lack of a convincing solution, I do not think this constitutes a sufficiently complete method paper for ICLR. I therefore maintain my original rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.