acceptodds
Under review as a conference paper at ICLR 2027

MF-SPECBO: MULTI-FIDELITY SPECTRAL BAYESIAN OPTIMIZATION FOR NEURAL OPERATORS

Abstract

Under Direct Spectral Evaluation (DSE), a neural operator’s cost is set by which Fourier modes it keeps, not just how many — yet whether informed mode selection can beat a plain isotropic truncation is untested. We give the question a screen: the gain of any selection rule over the matched-budget radial box is at most the spectrum’s radial defect ∆, a one-pass statistic (Theorem 1). We then build the machinery to search — a Sobolev chordal set kernel, a downward-closed restriction small enough for exact acquisition maximization, a calibrated MF-GP-UCB fidelity rule — and run the comparison at exactly matched cardinality. On standard benchmarks ∆ is tiny and, as predicted, no choice of mode set beats the box (a growth curriculum on the same set does — the win is in training, not selection); compression still yields 86.2–90.7% fewer modes, with 2.59–2.68× measured speedup on meshes above 103 nodes (the smallest mesh is launch-bound and gains none), but needs no search. On an anisotropic Darcy-flux family that passes the screen, MF-SpecBO improves on the box by up to 11.55 points and on a one-pass energy heuristic by up to 9.70 (p < 0.001 at κ=4), losing the advantage again at extreme anisotropy where the winning modes exceed what 800 samples can estimate. Mode selection pays in a regime ∆ locates in advance; every tabulated number is machine-verified against its recorded measurement.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
neural operators, Fourier neural operator, multi-fidelity Bayesian optimization, combinatorial Bayesian optimization, spectral methods, mode selection, Gaussian processes, partial differential equations

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_mHHyOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

I tried to read this for quite a while, but I cannot make any sense of this paper. It has something to do with neural operators. Without the slightest attempt to explain or motivate anything, the paper starts throwing around claims and undefined notation. The "related work" section is literally 4 lines and 4 citations.

Just from the abstract, I think this is about applying BO to select something undefined, and by some "theorem" involving all sorts of undefined notation, one often cannot do better than some "radial box" baseline, but I then trust the authors they are confident they can. Just I (and likely no other reader) will be able to follow them on their wild journey here.

I have no idea (and also no interest in speculating) whether AI was used here. If so, this reads like hallucination going off the rails.

What is quite funny (and I suppose unintendedly so) is the bloated vocabulary used here: "bites", "obstacle", "falsifiable", "organized so the answer can come out either way", "machine-checked numbers", "THE DECISION QUESTION", "like-for-like". What is any of this supposed to mean?

Critical strengths

I have literally absolutely no idea. And the authors make no effort conveying it. There are some claims, but the rest is drowned in undefined notation and super-short sections like 2.2, which suddenly stop nowhere.

Critical weaknesses

This paper is (excuse my French) a disorganized mess. Maybe 90% of all notation is not defined, or maybe defined later (at some point, I stopped checking). No motivation, no explanation of terms.

"(a) Downward-closed (enumerative). A symmetric downward-closed set in d = 2 is a non-increasing staircase b : {0,...,kmax}→ {0,...,kmax+1}with k active iff |ky|<b(|kx|); closure holds by construction. At fixed cardinality the admissible pool is tiny — 42 sets at M=27, 907 at M=145, 9,825 at M=221 — so zt is the exact maximizer of the acquisition, closing the oracle-versus-implementation gap that would otherwise void Theorem 3."

And it goes on and on like this.

Questions for authors

After reading the revision, I change my score to 2. I think with quite some effort, I could understand what is done here. Now, the authors need to work harder in order to tell me why I should spend the effort. This would be easiest by focussing on who is the audience here (maybe not even ML/AI, but rather applied physics?), and then give them some down-to-earth reasons why the considerable extra complexity of what is done here, would be worth their attention.

Reviewer_UB6POpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates how to select more effective Fourier modes for Fourier Neural Operators (FNOs) to reduce computational costs while maintaining prediction accuracy. The contributions are:

  1. Establishing a theoretical upper bound on the potential benefit of spectral mode selection.

  2. Proposing a physics-informed Sobolev kernel and a downward-closed search space to reduce the complexity of combinatorial Fourier mode selection.

  3. Developing a calibrated Bayesian optimization framework to reduce search costs, supported by theoretical guarantees.

  4. Comprehensivel Evaluation which demonstrates that the proposed method can achieve better performance on certain problems with strong anisotropy.

Critical strengths
  1. Meanful theoretical analysis. The authors propose Radial Defect and build theoretical upper bound to analysis potential gain of Fourier mode selection over isotropic truncation.

  2. Novel methodological design with theoretical support.

  3. Extensive validation and analysis, covering multiple PDE benchmarks and a variety of competitive baselines.

Critical weaknesses

The proposed method offers limited advantages on standard benchmarks in Table 1.Although the it has better performance on certain anisotropic problems (Table 3), in Appendix F.4 Table 6, there is no better performance shown with statistical significance on real dataset. I appreciate the authors' extensive experimental evaluation and insightful discussion of the results. However, the practical applicability of the proposed method appears to be rather limited.

Questions for authors

None

Reviewer_feSTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces a new automatic selection method for spectral mode sets based on multi-fidelity Bayesian optimization. The proposed method is specifically designed for Direct Spectral Evaluation (DSE) in Fourier Neural Operator (FNO), which was developed for efficiently computing PDE solvers with different system parameters. The authors developed a new method under certain assumptions to theoretically restrict the search space for spectral modes. Numerical experiments demonstrate the scenarios in which the proposed method outperforms existing methods.

Critical strengths

This study establishes a novel theoretical framework under reasonable assumptions and develops a method tailored specifically to DSE within the FNO architecture.

Critical weaknesses
  1. The motivation behind the proposed automatic mode selection is questionable for two main reasons. First, the necessity of explicit mode selection remains unclear; the standard FNO architecture already learns optimal mode coefficients from data and, as shown in the original work, effectively captures high-frequency components through its layered structure. Second, the approach introduces significant computational overhead. Given that FNO was designed to minimize optimization costs in iterative simulations, introducing an additional search budget directly contradicts this core objective.

  2. The simulation setup and reported performance remain questionable. Although the proposed method reduces computational overhead, simulation results exhibiting relative errors exceeding 10% are hardly acceptable for tasks demanding high-precision numerical computation.

  3. As shown in the numerical experiments, the performance gains offered by the proposed method are marginal. The authors should more clearly specify the target class of simulation problems where this approach provides practical advantages.

Questions for authors
  1. Could you clarify the motivation of this work, as raised in the above Critical Weakness?

  2. How is the convergent rate of the prediction accuracy when the proposed method can use the large number of modes?

  3. In Table 1, the prediction errors of the proposed method on the Airfoil and Elasticity benchmarks appear substantially higher than the results in the original DSE paper. Although a strict direct comparison is hindered by differing metrics (L1 versus L2), the performance should still be reasonably comparable since the original setup did not employ a particularly large number of frequency modes.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.