acceptodds
Under review as a conference paper at ICLR 2027

CAFE+FNO: Fourier Kernel Generation via Multiplicative Feature Composition

Abstract

The Fourier Neural Operator (FNO) learns solution operators of partial differential equations (PDEs) through Fourier-space kernel parameterization, but frequency truncation can limit the learning of high-frequency variations. AM-FNO and SirenFNO generate kernels for all grid modes from spectral coordinates using shared networks, making coordinate encoding and generator design important. Recent work on implicit neural representations (INRs) has proposed constructing frequency interactions through explicit feature composition rather than relying on subsequent MLPs to form them implicitly. Building on this approach, we propose CAFE+FNO, which incorporates Content-Aware Frequency Encoding+ (CAFE+) into Fourier kernel generation. CAFE+ combines Fourier–Chebyshev features through parallel affine branches and a Hadamard product, forming interactions within and across the two feature families. A kernel MLP maps the resulting representation of each normalized spectral coordinate to a complex channel-mixing matrix. Each layer shares its generator across all stored modes, making the number of trainable parameters independent of the number of modes for a fixed architecture. We compare CAFE+FNO with existing FNO variants on five PDE benchmarks and conduct ablation studies on basis configuration, multiplicative composition, and bandwidth learnability. Code and experimental configurations are available at https://anonymous.4open.science/r/CAFEPlusFNO-70D2.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Fourier Neural Operator, Neural Operator, Spectral Kernel Generation, Multiplicative Feature Composition, Fourier Features, Chebyshev Features

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_f14rOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CAFE+FNO for learning PDE solution operators. The method combines Fourier and Chebyshev features and multiplies the outputs of parallel branches. An MLP then uses these features to generate the Fourier kernel weights. Each layer uses the same generator for all stored frequency modes.

The authors test the method on five PDE datasets. The dense model has the lowest average error on four datasets, while SirenFNO performs better on reaction--diffusion. The paper also reports results over five random seeds and includes ablation studies on the feature choices, multiplication step, and bandwidth.

Critical strengths
  1. The idea is clear and easy to understand. Combining Fourier and Chebyshev features is a reasonable way to improve the kernel generator. The paper also explains how the method builds on earlier work.

  2. The reported results are promising. The dense model has the lowest average error on four of the five datasets. The authors report five-seed results with standard deviations, and the ablation studies support using both feature types and the multiplication step. They also clearly report the dataset where their method performs worse.

Critical weaknesses

My main concern is that the errors of several methods in the main table look larger than those reported in well-known papers for the same PDE problems. This makes me question whether the baselines are performing as well as they should. Training all methods with the same pipeline does not fully address this concern.

The authors should compare their reproduced baseline results with the published results and explain the differences. Please clarify the dataset versions, PDE parameters, resolution, prediction time, normalization, and error calculation. If the settings are intended to match, the authors should explain why the errors are larger. Without this check, it is difficult to judge whether the reported improvement is over strong baselines and how the proposed method compares with existing work.

Questions for authors
  1. Could the authors compare their reproduced baseline results with the published results for the same PDE problems? Why are some errors much larger, and which differences in the experimental settings explain this?

  2. Could the authors provide a short table showing the dataset versions, PDE parameters, resolution, prediction time, normalization, and error calculation used in this paper and the reference papers? If the settings are intended to match, could the authors reproduce the reference baseline results and explain any remaining differences?

Reviewer_JZQeOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work aims to improve the generation of Fourier kernels in Fourier Neural Operators (FNOs), in order to enhance their modeling capabilities for PDE solution operators. The proposed CAFE+FNO introduces Content-Aware Frequency Encoding + (CAFE+) into the Fourier kernel generation process. By combining Fourier–Chebyshev features and multiplicative feature composition, it explicitly constructs the interaction between different encoded features. Additionally, this method is extended to CP, TT, and Tucker tensor factorization. CAFE+FNO achieved competitive performance on five PDE benchmarks. Ablation experiments further validated the effectiveness of Fourier–Chebyshev encoding and multiplicative composition, and demonstrated the potential of tensor factorization in parameter compression. However, the verification of the core mechanism still has some key issues.

Critical strengths
  1. This work proposes clear architectural improvements. By combining Fourier–Chebyshev encoding with explicit multiplication features for kernel generation, it has a reasonable technical motivation.

  2. CAFE+FNO achieves competitive performance on five benchmarks. Additionally, ablation studies verifies the roles of the main architectural components.

Critical weaknesses
  1. The proposed framework lacks sufficient theoretical justification, such as approximation theory, efficiency, and expressivity. Moreover, due to the lack of further analysis of frequency, spectrum, or learned kernel, it is unclear whether the performance improvement stems from more effective high-frequency representation.

  2. The validation of the proposed method is insufficient. The current manuscript lacks comparisons with parameter-matched MLP or other nonlinear feature composition mechanisms. Additionally, resolution generalization and practical computational efficiency are also unclear.

Questions for authors

Why does the proposed method use normalized spectral coordinates that depend on the FFT grid size? Will this design affect cross-scale generalization? Why does CAFE+FNO perform worse than SirenFNO on ReacDiff?

Reviewer_c2t9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces CAFE+FNO, which retains the AM-FNO backbone while incorporating the CAFE+ encoding from the implicit neural representation (INR) literature into its kernel generator. At each layer, the normalized spectral coordinate ξk\xi_k is transformed into random Fourier and Chebyshev features. These features are then concatenated and passed through an MLP to produce a complex-valued dv×dvd_v \times d_v kernel for each mode kk. Since the same generator is used for all modes, the number of trainable parameters remains independent of the mode count. The authors also investigate CP-, TT-, and Tucker-based variants. Experiments on five 1D/2D benchmarks (Darcy, NS, Burgers, Airfoil, and ReacDiff), each evaluated over five random seeds, show that CAFE+FNO achieves the lowest mean error on four benchmarks, while SirenFNO performs better on ReacDiff.

Critical strengths

1. CAFE+FNO reduces the error over SirenFNO by 44%, 60%, 30%, and 18% on Darcy, NS, Burgers, and Airfoil, while using a similar number of parameters (0.35M vs. 0.31M on Darcy). The results are also stable across seeds. If confirmed by a fair comparison, this could be a simple and effective upgrade to amortized FNOs. This is the main reason I did not give a lower score.

2. The authors provide per-seed errors, parameter counts, and code. The claims are also carefully stated, especially regarding input-independent kernels and the frequency encoding. My concerns are mainly about the experimental evidence, not the presentation.

Critical weaknesses

1. It is hard to tell how much of the improvement actually comes from CAFE+. Some baselines may not be well tuned. For example, SirenFNO underperforms AM-FNO and TFNO-CP on NS, while AM-FNO diverges on one ReacDiff seed. The ablations also raise questions: AM-FNO (MLP) gets 62.02 on Darcy, compared with 27.28 for the no-composition variant, a larger gap than the gain from adding composition (27.28 to 13.63). The no-composition variant also changes the MLP input size, making the comparison less controlled. Moreover, Chebyshev-only performs within 6% of the full model on all five tasks. Since composition is the main contribution, this is my biggest concern.

2. I feel that the novelty and analysis are somewhat limited. The method mainly combines two existing ideas, and Sec. 4.2 largely repeats the product-to-sum argument from MFN-style methods. There is no analysis of frequency-wise errors or learned kernels to support the high-frequency motivation. The paper also does not test discretization effects, even though ξk=2k/L\xi_k = 2k/L depends on the grid size (Q4). Runtime and memory costs are missing as well.

Questions for authors

Some questions:

  1. How were ρℓ=1\rho_\ell = 1, m=n=32m = n = 32, and the settings in Table 5 selected without a validation split?
  2. How is the MLP input reduced to 32 (16 for ReacDiff) in the ∗\ast variant? Were mm and nn changed?
  3. Why do Tables 2 and 3 report slightly different results for the default model (Darcy: 13.70 vs. 13.63; NS: 17.20 vs. 17.25)? Are these separate runs? The differences are comparable to the learnable-bandwidth effect on NS (17.22).
  4. Since ξk=2k/L\xi_k = 2k/L changes with resolution for the same physical wavenumber, how does the model handle resolution transfer? Is it intended to be discretization-invariant? Using a fixed normalization could also push Chebyshev inputs beyond [−1,1][-1,1], where T31(1.1)≈4.7×105T_31(1.1) \approx 4.7 \times 10^5.
Reviewer_6VMZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CAFE+FNO, which replaces the spectral-coordinate encoding in an amortized Fourier kernel generator with the existing CAFE+ encoding. Fourier and Chebyshev features pass through parallel affine branches and a Hadamard product before an MLP generates a complex channel-mixing matrix for each mode. Dense and CP/TT/Tucker variants are evaluated on five PDE benchmarks. The dense model improves over SirenFNO on four benchmarks, while SirenFNO remains better on reaction–diffusion. Ablations examine basis selection, feature composition, and bandwidth learning.

Critical strengths

The strongest evidence is the substantial and repeatable improvement of the dense model on several established benchmarks. The implementation is also sufficiently explicit to make the design easy to assess.

  1. Dense CAFE+FNO reduces mean error relative to SirenFNO by 43.7% on Darcy and 60.2% on Navier–Stokes, with improvements also on Burgers and Airfoil. These gains are substantial compared with the reported seed variability (Table 2).
  2. Baselines are retrained with shared dataset-specific preprocessing and training protocols. Five-seed statistics, individual-seed results, and final-epoch evaluation improve the transparency of the empirical comparison (Section 5.2; Appendix E).
  3. The combined Fourier–Chebyshev configuration has lower mean error than either single-basis variant on all five tasks. This provides useful empirical evidence that complementary coordinate features can improve kernel generation (Table 3).
Critical weaknesses

The results are promising, but the methodological advance and the evidence isolating its proposed mechanism are currently limited.

  1. The main construction transfers the existing CAFE+ encoding into the established AM-FNO/SirenFNO kernel-generation framework; the tensorized extensions also follow SirenFNO. The product-to-sum analysis explains the inherited encoding, but offers limited additional insight into when this combination should improve PDE operator learning (Sections 3–4).
  2. The ablations do not isolate multiplicative composition from capacity and basis coverage. Matching the downstream MLP input dimension does not match total parameters, and the released input-32 branch-free control reduces the underlying Fourier and Chebyshev basis counts. A control retaining the same raw bases with a learned non-multiplicative projection would better establish the contribution of the Hadamard product (Section 5.4; released ablation documentation).
  3. The high-frequency motivation is evaluated mainly through aggregate relative L2 error, without frequency-resolved errors or cross-resolution tests. Appendix B normalizes modes by the current FFT-axis length, so the same physical Fourier index receives a different generator input when the grid changes. Resolution-independent parameter count therefore does not by itself establish consistent transfer across discretizations.
Questions for authors
  1. Can you provide a composition control retaining the same raw Fourier/Chebyshev bases and comparable parameter count, using a learned projection or additive combination instead of the Hadamard product? Please also specify the exact basis counts and configuration producing each starred Table 3 result, including the 16-dimensional ReacDiff case.

  2. Under the normalization in Appendix B, how should a trained kernel generator be evaluated at a different resolution on the same physical domain? Can you provide a cross-resolution comparison and frequency-resolved prediction errors to determine whether the gains specifically improve high-frequency solution components?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.