acceptodds
Under review as a conference paper at ICLR 2027

Structurally Aware Post-Training Quantization of Fourier Neural Operators

Abstract

Fourier neural operators (FNOs) provide an efficient framework for learning mappings between function spaces, but deploying trained operators on resource-constrained hardware requires effective model compression. Post-training quantization (PTQ) is particularly relevant here because, unlike quantization-aware training (QAT), it requires no retraining and can be applied directly to a trained model. Despite this, the effectiveness of PTQ for neural operators remains largely unexplored. This paper presents a systematic study of PTQ for FNOs on two three-dimensional problems: forced isotropic turbulence and Darcy flow. We evaluate quantization strategies across model capacities and bit-widths at equal storage and find that quantizer design has a substantial impact on accuracy. In particular, different Fourier modes have very different sensitivity to quantization, and the measured input energy closely tracks quantization sensitivity. Allocating bits according to this input energy reduces error across all configurations tested, with a median improvement over uniform allocation of overall and of , and at two, three and four bits. We also evaluate three other aspects of quantization: weight representation, scaling granularity, and quantization range. Once the bit budget is allocated according to input energy, these choices have substantially smaller effects. Nevertheless, at a fixed bit-width, different quantizer designs can produce errors that differ by up to two orders of magnitude. These results show that exploiting the structure of Fourier Neural Operators can substantially improve post-training quantization.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Fourier Neural Operators, Post-Training Quantization, Neural Operators, Model Compression, Quantization, Scientific Machine Learning, Computational Fluid Dynamics, Physics-Informed Machine Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_wqCQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates post-training quantization for Fourier Neural Operators and proposes a structure-aware mixed-precision strategy that allocates bits across frequency bands according to measured input spectral energy using reverse water-filling.

Critical strengths
  1. The paper exploits the Fourier-domain structure of FNOs to relate quantization sensitivity to input spectral energy. The resulting allocation rule is intuitive and supported by a single-layer error analysis, offering a meaningful connection between neural operator structure and mixed-precision quantization.
  2. The proposed input-energy allocation consistently outperforms uniform quantization across the evaluated configurations, with median error reductions of 14.9×, 5.9×, and 3.1× at two, three, and four bits, respectively.
Critical weaknesses
  1. The paper does not sufficiently demonstrate a fundamentally new quantization mechanism beyond identifying an appropriate sensitivity proxy for this particular architecture.
  2. Both tasks (turbulence and Darcy flow) exhibit relatively smooth spectral distributions with energy concentrated at low frequencies, which particularly favors the proposed allocation strategy. It remains unclear whether similar advantages would hold for more challenging settings involving high-frequency structures, discontinuities, or alternative neural operator architectures.
  3. The proposed allocation is derived from a single-layer error model assuming independent, zero-mean quantization errors and frequency-wise decoupling. However, nonlinear activations and repeated spectral operations introduce cross-frequency interactions and error accumulation across layers.
  4. The evaluation focuses on one-step operator prediction, whereas many important applications of neural PDE solvers involve recursively applying the learned operator over long temporal horizons. Quantization errors that appear small in single-step relative L2 metrics may accumulate and affect long-term stability, conservation properties, or physical consistency. Although the paper reports divergence-related measurements, it does not evaluate actual multi-step rollouts, leaving an important practical question unanswered.
Questions for authors
  1. Could the authors clarify the main methodological differences between the proposed energy-based allocation and existing Hessian-aware or activation-aware mixed-precision quantization methods?
  2. How does the proposed approach perform on PDE problems whose energy spectra are not strongly concentrated at low frequencies, such as high-Reynolds-number turbulence, discontinuous solutions, or multiscale physical systems?
  3. Given that nonlinear activations introduce cross-frequency coupling, how robust is the proposed single-layer sensitivity approximation when multiple layers are quantized simultaneously? Could the authors investigate whether incorporating downstream error amplification or layer-wise sensitivity information improves allocation, particularly for the smaller turbulence model?
  4. Could the authors provide an implementation of the proposed frequency-dependent mixed-precision representation and evaluate its actual storage footprint, inference latency, and GPU memory consumption?
2 comments

Thank you for the detailed review. We respond to each point below; the sections and tables cited are in the revised paper.

W1, Q1. No fundamentally new quantization mechanism; differences from Hessian-aware and activation-aware methods.

We agree that the paper introduces no new mechanism: the solver is the classical reverse water-filling solution. The contribution is the first study of post-training quantization for neural operators. It shows that uniform bit-widths degrade FNOs severely at two and three bits (Table 2), and that the Fourier structure gives a direct remedy: the weight at mode k multiplies only the input at mode k, so the output error of a layer separates exactly across modes, and the sensitivity of a band is its input energy, measured with forward passes alone.

We ran the Hessian-aware methods on all 20 checkpoints at equal storage (Table 1, Section 5; Appendix C). HAWQ assigns one bit-width per layer; it has 5.4× and 2.8× our error ratio at 3 and 4 bits, and at 2 bits it equals uniform bit-widths. The difference is the unit of allocation: bands, not layers. A Hessian trace per band ranks the bands as input energy does on three of four model groups, so the two estimate the same sensitivity, and input energy needs only forward passes. AWQ protects salient channels at one bit-width and does not allocate bit-widths.

W2, Q2. Spectra not concentrated at low frequencies; other architectures.

We trained nine models (width 32, three seeds each) on 3D Darcy flow with two-phase (discontinuous), multiscale and anisotropic coefficients, whose spectra are less concentrated at low frequencies (Section 6.2; Figure 6, Appendix H).

coefficientuniform / measured (3 bits, 4 bits)fitted power law / measured (2 bits)
two-phase4.2×, 2.0×1.05
multiscale6.4×, 2.9×1.05
anisotropic9.2×, 3.5×1.19

The advantage over uniform bit-widths holds (the paper reports 5.9× and 3.1× at 3 and 4 bits). The fitted power law in the last column is a two-parameter summary of the same measured spectrum. It comes close, but the measured spectrum is better on every seed. Other neural operator architectures are not covered, but the derivation applies wherever each retained Fourier mode has its own weight.

W3, Q3. Single-layer error model; downstream amplification.

We implemented your suggestion, and it helps most on the small turbulence model, as you anticipated. For every layer and band we measured how much an error in that band changes the network output relative to the layer output, and allocated with input energy times this gain (Section 6.3; Table 15, Appendix I). All results quantize all layers simultaneously. Error ratios:

modelsenergy only (2 / 3 / 4 bits)with downstream gain
turbulence, width 83.44 / 2.11 / 1.452.04 / 1.70 / 1.27
turbulence, width 321.25 / 1.12 / 1.041.23 / 1.12 / 1.04
Darcy, log-normal coefficient (2 bits)1.11–1.231.12–1.27

In the small turbulence models later layers damp errors in the outer bands, so input energy alone assigns them too many bits, and the gain corrects this. A limitation remains: hard low-pass truncation is still better there (1.17 against 2.04 at 2 bits), as the Limitations state.

W4. Multi-step rollouts.

We ran rollouts on the 8 turbulence models: 72 unseen trajectories and 8 autoregressive steps (Appendix J). Width-32 models at 2 bits, relative L² distance from the FP32 rollout (Table 16):

step148
uniform, same storage21%63%98%
our allocation2.5%4.3%25%
FP32 against the truth, for reference0.7%4.1%25%

Kinetic energy stays within 1% of the truth through step 8, whereas uniform bit-widths lose 69% of it. The quantized rollouts blow up less often than FP32 (0–6% of trajectories against 10–14%; we attribute this to damping by the pruned high-frequency bands, which we have not tested). Incompressibility is affected: the normalised divergence at step 1 is 0.036 against 0.013 for FP32. The width-8 FP32 models are themselves unstable over this horizon, so we draw no conclusion about quantization there.

Q4. Implementation, storage footprint, latency and GPU memory.

The submission reported one packed file. Appendix A now covers all three bit-widths, with pruned bands: at width 32 the file is 226.5 MB in FP32 and 14.2 / 21.3 / 28.4 MB at 2 / 3 / 4 bits, within 0.1% of the storage computed in the paper. The weights are unpacked at load time, so inference time and GPU memory are those of the FP32 model (Appendix A); the benefit is the size of the stored and transferred model. The packing code is in the supplementary material.

Thank you for the detailed rebuttal and additional experiments. I appreciate the authors' efforts in addressing my concerns.

Reviewer_5rpLOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The core of this work is the insight that when applying Post-Training Quantization (PTQ) to Fourier Neural Operators (FNOs), one should not assign the same bit-width to all Fourier modes; instead, quantization precision should be allocated based on the actual input energy carried by each mode.

Critical strengths
  1. The experimental framework is comprehensive, incorporating multiple checkpoints, storage analysis, and ablation studies.
  2. The method is highly lightweight, requiring only a small amount of unlabeled data for forward hooks.
  3. The results are robust and not dependent on any specific quantizer design.
Critical weaknesses
  1. Given my limited familiarity with FNOs, I cannot fully assess the state of mixed-precision research in this domain; however, it appears that no prior mixed-precision quantization methods have been applied in this context.
  2. While the authors demonstrate significantly lower numerical error for a given theoretical storage budget, they do not explicitly show or discuss actual reductions in device memory footprint or latency.
  3. The study focuses solely on the quantization of spectral weights, omitting activation quantization. A more detailed specification of the quantization configuration is needed.
Questions for authors

See Weaknesses.

1 comment

Thank you for the review and for the questions on practical cost. We respond to each point below; the sections and tables cited are in the revised paper.

W1. No prior mixed-precision methods appear to have been applied in this context.

Your impression is correct: to our knowledge, no prior work studies post-training quantization of neural operators, mixed-precision or otherwise. We therefore ran the standard approaches ourselves on all 20 checkpoints at equal storage (Table 1, Section 5; Appendix C):

baselineerror ratio relative to ours
HAWQ (one bit-width per layer, by Hessian trace)5.4× at 3 bits, 2.8× at 4 bits
uniform bit-widths with per-output-channel scales3.1–39×
Hessian trace used inside our per-band allocation0.93–1.00× on three of four model groups

At 2 bits HAWQ equals uniform bit-widths, because a per-layer rule must give every layer the smallest width. In an FNO the freedom that matters is across frequency bands inside a layer. The last row shows that a Hessian trace per band is another estimate of the same band sensitivity and leads to nearly the same allocation, while input energy needs only forward passes.

W2. Actual memory footprint and latency.

The submission reported one packed file (21.30 MB against 226.53 MB at 3 bits). Appendix A now reports all three bit-widths, with pruned bands, for the width-32 turbulence model:

FP322 bits3 bits4 bits
file size on disk226.5 MB14.2 MB21.3 MB28.4 MB
reduction1×15.9×10.6×8.0×

Each file is within 0.1% of the storage computed in the paper. The weights are unpacked once when the file is loaded, so inference time and GPU memory are those of the FP32 model; the benefit of the method is the size of the stored and transferred model.

W3. Activation quantization and the quantization configuration.

Table 18 in Appendix L gives the full configuration. On activations: our target is the stored model, and the spectral weights are over 99.9% of the parameters, so quantizing activations would not change its size. It would reduce memory during inference, where activations dominate at large batch sizes (1,034 MiB against 216 MiB of weights at batch size 16; Appendix A). This is a separate problem, because the activations pass through floating-point FFTs, and we do not address it in this work.

Reviewer_QdhWOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies post-training quantization (PTQ) for Fourier Neural Operators (FNOs) to reduce model storage without retraining. The authors show that Fourier modes have different sensitivities to quantization and that input spectral energy can be used to estimate their importance. Based on this observation, they use reverse water-filling to allocate different bit-widths across frequency bands and prune low-energy bands. The paper also examines several quantizer design choices, including weight representation, clipping, and scaling granularity.

Experiments on 3D turbulence and Darcy flow demonstrate that the proposed method significantly reduces prediction errors compared with uniform quantization under the same storage budget, achieving a median improvement of 5.2×. Additional analyses support the relationship between input spectral energy and quantization sensitivity.

Critical strengths
  1. The paper identifies a meaningful mismatch between weight statistics and quantization sensitivity in FNOs. By exploiting the frequency-wise structure of Fourier layers, the authors derive a relationship between input spectral energy and quantization-induced output error at the single-layer level, further supported by empirical sensitivity analyses. This provides a clear motivation for the proposed approach.

  2. The proposed method achieves substantial error reductions over uniform quantization under matched storage budgets, with a median improvement of 5.2×. Experiments across two PDE tasks, different model sizes, and bit-widths demonstrate its effectiveness. The extensive ablation studies further show that energy-aware bit allocation is the main contributor to the performance gains.

Critical weaknesses
  1. The fixed spectral prior ((1+|k|)^{-4}) achieves performance close to the proposed method, raising questions about the necessity of fine-grained input-energy estimation. Could the authors compare against a simple spectral decay function fitted using the same calibration samples to better demonstrate the additional benefits of their approach?

  2. Sensitivity-based bit-width allocation and the Reverse Water-Filling algorithm have both been explored in prior quantization research. Although this paper provides an FNO-specific sensitivity analysis, the overall optimization framework is largely built upon existing techniques. The authors are encouraged to further clarify the distinctions between the proposed method and prior quantization approaches, such as BAQ, as well as existing frequency-aware quantization studies, and to highlight its unique methodological contributions.

  3. The experiments are restricted to two PDE tasks with rapidly decaying spectra. While the authors suggest that measuring input spectral energy can adapt to different spectral distributions without assuming a fixed decay pattern, this advantage has not been sufficiently validated. Experiments on PDEs with non-smooth or non-monotonic spectral distributions would help demonstrate the generalizability of the proposed method.

Questions for authors
  1. The proposed method uses only 32 unlabeled calibration samples to estimate the input spectral energy for bit allocation. However, Figure. 2 reports spectral energy statistics measured using 128 training samples, while the actual bit allocation in Section 5 relies on only 32 samples. The rationale for using different sample sizes is not clearly explained.

  2. How sensitive is the resulting quantization performance to the number and selection of calibration samples? It would be helpful to provide a calibration-size ablation (e.g., 8, 16, 32, 64, and 128 samples) and evaluate the stability of the resulting bit allocation and prediction accuracy across different calibration subsets, particularly under low-bit settings where frequency-band pruning is involved.

  3. Given that the blue energy curve generally decreases monotonically with increasing frequency, could the proposed energy-aware quantization scheme essentially be reduced to a simple heuristic of assigning higher bit-widths to low-frequency components and lower bit-widths to high-frequency components?

2 comments

Thank you for the careful review and the concrete suggestions. We respond to each point below; the sections and tables cited are in the revised paper.

W1. Compare against a spectral decay function fitted on the same calibration samples.

We ran this comparison. A fitted law does not replace the measurement: it is fitted to the same measured spectrum and summarises it by two numbers per layer, an amplitude and a decay rate. Both are estimates of the same band sensitivity, given to the same solver. On turbulence and on Darcy flow with the log-normal coefficient the spectrum is close to a power law, so the two allocations nearly coincide (error ratio of the fitted law divided by that of the measured spectrum, geometric mean over checkpoints, so values above 1 favour the measured spectrum; Table 1, Section 5):

models2 bits3 bits4 bits
turbulence, width 320.981.011.01
Darcy, width 81.011.001.00
Darcy, width 321.031.021.00
turbulence, width 8, with downstream gain1.351.021.00

On these spectra the fine shape therefore adds little, as you suspected. Where the spectrum is not a power law, the fit loses information and the measured spectrum is better on every seed (W3 below). The last row needs a separate correction: in the small turbulence models later layers damp errors in the outer bands, so input energy alone overstates their sensitivity, and there the fitted law is 12% to 19% better than it (Appendix C). A basic correction, which multiplies the input energy by a measured downstream gain (Section 6.3), accounts for this, and the fitted law then no longer has an advantage. On the other models the allocation with the gain matches the one without it.

W2. Distinguish the method from BAQ and frequency-aware quantization.

We agree that the solver is not new: the paper uses the classical reverse water-filling solution, and the revised related work positions it against prior allocation work, including BAQ. Those methods need an approximation or an extra transform before the error separates. In an FNO it separates exactly, because the weight at mode k multiplies only the input at mode k, and the sensitivity is the input energy, measured with forward passes alone. FAT and CRAFT use frequency information at a single bit-width and do not allocate bit-widths across frequencies.

W3. Non-smooth or non-monotone spectra.

We trained nine models (width 32, three seeds each) on 3D Darcy flow with two-phase (discontinuous), multiscale (non-monotone) and anisotropic coefficients; Figure 6 in Appendix H shows their spectra. The allocation is applied to them unchanged (Section 6.2, Table 4):

coefficientuniform / measured (3 bits, 4 bits)fitted law / measured (2 bits)
two-phase4.2×, 2.0×1.05
multiscale6.4×, 2.9×1.05
anisotropic9.2×, 3.5×1.19

The gain over uniform bit-widths is of the same size as on the paper's two benchmarks (5.9× and 3.1× at 3 and 4 bits). The method is therefore not tied to smooth, steeply decaying spectra: it reads the spectrum from the calibration inputs and assumes no shape. A power law fitted to the same spectrum comes close, but the measured spectrum is better on every seed at two bits.

Q1, Q2. Calibration sample counts and a calibration-size ablation.

Thank you for catching the mismatch. It was an oversight: the number of inputs was a script setting that we did not hold fixed (32 inputs for seven of the eight turbulence checkpoints, 128 for the others). Section 5 now states the counts. We also ran the ablation with 8 to 128 inputs on three disjoint subsets (Table 17, Appendix K). The count matters little: even with 8 inputs, the error ratio at two bits changes by at most 2.0% outside the small turbulence models and by 8.9% there. We did not expect this, and it suggests that calibration could be made cheaper still.

Q3. Does the method reduce to assigning more bits to low frequencies?

No. The allocation assigns bits by measured input energy, not by frequency. On turbulence and on Darcy flow with the log-normal coefficient the energy decays with frequency, so the two orderings coincide, which is what Figure 2 shows. On the multiscale and anisotropic coefficients they do not coincide: the energy has a maximum at band 7 or depends on direction, and the allocation follows it. There the fixed rule (1+|k|)^-4, which orders bits by frequency alone, has a 28% and 63% higher error at two bits (Section 6.2, Table 4).

Thank you for the rebuttal and additional experiments. Most of my concerns have been addressed, although the methodological novelty remains somewhat limited.

The new results on non-monotonic spectra are useful, as they better demonstrate the benefits of the proposed method over simple spectral decay priors. I suggest giving these results more emphasis in the main paper.

Overall, I will maintain my original score

Comments on the paper

2 comments

We thank the reviewers for their careful evaluation. We are encouraged that they found the link between the Fourier structure of FNOs and quantization sensitivity well motivated, and we appreciate the constructive feedback. We ran experiments accordingly and have uploaded a revised paper. The main changes in the paper are:

  • The abstract and introduction are reframed slightly: they present the allocation rules as different estimates of band sensitivity, as summarised below.
  • A new Table 1 (Section 5) compares the allocation with other sensitivity estimates: a power law fitted to the same calibration samples, HAWQ, and uniform bit-widths with per-channel scales.
  • Section 6.2 tests nine new Darcy models with discontinuous, multiscale and anisotropic coefficients (Table 4, Appendix H).
  • Section 6.3 adds a basic correction for interaction between layers: the input energy multiplied by a measured downstream gain (Appendix I).
  • New appendices report packed file sizes, latency and GPU memory (A), eight-step autoregressive rollouts (J), a calibration-size ablation (K) and the full quantizer configuration (L).

Taken together, the results of these experiments let us state the allocation more simply. Every allocation rule our paper compares, from uniform bit-widths to the measured spectrum, distributes a fixed storage budget across frequency bands to minimise output error. The rules differ only in how they estimate each band’s sensitivity. Reverse water-filling, which we use to turn these estimates into bit-widths, is established in prior bit-allocation work. In an FNO each weight acts on a single Fourier mode, so a layer’s error separates across modes and band sensitivities can be measured from a few forward passes.

We thank the reviewers again for their time and feedback. We have uploaded a final revision updating only the abstract. It now states the methodology more clearly and summarizes the auto-regressive rollout results (Appendix J), showing that allocation maintains physical fidelity. The rest of the paper is unchanged.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.