- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates post-training quantization for Fourier Neural Operators and proposes a structure-aware mixed-precision strategy that allocates bits across frequency bands according to measured input spectral energy using reverse water-filling.
Critical strengths
- The paper exploits the Fourier-domain structure of FNOs to relate quantization sensitivity to input spectral energy. The resulting allocation rule is intuitive and supported by a single-layer error analysis, offering a meaningful connection between neural operator structure and mixed-precision quantization.
- The proposed input-energy allocation consistently outperforms uniform quantization across the evaluated configurations, with median error reductions of 14.9×, 5.9×, and 3.1× at two, three, and four bits, respectively.
Critical weaknesses
- The paper does not sufficiently demonstrate a fundamentally new quantization mechanism beyond identifying an appropriate sensitivity proxy for this particular architecture.
- Both tasks (turbulence and Darcy flow) exhibit relatively smooth spectral distributions with energy concentrated at low frequencies, which particularly favors the proposed allocation strategy. It remains unclear whether similar advantages would hold for more challenging settings involving high-frequency structures, discontinuities, or alternative neural operator architectures.
- The proposed allocation is derived from a single-layer error model assuming independent, zero-mean quantization errors and frequency-wise decoupling. However, nonlinear activations and repeated spectral operations introduce cross-frequency interactions and error accumulation across layers.
- The evaluation focuses on one-step operator prediction, whereas many important applications of neural PDE solvers involve recursively applying the learned operator over long temporal horizons. Quantization errors that appear small in single-step relative L2 metrics may accumulate and affect long-term stability, conservation properties, or physical consistency. Although the paper reports divergence-related measurements, it does not evaluate actual multi-step rollouts, leaving an important practical question unanswered.
Questions for authors
- Could the authors clarify the main methodological differences between the proposed energy-based allocation and existing Hessian-aware or activation-aware mixed-precision quantization methods?
- How does the proposed approach perform on PDE problems whose energy spectra are not strongly concentrated at low frequencies, such as high-Reynolds-number turbulence, discontinuous solutions, or multiscale physical systems?
- Given that nonlinear activations introduce cross-frequency coupling, how robust is the proposed single-layer sensitivity approximation when multiple layers are quantized simultaneously? Could the authors investigate whether incorporating downstream error amplification or layer-wise sensitivity information improves allocation, particularly for the smaller turbulence model?
- Could the authors provide an implementation of the proposed frequency-dependent mixed-precision representation and evaluate its actual storage footprint, inference latency, and GPU memory consumption?
2 comments
Thank you for the detailed rebuttal and additional experiments. I appreciate the authors' efforts in addressing my concerns.
Thank you for the detailed review. We respond to each point below; the sections and tables cited are in the revised paper.
W1, Q1. No fundamentally new quantization mechanism; differences from Hessian-aware and activation-aware methods.
We agree that the paper introduces no new mechanism: the solver is the classical reverse water-filling solution. The contribution is the first study of post-training quantization for neural operators. It shows that uniform bit-widths degrade FNOs severely at two and three bits (Table 2), and that the Fourier structure gives a direct remedy: the weight at mode k multiplies only the input at mode k, so the output error of a layer separates exactly across modes, and the sensitivity of a band is its input energy, measured with forward passes alone.
We ran the Hessian-aware methods on all 20 checkpoints at equal storage (Table 1, Section 5; Appendix C). HAWQ assigns one bit-width per layer; it has 5.4× and 2.8× our error ratio at 3 and 4 bits, and at 2 bits it equals uniform bit-widths. The difference is the unit of allocation: bands, not layers. A Hessian trace per band ranks the bands as input energy does on three of four model groups, so the two estimate the same sensitivity, and input energy needs only forward passes. AWQ protects salient channels at one bit-width and does not allocate bit-widths.
W2, Q2. Spectra not concentrated at low frequencies; other architectures.
We trained nine models (width 32, three seeds each) on 3D Darcy flow with two-phase (discontinuous), multiscale and anisotropic coefficients, whose spectra are less concentrated at low frequencies (Section 6.2; Figure 6, Appendix H).
The advantage over uniform bit-widths holds (the paper reports 5.9× and 3.1× at 3 and 4 bits). The fitted power law in the last column is a two-parameter summary of the same measured spectrum. It comes close, but the measured spectrum is better on every seed. Other neural operator architectures are not covered, but the derivation applies wherever each retained Fourier mode has its own weight.
W3, Q3. Single-layer error model; downstream amplification.
We implemented your suggestion, and it helps most on the small turbulence model, as you anticipated. For every layer and band we measured how much an error in that band changes the network output relative to the layer output, and allocated with input energy times this gain (Section 6.3; Table 15, Appendix I). All results quantize all layers simultaneously. Error ratios:
In the small turbulence models later layers damp errors in the outer bands, so input energy alone assigns them too many bits, and the gain corrects this. A limitation remains: hard low-pass truncation is still better there (1.17 against 2.04 at 2 bits), as the Limitations state.
W4. Multi-step rollouts.
We ran rollouts on the 8 turbulence models: 72 unseen trajectories and 8 autoregressive steps (Appendix J). Width-32 models at 2 bits, relative L² distance from the FP32 rollout (Table 16):
Kinetic energy stays within 1% of the truth through step 8, whereas uniform bit-widths lose 69% of it. The quantized rollouts blow up less often than FP32 (0–6% of trajectories against 10–14%; we attribute this to damping by the pruned high-frequency bands, which we have not tested). Incompressibility is affected: the normalised divergence at step 1 is 0.036 against 0.013 for FP32. The width-8 FP32 models are themselves unstable over this horizon, so we draw no conclusion about quantization there.
Q4. Implementation, storage footprint, latency and GPU memory.
The submission reported one packed file. Appendix A now covers all three bit-widths, with pruned bands: at width 32 the file is 226.5 MB in FP32 and 14.2 / 21.3 / 28.4 MB at 2 / 3 / 4 bits, within 0.1% of the storage computed in the paper. The weights are unpacked at load time, so inference time and GPU memory are those of the FP32 model (Appendix A); the benefit is the size of the stored and transferred model. The packing code is in the supplementary material.