- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates a targeted architectural question for Fourier Neural Operators (FNOs): whether low-dimensional system parameters should also participate in constructing spatial features, not just in input channels or by modulating Fourier layers. The authors propose a parameter-conditioned sinusoidal coordinate network (a modulated SIREN) that produces configuration-dependent spatial feature maps, which are then processed by FiLM-conditioned Fourier blocks. Through controlled heat-transport problems (translating vs expanding material rings) and an industrial radio-frequency hollow cathode discharge (HCD) application, they show that learning spatial features is beneficial over standard FNOs, and conditioning these features yields additional gains, especially when geometry translates. On HCD (fixed geometry, varying operating parameters), their shared-embedding variant achieves the lowest errors across four coupled physical fields compared with FNO, Transolver, and DeepONet.
Critical strengths
The paper uses a principled conditioning point: parameters modulate the coordinate-based spatial representation before Fourier processing, complementing standard FiLM conditioning in FNO layers. The architectural decomposition is clear between “spatial feature construction” and “nonlocal mixing” (Fourier blocks), enabling controlled ablations via a static-spatial variant. The method also represents a simple and general mechanism (amplitude and phase modulation of SIREN layers) that can be plugged into operator-learning backbones. The motivation (“where to condition?”) is well articulated, and the architecture and training setup are described with sufficient detail.
Critical weaknesses
- The technical novelty is very incremental in that conditional neural representations and FiLM conditioning are established, and several geometry-aware operator methods exist. The core innovation is the careful placement of conditioning rather than a fundamentally new operator class.
- No theoretical analysis or principled criteria for when spatial conditioning helps, beyond empirical differentiation between translation and expansion.
- Comparisons miss some relevant geometry-/representation-aware variants (e.g., Geo-FNO/DAFNO/GINO/ACT, hpFNO/HyperFNO); while not all are strictly necessary for the fixed-grid setting, such baselines could contextualize the gains more broadly.
- Extrapolation and OOD robustness (e.g., beyond the trained geometry/parameter ranges) are not evaluated.
- Computational/runtime overhead of the added SIREN branch and modulation is not reported. the scaling implications are also unclear.
Questions for authors
- How does the approach perform under parameter/geometry extrapolation (e.g., translation or expansion beyond the training interval) and under sparser training coverage? Any signs of overfitting or failure modes?
- Can you provide runtime and memory overhead of the conditioned SIREN branch relative to FNO/FiLM FNO, and scaling considerations for higher resolutions or 3D?
- What is the sensitivity to SIREN hyperparameters (depth/width/frequency multiplier) and to the choice of positional encoding (pure coordinates vs Fourier features)? Are phase vs amplitude modulations equally important?
- In the heat-transport setup, what is the contribution of Fourier-layer FiLM vs spatial SIREN conditioning? An ablation that removes operator FiLM (keeping spatial conditioning) and vice versa would help quantify their roles.
- How would your method compare to coordinate-adaptation (e.g., ACT) or learned-deformation FNOs (Geo-FNO) and DAFNO on the same translating/expanding ring tasks? Do you expect complementary gains if combined?
- On HCD, did you try training a single shared model across multiple geometries (e.g., altering bore radius or electrode segments) to test whether spatial conditioning scales to modest geometry changes?
- Are the localized metrics (transition/hotspot) robust to threshold choices (e.g., different occupancy bands or top-k% regions)? A sensitivity analysis would strengthen conclusions.
- Could you share per-field loss contributions and qualitative failure cases on HCD (e.g., near electrode edges) to better understand where improvements are concentrated?
2 comments
Part TWO
3. SIREN settings and modulation
Depth, width, frequency and encoding sensitivity studies, and amplitude-only/phase-only ablations, remain incomplete. The ring branch uses three width-16 sine layers, frequency multiplier one and periodic coordinates; HCD uses three width-128 layers, frequency multiplier 30, normalized coordinates with Gaussian Fourier features, and a shared 128-dimensional condition embedding.
The capacity-matched conditional SIREN control, without Fourier mixing, is less accurate on both ring tasks. It tests mixing, rather than the relative importance of amplitude and phase or the optimal SIREN settings (Appendices A.1, A.2 and F.2).
4. Spatial conditioning versus Fourier FiLM
The 2×2 translation experiment gives every model identical direct inputs, including all nine parameters, and matches data and training. Global scaled MSE (× 10⁻⁵) is:
| Spatial features | No Fourier FiLM | All-nine-parameter Fourier FiLM |
|---|---|---|
| Static | 11.581 ± 1.502 | 9.811 ± 1.269 |
| Conditioned | 3.664 ± 0.763 | 4.435 ± 0.882 |
Spatial conditioning reduces MSE by 68.57 ± 2.63% without Fourier FiLM and 54.49 ± 9.30% with it, improving every run. The latter pair has 117,058 real parameters each; the no-FiLM pair has 114,994 for Static and 117,058 for Conditioned.
Adding Fourier FiLM to Conditioned instead increases MSE by 22.71 ± 24.61% in aggregate and worsens every run. Conditioned without Fourier FiLM has the lowest mean error. Because the combined model shares its encoder and modulation heads between routes, the cause of this adverse interaction remains unresolved (Section 5.3, Table 2; Appendix F.1, Tables 9–10).
5. ACT and other geometry-aware baselines
We add the author's ACT blocks after each FiLM FNO block, without a SIREN branch. All models use all-nine-parameter Fourier conditioning and matched data, preprocessing, optimization, 32,000 updates and validation selection. Translation reuses the factorial Static/Joint controls; expansion uses the original single-source task. Global scaled MSE from two validated seeds is:
| Task | FiLM FNO | Static | Joint | ACT+FiLM |
|---|---|---|---|---|
| Translation MSE, × 10⁻⁵ | 10.709 ± 4.260 | 10.476 ± 0.749 | 4.386 ± 1.241 | 3.846 ± 0.044 |
| Expansion MSE, × 10⁻⁶ | 4.277 ± 1.500 | 4.491 ± 2.375 | 4.007 ± 2.289 | 0.809 ± 0.163 |
ACT and Joint each win one translation run. On expansion, ACT improves all four main field metrics over Joint on every test case in both runs. Joint improves all five field/response metrics over Static on both tasks.
These gains do not extend to every metric. Joint sometimes has larger peak-position errors than Static, and in one expansion run it has higher global, physical and transition errors than FiLM FNO. ACT also does not improve peak-value or peak-position errors on every expansion case.
ACT has 131,040 real parameters versus Joint's 117,058 and uses about 17.4× the fitting time and 5.45× the peak memory in this author-block adaptation with deterministic sampling. Updates are matched, not computing time. The third seed remains unvalidated (Appendix I.1, Tables 22–23, Figure 17; Appendix J, Table 26).
Joint constructs input features; ACT resamples intermediate features. Their combination remains untested, as do direct Geo-FNO, DAFNO and GINO comparisons. Our spatial-shift control adapts the official hpFNOx blockwise hyper-shift to two dimensions, rather than benchmarking the full hpFNO/HyperFNO family.
Joint consistently outperforms this shift control on translation. Expansion is mixed: the shift control has lower global, physical, transition and response errors in one run, while Joint consistently has lower hotspot and peak errors.
6. HCD across device geometries
HCD uses one fixed device geometry with varying pressure, RF voltage and SEEC. A shared model across bore radii will be added if the paper is accepted as agreed before.
7. Threshold sensitivity
We rescored saved predictions using occupancy bands (0.05,0.95), (0.1,0.9), (0.2,0.8), (0.3,0.7) and reference hotspot fractions 1%, 2.5%, 5%, 10%. The matched all-parameter Conditioned model improves on Static for all eight choices in every run: 40.65–65.20% lower transition MSE and 48.31–68.52% lower hotspot MSE. Masks are evaluation-only; thresholds are not independent training repetitions (Appendix H.2, Table 21).
8. HCD field losses and failures
Table 4 reports normalized-space test relative L², not contributions to training loss. Over five seeds, Ours gives 3.246 ± 0.370, 2.313 ± 0.190, 2.092 ± 0.160 and 2.491 ± 0.100 (× 10⁻³) for potential, electrons, Ar⁺ and Ar*, respectively.
The field loss averages four standardized-space relative L² terms and adds 0.1 times joint relative Frobenius error, with 10⁻⁸ in denominators. Training also adds a 0.25-weighted radial adjacent-grid penalty on the predicted metastable field; validation excludes it.
Dear reviewer g7Gy,
Thank you for the detailed questions. We have added ablations, generalization tests, and cost analyses, and included ACT—which we agree is interesting concurrent work—in both our empirical comparisons and related-work discussion. We also agree that HCD as a important and practical problem deserves more detailed discussion, we agree to expand a varying geometry setup for HCD if the paper is accepted --- we need nontrivial time to generate the data, due to limited time and access it is not possible in the current rebuttal period.
Part ONE
The contribution: a spatial-conditioning principle
Our goal is to demonstrate spatial conditioning as a design principle, rather than introduce a new neural-operator family: system parameters can help construct the spatial representation before information is mixed across locations. FNO is our main example. To test the same principle with a different mixing mechanism, we also add static and conditioned spatial branches to Transolver, leaving its Physics-Attention blocks unchanged.
Unless stated otherwise, results are mean ± sample SD over three training seeds on fixed splits; percentage reductions are averaged over matched seed pairs. ACT uses two seeds and HCD uses five.
For Transolver at 2,000 updates, global scaled MSE (× 10⁻³) is:
Conditioning improves all four field-error metrics over the static branch in every run, with 2,064 additional parameters. This supports the principle in both Fourier and attention-based backbones under the tested settings. The separate budget permits a within-Transolver comparison, not a ranking against the 32,000-update FNO results (Appendix I.2, Table 24).
A theory predicting when conditioning helps remains open. The results also go beyond a simple translation/expansion distinction: retaining translation's two-source forcing and diffusivity during expansion reduces Static-to-Joint MSE from 4.336 ± 0.774 to 3.193 ± 0.816 (× 10⁻⁵), with consistent global, physical, transition and response gains. Original single-source expansion is less consistent, indicating that forcing and material properties also matter (Appendix F.3, Tables 12–13).
1. Extrapolation and sparse coverage
We train and validate on ring positions s ∈ [0.1, 0.9], then test 128 in-range positions and 32 on each side. Models share cases, training-only normalization, 32,000 updates and validation selection. Global scaled MSE is:
Our OOD MSE is 59.24 ± 16.85% lower than Static, with both tail means improving in every run. This test retains the original Static/Ours Fourier conditioning on propagation and coupling parameters, separately from the all-nine-parameter control below.
Our OOD MSE remains 157–250 times our own in-range MSE. In one run, Static wins on 10/32 positions just above s = 0.9 despite our lower upper-tail mean; our pooled peak-value error is also higher, at 7.57% versus 5.10% (Appendix G.1, Tables 14–16, Figure 12).
With 16 training geometries, physical relative L² (× 10⁻³) is 1.791 ± 0.614 for Ours, versus 3.156 ± 0.365 for FiLM FNO and 3.217 ± 0.455 for Static. All five field/response metrics improve over both controls in every run with either 16 or 32 training geometries. Removing s ∈ (0.4, 0.6) from training and validation gives mixed results: physical/response gains persist, but Static wins global, transition and hotspot MSE in one run. This interpolation gap reveals sensitivity to missing coverage without establishing overfitting as its cause (Appendix G.2, Tables 17–18).
2. Runtime, memory and scaling
At 256×256, batch 16 and 32,000 updates on DGX Spark, spatial conditioning adds 6.34 ± 0.67% fitting time and 7.05% peak memory over the matched all-parameter Static model. In the separate extrapolation experiment, the complete Ours model adds 27.73 ± 0.35% time and 25.49% memory over FiLM FNO. Time includes validation, checkpoint saving and replay; memory measures peak PyTorch allocation (Appendix J, Table 25).
Higher-resolution and 3D scaling remain unmeasured. The Fourier-mode study changes bandwidth on the same grid; at 24×24 modes, Joint improves all five field/response metrics in every run (Appendix G.3, Table 19).