- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper investigates the relationship between nominal frequency hyperparameters and the spectrum realized by trained sinusoidal neural fields. The authors define effective bandwidth and systematically examine its behavior across frequency settings, network depths, architectures, and datasets. The paper presents three main findings: (1) the effective bandwidths of near-optimal configurations remain relatively stable across depths; (2) increasing network depth can amplify spectral expansion; and (3) a target-conditioned frequency rule achieves low regret across several held-out settings.
Critical strengths
-
The paper provides a systematic empirical analysis showing that the effective bandwidths of near-optimal configurations remain relatively stable across network depths. The depth-amplified spectral expansion results and layerwise analyses provide useful intuition for understanding this phenomenon.
-
The experimental setup is well-designed and clearly described.
Critical weaknesses
-
While the authors acknowledge relevant prior work on harmonic generation and spectral control, its findings remain somewhat incremental. A clearer distinction from existing spectral analyses and a stronger discussion of the conceptual contributions would strengthen the paper.
-
The study is primarily empirical, with limited explanation of the mechanisms behind the stability of near-optimal effective bandwidths and depth-amplified spectral expansion. Stronger connections to existing theoretical results would help explain these observations.
Questions for authors
-
Given that the paper aims to study sinusoidal neural fields in general, what is the motivation for focusing on the frequency-bank architecture in the primary analyses? In particular, the sharp spectral expansion crossover appears more pronounced in the frequency-bank network than in standard SIREN, raising questions about whether this behavior is a general property of sinusoidal neural fields.
-
Several key components of the paper rely on heuristic or empirically determined choices, including the 95% spectral-energy threshold used to define effective bandwidth, the 2$\times$ criterion for identifying the expansion boundary, and the calibrated ratio (). Could the authors clarify how sensitive the main conclusions are to these choices? In particular, what motivates the 95% threshold, and how well is the fixed bandwidth ratio expected to generalize beyond the evaluated settings?
-
The paper focuses exclusively on sinusoidal neural fields, and it remains unclear whether the findings generalize to other non-sinusoidal neural field architectures, such as Gaussian, or WIRE. Do the observed spectral behaviors and calibration rule extend to non-sinusoidal neural fields? Although the authors acknowledge this limitation, experiments with additional neural field architectures would help establish the applicability of the observed spectral behavior and calibration rule.
-
The experiments are limited to 2D image regression at a fixed resolution. How well do the findings transfer beyond fixed-resolution 2D image regression, particularly to 3D NeRF or PDE solving? Are the observations consistent across different image resolutions and training-coordinate sampling ratios?
2 comments
Thank you to the authors for the detailed rebuttal. I appreciate the clarifications regarding the architectural differences and sensitivity analyses, which address some of my concerns.
However, my main concerns still remain. While the empirical findings are well presented, the conceptual novelty over existing spectral analyses is still limited. Moreover, the underlying mechanisms remain insufficiently explained, and the generalizability beyond the evaluated settings is not clear.
Overall, the rebuttal clarifies several points but does not substantially change my assessment of the paper's contributions.
Response to Reviewer mNnm
e thank the reviewer for describing the analysis as systematic, the layerwise analyses as providing "useful intuition", and the setup as "well-designed and clearly described". We answer the questions in order and then the two weaknesses.
We used the bank network as the main case because the knob of this network is an explicit frequency set that can be frozen, and SIREN reproduces the general findings. In the bank network we compare a frozen arm with a free arm. Their test PSNR differs by at most 0.30 dB, and the free frequencies move by 0.34% on average (Section 3.5). The measured bandwidth therefore comes from training with fixed frequencies. SIREN shows the same depth trend. Its optimal ω₀ moves from 16 to 12, 8 and 4 for L = 2 to 5, its near-optimal values span 4.0× in ω₀ against 2.15× in B_eff, and the selector transfers to it with a mean regret of 0.52 dB. The sharpness does not carry over. The largest adjacent B_eff ratio exceeds 20× for the bank and stays between 2.0 and 3.0 for SIREN (Section 5.2). In our data the depth shift and the concentration in B_eff hold in both architectures, and the localized crossover is specific to the bank.
The selections change little under these choices, and Appendix D reports each case.
Both are outside the present study, and Section 7 lists them as open. B_eff is defined on the output of any network sampled on a dense grid, and it applies unchanged to Gaussian or WIRE activations and to 3D fields. We have not run these cases. All results use 2D images at 128 × 128 with 25% training coordinates. Five different 25% splits keep the L = 3 regret at 0.00 dB and give 0.27 to 0.54 dB at L = 5 (Appendix D). Other sampling ratios and resolutions remain untested.
Prior theory explains why composition expands the spectrum, and the paper measures what that theory leaves open. Yüce et al. (2022) and TUNER show that depth generates harmonics and bound the spectrum. They do not give the realized band of the good models or the depth at which the expansion turns steep in trained networks. The layerwise measurements support a cascade-like amplification account, with late-layer amplification in 32 of 32 past-boundary pairs and in 0 of 32 before the boundary (Section 5.3). They do not establish the mechanism, and the next version will connect them to the expansion of TUNER.
We hope these answers help, and we would be glad to discuss further.