- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Spectral Allocation and Routing (SAR) for physics-informed neural networks (PINNs), an adaptive mechanism that learns a global set of Fourier frequencies and controls their local contributions through continuous, frequency-conditioned spatial routing. Common Fourier feature embedding methods use prior bandwidth to fix the spectrum of the network before training, the network cannot adjust this bandwidth during training, which may lead to wasted high-frequency features. In contrast, SAR jointly learns trainable frequencies and the weights assigned to them. The paper also includes an analysis of the network’s neural tangent kernel (NTK). Together, these are intended to demonstrate that the proposed method mitigates the spectral bias of the network.
Critical strengths
- The proposed SAR introduces a joint formulation that couples globally trainable Fourier frequencies with continuous spatial-frequency routing, which enhances the effectiveness of Fourier feature embedding.
- The paper introduces optional geometric supports that organize groups of frequencies within a shared spectral structure.
- The paper uses NTK analysis to examine the architecture of the proposed method, which provides some interpretability regarding the mitigation of spectral bias.
Critical weaknesses
- Unlike the analysis given in Tancik et al. (2020), the NTK analysis in this paper does not examine the eigenvalue decay of the network’s NTK. Instead, the analysis is based only on relatively simple one-dimensional linear regression problem, which does not explain how SAR improves the NTK of the MLP during training process.
- The presentation of the method is not very clear. The content of Section 3.2 is not well supported by the background material presented earlier in the paper.
- The benchmarks and baselines are limited in this paper, with only two-dimensional heterogeneous Helmholtz problems under manufactured solutions. No other popular PINN benchmarks are considered (there are a lot popular benchmarks regarding the spectral bias and high-frequency problem). The choice of baselines and the comparison also limit the conclusions.
Questions for authors
- In Section 3.2, could the authors explain why the RBF formulation in frequency space was adopted for the Helmholtz problems instead of the routers studied in Section 4.2? More generally, how should one choose among these router designs and their hyperparameters for more experiments?
- Regarding Weakness 3, could the authors evaluate SAR and the baselines on a broader set of problems? It could include many commonly used PINN benchmarks. Including similar recent methods identified in the related work.
- The problem setting in Section 3.1 considers only spatial coordinates and steady-state equations. Can SAR be applied to time-dependent PDEs? Would time be treated as an additional input dimension with space-time wavevectors and space-time routing, or would temporal frequencies be handled separately?
Overall, the SAR method seems to be an extension based on the Fourier feature embedding. Therefore, it's good to see the authors can provide more general applicability of SAR similar to the Fourier feature methods.