- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes the Separable RWKV (S-RWKV) architecture, based on separable physics-informed neural networks (separable PINNs), whose branches apply a smooth matrix-memory update to groups of learned Fourier features. Starting from the question of what a separable PINN needs from a recurrent block, the paper conducts extensive experimental comparisons and ablation studies and concludes that (i) accurate approximation can be achieved without recurrent memory, (ii) channel mixing improves training reliability, and (iii) under the shared pipeline, the memory-free model has a smaller model size while achieving lower error.
Critical strengths
- The proposed S-RWKV architecture introduce a clean network architecture, clearly separates the channel-mixing path from the recurrent path.
- The experiment section in this paper consists of ten random seeds for each configuration (e.g. all float64, same training setting), with the results described in detail, and all separable variants share the same experimental setup.
- The experiment results show that a memory-free grouped channel-mixing branch can outperform more complex recurrent/attention-based PINNs, especially on Reaction, Convection, and Wave equations.
- The ablation study in this paper is systematic. With the four branch configurations (full, one block, no memory, no blocks), the paper includes a SPINN control that differs from S-RWKV only in the branch body, a parameter-matched non-separable control , and an input–assembly–body ablation.
Critical weaknesses
- The method performs very well on benchmarks with low-rank/separable structure. But on some non-separable problems, such as the NS equation benchmark, the method is a little bit weak.
- The no-memory branch proposed by the paper overlaps with the multi-scale Fourier feature (MFF) architecture of Wang et al. (2021b). In the algorithm, each coordinate has its own branch that maps groups of learned Fourier features. The relationship to (spatio-temporal) MFF should be discussed.
Questions for authors
- All experiments in this paper use float64 data, but now more PINNs paper are using float32 data instead. Could the authors report some float32 results?
- How important is learning the Fourier frequencies and phases? Could the authors report results with and fixed at their initialization, as in MFF (Wang et al., 2021b)? This would clarify whether learnable frequencies are different from MFF.
- Since S-RWKV employs learned Fourier features, the baseline should include some comparisons with recent Fourier-based method, such as [1].
- is a fixed sine–cosine combination, and SPINN and No memory differ in activation, weight sharing, and residual scaling. Is the large Convection gap vs SPINN attributable mainly to the activation function, to groupwise weight sharing, or to residual scaling?
[1] Y. Wu, M. Aguiar, K. H. Johansson, and M. Barreau. Iterative training of physics-informed neural networks with Fourier-enhanced features. In The Fourteenth International Conference on Learning Representations (ICLR), 2026.