- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Legendre Representation Networks (LERENs) to improve the representational capacity of physics-informed neural networks for high-frequency and structurally complex PDE solutions. Existing approaches for mitigating spectral bias in PINNs often rely on Fourier embeddings, periodic input encodings, or sinusoidal activations. LEREN offers an alternative by incorporating learnable Legendre polynomial expansions into the hidden layers of an MLP. Through multilayer composition, lower-degree polynomial components can generate higher-degree modes, which provides a non-Fourier mechanism for representing high-frequency structures. The method is evaluated on five PDE benchmarks, covering both sinusoidal or periodic problems and non-sinusoidal problems with high-dimensional or nonlinear dynamics. The reported results show that LEREN achieves the lowest errors among the compared methods across these benchmarks, demonstrating that Legendre-based representations can be competitive with existing Fourier-based approaches for PINNs.
Critical strengths
-
The proposed method is simple, yet it delivers substantial empirical gains across multiple PDE benchmarks with diverse characteristics.
-
The authors provide a relatively clean representation ablation that isolates the contribution of the Legendre expansion from parameter-count effects. This strengthens the evidence that the performance gains come from the proposed representation design.
Critical weaknesses
The universal approximation result has limited explanatory value. The universal approximation theorem is relatively weak in its current form. The proof effectively reduces LEREN to a standard tanh MLP by retaining only the first-order Legendre component. As a result, the theorem establishes expressivity, but it does not explain why the proposed Legendre representation should outperform conventional MLP-based PINNs in practice.
The optimization mechanism is not sufficiently understood. The paper shows that compositions of Legendre polynomials can generate higher-degree modes. However, this representational argument does not explain why gradient-based optimization can efficiently discover and exploit these modes during training. A deeper analysis of the optimization dynamics would strengthen the connection between the proposed representation and the observed empirical gains.
The central mechanism of higher-mode generation through composition is not directly validated. A central motivation of the paper is the following mechanism that depth leads to higher-mode generation. However, the experiments do not directly verify that deeper LEREN layers actually contain progressively higher-frequency or higher-degree components. The paper does not report the learned Legendre spectrum across layers, the Fourier spectrum of hidden features, the effective degree distribution, the learned coefficients , or the learned scaling parameters . At present, the main empirical evidence is essentially LEREN leads to lower final error. This result alone does not establish that the improvement comes from the proposed compositional mode-generation mechanism. A useful diagnostic would be to analyze the hidden representations at and project them onto a Fourier basis. The resulting spectra could reveal whether higher-frequency content actually emerges as depth increases. A complementary analysis could track the learned Legendre coefficients and effective polynomial degrees across layers. The existing ablation results make this question particularly important. Increasing the maximum degree from degrades performance, and increasing the depth from four to five layers also increases the error. The paper summarizes this observation by stating that an appropriate depth and polynomial degree are required. However, these results may also indicate a more fundamental issue: higher-degree composition may become poorly conditioned. This possibility creates tension with the simple narrative that generating higher-degree modes is inherently beneficial. Further analysis of conditioning, optimization stability, or derivative magnitudes would help clarify this point.
The architectural choices appear highly heuristic, and their generality is unclear. The default LEREN architecture uses in the first layer, while the subsequent three layers use only This is a highly specific design choice. The paper does not provide a clear theoretical or empirical explanation for excluding from the later layers while retaining and . It is also unclear why alternatives such as or an adaptive basis-selection mechanism would be less effective. Most of the degree and depth ablations are conducted on the convection benchmark. This makes it difficult to determine whether the selected architecture generalizes across PDE families or reflects benchmark-specific tuning. Additional ablations across several PDEs would provide stronger evidence for the robustness of these architectural choices.
The experimental scope is still limited. The evaluation covers only five PDE benchmarks. These experiments include several useful settings, but the overall scope remains relatively narrow for establishing broad effectiveness across challenging PDE systems. At minimum, I would like to see an additional benchmark with complexity comparable to a two-dimensional Navier–Stokes problem. Such an experiment would provide stronger evidence that LEREN remains effective on more complex nonlinear and multiscale dynamics.
The experiments do not establish that the advantage is specific to the Legendre basis. The current results show that polynomial-enriched hidden representations can be effective, but they do not clearly establish that Legendre polynomials are the critical ingredient. A particularly important missing comparison is AC-PKAN [1], which uses a Chebyshev Type-I polynomial basis for physics-informed PDE solving. Its Cheby1KAN layer can be written as , where and are learnable coefficients. LEREN uses and . The two constructions are conceptually close. Both combine normalization, an orthogonal polynomial basis, learnable polynomial coefficients, and deep composition. A direct comparison with AC-PKAN would therefore be important for clarifying whether the empirical advantage comes specifically from Legendre polynomials or from the broader design pattern of learnable orthogonal-polynomial representations.
[1] AC-PKAN: Attention-Enhanced and Chebyshev Polynomial-Based Physics-Informed Kolmogorov-Arnold Networks, TMLR 2026.
Questions for authors
Can the authors establish a more formal connection between the polynomial composition argument and the actual LEREN architecture? Equation 11 shows that composing Legendre polynomials can generate higher-degree Legendre modes. However, the actual LEREN architecture in Eqs. 13–14 does not directly compose Legendre polynomials. Each layer also contains an affine transformation and a tanh mapping before the next Legendre expansion. Can the authors clarify whether the higher-mode generation argument in Eq. 11 still applies to the full architecture? A theoretical characterization of how affine mixing and tanh transformations affect the compositional mode-generation mechanism would make the proposed motivation more convincing.
Please clarify the value of used in the Diffusion experiment. Section 3 derives the first-layer initialization Appendix D.6 also states that all experimental results use the first-layer odd-coefficient initialization . However, Table 6 reports the Diffusion result as with an RL2 error of , which corresponds to the Diffusion result reported in the main table. Which initialization was actually used for the reported Diffusion result? If was used, please explain why the theoretically motivated initialization was not adopted for this benchmark. It would also be important to clarify whether was selected through benchmark-specific tuning. This issue directly affects reproducibility and the claimed generality of the proposed initialization strategy.
Can the authors provide a frequency-scaling experiment? The current evidence for high-frequency approximation is mainly based on a fixed convection setting with and a fixed Helmholtz setting with . It would be useful to evaluate the method over a range of frequencies, for example and report how the errors of LEREN, Fourier-based baselines, and a standard tanh baseline change as the target frequency increases. If the paper aims to argue that LEREN alleviates high-frequency approximation difficulties, such a frequency-scaling experiment would provide substantially stronger evidence than evaluation at a single high-frequency operating point.
11 comments
W4. Choice and generality of the layerwise degree sets
We chose based on a simple denoising idea: retain the components needed to fit the signal while reducing unnecessary terms in each later layer. We keep two low-degree terms and two high-degree terms:
- provide the constant and linear terms.
- provide higher-degree changes. With maximum degree , these are the highest even and odd degrees, so we retain both types.
Further increases in depth and degree can make training harder by introducing more components that must be suppressed. For example, only needs two frequencies. If a network generates many additional components, it must learn to suppress them or cancel their contributions; unwanted components that remain cause fitting error. Figure 4 shows that four layers and maximum degree achieve the lowest error, while further increases raise the error.
We retain the full – set in the first layer because it acts directly on the input coordinates and provides a richer starting representation. Later layers combine these learned features using . All selected coefficients remain trainable.
The added ablation tests this choice on both convection and Helmholtz (Appendix F, Table 9). The sparse variant uses in every layer, and the full variant uses – in every layer. Relative errors are reported as Mean Std.
| Configuration | Convection | Helmholtz |
|---|---|---|
| Default | ||
| Sparse in all layers | ||
| Full in all layers |
Using all ten terms in every layer does not improve either benchmark. Using only four terms in every layer improves Helmholtz but worsens convection. We therefore retain the full first layer and sparse later layers as the shared default.
W5. A more challenging coupled-flow benchmark
To evaluate LEREN on more complex nonlinear flow with steep fronts, we added a one-dimensional viscous compressible Navier–Stokes shock-tube experiment (Appendix I, Table 13). The problem jointly predicts density, velocity, and pressure, with viscosity . We compare four methods over three seeds under approximately matched training-time budgets. LEREN achieves the lowest mean error and standard deviation, with a mean error 28.3% below JAX-PI.
| Method | Relative error (Mean Std.) |
|---|---|
| DGM | |
| JAX-PI | |
| IFeF-PINN | |
| LEREN |
W6. Comparison with AC-PKAN
We have added a direct comparison with the complete AC-PKAN method, retaining its Chebyshev basis and attention mechanism. Appendix H, Table 12 compares AC-PKAN, a B-spline KINN baseline, and LEREN on three benchmarks under matched training-time budgets, with each method retaining its own training strategy.
LEREN achieves lower mean errors than both KAN baselines on all three benchmarks. The relative errors are reported as Mean Std.
| Benchmark | AC-PKAN | KINN | LEREN |
|---|---|---|---|
| Convection | |||
| Burgers | |||
| Helmholtz |
LEREN learns Legendre representations within an MLP architecture. AC-PKAN learns polynomial coefficients on each edge, whereas LEREN first applies an affine transformation and then a Legendre combination with coefficients shared within each layer.
Q2. Initialization used in the diffusion experiment
is an old setting, which has been replaced by in our current default.
The current initialization follows the unit-slope condition . The added multi-seed evaluation in Appendix D uses across all five equations, including diffusion.
The five-benchmark comparison shows more stable results and better overall accuracy with . As reported in Appendix E, Table 8, it yields a lower error standard deviation on all five benchmarks and a lower mean error on four. Relative errors are reported as Mean Std.
| Benchmark | ||
|---|---|---|
| Convection | ||
| Burgers | ||
| Helmholtz | ||
| Heat-5D | ||
| Nonlinear diffusion |
Q3. Frequency-scaling experiments
To evaluate performance as the target frequency increases, we added the requested Helmholtz scan over and a convection scan over . Appendix G, Tables 10–11 compare LEREN, a standard MLP, and a fixed multiscale Fourier-feature MLP, all with four hidden layers of width . The Fourier projections remain fixed throughout each scan.
LEREN achieves the lowest mean error in most tested settings, demonstrating its ability to handle both high- and low-frequency solutions. The relative errors are reported as Mean Std.
Convection:
| MLP | Fourier-feature MLP | LEREN | |
|---|---|---|---|
| 40 | |||
| 80 | |||
| 100 | |||
| 200 |
Helmholtz:
| MLP | Fourier-feature MLP | LEREN | |
|---|---|---|---|
| 10 | |||
| 25 | |||
| 50 | |||
| 100 | |||
| 200 |
We thank the reviewer for the detailed feedback and suggestions. We have strengthened the theoretical analysis and added experiments, and address the related weaknesses (W) and questions (Q) together below.
To establish the connection between the actual layer and its Legendre representation, we have added Theorem 1 to the main text, with the full proof in Appendix N. Starting from the Legendre expansion of tanh(wx+b), the theorem expresses the complete layer in the Legendre basis of the input coordinate:
=\sum_{k=0}^{\infty}\left(\sum_{n=0}^{N}c_nT_{kn}(w,b)\right)P_k(x).$$ The coefficients $T_{kn}(w,b)$ are constructed using the Legendre recurrence, and the expansion is unique and converges absolutely and uniformly on $[-1,1]$. This gives an explicit connection between the trainable layer coefficients and the resulting Legendre coefficients, enabling composition analysis with the affine transformation and $\tanh$ included. Building on this representation, the revised universal-approximation argument retains a nonconstant Legendre combination, without setting the higher-degree terms to zero and reducing the model to a standard $\tanh$ network. The construction also extends to multidimensional inputs through tensor-product Legendre bases. We thank the reviewer for this suggestion, which helped us strengthen the theoretical foundation of our work. > W2. Learning higher-degree modes through gradient-based optimization Theorem 1 shows that the Legendre spectral coefficients of each layer are controlled by the network parameters $w$ and expansion coefficients $c$. Gradient updates to $w$ and $c$ therefore adjust these spectral coefficients, including those of higher-degree modes, allowing these modes to be learned from the PDE training loss. The component ablation in Appendix B, Table 5 supports the effectiveness of this joint optimization. We compare the $\tanh$ baseline, fixed Legendre coefficients, and learnable Legendre coefficients using the same four-layer backbone, trainable activation scales, and shared readout. All variants use Adam, seed $42$, and the same training budget within each task. Their relative $L_2$ errors are shown below. | Configuration | Convection | Helmholtz | | :--- | :--- | :--- | | $\tanh$ with the same backbone | $0.912$ | $0.247$ | | Fixed Legendre coefficients | $2.09\times10^{-4}$ | $2.35\times10^{-2}$ | | Full LEREN | **$9.33\times10^{-5}$** | **$7.10\times10^{-3}$** | With $c$ fixed, learning the network parameters already gives much lower errors than the $\tanh$ baseline. Jointly learning $w$ and $c$ further reduces the errors on both equations, demonstrating the benefit of learning the Legendre representation through gradient updates. > W3. Direct evidence of higher-degree modes in hidden representations To demonstrate that layer composition generates higher-degree modes, we projected the learned hidden features onto a Legendre basis layer by layer. On convection, we trained one- to five-layer models with width $256$ and maximum activation degree $9$, then projected their hidden features along time onto $P_n(2t-1)$ for $n=0,\ldots,128$. The projections show stronger high-degree components in deeper layers (Appendix J, Figure 6). In the four-layer model, the energy-weighted mean degrees increase from $1.64$ to $3.08$, $7.16$, and $12.29$, and the later layers contain more energy around degree $64$.