- The manuscript offers limited novelty relative to the existing literature on functional approximation using ReLU Neural Networks. Liu et al. [1] have shown how the decay of coordinate-dependent Fourier weights affects the tractability exponent and the resulting approximation rates of Fourier Functional Networks. Li et al. [2] have studies function spaces with fast coefficient decay and established approximation bounds involving the best s-term approximation error, with the decoder’s convergence rate depending on the sparsity level s rather than the dictionary size N. These results have already demonstrated that exploiting coordinate weights, coefficient decay, and sparse representations can improve neural-network approximation of nonlinear functionals. The authors should therefore compare their effective dimension D0, based on wdsd against these existing results and explain what additional approximation advantages their method would provide, if any.
[1] Liu, Peilin, Yuqing Liu, Xiang Zhou, and Ding-Xuan Zhou. ``Approximation of functionals on Korobov spaces with Fourier Functional Networks.'' Neural Networks 182 (2025): 106922. Neural Networks, 182:106922.
[2] Li, Jianfei, Shuo Huang, Han Feng, Ding-Xuan Zhou, and Gitta Kutyniok. ``Sparse-Aware Neural Networks for Nonlinear Functionals: Mitigating the Exponential Dependence on Dimension.'' arXiv preprint arXiv:2604.06774 (2026).
2 There is a potentially critical gap in the proof of Theorem 1. On Page 17, the proof chooses L through the balancing condition
(A.11) and, after setting ℓd=wd at line 870, obtains H(D0)=D0/L. It then derives the approximation bound using this relation. However, the proof does not establish that this balancing condition is compatible with the definition of D0 in Theorem 1.
- Minor issues:
Page 7: Theorem 1: For A=1, the defining set for D0 in Theorem 1 is empty when wdsd is nonincreasing.
Page 14: Line 707: Here D0 is different from D0 defined in Theorem 1. Their equivalence is not shown.
Page 14: The derivative bound used in (A.3) requires the segment Xθ+τ(PDX−Xθ) with τ∈[0,1] to lie in the domain where Assumption 2 holds; this is not established.
Page 14: In (A.4), it should be ∑d=D0+1Dwdsd≤H(D0).
Page 15: In Step 1, it requires Wψ,d2Mψ2=Ld, although a general integer Ld need not be a perfect square.
Page 15: In Step 2, ∏dLd=LD0∏dℓdsd ignores the fact that Ld=⌊Lℓdsd⌋.
Page 15: In (A.5)--(A.6), Lemma 2 is applied to approximate the coefficient maps Qd(s) and Q(0). However, Lemma 2 requires target values in [0,1], whereas these maps may take negative values or values greater than one. The required normalization is not explained.
Page 15: In (A.7), Lemma 4 is stated on [0,1]n, while the Taylor increments for d>D0 can be negative.
Page 16: The derivation of (A.9) assumes ∑d≥1wdsd<∞, but this condition is neither stated as an assumption nor established anywhere earlier in the paper.