- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies approximation of spectral projectors associated with parameter-dependent operators , using predictors of the form . Although spectral projectors can vary continuously, it is known that topological obstructions can prevent their exact representation by continuous factors with too few columns. The paper aims to quantify the resulting approximation error and its sensitivity to distribution shift.
In the canonical setting with Haar measure , it considers Lipschitz predictors approximating the orthogonal projector . For , with , the authors establish sharp rates:
\mathbb E_\mu\|WW^\top-P\|_{\mathrm{op}}^2 \asymp L^{-r},$$ and $$\inf_{\operatorname{Lip}(W)\le L, \sup_{x\in X}\|W(x)\|_F\le\sqrt{k}} \sup_{W_p(\mu,\nu)\le\varepsilon} \mathbb E_\nu\|WW^\top-P\|_{\mathrm{op}}^2 \asymp L^{-r}+\varepsilon^{pr/(p+r)}.$$ The results combine topological lower bounds with matching constructive upper bounds, and are supplemented by extensions to density-constrained shifts and numerical experiments on neural-operator approximations of spectral problems.Critical strengths
-
The paper provides a rigorous quantitative analysis of a known topological obstruction to continuous spectral-projector factorization.
-
The matching upper and lower bounds establish sharp approximation rates, with an explicit dependence on the output width.
-
The extension to Wasserstein distribution shift is mathematically interesting, yielding sharp robustness rates and additional results under density-ratio constraints.
Critical weaknesses
-
Restrictive distributional assumptions. The sharp rates rely on Haar measure or distributions with densities uniformly comparable to it. It is unclear whether similar results hold for more general distributions, particularly those supported on lower-dimensional subsets of the Grassmannian.
-
Novelty of the techniques. The analysis builds on well-known topological obstructions and geometric arguments. The paper could better clarify which technical ingredients are genuinely new.
-
Dimension dependence. Rates are sharp only for fixed ambient dimension , and the dependence of the constants on is not sufficiently characterized. This could be a significant limitation given that the motivating applications involve high-dimensional operators.
-
Presentation. Main body is overly dense and lacks pedagogical explanations. The intuition behind the results, the role of the assumptions, and the progression of the arguments are unnecessarily difficult to follow.
Questions for authors
-
To what extent do the rates depend on the Haar measure assumption? Can analogous results be established for distributions supported on lower-dimensional subsets of the Grassmannian?
-
What are the main new mathematical ingredients needed to derive the sharp rates, beyond the qualitative topological obstruction discussed in prior work such as Esmaeili et al. (2023)?
-
Can the authors characterize the dependence of the constants in their upper and lower bounds on the ambient dimension ? In particular, do the rates remain informative as grows?
-
Can the theoretical insights be used to design practical architectures that avoid or mitigate the identified topological obstruction, while retaining the computational advantages of low-rank factorization?
I would like to point out that I am willing to raise the score to weak accept if the presentation is significantly improved, and possibly further pending on the good answer to questions 3 and 4 above.
4 comments
Summary of the revised manuscript
We thank the reviewers for their detailed comments. We have uploaded a revised manuscript that preserves the original research question and the main approximation and Wasserstein-shift rates, while substantially clarifying and expanding their assumptions, scope, proofs, and practical implications.
The revised PDF has 42 pages rather than 35. The main technical text remains within 9 pages. The additional pages occur after the main text and primarily contain expanded proofs, ambient-dimension analysis, training protocols, and reproducibility documentation.
The principal revisions are:
-
Problem and presentation. Sections 1--3 now define the parameter-to-projector learning problem, the factor predictor, the training labels and loss, the distinction between projector continuity and factor continuity, and the exact meaning of paired FNO output-head comparisons before presenting the results.
-
Theoretical scope. Theorems 1--2 now state the quantifiers and admissible factor class explicitly. We distinguish lower bounds that hold for every admissible factor under the stated assumptions, constructive upper bounds attained by explicit factors, and architecture-specific claims about trained neural networks.
-
Distributional assumptions. Appendix C gives extensions to comparable densities and specified lower-dimensional projective families. It also gives a counterexample showing that matching lower rates cannot hold for arbitrary input laws.
-
Ambient-dimension dependence. Theorem 14 and Appendix C.2--C.3 give dimension-uniform bounds whose comparison constants are independent of the ambient dimension when the ranks are fixed. They identify the relevant ambient-dimension scales and state precisely when the optimal errors vanish as the ambient dimension grows.
-
Practical architecture design. Corollary 3 connects the theory to frozen neural-operator output heads. The revised experiments compare output width with hidden capacity and report paired-seed controls, initialization and expansion controls, prediction time, memory, and factor-storage costs.
-
Reproducibility. Appendix F specifies the inputs, reference solvers, architectures, losses, normalizations, seeds, training schedules, checkpoint selection, and evaluation distributions. We also provide an anonymous reproducibility archive containing code, configurations, per-seed results, diagnostic scripts, retained checkpoints, and verification instructions.
Our individual responses identify the theorem, section, figure, table, and appendix addressing each reviewer question. Copyediting, notation alignment, and proof reorganization are visible in the automatic PDF diff but are not listed individually here.
Response to reviewer C4jA
Thank you again for the constructive review. Since posting our original response, we have uploaded a further revised manuscript focused specifically on presentation and readability.
The new version substantially reorganizes the exposition while preserving the research question and main rate statements. The Introduction now starts from the parameter-to-projector learning task and its computational motivation, followed by a concrete sign-reversal example, the geometric obstruction, the sharp rates, and their architecture implications. Figures 1 and 2 provide visual overviews of the learning problem and the rate mechanism.
The main argument is now organized as follows: Section 2 defines the admissible factor class, reference risk, Wasserstein distance, shifted risk, and theorem assumptions before stating the canonical results; Section 3 distinguishes contractible parameter domains from obstructed mode loops and frozen neural-operator output heads; and Section 4.1 defines the labels, training loss, shifted evaluation, and paired-FNO-head protocol before presenting the experiments. Detailed proofs, dimension-dependence arguments, and training protocols are moved to Appendices A--G, leaving the main technical narrative within nine pages.
We have also aligned the results emphasized in Questions 3 and 4 with this new organization. The ambient-dimension result is now Theorem C.2, with its matching proof in Appendices C.2--C.3 and the fixed-intrinsic-family result in Proposition C.1. The practical frozen-head implication is now Corollary 3.5 and is evaluated in Sections 4.1--4.3, particularly Experiment 5.
We hope this further revision directly addresses the presentation concern as well as the requested ambient-dimension and practical-architecture questions.
I thank the authors for their careful response, which I have read in detail. I found the new dimension-dependent analysis (Theorem 14) a genuinely interesting addition. However, the main proof ingredients seem standard, and the implications for learning (estimation error, sample complexity) are left unexplored. I therefore keep my score unchanged.
Response to Reviewer C4jA
Thank you for the constructive review and for indicating what could change your assessment. We reorganized the Introduction around the learning task, sign-reversal example, mechanism, and rates. Section 3 now defines labels, loss, Error, and paired heads before the results; technical and training details are in the appendices.
Q1. To what extent do the rates depend on the Haar measure assumption? Can analogous results be established for distributions supported on lower-dimensional subsets of the Grassmannian?
A1. The lower rates require both obstructed geometry and sufficient local mass; they are not universal over all laws. If dμ=fdμHaar with 0<c≤f≤C, the exponents are unchanged, with constants depending on c,C. For lower-dimensional families, an embedded RPd−1 has rank-one exponent r=m for m<d, while a sign-reversing loop has r=1 at width one and exact width two. Conversely, Appendix C gives a full-support density for which the Haar exponent fails, and a contractible domain where width k is exact. Thus support dimension alone is insufficient; Appendix C.1 states the positive extensions and counterexample.
Q2. What are the main new mathematical ingredients needed to derive the sharp rates....
A2. The contribution is the sharp subthreshold approximation and robustness law, beyond the qualitative obstruction. Four ingredients give the matching rates: (i) averaging unavoidable unit-error points over rotated projective r-slices yields the sharp L−r mass; (ii) clipped polar factors and small-singular-value estimates attain the upper order; (iii) transporting mass tr over distance t yields exponent pr/(p+r), while a capped coupling gives the M-dependent crossover; and (iv) an equivariant partition and concentration estimates expose the N scale. The revised related work and Appendix A separate these results from prior obstruction and exact-representation results.
Q3. Can the authors characterize the dependence of the constants in their upper and lower bounds on the ambient dimension N? In particular, do the rates remain informative as N grows?
A3. Theorem 14 now characterizes this dependence. For full Haar sampling on Gr(k,N), fixed k,m,p, r=m−k+1, and finite p, set a=(N/L)r,b=(Nε)pr/(p+r). For every N>m, the optimal reference error is of order min(1,a), with comparison constants depending only on k,m. The optimal density-capped shifted error is of order min(1,Ma,a+b), with comparison constants depending only on k,m,p. For M=∞, the term Ma is omitted. The comparison constants are independent of N: they depend only on k,m for reference error and on k,m,p for shifted error. Therefore the reference error vanishes as N→∞ exactly when LN/N→∞. The unrestricted shifted error vanishes exactly when this holds and NεN→0. Appendix C.2--C.3 proves the matching bounds and coefficient scale Nr/2. Proposition 13 separately proves that a fixed intrinsic family embedded isometrically into increasing ambient spaces has unchanged norm-bounded optimal error. Thus the N growth is specific to full-Haar sampling, not merely to extra coordinates.
Q4. Can the theoretical insights be used to design practical architectures ...
A4. The theory suggests changing factor-output width rather than indiscriminately increasing hidden capacity. For a verified intrinsic family, use the smallest width that removes its obstruction and allow dependent columns, so WW⊤ remains low rank. For a sign-reversing rank-one loop, two columns give W(ξ)=u(ξ)[cos(πξ/T),sin(πξ/T)] and WW⊤=P. This motivates a two-column pointwise head on a frozen neural-operator backbone. It retains O(Nm) factor storage and application cost and needs no test-time eigensolve. Corollary 3 formalizes this on Y×RPd−1: every m<d continuous norm-bounded head obeys the predicted lower rates independently of ambient N and backbone size, while width d is exact.
Experiment 5 tests the intervention. Across five paired plate seeds with identical frozen features, changing only the head from one to two columns adds 25 parameters and reduces median shifted Error from 0.152 to 0.00854. Increasing hidden capacity at width one leaves shifted Error at 0.145--0.147. Prediction-plus-application time is 3.543/3.382 ms per batch of 32, with equal measured peak memory; factor storage changes from 0.5625 to 1.125 KiB per sample. The Darcy control also finds wide initialization best in all ten matched comparisons, with small nonzero expansion providing a recovery path. These experiments support the design prescription; the theorems provide the class-level guarantee.