- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper under review discusses the minimal width for universal approximation in Sobolev spaces and find some priorly unknown relationships between the index and the minimum width required. This is in part due to topological effects in combination with Sobolev embedding theorems which say that for sufficiently high and thus topological restictions for continuous functions materialize. Going beyond this single observation, the authors indentify a 'phase diagram' with four regions.
Extensive, AI supported poofs are given in the appendix and explanatory sketches are given in the main text.
Critical strengths
- The topic of universal approximation is at the core of neural network theory and the simultaneous approximation of derivatives and the function itself is of interest for deeper studies, e.g., of NeuralODE where the densities are expressed via Liouville's formula by the divergence of the vector field.
- The topological effects underlying the found structure are studies in detail, are nicely explained and are definitively interesting.
Critical weaknesses
- As the main idea to the proof is an AI assisted mathematical technique, the paper has a special challenge in distinguishing itself from the mathematical proofs as a commodity thing. Here the easiest way would be to argue better why the found results embed themselves into a bigger theory picture, e.g. by relating the findings to the statistical learning theory of neural ODE or FlowMatching. This is however quite indirect and not really visible. In particular, the -results do not easily help in this regard, as they might be too week to guarantee a quantitative analysis. Also, the improvement over the Results of Hwang et al (2025) seems to be gradual.
- I acknowledge the use of AI declaration as absolutely professional. But the human checking of the proofs, which are not too detailed, can be executed on various levels. Formal checking would have been a plus, as on the grade of details the proofs have been given, LLM inaccuracies are not easy to detect.
Questions for authors
- Where do you see the larger theoretical context of your result?
- How do you compare your results to Denis Belomestny, Alexey Naumov, Nikita Puchkin, Sergey Samsonov, Simultaneous approximation of a smooth function and its derivatives by deep neural networks with piecewise-polynomial activations, Neural Networks, Volume 161, 2023, Pages 242-253.
4 comments
C5. Human checking of the proofs and formal checking.
R5. We thank the reviewer for acknowledging our AI use statement. We agree that the correctness of proofs is essential, and we would like to clarify the following points.
- Level of detail. We provide complete proofs in Appendices A–C (about 20 pages) and proof sketches of the main ideas in Section 4. If there is any step that is not detailed enough, we would greatly appreciate it if the reviewer could point it out. We will add more details in the revised draft.
- Human verification. All arguments in our paper were verified by the authors. That is, the parts assisted by AI were verified under the same standard as other mathematical papers, and the authors take full responsibility for the correctness of all results.
- Formal verification. We agree that formal verification can further improve the reliability of our proofs. However, we think that formal verification is a higher standard than what is usually required for mathematical papers, and formalizing our proofs would be a separate project on its own. In particular, to our knowledge, many of the basic definitions and results used in our proofs (e.g., Sobolev spaces on bounded domains, transversality, linking numbers, and Morrey's inequality) are not yet available in Mathlib, the main mathematical library of Lean. Hence, given the scale of such a project, it is nontrivial to complete it within the discussion period. Nevertheless, we believe that the correctness of our proofs can be evaluated based on their content.
C6. Comparison with Belomestny et al. (2023).
R6. We thank the reviewer for pointing out this work. Belomestny et al. (2023) establish quantitative simultaneous approximation results for smooth functions and their derivatives using a neural network with piecewise-polynomial activation functions. Their results control network depth, width, sparsity, and weight magnitudes in terms of the target smoothness and approximation accuracy.
Our work addresses a complementary question: determining the smallest fixed width that permits universal approximation in with unrestricted depth. In particular, this result would rule out arbitrary approximation accuracy below certain widths, regardless of depth. We thank the reviewer for pointing us to this relevant reference, which we will cite and discuss in the revised manuscript.
Thank you for the reply, I will consider your points.
Thank you for the reply. I especially find the relation to the neural ODE in your new Theorem E.1 of interest. I think this is moving in the right direction. The reason, why one would like to control derivatives and values at one in generative learning lied in the change of variables formula. To control the determinant of the Jacobian matrix, you presumably need approximations for the generator with , the dimension of the problem. This could presumably link-up with the theory of generative learning. However I think that this could be further refined.
It is well known from the theory of (optimal) transport that you can reach any target distribution with sufficiently regular density from a smooth source distribution via a transformation generated with a vector field. Your result would then be useful to show that for such generators, densities can be approximated in along with the generator itself. If you could further contextualize, your paper would be better motivated and I would consider C2, C3 to be sufficiently treated, although the results are still not strong enough to make a reasonable impact to the theory of neural ODE where you have to evaluate the divergences of vector fields on 1D substructures.
Thank you for clarification on C4. In fact you find some new effects by the variation of (C4), although quite some techniques are recycled.
I admit that the discussion on C1 and C5 is difficult to resolve.
The answer to C6 is fine.
In conclusion, I consider raising the score.
We are pleased that our response has addressed several of your concerns. Following the reviewer’s suggestion, we have added Lemma E.4 and Corollary E.5 to further address C2 and C3.
Lemma E.4 bounds the distance between generated densities by the distance between their generators. This directly connects Sobolev approximation of generators to density approximation in generative modeling. The bound holds for every ; in particular, suffices, without requiring .
Corollary E.5 provides a quantitative version of this connection: a width- network can simultaneously approximate the generator in and its generated density in , with the same parameter bound as Theorem E.1.
We hope these additions help contextualize our results within the approximation theory of generative models.
C1. The main idea to the proof is an AI assisted mathematical technique.
R1. We respectfully disagree with this comment. As described in our AI use statement, the main ideas of our proofs were developed by the authors, e.g., tracking singularities via the generic-singularity theorem of Whitney (1944) (Appendix A.1) and the linking number argument for the lower bound in Theorem 3.2 (Appendix A.4), and AI contributed to the proof ideas of only a small number of technical lemmas. Specifically, we used AI for the following:
C2. Larger theoretical context.
R2. Sobolev norms are standard error measures in the analysis and numerical approximation of PDEs. For example, H1=W1,2 is the natural energy space for many second-order elliptic problems, and approximation in this norm controls both the solution and its gradient. Thus, finite-p Sobolev approximation is meaningful in applications, and an important question is whether quantitative approximation guarantees can be obtained under architectural constraints such as fixed hidden width.
C3. Quantitative analysis.
R3. We have added Theorem E.1 in Appendix E, which connects shallow approximation of a vector field to approximation of its flow map by a fixed-width network. We show that if the generating vector fields admit shallow C1 approximations with M(δ) terms and the uniform coefficient bounds stated in the theorem, then the flow map can be approximated to W1,p accuracy ε using O(ε−1M(cε)) parameters and hidden layers. This provides a quantitative connection to neural ODEs at the level of approximation complexity.
For example, for generating vector fields with a uniformly stable Barron-type shallow approximation rate M(δ)=O(δ−2), shallow approximation uses O(ε−2) parameters, while our fixed-width construction uses O(ε−3). Thus, the polynomial exponent increases by one while the width remains fixed.
An important limitation remains when applying this estimate to a general target function. The perturbations used to avoid self-intersections can change the regularity of the resulting diffeomorphism and its generating vector fields. Consequently, the constants in the fixed-flow estimate may grow as these perturbations become smaller, so the theorem does not yet provide a complete complexity bound in terms of the original target alone. Nevertheless, it gives a quantitative approximation result when the associated flows remain well controlled. We believe that quantifying the cost of the geometric perturbations and developing statistical learning guarantees for neural ODEs and flow matching from these approximation estimates are important directions for future work.
C4. Improvement over Hwang et al. (2025).
R4. Our main novelty is understanding how these obstructions behave under finite-p Sobolev approximation, which introduces new opportunity or difficulty in the analysis that has not been discussed before in Hwang (2025b) under the infinity Sobolev approximation.
Indeed, topological obstructions that are stable under uniform approximation need not remain stable under W1,p approximation. Unlike the W1,∞ setting, finite-p approximation allows large local deformations to have arbitrarily small Sobolev cost, depending on p and the codimension of the singular set. This establishes a connection to the critical exponent threshold determined by the geometry of generic self-intersections.
Our results characterize when such obstructions can be removed and when they persist, leading to sharp p-dependent minimum-width transitions that cannot be inferred from Hwang (2025b).