acceptodds
Under review as a conference paper at ICLR 2027

Minimum Width for Universal Approximation in Spaces

Abstract

The minimum width for universal approximation has been extensively studied for the norm, the norm, and more recently the norm. A notable observation is that the minimum width for the norm is strictly larger than that for the norm for some input/output dimensions; however, no other non-trivial separation between norms is known. In this work, we study the minimum width for the norm with finite and ask whether it is separated from the norm and from the norm. We first show that for networks with leaky-ReLU-like activation functions, if the output dimension is larger than times the input dimension and , then the minimum width for the norm is . This value coincides with that for the norm and if , it is strictly smaller than that for the norm. We then show that the norm is separated from the norm through a finer analysis of the case with , where we characterize the minimum width for all . Here, the minimum width coincides with that for the norm if , increases by one when exceeds , and coincides with that for the norm if . Furthermore, if is odd, it increases by one once more when exceeds . Hence, the norm is separated from both the norm and the norm. In addition, unlike the norm, the minimum width for the norm depends on and even on the parity of . We also extend our results to general input and output dimensions, and a general class of activation functions.

Primary area:
learning theory
Keywords:
Universal Approximation, Sobolev Space, Minimum Width, Neural Network

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_AXwEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper under review discusses the minimal width for universal approximation in W1,pW^{1,p} Sobolev spaces and find some priorly unknown relationships between the index pp and the minimum width required. This is in part due to topological effects in combination with Sobolev embedding theorems which say that for sufficiently high pp and thus topological restictions for continuous functions materialize. Going beyond this single observation, the authors indentify a 'phase diagram' with four regions.

Extensive, AI supported poofs are given in the appendix and explanatory sketches are given in the main text.

Critical strengths
  • The topic of universal approximation is at the core of neural network theory and the simultaneous approximation of derivatives and the function itself is of interest for deeper studies, e.g., of NeuralODE where the densities are expressed via Liouville's formula by the divergence of the vector field.
  • The topological effects underlying the found structure are studies in detail, are nicely explained and are definitively interesting.
Critical weaknesses
  • As the main idea to the proof is an AI assisted mathematical technique, the paper has a special challenge in distinguishing itself from the mathematical proofs as a commodity thing. Here the easiest way would be to argue better why the found results embed themselves into a bigger theory picture, e.g. by relating the findings to the statistical learning theory of neural ODE or FlowMatching. This is however quite indirect and not really visible. In particular, the W1,pW^{1,p}-results do not easily help in this regard, as they might be too week to guarantee a quantitative analysis. Also, the improvement over the Results of Hwang et al (2025) seems to be gradual.
  • I acknowledge the use of AI declaration as absolutely professional. But the human checking of the proofs, which are not too detailed, can be executed on various levels. Formal checking would have been a plus, as on the grade of details the proofs have been given, LLM inaccuracies are not easy to detect.
Questions for authors
  • Where do you see the larger theoretical context of your result?
  • How do you compare your results to Denis Belomestny, Alexey Naumov, Nikita Puchkin, Sergey Samsonov, Simultaneous approximation of a smooth function and its derivatives by deep neural networks with piecewise-polynomial activations, Neural Networks, Volume 161, 2023, Pages 242-253.
4 comments

C1. The main idea to the proof is an AI assisted mathematical technique.

R1. We respectfully disagree with this comment. As described in our AI use statement, the main ideas of our proofs were developed by the authors, e.g., tracking singularities via the generic-singularity theorem of Whitney (1944) (Appendix A.1) and the linking number argument for the lower bound in Theorem 3.2 (Appendix A.4), and AI contributed to the proof ideas of only a small number of technical lemmas. Specifically, we used AI for the following:

  • Polishing the writing and checking the text for accuracy and consistency.
  • Turning our detailed proof sketches into formal arguments and checking their geometric and analytic details.
  • Identifying relevant literature for Theorem 3.1 (Appendix C), i.e., on the existence of isometric bundle embeddings (Theorem C.3) and on relative multijet transversality used in Lemma C.5. The core idea of the arguments involving the spine lemma (Lemma C.8) also came from AI.
  • Our initial approach for Theorem 3.2 did not resolve the case din−1<p≤dind_{\mathrm{in}}-1<p\le d_{\mathrm{in}} for odd dind_{\mathrm{in}}. AI contributed to the argument establishing the density of smooth embeddings into R2din\mathbb{R}^{2d_{\mathrm{in}}} in W1,pW^{1,p} for p≤dinp\le d_{\mathrm{in}} (Lemma 4.3), which closes this gap.
  • The core argument for the upper bound in Theorem 3.4 (Appendix D.2), which we added during the review process, was developed with AI assistance.

C2. Larger theoretical context.

R2. Sobolev norms are standard error measures in the analysis and numerical approximation of PDEs. For example, H1=W1,2H^1=W^{1,2} is the natural energy space for many second-order elliptic problems, and approximation in this norm controls both the solution and its gradient. Thus, finite-pp Sobolev approximation is meaningful in applications, and an important question is whether quantitative approximation guarantees can be obtained under architectural constraints such as fixed hidden width.

C3. Quantitative analysis.

R3. We have added Theorem E.1 in Appendix E, which connects shallow approximation of a vector field to approximation of its flow map by a fixed-width network. We show that if the generating vector fields admit shallow C1C^1 approximations with M(δ)\mathcal M(\delta) terms and the uniform coefficient bounds stated in the theorem, then the flow map can be approximated to W1,pW^{1,p} accuracy ε\varepsilon using O(ε−1M(cε))O(\varepsilon^{-1}\mathcal M(c\varepsilon)) parameters and hidden layers. This provides a quantitative connection to neural ODEs at the level of approximation complexity.

For example, for generating vector fields with a uniformly stable Barron-type shallow approximation rate M(δ)=O(δ−2)\mathcal M(\delta)=O(\delta^{-2}), shallow approximation uses O(ε−2)O(\varepsilon^{-2}) parameters, while our fixed-width construction uses O(ε−3)O(\varepsilon^{-3}). Thus, the polynomial exponent increases by one while the width remains fixed.

An important limitation remains when applying this estimate to a general target function. The perturbations used to avoid self-intersections can change the regularity of the resulting diffeomorphism and its generating vector fields. Consequently, the constants in the fixed-flow estimate may grow as these perturbations become smaller, so the theorem does not yet provide a complete complexity bound in terms of the original target alone. Nevertheless, it gives a quantitative approximation result when the associated flows remain well controlled. We believe that quantifying the cost of the geometric perturbations and developing statistical learning guarantees for neural ODEs and flow matching from these approximation estimates are important directions for future work.

C4. Improvement over Hwang et al. (2025).

R4. Our main novelty is understanding how these obstructions behave under finite-pp Sobolev approximation, which introduces new opportunity or difficulty in the analysis that has not been discussed before in Hwang (2025b) under the infinity Sobolev approximation.

Indeed, topological obstructions that are stable under uniform approximation need not remain stable under W1,pW^{1,p} approximation. Unlike the W1,∞W^{1,\infty} setting, finite-pp approximation allows large local deformations to have arbitrarily small Sobolev cost, depending on pp and the codimension of the singular set. This establishes a connection to the critical exponent threshold determined by the geometry of generic self-intersections.

Our results characterize when such obstructions can be removed and when they persist, leading to sharp pp-dependent minimum-width transitions that cannot be inferred from Hwang (2025b).

C5. Human checking of the proofs and formal checking.

R5. We thank the reviewer for acknowledging our AI use statement. We agree that the correctness of proofs is essential, and we would like to clarify the following points.

  • Level of detail. We provide complete proofs in Appendices A–C (about 20 pages) and proof sketches of the main ideas in Section 4. If there is any step that is not detailed enough, we would greatly appreciate it if the reviewer could point it out. We will add more details in the revised draft.
  • Human verification. All arguments in our paper were verified by the authors. That is, the parts assisted by AI were verified under the same standard as other mathematical papers, and the authors take full responsibility for the correctness of all results.
  • Formal verification. We agree that formal verification can further improve the reliability of our proofs. However, we think that formal verification is a higher standard than what is usually required for mathematical papers, and formalizing our proofs would be a separate project on its own. In particular, to our knowledge, many of the basic definitions and results used in our proofs (e.g., Sobolev spaces on bounded domains, transversality, linking numbers, and Morrey's inequality) are not yet available in Mathlib, the main mathematical library of Lean. Hence, given the scale of such a project, it is nontrivial to complete it within the discussion period. Nevertheless, we believe that the correctness of our proofs can be evaluated based on their content.

C6. Comparison with Belomestny et al. (2023).

R6. We thank the reviewer for pointing out this work. Belomestny et al. (2023) establish quantitative simultaneous approximation results for smooth functions and their derivatives using a neural network with piecewise-polynomial activation functions. Their results control network depth, width, sparsity, and weight magnitudes in terms of the target smoothness and approximation accuracy.

Our work addresses a complementary question: determining the smallest fixed width that permits universal approximation in W1,pW^{1,p} with unrestricted depth. In particular, this result would rule out arbitrary approximation accuracy below certain widths, regardless of depth. We thank the reviewer for pointing us to this relevant reference, which we will cite and discuss in the revised manuscript.

Reviewer_AXwEOpenReview ↗

Thank you for the reply, I will consider your points.

Thank you for the reply. I especially find the relation to the neural ODE in your new Theorem E.1 of interest. I think this is moving in the right direction. The reason, why one would like to control derivatives and values at one in generative learning lied in the change of variables formula. To control the determinant of the Jacobian matrix, you presumably need W1,pW^{1,p} approximations for the generator with p>=dp>=d, the dimension of the problem. This could presumably link-up with the theory of generative learning. However I think that this could be further refined.

It is well known from the theory of (optimal) transport that you can reach any target distribution with sufficiently regular density from a smooth source distribution via a transformation generated with a vector field. Your result would then be useful to show that for such generators, densities can be approximated in L1L^1 along with the generator itself. If you could further contextualize, your paper would be better motivated and I would consider C2, C3 to be sufficiently treated, although the results are still not strong enough to make a reasonable impact to the theory of neural ODE where you have to evaluate the divergences of vector fields on 1D substructures.

Thank you for clarification on C4. In fact you find some new effects by the variation of pp (C4), although quite some techniques are recycled.

I admit that the discussion on C1 and C5 is difficult to resolve.

The answer to C6 is fine.

In conclusion, I consider raising the score.

We are pleased that our response has addressed several of your concerns. Following the reviewer’s suggestion, we have added Lemma E.4 and Corollary E.5 to further address C2 and C3.

Lemma E.4 bounds the L1L^1 distance between generated densities by the W1,pW^{1,p} distance between their generators. This directly connects Sobolev approximation of generators to density approximation in generative modeling. The bound holds for every p≥1p\ge1; in particular, p=1p=1 suffices, without requiring p≥dp\ge d.

Corollary E.5 provides a quantitative version of this connection: a width-dd network can simultaneously approximate the generator in W1,pW^{1,p} and its generated density in L1L^1, with the same parameter bound as Theorem E.1.

We hope these additions help contextualize our results within the approximation theory of generative models.

Reviewer_HBV2OpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The manuscript studies how wide neural networks need to be to facilitate universality when approximation is required in Sobolev norms. The authors establish a range of results here, depending on the input and output dimensions of the underlying functions as well as on the parameters of the Sobolev space.

A central insight is a characterization of the minimum width when the output dimension is 2d_{in} -1. Here, the results reveal an interesting dependence on the Sobolev exponent. Moreover, the parity of the input dimension is essential here.

Critical strengths

I think the underlying problem is fully resolved without any edge cases left open. I think the effect of the Sobolev exponent on the minimum depth is, at first, surprising, which shows that the result is very interesting. There seem to be no mathematical issues that I could find.

Critical weaknesses

There are essentially three main weaknesses:

A) The literature on ReLU approximation in Sobolev norms seems to not be discussed at all. It appears as if this paper falls into this category, so a proper discussion of related work would be appropriate. This should be repaired.

B) The results are a bit strange when one wants to apply them to small output dimensions. It is always required that d_{out} is large compared to the input dimension. Often in situations where Sobolev norms are relevant, neural networks are used to solve some sort of PDE, where we usually have scalar-valued networks. I think it would be great if the authors could discuss how important this assumption is. At first (judging also from the sentence immediately after the statement of Theorem 3.1), it seems as if the choice of d_{out} actually simplifies the arguments quite a bit. So it would be good to find out if completely different tools are necessary for scalar-valued functions.

C) There is another work (Hwang, 2025b), which is repeatedly cited, and which appears to have quite a few relatively similar arguments in spirit. Concretely, it already identifies a parity dependence in a similar set-up through a topological obstruction. Maybe I missed this, but if not, I would suggest that the authors give a clearer account of how this work goes beyond (Hwang, 2025b).

To prevent any misunderstandings, I am not asking the authors to produce results for scalar functions or an entirely new approach compared to (Hwang, 2025b). For a conference, the novelty in this work seems clearly enough. But they could address their limitations better.

Questions for authors

How do the authors plan to address the weaknesses?

3 comments

A. Related work on ReLU approximation in Sobolev norms.

R1. Our upper bounds extend straightforwardly to ReLU networks with one additional unit of width: whenever width ww suffices in our setting, width w+1w+1 suffices for ReLU networks. This follows by a straightforward adaptation of our existing proofs. We have clarified the weak-derivative convention in Section 2.2 (p. 5) and added the ReLU extension to Theorem 3.5 and its proof in Section 3.5 (p. 8), with the corresponding additions highlighted in red.

We have also expanded the related-work discussion to include the Sobolev approximation results of Gühring, Kutyniok, and Petersen (2020), Opschoor, Petersen, and Schwab (2020), Hon and Yang (2022), and Yang, Yang, and Xiang (2023). We now explain how these quantitative approximation results relate to our study of the minimum hidden width required for universal approximation in W1,pW^{1,p}.

B. Small output dimensions and scalar-valued functions.

R2. Regarding point B, we agree that the role of the large-output assumption and the limitations of our proof should be explained more clearly, especially given the relevance of scalar-valued networks to PDE applications.

The assumption dout>3din/2d_{\mathrm{out}}>3d_{\mathrm{in}}/2 substantially simplifies the proof of Theorem 3.1. It ensures a tractable geometry for the self-intersections of generic maps and excludes triple points. Our construction relies on these properties, so this proof does not directly extend to scalar output.

We appreciate that you are asking for clarification of these limitations rather than additional results. Separately, in response to Reviewer 9hM1’s question, we have added a scalar-output result as Theorem 3.4, which also helps clarify the different tools used in this regime. For n=din≥4n=d_{\mathrm{in}}\ge4, dout=1d_{\mathrm{out}}=1, and the leaky-ReLU-like activation class Σ\Sigma considered in the paper,

wmin⁡1,p(n,1;Σ)={n,1≤p≤⌊n/2⌋,n+1,⌊n/2⌋<p<∞.w_{\min}^{1,p}(n,1;\Sigma)=\begin{cases} n, & 1\le p\le\lfloor n/2\rfloor, \cr n+1, & \lfloor n/2\rfloor<p<\infty. \end{cases}

For p>⌊n/2⌋p>\lfloor n/2\rfloor, the lower bound n+1n+1 follows from a small modification of our existing linking argument and matches the existing upper bound.

For the width-nn upper bound, however, we use a substantially different, Morse-theoretic construction. GPT Astra generated the initial argument, which we developed into the proof in Appendix D.

In our earlier arguments, the main geometric task is to avoid self-intersections and obtain an embedding. For the scalar width-nn upper bound, we seek W1,pW^{1,p} approximations of the target that can serve as the first coordinate of a global diffeomorphism. Such functions must have nonvanishing gradients, so controlling critical points, where the gradient vanishes, becomes central. We therefore use Morse theory, which studies the critical points of smooth functions and classifies their nondegenerate local forms.

The added theorem is restricted to scalar output; it does not provide a general treatment of small output dimensions. Extending this Morse-theoretic construction to higher-dimensional outputs appears to require more sophisticated geometric techniques.

We have incorporated these consequences and revised the abstract and introduction. We have also added the exact scalar-output characterization in Section 3.4 (Theorem 3.4) together with a summary table, and provided the full proof in Appendix D.

C. How this work goes beyond Hwang (2025b).

R3. Our main novelty is understanding how these obstructions behave under finite-pp Sobolev approximation, which introduces new opportunity or difficulty in the analysis that has not been discussed before in Hwang (2025b) under the infinity Sobolev approximation.

Indeed, topological obstructions that are stable under uniform approximation need not remain stable under W1,pW^{1,p} approximation. Unlike the W1,∞W^{1,\infty} setting, finite-pp approximation allows large local deformations to have arbitrarily small Sobolev cost, depending on pp and the codimension of the singular set. This establishes a connection to the critical exponent threshold determined by the geometry of generic self-intersections.

Our results characterize when such obstructions can be removed and when they persist, leading to sharp pp-dependent minimum-width transitions that cannot be inferred from Hwang (2025b).

[1] Laudenbach, Francois. "A proof of Reidemeister-Singer's theorem by Cerf's methods." arXiv preprint arXiv:1202.1130 (2012).

Reviewer_HBV2OpenReview ↗

Thanks for the clarifications

All my points were addressed adequately. I think this is solid work, and the result is very interesting. I am happy to raise my score to 4.

We thank the reviewer for the positive assessment and the constructive feedback that helped improve our manuscript.

Reviewer_9hM1OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies the minimum width needed for deep networks to be universal approximators in the W1,pW^{1,p} norm with finite pp, i.e., approximating both the function and its gradient. This fills the gap between the known results for the LpL^p and W1,∞W^{1,\infty} norms. The paper gives the exact minimum width in two settings (Theorems 3.1 and 3.2), and partial bounds for general dimensions. Notably, the minimum width depends on pp and even on the parity of dind_{\mathrm{in}}, which separates the W1,pW^{1,p} case from both the LpL^p and W1,∞W^{1,\infty} cases. The proofs reduce the problem to a geometric question about embeddings and combine Sobolev estimates with topological arguments.

Critical strengths
  • The results are new. This seems to be the first time the minimum width has been shown to differ between norms beyond the known LpL^p vs. L∞L^\infty gap: here W1,pW^{1,p} differs from both LpL^p and W1,∞W^{1,\infty}. It is also surprising that the minimum width depends on pp, and even on whether dind_{\mathrm{in}} is odd or even.

  • The work is solid. The reduction to a geometric problem (Proposition 2.2) is carefully carried over to networks, the thresholds where the minimum width jumps are given a clear geometric explanation, and the upper and lower bounds match in the settings the paper characterizes.

  • The proofs in the appendix are complete and detailed, and the proof sketches in Section 4 make the main ideas easy to follow.

Critical weaknesses
  1. ReLU is not covered. The main results only apply to leaky-ReLU-like activations (Definition 2.1), which must be C1C^1; even the extension in Theorem 3.4 needs a C1,1C^{1,1} activation. So the results do not cover ReLU itself, which is the most widely used activation and is covered by the earlier LpL^p results in Table 1. This limits how broadly the results apply. Since the W1,pW^{1,p} norm of a ReLU network is still well defined, it would help if the authors explained what exactly goes wrong for ReLU in their arguments.

  2. The characterization is incomplete for general dimensions. The exact minimum width is only pinned down in two cases: dout>3din/2d_{\mathrm{out}} > 3d_{\mathrm{in}}/2 with small pp (Theorem 3.1), and dout=2din−1d_{\mathrm{out}} = 2d_{\mathrm{in}} - 1 with din≥3d_{\mathrm{in}} \ge 3 (Theorem 3.2). For other dimension pairs, Corollary 3.3 only gives upper and lower bounds that do not match; for example, when p≤din−1p \le d_{\mathrm{in}} - 1 the lower bound is just the trivial one, while the upper bound can be much larger. The paper is upfront about this, but the title and abstract may suggest a more complete picture than the paper actually delivers.

  3. No bounds on depth or network size. The results allow the network to be arbitrarily deep, and the constructions give no estimate of how deep the network must be or how the depth grows as the target accuracy improves. This weakens the connection to the practical motivation (PINNs and Sobolev training) given in the introduction.

Questions for authors
  1. Can the results be extended to networks with ReLU activations? The W1,pW^{1,p} norm of a ReLU network is still well defined, and the earlier LpL^p results do cover ReLU. Could the authors explain which parts of the proof break down for ReLU (beyond it not being C1C^1), and whether they expect the same minimum-width values to hold, e.g., via smoothing or approximating ReLU by leaky-ReLU-like activations?

  2. While the minimum width needed for universal approximation in various norms is now fairly well studied, is there any high-level intuition for what determines the minimum width required to approximate a mapping from Rdin\mathbb{R}^{d_{\mathrm{in}}} to Rdout\mathbb{R}^{d_{\mathrm{out}}}? In particular, is there an intuitive explanation for why approximating gradients as well (W1,pW^{1,p} vs. LpL^p) can require extra width, and why the answer depends on pp and even on the parity of dind_{\mathrm{in}}?

4 comments

C1. Extension to ReLU activations.

R1. Our main goal was to understand how geometry determines the minimum width. We therefore initially focused on leaky-ReLU-like activations, which allow us to implement the geometric construction without adding extra neurons. However, our upper bounds (Theorem 3.5) extend straightforwardly to ReLU networks with one additional unit of width: whenever width ww suffices in our setting, width w+1w+1 suffices for ReLU networks. This follows by a straightforward adaptation of our existing proofs. We have clarified the weak-derivative convention in Section 2.2 (p. 5) and added the ReLU extension to Theorem 3.5 and its proof in Section 3.5 (p. 8), with the corresponding additions highlighted in red.

In Hwang (2025b), a key step in approximating diffeomorphisms is to approximate arbitrary increasing scalar C1C^1 functions using width-one networks with sufficiently large depth. This is possible with leaky-ReLU-like activations, but not with ReLU, so the same argument does not apply directly. However, one additional hidden neuron is enough to carry out the construction with ReLU, giving width w+1w+1 instead of ww.

C2. Incomplete characterization for general dimensions.

R2.

Thank you for pointing this out. We have now obtained an exact characterization for scalar output. Writing n=dinn=d_{\mathrm{in}}, for n≥4n\ge4, 1≤p<∞1\le p<\infty, and the leaky-ReLU-like activation class Σ\Sigma considered in the paper, we prove

wmin⁡1,p(n,1;Σ)={n,1≤p≤⌊n/2⌋,n+1,⌊n/2⌋<p<∞.w_{\min}^{1,p}(n,1;\Sigma)=\begin{cases} n, & 1\le p\le\lfloor n/2\rfloor, \cr n+1, & \lfloor n/2\rfloor<p<\infty. \end{cases}

For p>⌊n/2⌋p>\lfloor n/2\rfloor, the lower bound n+1n+1 follows from a small modification of our original linking argument: we use linked spheres of dimensions ⌊(n−1)/2⌋\lfloor(n-1)/2\rfloor and ⌈(n−1)/2⌉\lceil(n-1)/2\rceil and apply the same Fubini–Sobolev estimate. Together with the existing width-n+1n+1 upper bound, this settles that regime.

The new ingredient is the width-nn upper bound. GPT Astra generated the initial Morse-theoretic argument, which we developed into the proof provided in Appendix D of the revised manuscript. Taking Laudenbach’s path constructions [1] and the standard genericity and normal-form results for one-parameter families of functions as inputs, we regard the remaining argument as relatively elementary: regular portions of the path are handled by changes of coordinates, while births, deaths, and critical-value crossings are treated through explicit local modifications and Sobolev cutoff estimates.

We have added the exact scalar-output characterization in Section 3.4 (Theorem 3.4) together with a summary table, and provided the full proof in Appendix D.

Besides the scalar case, for general vector output, a basic difficulty is that the geometric and topological obstructions underlying minimum-width lower bounds are not fully understood even in the C0C^0 setting. When p>np>n, Sobolev–Morrey embedding gives

∥u∥C0([0,1]n)≤C∥u∥W1,p([0,1]n),\lVert u\rVert_{C^0([0,1]^n)} \le C \lVert u\rVert_{W^{1,p}([0,1]^n)},

so uniform-approximation obstructions for smooth target functions immediately yield Sobolev lower bounds. For example, Chu and Weinkove’s recent preprint [2] establishes such obstructions in several low-dimensional cases. Combining their lower bounds with our existing width-(n+m)(n+m) Sobolev upper bound yields the further exact characterization

\quad\text{for }n\in\{2,3,4,7,8\},\quad 1\le m\le n,\quad n<p<\infty.$$ These additions resolve further dimension regimes, but the uniform minimum-width problem itself remains open in general, and the corresponding topological difficulties also affect the Sobolev problem. We have incorporated these consequences and revised the abstract and introduction to distinguish the exact characterizations from the bounds for unresolved cases. To better reflect the scope of our results, we revised the title to “Minimum Width for Universal Approximation in W1,p Spaces: A Focus on the Morse-Theoretic and Double-Point Intersection Regimes.” Also, we revised the abstract to remove the misleading wording about general dimensions and explicitly state the scalar-output result. Specifically, we replaced the sentence > We also extend our results to general input and output dimensions. with > For scalar output $d_{\mathrm{out}}=1$ with $d_{\mathrm{in}}\ge4$, the minimum width is $d_{\mathrm{in}}$ for $1\le p\le\lfloor d_{\mathrm{in}}/2\rfloor$ and $d_{\mathrm{in}}+1$ for $\lfloor d_{\mathrm{in}}/2\rfloor<p<\infty$. We also extend our results to a general class of activation functions.

C3. No bounds on depth or network size.

R3. Now, we have obtained a quantitative bound closely related to shallow approximation rates. We have included this result as Theorem E.1 in Appendix E. For a fixed flow, if its generating vector fields admit shallow C1C^1 approximations with M(δ)\mathcal M(\delta) terms and the uniform coefficient bounds stated there, then a network of fixed width achieves W1,pW^{1,p} error ε\varepsilon using O(ε−1M(cε))O(\varepsilon^{-1}\mathcal M(c\varepsilon)) parameters and hidden layers.

For example, for generating vector fields with a uniformly stable Barron-type shallow approximation rate M(δ)=O(δ−2)\mathcal M(\delta)=O(\delta^{-2}), the shallow approximation uses O(ε−2)O(\varepsilon^{-2}) parameters, while our fixed-width construction uses O(ε−3)O(\varepsilon^{-3}). Thus, the polynomial exponent increases by one while the width remains fixed.

For approximation of a general target function, an additional difficulty remains. The perturbations used to avoid self-intersections can change the regularity of the resulting diffeomorphism and its generating vector fields. Consequently, the constants in the fixed-flow estimate may grow as these perturbations become smaller, and Theorem E.1 does not yet give a complete complexity bound in terms of the original target alone. We nevertheless expect the estimate to be useful for practical targets whose geometric constructions remain well controlled. We believe that quantifying the cost of these perturbations and its implications for PINNs and Sobolev training is an important direction for future work.

C4. Intuition for the minimum width, dependence on p, and parity of din.

R4. One main insight is that the minimum width is governed by interaction between geometric obstructions and the strength of the approximation norm. Through our geometric reduction framework, this reduces to a question about projection of embeddings, which has different behavior throughout various regimes of pp and parity of dind_{\mathrm{in}}.

Indeed, the dependence on pp comes from the Sobolev cost of removing such intersections. Intuitively, one can move intersecting pieces apart by modifying the function in a small neighborhood. Such a deformation may have large derivatives, but its W1,pW^{1,p} cost can still vanish when pp is sufficiently small relative to the codimension of intersection. For larger pp, the same geometric obstruction can no longer be removed at arbitrarily small cost.

The parity reflects topological obstructions: resolving self-intersections may require additional coordinates, and whether one extra coordinate can consistently separate double-point pairs depends on the parity of dind_{\mathrm{in}}.

[1] Laudenbach, Francois. "A proof of Reidemeister-Singer's theorem by Cerf's methods." arXiv preprint arXiv:1202.1130 (2012).

[2] Chu, BaoZhi, and Ben Weinkove. "The minimal width of neural networks and immersions of real projective space." arXiv preprint arXiv:2610.20229 (2026).

Reviewer_9hM1OpenReview ↗

Update upon the rebuttal response

I thank the authors for the detailed response and the effort put into updating the paper. The intuitive explanation of the role of p and the parity of din is helpful, and the updated title and abstract better reflect the scope of the work. The new results also strengthen the manuscript, and the discussion of the remaining challenges is much appreciated. Meanwhile, gaps still remain for general multi-dimensional outputs, and the new proof is quite involved to fully verify. Overall, the paper presents new and interesting results, and I will maintain my score of 3.

We thank the reviewer for the positive assessment and the constructive feedback that helped improve our manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.