acceptodds
Under review as a conference paper at ICLR 2027

Universality of Autonomous Neural ODEs

Abstract

We show that using composition of autonomous-neural-ODEs, one can universally approximate diffeomorphisms in the identity component defined on the cube with rate with parameters. On the other hand, we show that by using only a single autonomous flow, the class of neural ODEs is nowhere dense on the cube in dimension . Under a compact-support condition on , we show by a composition of at most autonomous-neural-ODEs with the same support, where depends only on the dimension one can universally approximate compactly supported diffeomorphisms on for any dimension with rate , and also compactly supported homeomorphisms on in dimension . Moreover, we show that the class of single autonomous-neural-ODEs compactly supported on is meagre in the space of compactly supported homeomorphisms on for . By linearly lifting, we obtain a universal approximation result for Lipschitz functions compactly supported on with rate with parameters.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Autonomous Flow, Neural ODE, Universal Approximation, Dynamical System, Homeomorphism and Diffeomorphism Groups, Structure-Aware AI

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_zhRiOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper explores the universal approximation property (UAP) of Neural ODEs (NODEs). It demonstrates (with a series of theoretical proofs) that while a single autonomous model is limited in its ability to approximate time-varying vector fields or complex transformations (being "nowhere dense"), a finite composition of NODEs can satisfy the UAP.

Critical strengths

The universal approximation theorem (UAT) has long been a subject of extensive research. Although Neural ODEs may not be a particularly trending topic at the moment, establishing the UAT for a finite composition of autonomous Neural ODEs represents a solid and meaningful contribution.

Critical weaknesses
  1. The fact that a single autonomous NODE is "nowhere dense" is perhaps an intuitive, expected result.

  2. The proof that a composition of autonomous NODEs satisfies the UAP relies on a structure that (at least from a practical standpoint) appears entirely unfeasible.

In short, while the paper presents an interesting theoretical finding, it does not have novelty with practical utility at the same time.

Questions for authors

Are there any examples of using Composition of autonomous neural ODE in practice?

Reviewer_GAVTOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proves universal approximation results for models built by composing autonomous neural ODEs.

I am not an expert in this field, but I found the paper poorly written, with substantial weaknesses in its motivation.

Critical strengths
  • I think an interesting question is whether a single NODE with a time-independent vector field is a universal approximator. The paper’s positive result concerns compositions, which the title should make clearer.
Critical weaknesses
  • The paper is poorly written and difficult to follow. The motivation and main contribution are unclear, while the apparently unused Whitney-topology discussion (p. 2), too long notation section, and elementary rotation example (p. 3) distract from the main argument.

  • The positive result concerns composing several autonomous NODEs, rather than a single autonomous NODE as the title might suggest.

Questions for authors
  • Why is this problem important, and how do the results advance the understanding or practical use of NODEs?
Reviewer_EmfEOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies what autonomous (time-independent) neural ODEs can represent, in two settings. On (0,1)d(0,1)^d with compact support, Thm 3.1 & Thm 3.2 record that the flowable homeomorphisms are nowhere dense / meagre; Thm 3.3 states that every Cinf⁡C^{\inf} compactly supported diffeomorphism is a composition of at most I_d time-one flows with I_d depending only on d; Thm 3.4 gives a uniform approximation rate O((PlogP)−2k/d)O((P log P)^{−2k/d}) for a composition of ReLU neural ODEs; Thm 3.6 extends this to orientation-preserving homeomorphisms for d≥5d \geq 5; and Cor. 3.7 gives a rate O(P−1/(d+1))O(P^{−1/(d+1)}) for compactly supported Lipschitz functions through a lifting construction. On the closed cube [0,1]d[0,1]^d, Thm 4.1 states that single neural ODEs are nowhere dense, Thm 4.2 that diffeomorphisms are not dense in homeomorphisms, and Thms 4.3 & 4.4 that composition attains universality.

Critical strengths

The question is a real one, and the contrast between the two domains is informative: on the open manifold with compact support the negative result is only meagreness, whereas on the cube it is nowhere density, and the authors say so plainly — "Considering only the cube results a stronger negative result (nowhere density) while considering compactly supported_id on (0,1)d(0,1)^d only gives meagreness, which is not an obstruction for universality" (secion 5.1.2). The logical architecture of the positive results is also clearly laid out and reusable: fragmentation => finitely many flows => RELU approximation of the vector fields => Gronwall propagation of error (Lem. A.7, Prop. A.8). The use of the topological literature is competent — portable manifolds, BIP-meagre groups, fragmentation norms — and Lemma A.5 (Munkres–Connell–Bing) is a good idea for getting from homeomorphisms to diffeomorphisms when d ≥ 5. Finally, the AI-use disclosure is unusually explicit, including "proving math claims", which is the right thing to do and is relevant to how carefully a reader should check the arguments.

Critical weaknesses

the headline rate is neither derived from the paper's own bounds nor quantified. Everything quantitative in the abstract rests on two objects: the rate O ⁣((Plog⁡P)−2k/d)\mathcal{O}\!\left((P\log P)^{-2k/d}\right) and the number of composing flows IdI_d. The transfer lemma the proofs actually use gives ∥φ−Ψ∥L∞  ≤  ∑i=1I(ωi ⁣(d2n)∏j≥ieLVj)\|\varphi-\Psi\|{L^\infty}\;\le\;\sum{i=1}^{I}\Big(\omega_i\!\left(\tfrac{d}{2n}\right)\prod_{j\ge i} e^{L_{V_j}}\Big) (Prop. A.8). Since Lemma A.6 makes the width grow like ndn^{d}, the resulting exponent is that of the modulus ω\omega; with ω(t)≍tk\omega(t)\asymp t^{k}, natural for CkC^{k} vector fields, the bound gives O(P−k/d)\mathcal{O}(P^{-k/d}), not O(P−2k/d)\mathcal{O}(P^{-2k/d}). The symbol kk is never defined and the step is never shown. In parallel, IdI_d is asserted to exist (Lem. A.15) and never bounded, so the composition error carries a constant exp⁡(c Id)\exp(c\,I_d) with c>0c>0 unknown, which makes the rate incomparable with any known MLP result. The central claim is thus unsupported on both ends, and there are no experiments to compensate.

Questions for authors

May you define kk and derive (7)–(8) from Lemma A.6 and Prop. A.8, naming the result that supplies the factor 22 in the exponent; And bound IdI_d explicitly or estimate it numerically for d=2,3d=2,3, and state whether ∏j≥ieLVj\prod_{j\ge i} e^{L_{V_j}} can be absorbed into the rate. If the exponent is meant to be the 2S/d2S/d rate from the smooth-function approximation literature, please cite it together with its hypotheses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.