Universality of Autonomous Neural ODEs

We show that using composition of autonomous-neural-ODEs, one can universally approximate diffeomorphisms in the identity component defined on the cube with rate $\mathcal{O}(P^{-1/d})$ with $P$ parameters. On the other hand, we show that by using only a single autonomous flow, the class of neural ODEs is nowhere dense on the cube in dimension $d\ge 2$ . Under a compact-support$\_\mathrm{id}$ condition on $(0,1{)}^d$, we show by a composition of at most $I_d$ autonomous-neural-ODEs with the same support$\_\mathrm{id}$, where $I_d$ depends only on the dimension one can universally approximate compactly supported$\_\mathrm{id}$ diffeomorphisms on $(0,1{)}^d$ for any dimension with rate $\mathcal{O}((\frac{P}{\log P}{)}^{-2/d})$, and also compactly supported$\_\mathrm{id}$ homeomorphisms on $(0,1{)}^d$ in dimension $d\ge 5$. Moreover, we show that the class of single autonomous-neural-ODEs compactly supported$\_\mathrm{id}$ on $(0,1{)}^d$ is meagre in the space of compactly supported$\_\mathrm{id}$ homeomorphisms on $(0,1{)}^d$ for $d\ge 2$. By linearly lifting, we obtain a universal approximation result for Lipschitz functions compactly supported on $(0,1{)}^d$ with rate $\mathcal{O}(P^{-1/(d+1)})$ with $P$ parameters.