acceptodds
Under review as a conference paper at ICLR 2027

Resolving Homotopy Obstructions in Normalizing Flows via Contractible Covers

Abstract

Normalizing Flows are a powerful class of likelihood-based generative models that map complex target distributions into simple, tractable base distributions, usually Gaussians. As differentiably invertible functions trained via likelihood maximization, they enable exact density evaluation and efficient sampling. However, they also suffer from the fundamental topological rigidity of homeomorphic transformations: they cannot alter the topology, and consequently homotopy, of the space they transform without severely increasing their bi-Lipschitz constant. resolves these homotopy obstructions by partitioning the target support into a data-driven and then training a posterior-regularized confined to the prescribed cover elements, which are guaranteed to be homotopically compatible with the Gaussian base by construction. The mixture model preserves exact likelihood evaluation, deterministic maximum likelihood training, and efficient sampling of standard Normalizing Flows. Experiments on synthetic and real-world data distributions demonstrate improved or competitive density estimation with increased continuity of the generative map compared to standard baselines. To our knowledge, CoCoFlow is the first mixture-of-normalizing-flows framework to provide per-component contractibility guarantees through a topology-aware, data-driven partition of the target support, offering a principled resolution to a long-standing structural limitation of the field.

Primary area:
generative models
Keywords:
topology, homotopy, topological data analysis, normalizing flows, mixture models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_3ALdOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
3
Presentation
2
Soundness
1
Read review
Paper summary

The paper deals with the problem of topological constraints in the normalizing flow framework. It has well known, that there is a Lipschitz tradeoff in handling multimodal (or more generally topologically different) distributions from e.g. a standard Gaussian. The authors deal with this issue with defining a graph cover, which essentially defines a graph structure on the data points and then finds an overlapping partition of such which is then used to learn a mixture model. Different ways (cover versus learning a mixture model) are ablated in the paper.

Critical strengths
  1. I think that the framework of graph covers is genuinely novel and the right stab at the problem. It is very reminiscent of [Cornish et al], but a good new rephrasing of this problem.

  2. The numerical results are encouraging, and are a clean ablation that help a deeper understanding isntead of improving just numbers.

Critical weaknesses
  1. There is a reason why this line of literature has received less attention in the last years. Of course when learning a NF there is a topological mismatch, but the question is whether it hinders training/inference. the rise of diffusion and flow matching models split up the prolbem into smart subproblems, see e.g., [1]. This allows making deeper networks that still train nicely and I would guess a comparison with such would be warranted.

  2. The EM problem formulation seems off, usually you would model it as maximizing the ELBO which cancels the evidence term. In the α=1\alpha = 1 case, it seems to require a normalization constant, which makes it not a EM algorithm. At α=0\alpha = 0 it just reduces to a ML objective for which the EM proecdure is not needed.

  3. The paper misses some important comparisons in my opinion, such as the piecewise normalizing flow [2] and the rejection sampling framework by stimper et al [3]. I would also appreciate a flow matching/diffusion comparison to mitigate my first point.

  4. It is unclear how much heavy lifting the cover does. If the density is a GMM, does not the prior already kind of solve it? Wouldnt the algorithm recover the separated modes?

[1] Can Push-forward Generative Models Fit Multimodal Distributions?, Salmona et al

[2] Piecewise Normalizing Flows, Bevin et al

[3] Resampling Base Distributions of Normalizing Flows, Stimper et al

Questions for authors

Basically discuss all the points I outlined in weaknesses. Two more:

  1. Of course a larger Lipschitz constant per se is bad, but in what effects does it materialize? Does it affect sampling std, training dynamics?

  2. Is there a straightforward way to generalize this method to cond generalization?

8 comments
AuthorsOpenReview ↗

Response (1/2)

We thank the reviewer for their feedback. We are pleased that the reviewer found our method genuinely novel and the results encouraging. We address the reviewer’s concerns and questions below.

Additional flow matching baseline (W1)

We follow the reviewer’s advice and evaluated the performance of the flow matching model and added it as a valuable baseline in the Tables 3, 4, 5, 6, 7, 8, 10. The flow matching model is parametrized by a velocity network with approximately 1.4⋅1041.4\cdot10^4 parameters (details in new Appendix E in the revised pdf), compared to small affine model with 1.3⋅1041.3\cdot10^4 parameters and big spline model with 6.3⋅1046.3\cdot10^4 parameters. The flow matching model performs better than the normalizing flow models on a large subset of datasets, which is expected, especially in the adversarial setup of our work, which is tailored to point out the limitations of normalizing flows. However, on the topologically complex Mickey and Rings datasets, the flow matching model is outperformed even by the relatively small affine CoCoFlow model (Mickey: −1.04±0.06-1.04\pm 0.06 vs 0.03±0.010.03\pm 0.01; Rings: −2.35±0.02-2.35\pm 0.02 vs −2.08±0.05-2.08\pm 0.05). These results highlight that, on datasets with substantial topological structure, even a simple multi-chart model can outperform more expressive architectures that rely on a single chart.

At the same time, we would like to point out that our work, as in the title, is concerned with normalizing flows specifically. The motivation, as framed in the Introduction, was to strengthen the performance of one-step, exact likelihood normalizing flows from standard Gaussian base distribution. Although the flow matching is naturally viewed as a method stemming from the normalizing flows, their computational behavior is distinct from classical normalizing flows. In particular, they have one downside: the log-probability estimates of the flow matching models are expensive to obtain, since they require multi-step integration of the flow, contrary to one-step evaluation in normalizing flows. Moreover, the log-probability returned by the above procedure is approximate in nature because of probability flow ODE discretization error and Jacobian approximation, which exact computation is intractable in high-dimensional cases. Normalizing flows employ constrained architectures which are invertible and have tractable Jacobian determinants that can be evaluated exactly and efficiently.

Clarifications on the training procedure (W2)

We thank the reviewer for pointing out the distinction between our training procedure and standard EM. We agree that our original terminology was imprecise, and we have revised the description to clarify the relationship.

In the special case of α=0\alpha=0 and a trivial cover (i.e., every data point belongs to every cover element), our procedure reduces to generalized EM for a mixture of normalizing flows. In this setting, the E-step computes the standard mixture posterior responsibilities, while the M-step maximizes the expected complete-data log-likelihood. Since the component densities are parameterized by normalizing flows, their parameters are optimized numerically using gradient-based optimization, whereas the mixture weights admit a closed-form update given by the empirical mean of the responsibilities. Provided that the numerical M-step increases its objective, this corresponds to generalized EM.

For a non-trivial cover, however, the membership assignments are restricted to the cover elements containing each data point. Consequently, they no longer correspond to the unrestricted posterior of the mixture model. The resulting alternating updates therefore do not inherit the standard EM guarantee of monotonically increasing the mixture log-likelihood. For α>0\alpha>0, our objective additionally incorporates posterior guidance by balancing mixture likelihood maximization with assignment-weighted conditional likelihood maximization. In particular, at α=1\alpha=1, the objective reduces to weighted likelihood maximization, as employed in Piecewise Normalizing Flows (Bevin et al., 2023). The cover-constrained assignments guide the specialization of individual components to the corresponding regions, without imposing explicit support constraints on the component densities.

We therefore agree that a more precise characterization of our method would be “posterior-guided alternating optimization”, which recovers generalized EM in the limiting case of a trivial cover and α=0\alpha=0. We will update the terminology and mathematical description in the final manuscript accordingly.

AuthorsOpenReview ↗

Response (2/2)

Missing comparisons (W3)

We acknowledge the importance of the flow matching baseline raised by the reviewer and have included flow matching results as an additional baseline, as described in our response above.

We would also like to clarify that Piecewise Normalizing Flows (Bevin et al., 2023) were already included in the original version of our work and are reported in Tables 1 and 2 under the name “k-means PNF.”

Furthermore, the rejection sampling framework of Stimper et al. (2022) was already cited in the original manuscript. As discussed in the Related Work section, rejection sampling from the base distribution comes at the cost of sacrificing its tractability. Our method is designed to preserve the core computational properties of normalizing flows, namely: (i) a simple, tractable base distribution, (ii) efficient and exact likelihood computation, and (iii) efficient one-step sampling. The rejection sampling framework sacrifices all three properties. Flow matching, in turn, retains property (i) but sacrifices property (ii) and, unless an additional flow map is trained, property (iii).

Impact of the contractible cover algorithm (W4)

Gaussian mixture distributions are a standard synthetic benchmark for generative models. They also exhibit non-trivial topology due to their disconnected modes (non-trivial zeroth-order homology). However, in this particular setting, k-means clustering would, in principle, suffice, since its centroids can be interpreted as estimates of the component means of a Gaussian mixture model.

In contrast, the synthetic datasets introduced in our work exhibit non-trivial zeroth- and first-order homology, corresponding to disconnected modes and holes, respectively. The results in Table 1 demonstrate that mixture models achieve better likelihood scores than single-chart models. More importantly, topology-aware partitioning based on contractible covers consistently improves model performance compared to topology-unaware baselines, such as k-means clustering and unregularized mixture models.

We believe these results provide clear evidence of the benefits of topology-aware data partitioning and demonstrate the added value of our cover algorithm beyond conventional clustering approaches.

The role of Lipschitz constants in the evaluation (Q1)

We report the Lipschitz constants of the evaluated methods because prior work (Cornish et al., 2020) developed theoretical results showing that a topological mismatch between the source and target distributions has to manifest as large Lipschitz constants as the model increasingly accurately approximates the target distribution. We therefore use Lipschitz constants as a theory-motivated diagnostic for assessing potential topological mismatch. Nevertheless, as discussed in Section 5.3, large Lipschitz constants are not necessarily indicative of a topological mismatch. They may also arise when the source and target distributions have matching topologies but the target distribution exhibits high curvature. During the rebuttal, we made the Lipschitz constant comparison more principled by normalizing the scores (new Appendix F in the revised pdf) and we compared the topological faithfulness of the learned models by comparing the persistent homology features of their learned density maps (new Appendix I).

Extending the Method to Conditional Generation (Q2)

For a finite set of conditioning values cc, conditional generation p(x∣c)p(x \mid c) can be viewed as modeling a collection of distributions indexed by cc. One possible extension would therefore be to partition the training data corresponding to each conditional subpopulation independently and use the union of the resulting labels to index the mixture model. In this way, each conditional distribution would be associated with its own set of mixture components. When cc is continuous, a possible extension would be to partition the distributions p(x∣c∈I)p(x \mid c \in I) over intervals II, potentially under suitable continuity assumptions on the family of conditional distributions p(x∣c)p(x \mid c). We acknowledge conditional generative modeling as an interesting direction for future research and leave its systematic investigation to future work.

We hope the additional analyses and theoretical results address your questions. We would be happy to clarify anything that remains open. If our responses have resolved your concerns, we would be grateful if you could consider updating your score accordingly.

Reviewer_3ALdOpenReview ↗

Response

I thank the authors for their response and additional material. I am keeping my score, as the flow matching comparison shows that my concerns are somehow justified. I remain unconvinced of the significance of the approach given that flow matching performs well out of the box. Note also that 10^4 parameters is extremely low (even for the low dimensions). As you correctly pointed out, flow matching is a kind of normalizing flow and there has been recently a lot of progress in one-step generators with flow matching.

Further, I do not concur with the whole Lipschitz constant as a theoretical measure of the well-posedness. There is likely real effects that a high Lipschitz constants has and showing it is crucial to justify the problem.

EDIT: I thought about it and I like the idea and I am raising my score in light of the nice idea. I would still like a discussion of the practical impact of the Lipschitz constant.

Reviewer_rNEXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses an important problem: normalizing flows and flow matching methods preserve, by construction, the properties that are invariant under diffeomorphism. When the support of the source distribution (for example a Gaussian) is topologically different from that of the target data distribution (number of connected components, fundamental group, etc.), matching it is difficult and leads to large Jacobian norms, since the map must effectively tear the space.

In this work, the authors propose to use a mixture of diffeomorphisms to achieve this goal. The data support is approximated by the clique complex of a kNN graph on landmark points. A cover of this complex by contractible regions is built by greedy expansion from several roots: a vertex is added to a region only if its link is a dismantlable graph. A mixture of flows is then trained with EM, with each component’s posterior restricted to one region early in training. This keeps exact likelihood and efficient sampling. Experiments on synthetic and Earth-science datasets show competitive or better log-likelihoods and often lower Jacobian norms than mixture and k-means baselines.

Critical strengths

-The paper address an important problem, normalizing flow and flow matching method preserves by construction diffeomorphic invariant property.

-The methodology seems original but I am not an expert of this domain.

Critical weaknesses
  • Clarity. Section 3, which contains the core contribution, is hard to follow for readers who are not specialists in topological data analysis (which is my case). Key notions (dismantlability, links, stars) would benefit from illustrations, and Algorithms 1 and 2 would be much easier to understand with a worked toy example. The appendix is quite thin: additional background and diagrams would make the paper accessible to the broader generative-modeling audience.

  • Experimental validation (main concern). I am not convinced that the experimental protocol validates the proposed methodology.

    • The evaluation relies on log-likelihood and Lip.99. Lip.99 depends on the size of the regions modeled by each component, so splitting the data into smaller pieces mechanically lowers it, independently of topology. It is also only measured in the generative direction, whereas the obstruction result of Cornish et al. concerns the bi-Lipschitz constant.
    • A natural baseline is missing: a standard mixture of KK flows trained by minimizing −LL+λR-\mathrm{LL} + \lambda R, where RR penalizes large Jacobian norms of the generative map (e.g., via a Hutchinson estimator), for several values of KK and λ\lambda. If such a model reaches similar LL and Lip.99, the topological cover would not be necessary.
  • Theoretical support. The paper repeatedly claims provable guarantees, but contains no formal theorem: the only argument is an informal induction for the single-root case. Combined with the heuristic choice of kk (longest plateau of the cover size), the overall contribution currently looks more heuristic than principled.

  • Scalability. All experiments involve data of intrinsic dimension at most 2 (the 24-dimensional cyclooctane data lies on a 2-dimensional set). It is unclear whether the cover construction remains meaningful and affordable in higher intrinsic dimension, where kk must presumably grow quickly. This seems to be a common limitation of TDA-based methods, but it should be discussed or tested.

Questions for authors

-Can you train a standard mixture of K flows (K = 1, …, m) by minimizing −LL + λ · R, where R penalizes large Jacobian norms of the generative map (e.g., a Hutchinson estimate of the Jacobian norm, averaged over the largest values), for a few values of λ. How does CoCoFlow compare to these models in terms of LL and Lip.99? If such a regularized mixture reaches similar results, the topological cover would not be necessary.

-How would you compare a topological invariant of your data distribution with that of your learned density ? Preserving the topology of the data seems to be the main motivation of this work, but no such comparison is reported. If this is not feasible for some technical reason, could you discuss it?

6 comments
AuthorsOpenReview ↗

Response (1/2)

We thank the reviewer for their helpful and constructive feedback. We are pleased that the reviewer recognizes the importance of the problem we study and the originality of our methodology. We address the reviewer’s comments and questions below.

Extended empirical validation

Scale-adjusted Lipschitz constant

“[...] Lip.99 depends on the size of the regions modeled by each component, so splitting the data into smaller pieces mechanically lowers it. [...]”

We agree with the reviewer’s critique regarding the direct comparison of Lipschitz constants between models trained on different dataset partitions. In particular, the Lipschitz constant of a map can be trivially reduced by rescaling its output, making direct comparisons potentially misleading.

To account for this effect, we estimate the covariance matrix Σc\Sigma_c of data sampled from each mixture component fθ(z∣c)f_\theta(z \mid c) and compute the Lipschitz constants of the corresponding whitened component Jacobians: Σc−1/2∇zfθ(z∣c).\Sigma_c^{-1/2}\nabla_z f_\theta(z \mid c). This normalization is intended to make the resulting Lipschitz constants measure local map distortion relative to the global scale of each component, providing a more meaningful basis for comparison across models.

We report updated versions of Tables 1 and 2, including the whitened Lipschitz constants, in the revised pdf, Appendix F (Tables 5 and 6). The resulting rankings are consistent with those obtained using the originally reported Lipschitz constants in all scenarios except Small Affine Flood. In this case, after whitening, CoCoFlow achieves the best (lowest) Lipschitz constant.

CoCoFlow improves the performance of Jacobian-regularized baselines

“[...] A natural baseline is missing: a standard mixture of flows trained by minimizing, which penalizes large Jacobian norms of the generative map (e.g., via a Hutchinson estimator). [...]”

Following the reviewer’s suggestion, we trained additional models with Jacobian regularization using Hutchinson’s estimator.

Specifically, we trained Small Affine and Big Spline models using both the Mixture and CoCoFlow methods, optimizing the regularized objective with λ∈{0.01,0.1,1.}\lambda\in\{0.01, 0.1, 1.\}. We also included unregularized models (λ=0\lambda=0) to provide a comprehensive comparison. The results are reported in the revised pdf, Appendix H, Table 9. The results show that CoCoFlow achieves better log-likelihood (LL) than the Mixture baseline in 7 out of 8 scenarios. Furthermore, in 7 out of 8 scenarios, CoCoFlow achieves better Lipschitz constants among the models with the best and second-best LL performance. Finally, in 5 out of 8 scenarios, CoCoFlow simultaneously achieves higher LL and a lower Lipschitz constant.

Measuring the topological faithfulness of the methods

“[...] How would you compare a topological invariant of your data distribution with that of your learned density? [...]”

Assessing the topological faithfulness of a model is generally challenging. One possible approach would be to estimate the ground-truth topology using graph-based topological data analysis (TDA) methods. However, this could introduce bias into the evaluation, since our method itself relies on graph-based TDA. To obtain a more independent assessment of the topological faithfulness of the learned maps, we therefore employ a density-map-based comparison. This approach, however, is not scalable to higher dimensions and requires access to the ground-truth density of the target distribution.

For the 2D synthetic datasets, we evaluate the models' log-probability estimates on a 200×200200\times 200 grid covering the support of each dataset. The resulting density maps are presented in the revised pdf, Appendix I (Figures 8 and 9).

We compute persistent homology barcodes using cubical complexes constructed from the superlevel sets of the density maps, restricting the log-probability range to (−10,−2)(-10,-2). We then compare the H0H_0 and H1H_1 persistence features of each model with those of the ground-truth data density. The comparison is quantified using the 1-Wasserstein distances between the corresponding persistence barcodes. The results are reported in Appendix I, Table 10.

The results show that CoCoFlow achieves the best H1H_1 topological fidelity in 5 out of 6 scenarios with the biggest gains on the Concentric dataset (1.40±1.391.40 \pm 1.39 vs 4.66±1.064.66 \pm 1.06 for Big Spline). In particular, the Big Spline CoCoFlow model achieves better H1H_1 fidelity than the flow matching baseline across all datasets.

The relatively large H0H_0 distances can be attributed to noisy density estimates in regions outside the main high-density regions of the data distribution. In these regions, oscillations in the model predictions can produce numerous small, disconnected components, thereby increasing the discrepancy between the corresponding H0H_0 persistence barcodes.

AuthorsOpenReview ↗

Response (2/2)

Lipschitz constants are measured only in generative direction

“[...] It is also only measured in the generative direction. [...]”

Since all datasets considered in this work are concentrated on lower-dimensional subsets of the ambient space, large Lipschitz constants in the noising direction are expected. The results are reported in the revised pdf, Appendix G, Tables 7 and 8. As the dimensionality mismatch between the source and target distributions is the dominant factor contributing to these large values, we do not consider Lipschitz constants in the noising direction to be informative measures of homotopy mismatch.

Theoretical guarantees for the algorithm

“[...] The paper repeatedly claims provable guarantees, but contains no formal theorem. [...]”

We would like to clarify that the theoretical guarantees of our approach hold with respect to the chosen graph-based representation of the topology. For any specified graph, we formally prove by induction that the invariant maintained by our algorithm is that the expanded region is contractible with respect to the graph's clique complex. Extending this result to multiple roots is straightforward, since the invariant is preserved independently for each root.

In our method, we employ a longest-plateau strategy to select a stable graph-based representation, which is subsequently used downstream to limit variance and reduce the degrees of freedom in the modelling process. The choice of graph used to capture the topology of the dataset remains a modelling assumption.

Importantly, our contractible cover algorithm is not restricted to a particular graph-selection strategy: it can operate on any graph-based representation of the data topology and provably constructs a contractible cover with respect to that graph.

Scalability of the algorithm

“[...] It is unclear whether the cover construction remains meaningful and affordable in higher intrinsic dimension. [...]”

There are two aspects of scalability to consider: computational and representational.

Computational scalability. Capturing high-dimensional topological features of a given complex typically requires enumerating all its d−d-dimensional simplices, whose number can grow exponentially with the dimension dd. To overcome this computational bottleneck, we designed our algorithm to operate entirely on the graph representation of the clique complex, without ever explicitly constructing its simplices. Consequently, our contractible cover algorithm runs in polynomial time with respect to the graph size, without an explicit dependence on the dimension of the topological features being captured. This qualitative difference is illustrated in Figure 5(a), where we fix the graph size while increasing the dimension of the data. The running time of our algorithm remains stable across dimensions, whereas computing Betti numbers becomes orders of magnitude more expensive as the dimension increases. To further demonstrate the computational scalability of our approach, we provide an explicit construction of a clique complex with arbitrarily high-dimensional topological features for which the contractible cover can be computed in polynomial time (new Appendix J in the revised pdf.

Representational scalability. The ability of graph-based representations to faithfully capture high-dimensional topological features is subject to an inherent limitation of the Topological Data Analysis (TDA) framework. The results in Figures 5(b) and 5(c) illustrate this limitation: as the dimension of the topological features increases, larger graphs, with more edges, are required to recover the correct topology of the underlying data. Although this limitation is inherent to graph-based representations, such representations remain one of the most principled and widely adopted approaches in the TDA community for studying the topology of data.

Clarity

“[...] Section 3, which contains the core contribution, is hard to follow for readers who are not specialists in topological data analysis. [...]”

We acknowledge the reviewer’s concerns regarding the presentation of our method in Section 3.2. We recognize that Topological Data Analysis (TDA) is not yet a standard tool in the machine learning community and that certain aspects of our presentation could benefit from clearer visualizations and additional motivation. In the camera-ready version, we will include a dedicated method walkthrough in the appendix, featuring illustrations of the TDA concepts used in our work, namely links, stars, and dismantlability. We will also provide a toy example demonstrating how our algorithm operates, step by step.

We hope the additional analyses and theoretical results address your questions. We would be happy to clarify anything that remains open. If our responses have resolved your concerns, we would be grateful if you could consider updating your score accordingly.

Dear authors, thanks for your detailed answer, I will take time to digest this by tomorrow (it's too late here), this will leave us a very few time before the end of the rebuttal.

I read other's reviewer discussion about flow matching, in my opinion normalizing flows & any likelihood based generative models remain useful for some specific application including for example Gibbs sampling when you have no or very few samples but you'd like to train a model by reverse KL and function evaluation; I don't know if your method would be applicable in this regime but just to say that there is still reasons do work on NF.

Reviewer_CQCWOpenReview ↗
Rating
3
Confidence
3
Informativeness
3(+1)
Presentation
2
Soundness
3
Read review
Paper summary

The paper tackles the problem of training normalizing flows when the target data-topology is non-trivial. It introduces a method to decompose the target data into a union of contractible components, and uses a mixture of Normalizing flows to model each contractible component separately. The method is evaluated on a range on synthetic and real world datasets, and the Jacobian of the learned normalizing flow is analyzed empirically.

Critical strengths
  1. The problem setting is interesting. Training normalizing flows is hard. Understanding why it is hard matters. This paper tackles one aspect which is caused by the non-trivial topology of some datasets.
  2. The proposed TDA-based algorithm to extract a contractible cover is interesting.
  3. The experiments are rigorous. I particularly enjoyed the empirical analysis of the Jacobian which highlights the strength of the method in practice.
Critical weaknesses

W1. The method is based on k-NN graphs and TDA which are known to fail to scale to high dimensions. This limits the versatility of the approach. W2. I found section 3.2 hard to read. It currently is a list of paragraphs with little effort to link them. W3. I missed a formal statement characterising the limitations of normalizing flows. Ideally you would have a statement saying that the image of a normalizing flow has to be a contractible open subset of R^d. I do not know if such result exists in the literature, but it would be great to state clearly, even if the statement is weaker (e.g. the image has to be connected). This would then motivated why your method allows to expand the expressivity by allowing to capture data with disconnected connected components and other more complicated topological obstructions. W3. While the paper focuses on topological obstructions in terms of contractibility, another important topological obstruction that normalizing flows fail to capture is that data often lie on a low-dimension manifold. Since normalizing flows are continuous and bijective onto their image, the support of the target need to be d-dimensinoal and therefore the support cannot consist of a lower-dimensional manifold.

Questions for authors

Q1. How important is the \alpha scheduler? A small ablation could be helpful here. I am also confused if some shows of \alpha corresponds to the baselines? Q2. In practice, how do you chose n, the number of landmark points? Q3. Some ablations on the impact of the choice of n, m, and alpha would improve the experimental results.

4 comments
AuthorsOpenReview ↗

Response (1/2)

We thank the reviewer for their positive assessment of our work. We are pleased that the reviewer found the proposed algorithm interesting and appreciated the rigor of our experimental evaluation. We address the reviewer’s concerns and questions below.

Contractible cover algorithm is scalable with dimension (W1)

“The method is based on k-NN graphs and TDA which are known to fail to scale to high dimensions.”

There are two aspects of scalability to consider: computational and representational. Computational scalability. Capturing high-dimensional topological features of a given complex typically requires enumerating all its d−d-dimensional simplices, whose number can grow exponentially with the dimension dd. To overcome this computational bottleneck, we designed our algorithm to operate entirely on the graph representation of the clique complex, without ever explicitly constructing its simplices. Consequently, our contractible cover algorithm runs in polynomial time with respect to the graph size, without an explicit dependence on the dimension of the topological features being captured.

This qualitative difference is illustrated in Figure 5(a), where we fix the graph size while increasing the dimension of the data. The running time of our algorithm remains stable across dimensions, whereas computing Betti numbers becomes orders of magnitude more expensive as the dimension increases.

To complement these empirical results with a formal derivation, in the revised pdf we provide an explicit construction of a clique complex with arbitrarily high-dimensional topological features for which the contractible cover can be computed in polynomial time (new Appendix J).

Representational scalability. The ability of graph-based representations to faithfully capture high-dimensional topological features is subject to an inherent limitation of the Topological Data Analysis (TDA) framework. The results in Figures 5(b) and 5(b) illustrate this limitation: as the dimension of the topological features increases, larger graphs, with more edges, are required to recover the correct topology of the underlying data. Although this limitation is inherent to graph-based representations, such representations remain one of the most principled and widely adopted approaches in the TDA community for studying the topology of data. This limitation will be listed in the “Discussion” section in the camera-ready version of the paper.

Improving the presentation of our method (W2)

“[...] found section 3.2 hard to read. [...]”

We acknowledge the reviewer’s concerns regarding the presentation of our method in Section 3.2. We recognize that Topological Data Analysis (TDA) is not yet a standard tool in the machine learning community and that certain aspects of our presentation could benefit from clearer visualizations and additional motivation. In the camera-ready version, we will include a dedicated method walkthrough in the appendix, featuring illustrations of the TDA concepts used in our work, namely links, stars, and dismantlability. We will also provide a toy example demonstrating how our algorithm operates, step by step.

Formalizing the limitations of Normalizing Flows (W3)

“[...] formal statement characterising the limitations of normalizing flows [...]”

In Section 2, we explain that normalizing flows are homeomorphisms by construction, i.e., continuous and invertible transformations with continuous inverses. Since homeomorphisms preserve homotopy type, the image of a contractible set under a normalizing flow must also be contractible. This yields an immediate impossibility result: a normalizing flow cannot exactly represent a target distribution whose support exhibits nontrivial topological features absent from the support of the base distribution. To extend this observation beyond exact modeling and characterize the limitations of approximate modeling, we invoke the result of Cornish et al. (2020). They show that, when the support of PXP_X is not homeomorphic to the support of PZP_Z, weak convergence of the pushforward distributions, f_{\theta}\\# P_Z \rightarrow P_X, necessarily forces the bi-Lipschitz constants of fθf_\theta to diverge to infinity. We further complement these results with an additional quantitative result in the revised pdf, Appendix K. While the preceding arguments concern topological obstructions arising from the supports of the distributions, our new proposition considers distributions whose density superlevel sets exhibit persistent nontrivial topology, even when their support is the entire ambient space. We show that exactly representing such a distribution with a normalizing flow requires the ratio between the upper and lower bounds on the Jacobian determinant to exceed the ratio between the corresponding density thresholds.

AuthorsOpenReview ↗

Response (2/2)

Dimensionality mismatch is orthogonal to homotopy mismatch (W4)

“[...] another important topological obstruction that normalizing flows fail to capture is that data often lie on a low-dimension manifold [...]”.

Homotopy mismatch and dimensionality mismatch constitute two independent obstacles to modelling target distributions with normalizing flows. In particular, dimensionality mismatch does not necessarily imply homotopy mismatch, and vice versa. For example, a one-dimensional line embedded in a two-dimensional plane exhibits dimensionality mismatch without homotopy mismatch. Conversely, a distribution supported on an open annulus in two dimensions exhibits homotopy mismatch without dimensionality mismatch. The dimensionality mismatch between source and target distributions is a well-recognized limitation of normalizing flows. It can be addressed by adding a small amount of ambient noise to the target distribution, as is commonly done through noise dequantization (Zhai et al., 2024; Gu et al., 2025). However, provided that the noise scale is sufficiently small relative to the size of the topological features, and under suitable separation/persistence conditions, the high-density regions of the resulting Gaussian-smoothed distribution should retain the relevant topological structure of the original target distribution.

Our work focuses specifically on addressing the homotopy mismatch. It is worth emphasizing that all datasets considered in our experiments are concentrated on lower-dimensional subsets of the ambient space. We therefore employ noise dequantization when training all evaluated models to mitigate the dimensionality mismatch. Despite this, our experimental results demonstrate that CoCoFlow consistently outperforms competing approaches, highlighting the substantial benefits of explicitly accounting for the non-trivial homotopy of the target distribution's support, even after noise dequantization.

Method hyperparameters (Q1-Q3)

We address the reviewer's questions Q1–Q3 below.

α\alpha scheduler

The importance of the α\alpha scheduler is most clearly demonstrated by comparing the performance of the k-means-based model under a constant α\alpha schedule with its performance under our proposed annealing schedule, Table 1, 2 in the original version of our work. Across all evaluated scenarios, the annealed schedule consistently outperforms the constant-schedule variant. It is also worth noting that the constant schedule with α=0\alpha=0 removes the regularization term from our method, making the resulting mixture model a particular instance of our framework. Conversely, setting α=1\alpha=1 throughout training, together with k-means-based cover construction, recovers the k-means PNF model from the literature (Bevin et al. 2023).

Number of landmarks

In principle, the required number of landmarks could be estimated by monitoring the asymptotic behaviour of the distances from newly added points to the existing landmark set. In particular, under standard regularity assumptions, these distances should asymptotically scale as O(n−1/d)O(n^{-1/d}), where nn is the number of landmarks and dd is the intrinsic dimension of the data. However, we deliberately opted for a simpler strategy and used a fixed number of landmarks for all datasets, as selecting the number of landmarks is not the main focus of our contribution. More generally, the number of landmarks should be chosen to ensure that the topology of the target distribution is recovered at the desired level of granularity. This level of granularity is ultimately a modelling choice and we defer it to the practitioner based on the requirements of the application.

Number of clusters

The primary objective of this work is to evaluate the benefits of using a homotopy-aware partition of the target distribution. To isolate the contribution of this component, we designed our experiments to compare models with equivalent computational budgets and expressive capacities.

Specifically, all models use the same normalizing flow architecture and the same number of components (except for the single-component baseline). Consequently, differences in performance can be attributed to the data partitioning strategy and the corresponding training procedures, rather than to differences in model architecture or expressive capacity. This experimental design allows us to isolate the contribution of the proposed homotopy-aware partitioning strategy and provide a fair comparison between the considered approaches.

We hope the additional analyses and theoretical results address your questions. We would be happy to clarify anything that remains open. If our responses have resolved your concerns, we would be grateful if you could consider updating your score accordingly.

I thank the authors for their rebuttal. I maintain my positive assessment, and have also increased my informativeness score to 3.

Concerning the hyperparameters, I appreciate that the main contribution is to study the benefits of homotopy-aware partition. However, I believe that providing clearer guidance on how to select the hyperparameters (alpha schedule, number of landmarks, and number of clusters) could make the method more accessible and usable for practictioners.

We thank the reviewer for the positive assessment of our work and are glad that the informativeness score has improved to 3.

We agree with the reviewer that a more detailed discussion of hyperparameter selection would enhance the accessibility and practical applicability of our method. Accordingly, we will expand Appendix B in the camera-ready version to include the aforementioned discussion of hyperparameter choices and their selection.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary

We sincerely thank all reviewers for their insightful comments and constructive discussions throughout the rebuttal period. We believe that the manuscript has benefited substantially from their feedback, which has helped us strengthen both its empirical and theoretical contributions. In particular, we have incorporated several additional experiments, theoretical results, and clarifications into the revised manuscript. Below, we summarize the main additions and improvements.

Summary of Rebuttal Contributions

  • Additional Experiment: Demonstrated that CoCoFlow improves the performance of Jacobian-regularized baselines.

  • Additional Evaluation: Assessed the topological faithfulness of the methods using persistent homology features.

  • Additional Theoretical Result: Provided further theoretical results highlighting the limitations of normalizing flows.

  • Additional Baseline: Included Flow Matching as an additional baseline.

  • Additional Metric: Introduced the Scale-Adjusted Lipschitz Constant as an additional evaluation metric.

Reviewer Evaluations

Reviewer CQCW appreciated the rigor of our experiments, the importance of the problem we study, and the novelty of our approach. Our expanded discussions of the theoretical limitations of normalizing flows, the scalability of our algorithm to high dimensions, and hyperparameter selection helped increase the reviewer's informativeness score to 3.

Reviewer rNEX appreciated the importance of the problem and the novelty of our approach. In response to the reviewer's feedback, we conducted additional experiments, including reporting additional Lipschitz metrics, evaluating the topological faithfulness of the generative models, and explicitly assessing Jacobian-regularized models. We also expanded our discussion of the theoretical guarantees and scalability of our algorithm and clarified why measuring the Jacobian in the noising direction is uninformative in our problem setting. The reviewer maintained their original score of 2.

Reviewer 3ALd found our method genuinely novel and appreciated the experimental results, particularly the clean ablations and the deeper insights they provide. Our expanded discussion of the significance of our work and its positioning relative to the flow matching literature contributed to an increase in the reviewer's overall score from 2 to 3.

Significance of This Work

Our work makes three main contributions:

  • Addressing a fundamental limitation of normalizing flows: We tackle the long-standing structural limitation that normalizing flows are diffeomorphisms and therefore cannot change the topology of the source distribution.
  • Introducing a principled topology-aware framework: We propose a novel Contractible Cover Algorithm that partitions the target distribution into contractible subsets. When combined with a mixture model conditioned on these components, this partitioning provides a principled solution to the homotopy mismatch problem.
  • Enabling practical scalability and improved model properties: Our algorithm scales with the dimensionality of the topological features, enabling its application to practical settings. Models conditioned on the resulting topology-aware partition achieve improved or comparable likelihood performance while exhibiting better Lipschitz continuity.

Taken together, these contributions address an important structural limitation of existing generative modeling approaches and provide both theoretical insights and empirical evidence for the effectiveness of our method. We have made a substantial effort to address the reviewers' concerns and incorporate their feedback into the revised manuscript. We hope that the resulting improvements make the contributions, significance, and practical relevance of our work clearer.

We thank the reviewers again for their time and thoughtful feedback, and we believe that this work will be of interest to the ICLR community.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.