acceptodds
Under review as a conference paper at ICLR 2027

The Shape of Flows: Topological Regularization of Vector Fields in Flow Matching for Better Diversity and Fidelity.

Abstract

Flow Matching models have become the de facto paradigm for the generation of continuous data. Despite impressive results, conditional sampling from these models implies a fidelity/diversity tradeoff, which greatly impacts their applicability. To address this issue, we propose the use of persistent homology to identify relevant topological features both at the pixel level and at the sample level, from which we derive two training objectives. On the one hand, we use filtrations on cubical complexes to align the topological persistence of pixel velocity sublevel sets with that of the ground truths; on the other hand, we employ filtrations on Vietoris-Rips complexes to ensure the topology of the generated distribution approximates that of the training set. We demonstrate these approaches improve both the diversity and fidelity of generations, showing the promise of persistent homology and geometric invariants to improve the quality of Flow Matching models.

Primary area:
generative models
Keywords:
Flow Matching, Persistent Homology, Topology, Diversity, Fidelity, Post-training

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_HCSeOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes two persistent homology-based regularization objectives for flow matching models. The fidelity loss uses cubical complexes to align the topological features of predicted and target velocity fields, while the diversity loss uses Vietoris–Rips complexes to preserve the topology of velocity predictions across a batch. Experiments on MNIST, MSCOCO, and ImageNet show that these objectives can improve generation quality and diversity through additional fine-tuning, particularly when using RAE latents.

Critical strengths
  1. The paper explores an interesting direction for flow matching regularization by incorporating topological information beyond standard pointwise losses.
  2. The proposed batch-level diversity loss is a useful contribution, with encouraging improvements in FID and distribution coverage on RAE-based models.
  3. The paper provides theoretical analysis, ablation studies, and computational evaluations to support the proposed approach.
Critical weaknesses
  1. The benefit of topological regularization over simpler geometric constraints is not fully established. Since the diversity loss also includes geometric alignment between samples, it remains unclear how much of the improvement comes specifically from persistent homology.
  2. The improvements are not consistent across different settings. While the RAE experiments show promising results, the method does not improve performance with SD-VAE latents or FID in the additional SiT/XL-2 experiment. A clearer understanding of when the proposed losses are effective would strengthen the paper.
Questions for authors
  1. Could the authors compare the proposed diversity loss with a simpler pairwise distance-matching loss, such as minimizing the differences between pairwise cosine distances of predicted and target velocity vectors within a batch, without using persistent homology? This comparison would help clarify the benefit of the proposed topological formulation.
  2. Could the authors further explain why the method performs differently with RAE and SD-VAE latents? What might account for the limited improvement in the additional SiT/XL-2 experiment?
  3. The theoretical analysis focuses on simplified forms of the topological objective. To what extent do these results support the multiplicative diversity loss used in the experiments?
  4. How sensitive is the proposed diversity loss to batch size? Could the authors examine whether smaller batches affect the reliability of the estimated topology and the resulting generation performance?
4 comments
AuthorsOpenReview ↗

Comparison with simpler constrains, and batch size analysis

We thank the reviewer for adding relevant questions and comments. We address your concerns below.

W1/Q1. The benefit of topological regularization over simpler geometric constraints is not fully established.

We thank the reviewer for the suggestion to compare with simpler regularizers. We add results below. We compare (1) cosine distance regularization (pred vs tgt), (2) random pairwise cosine distance (between randomly selected pairs in the batch, n=16), (3) geometric alingment component of the diversity loss only (equivalent to pairwise cosine distance matching, but between critical indices), (4) geometric alignment component + MSE and (5) only the diversity loss. All results for 10k finetuning steps on the DiTDH-S RAE baseline, cfg 1.5.

methodfidprecisionrecalldensitycoverage
Cosine distance8.5670.8580.4821.2250.884
Random pairwise distance8.6430.8580.4861.2150.883
GeoAlign only8.7990.8570.4841.2110.882
MSE+ GeoAlign8.4250.8630.4911.2510.891
MSE + Fid8.5370.8800.4761.3400.905
Div only (proj)7.9200.8620.4971.2270.889

We observe the div loss achieves better FID and Recall than other regularization methods. MSE + Fid wins in precision and density (higher loss signal). Note: due to computational limitations, we had to compute the diversity loss with Johnson-Lindenstrauss projection, more details below. In most of our experiments, the method after projection is an upper bound for the results without projection, so we could probably expect even better numbers. MSE + Div (without proj.) achieves FID of 7.203, see paper.

W2/Q2. The improvements are not consistent across different settings (VAE latents).

We acknowledge this limitation, and are aware of its importance. We admit we have no clear solution at this point. We add some results we got below, as well as our research process and the hypothesis we tested in case it can shed some light on the issue.

We believe the root cause to be the curse of dimensionality when computing VR-filtrations, as discussed in Appendix B.3. Given that VR computations depend only on pairwise distances, noise in high dimensional spaces can produce large artifacts in the PD that renders it unusable. In RAE spaces, where cosine distances are meaningful, this problem is alleviated, but we do not have that privilege in VAE or Pixel space. We've taken this limitation very seriously, and have proposed two solutions: (1) project data to a lower dimensional space via a random sparse projection (leveraging Johnson-Lindenstrauss lemma, which ensures such a projection nearly preserves pairwise distances), and (2) using different distance metrics less sensitive to this issue, such as spectral distances. Both approaches work good with RAE latents (improved computational performance, at the price of slight degradation of the improvements).

However, this does not seem to be enough. Here some results with SiT/XL-2 [1], a famous VAE state-of-the-art model:

methodfidprecisionrecalldensitycoverage
MSE finetine3.9680.8330.6661.1940.947
MSE + Div (cos dist)4.9480.8460.6011.2080.933
MSE + Div (spectral dist)3.9890.8370.6631.2020.948
MSE + Div (l2 after proj)3.9820.8320.6681.1940.945

Corresponding to 8-10k steps of finetuning, CFG 1.5. It may be (1) that 10k ft steps for a model pretrained at a complete different scale (SiT/XL is trained 7M steps) is not enough to appreciate differences wrt MSE, or (2) that neither spectral, l2 or cosine distances are optimal for VAE spaces (in fact, cosine even increases recall). In any case, we do not think there is a fundamental limitation with our method (which is a very general idea that should work as long as we can compute meaningful PDs), but rather a lack of understanding of the right approach to extract topological information from VAE spaces, a direction in which more work is needed. Due to computational limitations we could not test 1, and are still trying alternatives to 2. Nonetheless, we still believe this work to be a useful proof-of-concept, that consistently and cheaply improves models in RAE space (see response to reviewer M4jn for statistical tests), and a step towards geometrical regularization of FM models.

AuthorsOpenReview ↗

Batch size analysis

We apologize for the lengthy response. We continue in this comment.

Q3. The theoretical analysis focuses on simplified forms of the topological objective. To what extent do these results support the multiplicative diversity loss used in the experiments?

Good question. The short answer is we cannot with certainty guarantee that a similar bound to Claim B.8 exists for the product version of the loss. We only suspect it to be the case due to two facts: (1) empirically, we obtain better results for the product than the sum, and (2) results for the product loss are close but slightly better to the case when we detach the geometrical alignment component so gradients only flow through the topological component (and we do have theoretical guarantees for this case, see Corollary B.4).

In more detail, under the setup of Claim B.8, let A=1{i,j,k∈C(x∣z)},B=Lgeo and C=Ltopo.A = 1_{\{i,j,k \in C(x \mid z)\}}, \qquad B = L_{geo} \quad \text{ and } \quad C = L_{topo}. Similarly let μA=E[A]\mu_A = E[A] etc. Then, the proof of the bound for the product loss follows that of Claim B.8, and we have: E[ABC]=μAμBμC+μACov(B,C)+μBCov(A,C)+μCCov(A,B)+E[(A−μA)(B−μB)(C−μC)]E[ABC] = \mu_A \mu_B \mu_C + \mu_A Cov(B, C) + \mu_B Cov(A, C) + \mu_C Cov(A,B) + E[(A - \mu_A)(B - \mu_B)(C - \mu_C)]

where the last term is the joint central moment. This, together with μACov(B,C)\mu_A Cov(B, C) are the two elements not present in the proof of Claim B.8. Now, B and C are not independent, and when expanding this covariance things are difficult to untangle. On the other hand, the joint moment is also very complex. It would be interesting to let an AI agent work on this problem, but we have not had the time and resources to do it. In any case, as we mention in the appendix, we believe the empirical evidence in favor of the product version to be substantial. For any use-case requiring stronger guarantees, the sum version can be used without a problem.

Q4. How sensitive is the proposed diversity loss to batch size?

This is an interesting question, that helps illustrate the way in which the proposed Vietoris-Rips filtration loss regularizes a FM model. Overall, smaller batches do not greatly degrade performance. We provide some analysis below, experiments with RAE baseline, 10k steps ft with div loss only (using JL projections), cfg1.5. For a fair comparison, we use gradient accumulation of 2 when choosing bs 32, and of 4 for bs 16, so that the VR-filtration is computed over a smaller sample, but overall we have comparable gradient scales.

methodfidprecisionrecalldensitycoverage
bs 647.9210.8610.4971.2270.889
bs 327.9570.8570.5051.2140.886
bs 167.9490.8590.5041.2230.885

Explanation: When optimizing with MSE, gradients are spread out across pixels, irrespective of their "importance". In essence, our proposed Vietoris-Rips (div) loss assigns topological importance to pixels, so that optimization happens over those that help match the topological signatures of prediction and target. But at this scale (bs << num_classes, with high probability only one sample per class per batch at all batch sizes tested) we only capture global topological features of the data (such as class directions with respect to one another). Intuitively, classes where the model underperforms receive a stronger signal that pushes specific pixels so that it gets correctly positioned with respect to the others in the batch. This is why even with smaller batches, the loss still provides meaningful signals. We do not know what would happen in larger bs regimes, where more fine-grained intra-class topology would start playing a role.


References: [1] Ma, Nanye, et al. "Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers." ECCV 2024.

I appreciate the authors' detailed response and supplementary experiments, which alleviated some of my concerns regarding the simple method comparison and batch size. My primary concern remains that the method's effectiveness seems heavily dependent on the RAE representation; its gains in other latent spaces and the reasons for this are not yet fully understood. Therefore, I am maintaining my original rating for now, while acknowledging that the supplementary experiments further support the method's value in the RAE setting.

Thank you for taking the time to consider our response and supplementary experiments. We appreciate that the additional results have helped alleviate some of your initial concerns.

Regarding the dependence on RAE representations, we have revised the manuscript to better articulate the contribution of our work, highlighting the effectiveness of the proposed regularization in the RAE setting while more explicitly discussing the limitations of our evaluation in VAE spaces.

With respect to the underlying reasons for this lack of generalization, as we mentioned in our previous response, we suspect high-dimensional artifacts in the persistence diagrams (PDs) to play a role. To further investigate this possibility, we conducted an additional experiment using SiT/XL-2 (following the same setup as in our previous response), trained for 15k steps with MSE and MSE + Div, this time applying a Johnson–Lindenstrauss projection to 128 dimensions (rather than 512 dimensions in the previous experiment).

MethodFIDPrecisionRecallDensityCoverage
MSE3.81430.83360.66991.20530.9507
MSE + Div (l2 dist & proj)3.80700.82040.67711.14710.9417

Interestingly, we begin to observe a behavior more similar to that in our RAE experiments, with regularization improving Recall and FID at the expense of Precision. While these results are still preliminary and insufficient to establish the potential effectiveness of our method in VAE spaces, they suggest that dimensionality reduction, together with the right choice of metric, could be a promising way to address this issue. We have incorporated this discussion into the revised manuscript (Appendices B.3 and B.4)..

Thank you again for your constructive feedback.

Reviewer_DAC7OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes persistent-homology-based topological regularization for Flow Matching models to improve generation quality and diversity. It introduces an intra-sample loss based on cubical complexes and a batch-level loss based on Vietoris–Rips complexes to constrain the spatial structure of predicted velocity fields and inter-sample relationships, respectively. Experiments span MNIST, MSCOCO, and ImageNet and use a broad set of evaluation metrics. The batch-level loss yields promising results when fine-tuning in the RAE latent space.

----- after rebuttal ----- The response has addressed most of my concerns. Thus I raise the rating to weak acceptance.

Critical strengths
  • The paper presents an interesting approach that complements pointwise MSE supervision with topological constraints, offering a meaningful perspective on regularization for generative models.

  • In the MSCOCO experiments with RAE representations, MSE + Diversity reduces FID from 15.798 to 13.779 and increases Recall from 0.557 to 0.584 compared with continued MSE training. These results provide encouraging evidence of improved generation quality and distribution coverage.

  • The evaluation considers multiple complementary metrics, including FID, Precision, Recall, and Density. Ablations of the loss components further help assess their respective contributions.

Critical weaknesses
  • At the end of Section 4.2, the discussion of the multiplicative rather than additive loss formulation contains an unresolved reference, “Appendix ??”. This reference should be corrected to identify the supplementary analysis supporting this design choice.

  • The results do not consistently support the claim of simultaneously improving fidelity and diversity. On MSCOCO, MSE + Diversity improves Recall but reduces Precision from 0.804 to 0.793. The Fidelity loss also does not consistently improve FID over MSE. A more qualified discussion of these trade-offs would better reflect the experimental findings.

  • The specific contribution of persistent homology remains insufficiently established. Comparisons with simpler regularizers, such as standard pairwise distance matching or distance matching over randomly selected sample pairs, would help clarify whether the gains arise from topological structure or more general inter-sample geometric constraints.

  • The effectiveness of the proposed regularization appears to depend on the representation space. The main gains occur in the RAE latent space, while the Stable Diffusion 2.1 VAE experiments show no improvement and the CIFAR pixel-space gains are limited. Further discussion of this dependence would help clarify the method’s applicability and limitations.

Questions for authors

see weakness

2 comments
AuthorsOpenReview ↗

Coparison against geometrical regularizers and VAE space discussion

We thank the reviewer for their detailed and constructive comments. We address your concerns as follows.

W1: Additive loss formulation missing reference.

We thank the reviewer for noticing this crucial omission. The broken reference pointed to Appendix B, where we study the properties of the additive version of the loss, in particular the fact that optimizing it leads to optimization of the topology of the marginal vector field. We cannot with certainty guarantee that a similar bound to Claim B.8 exists for the product version of the loss (more details below). We only suspect it to be the case due to two facts: (1) empirically, we obtain better results for the product than the sum, and (2) results for the product loss are close but slightly better to the case when we detach the geometrical alignment component so gradients only flow through the topological component (and we do have theoretical guarantees for this case, see Corollary B.4). Based on this, we settle for the product version of the loss, but note that for any use-case requiring stronger guarantees the sum version can be used wo a problem.

W2: The results do not consistently support the claim of simultaneously improving fidelity and diversity.

We acknowledge this limitation. There's often a trade-off between diversity and fidelity, hidden when considering only FID as an optimization metric, and we do not pretend to propose a single recipe to improve both quantities simultaneously. Rather, we propose two different post-training methods that target one quantity or the other. But they can be combined. For example, MSE + Fid + Div achieves the second highest precision and recall (it is bested by diversity and fidelity losses separately), as well as second highest FID.

If it is of interest for the reviewer, we have computed statistical significance tests with different random seeds to further support our claims (see response to reviewer M4jn for the full table). MSE + Diversity consistently improves in FID over MSE: 7.0667 ± 0.1942 vs 7.5685 ± 0.2531.

W3: The specific contribution of persistent homology remains insufficiently established.

We thank the reviewer for the suggestion to add an ablation against simpler regularizers. We add results below. We compare (1) cosine distance (pred vs tgt), (2) random pairwise cosine distance (between randomly selected pairs in the batch, n=16), (3) geometric alingment component of the diversity loss only, (4) geometric alignment component + MSE, (5) MSE + Fid and, (6) only the diversity loss. All results for 10k ft steps on the DiTDH-S RAE baseline, cfg 1.5.

methodfidprecisionrecalldensitycoverage
Cosine8.5670.8580.4821.2250.884
Random pairwise8.6430.8580.4861.2150.883
GeoAlign only8.7990.8570.4841.2110.882
MSE+ GeoAlign8.4250.8630.4911.2510.891
MSE + Fid8.5370.8800.4761.3400.905
Div only (proj)7.9200.8620.4971.2270.889

Div loss achieves better FID and Recall. MSE + Fid wins in precision and density. Note: due to computational limitations, we had to compute the diversity loss with JL projection, more details below. MSE + Div (without proj.) achieves FID of 7.203, see paper.

W4: The effectiveness of the proposed regularization appears to depend on the representation space.

Indeed, several experiments with VAE-latents with the Div loss showed no meaningful improvements over MSE finetuning. We add numbers below, as well as our research process and the hypothesis we tested in case it can shed some light on the issue.

We believe the root cause to be the curse of dimensionality when computing VR-filtrations, as discussed in Appendix B.3. Given that VR computations depend only on pairwise distances, noise in high dimensional spaces can produce large artifacts in the PD that renders it unusable. In RAE spaces, where cosine distances are meaningful, this problem is alleviated, but we do not have that privilege in VAE or Pixel space. We've taken this limitation very seriously, and have proposed two solutions: (1) project data to a lower dimensional space via a random sparse projection (leveraging Johnson-Lindenstrauss lemma, which ensures such a projection nearly preserves pairwise distances), and (2) using different distance metrics less sensitive to this issue, such as spectral distances. Both approaches work good with RAE latents (better computational performance, at the price of slight degradation of the improvements).

However, this does not seem to be enough. It may be that 10k finetuning steps for a model trained for over 1M steps is not enough to appreciate differences wrt MSE. Or that neither spectral or cosine distances are optimal for VAE spaces. Due to computational limitations we could not test 1, and are still trying alternatives to 2. Nonetheless, we still believe this work to be a useful proof-of-concept, and a step towards geometrical regularization of FM models.

Thanks for this careful response, which addresses a large part of my concern. I will raise the rating to weak acceptance.

Reviewer_M4jnOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper uses Persistent Homology (PH)​ to regularize Flow Matching models during post-training. It proposes two differentiable topological losses meant to improve both fidelity and diversity of generated samples, and to ease the fidelity/diversity trade-off that CFG introduces. The authors say this is the first work to post-train FM models with differentiable PH features.

Critical strengths

Strength 1: A novel, well-motivated batch-level topological regularizer. The Vietoris-Rips diversity loss treats a batch of predicted velocities as a point cloud and aligns its persistence diagram with the ground-truth diagram. This gives a distributional signal that pixel-wise MSE lacks, and it targets the fidelity/diversity trade-off introduced by CFG. The design goes beyond a heuristic for two reasons. Appendix B.1 links the conditional loss to the topology of the marginal velocity field. The ablation shows that both the topological and the geometric terms are needed: FID rises to 15.117 or 9.610 when either gradient is removed. This originality is the main reason I lean toward acceptance.

Strength 2: Consistent, low-cost gains in post-training. Short Div fine-tuning beats continued MSE training on three settings:

MNIST: 4.077→3.697 COCO-RAE: 15.798→13.779 ImageNet-RAE: 9.101→8.598, using 5× fewer steps It also outperforms the Spectral and Disperse regularizers. Without CFG it scores 8.598 against 8.699 and 9.364; at cfg=1.5 it scores 7.203 against 8.454 and 8.764. The overhead is only about 1.3×, so the method is practical to adopt. The authors also report the SD-VAE failure openly, which makes these results more credible.

These strengths count for less because the gains are small and reported without variance, the step budgets differ between methods, and the method depends on RAE latents.

Critical weaknesses

Weakness 1: Only half of the claimed contribution holds, and only in a narrow setting. The title and abstract promise better diversity and fidelity. The fidelity loss does not deliver this. Adding it to MSE makes FID worse on both MNIST (4.077→4.257) and COCO-RAE (15.798→16.649). Combining it with Div is also worse than Div alone (14.703 vs. 13.779). The Div loss itself works only in RAE (DINOv2) latents with cosine distance. It gives no gain on SD-2.1 VAE latents and only limited gains in CIFAR pixel space, which the authors acknowledge in §5 and Appendix C. The underlying cause is the curse of dimensionality for Vietoris-Rips persistent homology. The paper names this problem but does not solve it. This weighs heavily against acceptance: the paper presents a general flow-matching regularizer, but the evidence supports one loss in one latent space.

Weakness 2: The empirical evidence is too thin to establish the gains. The improvements are modest; on ImageNet the gain is about 0.5 FID. No result reports seeds or variance, so it is unclear whether the differences are statistically meaningful.

Controls:​ the ImageNet comparison sets 10k Div steps against 50k MSE steps. There is no step-matched MSE control, so the gain cannot be cleanly attributed to the loss. Baselines:​ the ImageNet RAE baseline (FID 9.45) appears far from the numbers reported for RAE models. The Spectral and Disperse baselines are AI-assisted reimplementations without official code. Topology metrics:​ the paper's own topology metrics are mixed. At cfg=1.5, MSE+Div has a worse Wasserstein-H0 distance (8.700) than the baseline (7.986) and the MSE fine-tune (6.820).

Questions for authors

In weakness

2 comments
AuthorsOpenReview ↗

Statistical significance tests and clarifications

We thank the reviewer for the useful comments, and appreciate the recognition of the originality of the work. We address your concerns as follows:

W1. Only half of the claimed contribution holds, and only in a narrow setting

Adding it to MSE makes FID worse on both MNIST (4.077→4.257) and COCO-RAE (15.798→16.649)

The FID metric captures fidelity and diversity simultaneously. The Fid loss raises fidelity as claimed: highest density in COCO experiments (1.091-> 1.202), highest precision (0.804 -> 0.826), but reduced diversity leads to worse FID.

Combining it with Div is also worse than Div alone (14.703 vs. 13.779)

We acknowledge this effect, which can be due to gradients pushing in conflicting directions, but note that MSE_Fid_Div achieves highest coverage, and second highest density, precision and recall on that experiment.

No gain on SD-2.1 VAE latents and only limited gains in CIFAR pixel space

As the reviewer correctly identifies, the root cause of this is curse of dimensionality for Vietoris-Rips complexes. Working in RAE space, where cosine distances are meaningful, alleviates this issue, but we do not have that privilege in VAE space. We mention a potential solution in Appendix B.3, but acknowledge the limitation and admit we have not identified the optimal distance metrics for working in VAE spaces. For a more detailed discussion, see response to reviewer HCSe.

W2. The empirical evidence is too thin to establish the gains.

It is unclear whether the differences are statistically meaningful.

We have computed statistical tests finetuning with different random seeds for CFG 1.5 (best for MSE) and CFG 2.0 (best for MSE_Div). Setup: DiTDH-S ckpt, 10k steps ft with mse_div (using Johnsons-Lindenstrauss projection before computing filtrations, the method explained in App. B.3) and mse, same parameters as in App D2. We sample using Variance Preserving probability paths instead of linear, which is why results differ slightly from the main table (higher recall -> lower FID).

Metric: FID

CFGComparisonMean ± StdTestStatisticp-value
1.5mse_div vs baseline7.0667 ± 0.1942 vs 8.9673One-sample t-testt = -30.94< 1e-4
mse_div vs mse (n=10)7.0667 ± 0.1942 vs 7.5685 ± 0.2531Welch's t-testt = -4.812.0e-4
mse_div vs mse (n=10)median: 7.0720 vs 7.4997Mann–Whitney UU = 2.05.0e-4
2.0mse_div vs baseline6.5815 ± 0.2663 vs 8.9673One-sample t-testt = -26.87< 1e-4
mse_div vs. mse (n=9)6.5815 ± 0.2663 vs 8.0125 ± 0.3594Welch's t-testt = -9.60< 1e-4
mse_div vs. mse (n=9)Median: 6.4644 vs. 8.0764Mann–Whitney UU = 0.04.0e-4

Metric: Recall

CFGComparisonMean ± StdTestStatisticp-value
1.5mse_div vs. baseline0.6562 ± 0.0097 vs 0.4783One-sample t-testt = 58.01< 1e-4
mse_div vs. mse (n=10)0.6562 ± 0.0097 vs 0.5629 ± 0.0137Welch's t-testt = 16.95< 1e-4
mse_div vs. mse (n=10)Median: 0.6523 vs. 0.5603Mann–Whitney UU = 90.03.0e-4
2.0mse_div vs. baseline0.6170 ± 0.0100 vs 0.4783One-sample t-testt = 41.52< 1e-4
mse_div vs. mse (n=10)0.6170 ± 0.0100 vs 0.5091 ± 0.0268Welch's t-testt = 10.74< 1e-4
mse_div vs. mse (n=9)Median: 0.6184 vs. 0.4994Mann–Whitney UU = 72.01.0e-4

W3. Controls and baseline.

The ImageNet comparison sets 10k Div steps against 50k MSE steps.

Here is the complete comparison:

RAE Baseline10k MSE finetune50k MSE finetune10k MSE + Div ft20k MSE + Div ft10k MSE + Div (proj)50k MSE + Div ft
FID ↓8.9678.4527.3677.2036.5977.6476.979

Note: all computed with CFG 1.5. 50k MSE_Div achieves worse FID than 20k, but higher recall.

The ImageNet RAE baseline (FID 9.45) appears far from the numbers reported for RAE models.

The reported ImageNet RAE baseline is computed using the DiTDH-S checkpoint instead of DiTDH-XL which is the bigger model reported in the RAE paper. We thank the reviewer for pointing this out as the information was not in the paper, and we have immediately corrected this grave omision.

The Spectral and Disperse baselines are AI-assisted reimplementations without official code.

We acknowledge this limitation. To the best of our knowledge, no similar methods with open code exist in the literature.

At cfg=1.5, MSE+Div has a worse Wasserstein-H0 distance (8.700) than the baseline (7.986) and the MSE fine-tune (6.820).

It is possible to improve Wasserstein H1 at the cost of H0, and vice versa. This is what we observe in the evals, where MSE 50k achieves the best Wasserstein H0, but at the cost of H1.

AuthorsOpenReview ↗

Gentle reminder about our response

Dear Reviewer M4jn,

Thank you again for your thoughtful review. As the discussion period closes on November 18 (AoE), we wanted to kindly check whether you have had a chance to read our response.

Following your comments, we conducted additional experiments and clarifications on (1) empirical evidence, demonstrating statistically significant improvements over MSE on FID and recall; (2) experiment results, admitting the tradeoff between fidelity and diversity, but showing our methods can be combined; and (3) complementing the information regarding controls and baseline.

If any concern remains, we would be glad to provide further clarification while the discussion is still open.

Best regards, The Authors

Comments on the paper

1 comment

We thank all reviewers for their efforts and constructive feedback and suggestions. We have revised the manuscript to strengthen the experimental evidence and more clearly state the method’s scope and limitations. The major revisions are:

  1. Added a comparison against purely geometrical baselines (Appendix C.2). We computed a comparison against other purely-geometrical batch-wise metrics, including the Geometrical Alignment component of the Vietoris-Rips filtration loss, to illustrate the effect of the topological component.
  2. Strengthen the experimental evidence in RAE latents (§§4.1, Appendix C.2). We have added the full table of results for experiments with RAE latents, including statistical significance tests.
  3. Clarified the role of the representation space (§§5.2, §6, Appendix B.3, Appendix B.4). We have expanded our discussion of how the representation space affects the method, clarified the scope and limitations of our empirical evaluation, and discussed potential directions for extending the approach to VAE spaces.

We hope that these revisions have adequately addressed the reviewers' concerns. We sincerely appreciate the reviewers' constructive feedback, which has helped us substantially improve the manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.