acceptodds
Under review as a conference paper at ICLR 2027

Aligning Latent Geometry for Spherical Flow Matching in Image Generation

Abstract

Latent flow matching typically treats pretrained variational autoencoder (VAE) representations as fixed, although accurate reconstruction does not ensure that their geometry is well suited to downstream generative modeling. We study whether adapting the latent geometry improves image generation while retaining the encoder and diffusion architecture. Across three tokenizer families, component-swap probes show that preserving token direction retains substantially more decoded perceptual and semantic similarity than preserving radius. Based on this observation, we project each latent token onto a fixed-radius sphere, with the encoder frozen, and lightly finetune the pretrained decoder to reconstruct from the projected representation. We then train spherical flow matching on the induced spheres. On class-conditional ImageNet-256 with FLUX.2 SiT-B/2, the complete method reduces FID from to under a matched training recipe and a fixed 50-update sampling protocol, and reaches FID in approximately fewer flow-training steps. Matched decoder controls support the combined latent construction and adaptation. Recipe-transfer comparisons yield lower FID across three tested tokenizer families and SiT-B/XL backbones, and the gain remains under a matched representation-alignment objective. Additional controls characterize reconstruction across the evaluated tokenizers.

Primary area:
generative models
Keywords:
Flow matching, image generation, spherical flow matching

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_UWnJOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a spherical flow matching framework. It improves image generation by aligning the geometry of pretrained VAE latents with the generative process. Specifically, the authors project latent tokens onto fixed-radius spheres, adapt the pretrained decoder, and perform flow matching along spherical trajectories.

Critical strengths

1.The paper provides an interesting and well-motivated perspective on the relationship between latent geometry and generative modeling.

2.The proposed framework is conceptually simple and can be applied to existing pretrained VAEs without modifying the diffusion backbone.

Critical weaknesses

1.Longer training evaluation. Although the authors provide results up to 200 epochs, it would be valuable to investigate whether the improvements persist with substantially longer training, such as 800 epochs, if computational resources permit. Such results would help establish the effectiveness of the proposed method at convergence. I would be happy to increase my score if the authors could provide these additional experiments.

2.Limited evaluation scenarios. The experiments are primarily conducted on ImageNet-256. Extending the evaluation to ImageNet-512 and text-to-image (T2I) generation would provide stronger evidence of the method's generalizability and practical applicability. I would be willing to raise my score if the authors could demonstrate consistent improvements in these settings.

3.More comprehensive CFG evaluation. While the paper reports results with classifier-free guidance (CFG), a more systematic comparison across different guidance scales would be valuable. This would help clarify whether the proposed method consistently improves generation quality under stronger guidance and whether its optimal guidance scale differs from that of the baseline.

Questions for authors

1.Longer training evaluation.

2.Limited evaluation scenarios.

3.More comprehensive CFG evaluation.

If the authors provide additional experimental results or analysis, I would be very willing to raise the score.

3 comments
AuthorsOpenReview ↗

Longer training, ImageNet-512 and guidance results

We thank the reviewer for finding our perspective on the relationship between latent geometry and generative modeling interesting and well motivated. We are glad that the reviewer sees the framework as conceptually simple and applicable to existing pretrained VAEs without modifying the diffusion backbone. We address the reviewer's three concrete suggestions below.

W1: Longer training. We continued training the FLUX.2 SiT-B/2 pair of Tab. 3 with the same recipe (Tab. A1). FID-50K at CFG 1.0, Euler with 250 steps:

EpochsLinearSpherical
80 (Tab. 3)26.1420.21
20021.5816.83

The relative improvement stays the same with longer training: 23% at 80 epochs and 22% at 200 epochs. The gain also holds for SiT-XL/2 at 200 epochs (Tab. A3): 9.15 vs. 8.35 at CFG 1.0 and 3.22 vs. 2.91 at CFG 1.5. We will add the full 800-epoch comparison to the final version.

W2: ImageNet-512 and T2I. Our method only changes the latent space, the decoder and the sampling path, so the backbone and the conditioning stay as they are. The projection is applied to each token separately (Eq. 5), so it does not depend on the resolution. Our spherical decoder, trained at 256, reaches rFID 0.813 on ImageNet-512 without any training at 512, compared with 0.815 for the original decoder (50K images). For T2I, the gains already hold with the FLUX autoencoders that current T2I models build on: FID improves from 38.00 to 26.07 with REPA-E FLUX.1 and from 26.14 to 20.21 with FLUX.2 (Tab. 3). We will also add a text-to-image comparison on COCO to the final version.

W3: Guidance. We swept eight guidance scales between 1.0 and 4.0 for the 80-epoch pair, with the same solver, steps, labels and seeds for both methods. At each method's best scale (50K samples):

MethodBest scaleFID-50KsFIDRecall
Linear2.05.266.800.42
Spherical1.754.616.080.48

Spherical also has higher IS and precision than Linear at all eight scales.

Along with this response, we have uploaded a revised PDF in which new text is marked in blue. In the revision, Sec. 5 summarizes the longer training and guidance results, with details in Tab. A3 and Sec. C.14. Sec. C.15 gives the 512 decoder result. We hope this addresses the reviewer's concerns, and we would appreciate it if the reviewer would consider raising the score.

Dear Reviewer UWnJ,

As the discussion period is ending soon, we wanted to check whether the new results address your three points.

To summarize briefly: at 200 epochs, FID improves from 21.58 to 16.83, the same relative gain as at 80 epochs. On ImageNet-512, our decoder reaches rFID 0.813 without any training at 512, compared to 0.815 for the original decoder. For guidance, we swept eight scales, and at each method's best scale Spherical reaches FID 4.61 vs. 5.26 for Linear.

We have uploaded a revised manuscript, with new text marked in blue. These results are in Sec. 5, Tab. A3, and Sec. C.14 and C.15.

You mentioned you would consider raising your score if these results were provided, so we hope they are helpful. If anything is still unclear, please let us know and we will answer before the deadline.

Best regards,

The Authors

Thank you very much for your efforts; the authors have provided additional experiments demonstrating the effectiveness of the proposed approach. I have decided to increase my score.

Reviewer_vFfQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper’s idea changing the geometry of a pretrained VAE’s latent space from linear to spherical can make image generation easier. It is based on observation that each latent token sits near a spherical shell, and a token’s direction carries most of what the decoder uses. It normalizes latent tokens onto spheres, trains/adapts the decoder to those new tokens. Then the paper trains a spherical flow matching model along spherical paths. The strongest controlled experiment improves FID from 26.35 to 20.55.

Critical strengths

(+) the design has a clear empirical motivation which is discussed

(+) The main result is reasonable

(+) It’s cheap and it can be dropped into existing diffusion models, such as DiT or SiT or MM-DiT. It is architecture-agnostic.

Critical weaknesses

(-) The improvement seems to shrink with scale and longer training, and reconstruction is getting worse.

(-) the practicality is a bit untested. All reported generation experiments use ImageNet-256, with SiT models. Although the paper uses VAEs from FLUX.2 and REPA-E FLUX.1, it does not evaluate the method within a complete text-to-image system. whether such a method improves a practical t2i or i2i setting on prompt adherence, compositional generation, higher resolutions, and varying aspect ratios remain untested

(-) the paper tests a few VAEs, and benefit is somewhat tokenizer-dependent. The paper does not establish a reliable criterion for when the method will help, and do analysis on when to use spherical and when to use linear.

Questions for authors

None

2 comments
AuthorsOpenReview ↗

Longer training, practicality and when to use spherical latents

We thank the reviewer for noting the clear empirical motivation of our design and the reasonable main result. The reviewer also points out that the method is cheap and architecture-agnostic, so it can be dropped into existing diffusion models such as DiT, SiT or MM-DiT.

W1: Scale, training length and reconstruction. The gain remains with longer training. When we continue the FLUX.2 SiT-B/2 pair of Tab. 3, the relative FID improvement is 22% at 200 epochs, about the same as at 80 epochs (23%, details in our response to UWnJ). The gain also holds for the larger model: SiT-XL/2 improves FID from 13.00 to 10.71 at 80 epochs and from 9.15 to 8.35 at 200 epochs (CFG 1.0, Tab. 3 and A3). At CFG 1.5, its relative gain even grows with training, from 8.6% to 9.6%. With each method at its own best guidance scale, Spherical also has the lower FID-50K (4.61 vs. 5.26).

Reconstruction is preserved. In the matched comparison of Tab. A2, the adapted decoder reconstructs as well as the original autoencoder (FLUX.2 rFID 0.882 vs. 0.894). At 512, without any training at that resolution, it reaches 0.813 vs. 0.815. The adaptation takes only 5 epochs, with the encoder frozen.

W2: Practicality. Our method only changes the latent space, the decoder and the sampling path, so the backbone and the conditioning, including the text encoder, stay as they are. The projection is applied per token and the adapted decoder keeps the original architecture, so every resolution and aspect ratio that the original autoencoder supports works the same way. The gains hold with the FLUX.1 and FLUX.2 autoencoders that current T2I models build on (table below). The final version will also add a text-to-image comparison on MS-COCO. For inversion-based editing, which relies on an accurate data-to-noise-to-data round trip, Spherical's round-trip error is 22% lower than Linear's (4.5×10−54.5\times10^{-5} vs. 5.8×10−55.8\times10^{-5}, Tab. A12).

W3: When to use spherical latents. The encoder statistics give a simple rule that can be checked before any training: use spherical latents when the token radii are concentrated, with a radius CV below 0.25 and less than 1% of tokens below half the mean radius Rˉ\bar{R} (Tab. A10).

TokenizerRadius CVTokens below Rˉ/2\bar{R}/2FID Linear / Spherical (Tab. 3)
REPA-E FLUX.10.2340.20%38.00 / 26.07
FLUX.20.2010.014%26.14 / 20.21
VA-VAE0.1660.0004%26.99 / 21.96

SD-VAE (d=4d = 4, radius CV 0.407, 8.77% of tokens below Rˉ/2\bar{R}/2) does not meet the rule (Sec. C.8).

Along with this response, we have uploaded a revised PDF in which new text is marked in blue. In the revision, Sec. 5 includes the longer training and guidance results and states this criterion. Tab. A3 and Sec. C.14 give the details, and Sec. C.15 gives the 512 decoder result.

Dear Reviewer vFfQ,

As the discussion period is ending soon, we wanted to check whether our response addressed your concerns.

To summarize briefly: the gain holds with longer training (22% relative FID improvement at 200 epochs, vs. 23% at 80) and with SiT-XL/2 (9.15 to 8.35 at 200 epochs). The adapted decoder reconstructs as well as the original one (rFID 0.882 vs. 0.894). The gains also hold with the FLUX.1 and FLUX.2 autoencoders that current text-to-image models use. Finally, we give a simple rule for when to use spherical latents, which can be checked from the encoder statistics before any training.

We have uploaded a revised manuscript, with new text marked in blue. These results are in Sec. 5, Tab. A3, and Sec. C.14 and C.15.

If anything is still unclear, please let us know and we will answer before the deadline. If your concerns are resolved, we would appreciate it if you could take this into account in your final rating.

Best regards,

The Authors

Reviewer_ncSeOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies spheric latents adapted and encoded by pretrained VAE encoders and Riemannian flow matching on spheric latents. First, this paper studies which factor impacts larger on reconstruction quality, radius or angle, and then find that latent angles determine most of the reconstruction content and changing radius brings little degradation. Therefore, the authors propose to normalize pretrained latents into spheric latents whose radius all equals, and only finetune decoders to adapt to these latents. As a suite, the paper furthermore train Riemannian flow matching on these spheric latents and obtain competitive results.

Critical strengths
  1. This paper has conducted extensive experiments with step-by-step decomposition of the performance of spheric latents and Riemannian flow matching on such latents. In Table 2, the spheric latent only comes with FID drop from 26.35 to 22.85, and RFM on spheric latents also reduce it to 20.55, which is appealing.

  2. This paper explores a new perspective on how to design latents, especially on systematic studies on latent geometry, where previous research have few explored this point.

Critical weaknesses
  1. The key concerning and critical weakness is that this paper shares too much similarity with Geometry-Aware Image Flow Matching [1](ICML 2026), from what components of latent geometry impact the most (direction or radius), to RFM on such spheric latents, especially slerp methods. Given the previous research, this paper seems to present almost few new information and finding. Also, [1] is first publicly available on openreview.net as ICLR 2026 submission on Sep, 2025.

[1] Geometry-Aware Image Flow Matching, ICML 2026

Questions for authors
  1. Besides the compact manifold, what factors does the author think are the reasons that the spheric latents are better than latents with just no geometry assumptions?
2 comments
AuthorsOpenReview ↗

Relation to related work and why per-token spheres help

We thank the reviewer for finding the paper clear and easy to follow, the experiments extensive, and the step-by-step decomposition of Tab. 2 appealing. We are glad that the reviewer sees our systematic study of latent geometry as a new perspective that previous work has rarely explored.

We also thank the reviewer for pointing us to [1], which we now cite and discuss. Our paper is not an extension of [1]. [1] puts each image on a single sphere, while we put each token on its own sphere, and all of our contributions follow from this token-level construction. We developed this work independently of [1].

W1: Relation to [1]. What the two papers share is the spherical flow matching machinery, i.e. slerp paths and tangent targets, which is Riemannian flow matching (Chen & Lipman, 2023). Our paper calls it established (l. 095) and does not claim it as a contribution. The differences are:

[1]Ours
What lies on the sphereeach image (1 sphere)each latent token, Eq. 5 (256 to 1,024 spheres per image)
Direction vs. radiuswhole-image direction carries the content ([1] Tab. 1)measured per token on three tokenizers: keeping directions and resetting radii gives DINOv2 cosine 0.86-0.97 to the original decoding, keeping radii gives 0.01-0.02
Decoderoriginal decoderper-token projection breaks the original decoder (rFID 17.37), and adapting only the decoder with the encoder frozen brings it to 0.882, vs. 0.894 for the original autoencoder (Tab. A2)
When it appliesnot studiedtoken radius statistics (Tab. A10, Sec. C.8)

We also compared the two directly. We trained [1]'s method with its own settings (one sphere per latent, [1]'s loss, original decoder), using the backbone, data, training length, sampler and evaluation of Tab. 2 (FLUX.2, SiT-B/2, 80 epochs). [1] reaches FID-50K 24.83, while ours reaches 20.55. With linear transport, our per-token construction alone already reaches 22.85.

Q1: Why spherical latents help beyond compactness. Tab. 2 splits our gain into two steps: the per-token construction (26.35 to 22.85) and spherical transport (22.85 to 20.55). We see four reasons, each supported by a measurement:

  1. The decoder is trained on the same kind of latents that the flow generates. Generated tokens stay at radius d\sqrt{d} (maximum deviation 1.4×10−61.4\times10^{-6}, Tab. A12). With the original decoder, the same projected latents give FID 43.17 (Tab. 2).
  2. The regression target is purely angular. On linear paths, about half of the squared target velocity near the endpoints is radial for FLUX.2 and VA-VAE, and about 90% near the noise endpoint for REPA-E FLUX.1 (Fig. A1), even though the radius carries little content (see the table above). Removing the tangent projection increases FID from 20.55 to 25.79 (Tab. A4).
  3. All tokens have the same log-SNR at time tt. With unequal radii, the log-SNR of a token zz is shifted by 2ln⁡(∥z∥/d)2\ln(\|z\|/\sqrt{d}), and this shift varies by 0.26-0.41 nats within an image. Per-token projection removes the shift. An image-level projection such as [1] keeps it, which is consistent with the gap in our direct comparison.
  4. Pretrained tokens already lie close to a shell (radius CV at most 0.234, Tab. A10), so fixing the radius removes a coordinate that carries little content.

Along with this response, we have uploaded a revised PDF in which new text is marked in blue. In the revision, Sec. 2 discusses [1] together with Yue et al. (2026) as image-level spherical latents. Sec. 5 now reports the direct comparison with [1] (Tab. 5) and the Q1 discussion, and Sec. C.16 and C.1 give the details.

We hope this clarifies that our contributions are distinct from [1], and we would appreciate it if the reviewer would reconsider their rating.

Dear Reviewer ncSe,

As the discussion period is ending soon, we wanted to check whether our response addressed your concern about [1].

To summarize briefly: [1] puts each image on a single sphere, while we put each latent token on its own sphere, and our contributions build on this token-level construction. We also ran [1]'s method in our setting with everything else matched, and it reaches FID 24.83 compared to 20.55 for ours.

We have uploaded a revised manuscript, with new text marked in blue. It discusses [1] in Sec. 2 and includes the direct comparison in Tab. 5.

If anything is still unclear, please let us know and we will answer before the deadline. If your concern is resolved, we would appreciate it if you could take this into account in your final rating.

Best regards,

The Authors

Comments on the paper

1 comment
Area_Chair_hJG2OpenReview ↗

Please engage in the discussion

Dear Reviewers,

Thank you for your reviews and contributions so far. The authors have now posted their response, and I would like to kindly remind you to review it and engage in the discussion where appropriate.

Please also consider whether the authors' clarifications address any of your concerns and update your assessment if warranted.

Thank you for your time and effort.

Best regards,

Your AC.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.