- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes Sphere Encoder 2 (SE2), which considers two limitations of Sphere Encoder (SE): insufficient sphere coverage during training and the use of a per-pixel reconstruction loss for generation. Specifically, it reparameterizes the noise so that training covers the full [0°, 90°] angle range, and decouples the decoder's task into two regimes: low-noise reconstruction and high-noise generation. Experiments show improvements over SE.
Critical strengths
-
Linking reconstruction and generation through the rotation angle is an interesting view.
-
It demonstrates performance improvement over SE.
Critical weaknesses
- Potential self-contradictions and inconsistencies.
- The paper identifies SE's [0°, 85°] angle range as a limitation and proposes a reparameterization that extends it to [0°, 90°]. However, Sec. 3.2 sets α = 85° by default, and Fig. 15 shows that 86° gives the best performance.
- For Oxford Flowers 256, the default configuration of SE2-B yields FDr6 values of 3.92/3.04 in Tab.3,4,6. However, the values in Tab.1 are 2.67/2.17.
- On ImageNet, the 4-step gFID for SE2-B is 3.66 in Fig.7, whereas it is 2.79 in Tab.2.
- Sec.C.2 states that "...gives the best rFID" in Fig. 13. However, there seems to be no rFID in that figure.
- Sec.C.3 mentions "Sphere-H to report the main results in Tab. 2". However, no Sphere-H results are reported there.
- Design and ablation issues.
- The authors introduce SE in Sec.2, but lack sufficient background to demonstrate their differences and provide experimental advantages compared with the standard VAE.
- The ablations of losses are merged. Detailed ablations of each component are needed.
- Section 3.4.2 claims that "adversarial loss...prone to mode collapse". Please provide a comparison with adversarial loss or training curves.
-
Overclaim and missing SOTA. SE2 significantly lags behind pMF-H and Drift-L at 1 NFE. Also, comparisons with strong baselines like PixelDiT are encouraged.
-
There are many typos. For example,
a autoencoderin the abstract,We replacesin L152.
Questions for authors
-
In L342, the author claims that DINOv3-small has only 29M parameters with trivial FLOPs. Detailed training cost comparisons are necessary.
-
I am not sure what the footnote "We release Sphere2-B checkpoints ... in our repo ..." on page 7 means. The supplementary materials contain no code or checkpoint files.
-
Can this method extend to text-to-image scenarios?
-
Tab. 2 only reports 1 and 4 steps on ImageNet. How about increasing steps to 8, 32, or more?
3 comments
Thanks for the response. My concerns remain:
- The numerical inconsistencies raised in my original review remain unexplained.
- Quantitative comparisons with standard VAEs and adversarial losses, as well as an analysis of DINO training costs, are still missing.
- Multiple inconsistencies exist between the original and revised versions. For example, Sphere2-B's 4-step FDr6/gFID changed from 7.82/2.79 to 7.95/3.07, while training epochs changed from 400 to 300, warmup from 5 to 10, and EMA decay from 0.9998 to 0.99985, without explanation.
We thank the reviewer for the response to our rebuttal.
For Oxford Flowers 256, the default configuration of SE2-B yields FDr6 values of 3.92/3.04 in Tab.3,4,6. However, the values in Tab.1 are 2.67/2.17.
This is explained in Q4 response to reviewer GEM3. Please forgive us not pointing this in the reponse.
On ImageNet, the 4-step gFID for SE2-B is 3.66 in Fig.7, whereas it is 2.79 in Tab.2.
This a typo in Fig.7. Thanks for catching this. We fix all numbers in Fig.7 (now Fig. A15) for both sides.
Multiple inconsistencies exist between the original and revised versions. For example, Sphere2-B's 4-step FDr6/gFID changed from 7.82/2.79 to 7.95/3.07, while training epochs changed from 400 to 300, warmup from 5 to 10, and EMA decay from 0.9998 to 0.99985, without explanation.
The old Table 7 in the original version (now Table A14) was prepared in a rush near the submission deadline and contained typos. The expanded Table A14 in the revised version is the final version, and it directly reflects the results in Table 2. We are happy to provide all checkpoints and code for verification if needed.
Warmup epoch numbers. The value of 5 epochs was a typo for ImageNet. We first used the hyperparameters on Oxford Flowers with 10 warmup epochs, then reused them on ImageNet. While debugging, we tried a shorter warmup of 5 epochs on ImageNet, since ImageNet is much larger than Oxford Flowers. However, this change was never applied on our main training cluster (we didn't share the same training script due to diffetent GPU environments between two clusters, one is Nvidia and the other is Amd). Then all final ImageNet models were still trained with 10 warmup epochs when we check the config to finish the Tabel A14.
EMA decay numbers. Behind the table, we swept EMA decay values of 0.9998, 0.99985, and 0.9999 for both the base and large models (by resuming from checkpoints at ep100 using 0.9998, to train the rest 300 epochs, total 400 epochs). As explained in General Response G1, not all evaluations were finished by the submission deadline. In the original version, we reported Sphere2-B with 0.9998 (400ep) and Sphere2-L with 0.99985 (300ep, because it didn't finish 400ep training yet when we submitted the paper) and the old Table 7 did not reflect this difference. In the revised version, we use a single EMA decay of 0.99985 for both model sizes. Sphere2-B therefore now uses 0.99985 (300ep), which gives slightly worse results than 0.9998 (400ep). We chose consistency across model sizes over the best per-model number, and our conclusions are unchanged.
Compare with adversarial loss
Table 2 already includes a modern GAN baseline: GAT-XL/2 (ICML 2026), a latent transformer GAN. It uses a large model (602M parameters) and optimal GAN training recipes, and it achieves sota FID on ImageNet.
We do not expect that we could have a GAN loss that outperforms theirs when applied to our decoder (487M parameters), so GAT-XL/2 already serves as a strong GAN baseline. Moreover, latent baselines generally outperform pixel-based baselines overall on FDr6 in Table 2. So we assume this modern GAN is already strong enough to present the best GAN we could get in pixel-base space.
Frozen encoder DINOv3-small-plus cost in training
| Module | Image size | Params | GFLOPs/img fwd | GFLOPs/img bwd | GFLOPs/img total | ms fwd | ms bwd | ms total (batch 4) | Peak mem |
|---|---|---|---|---|---|---|---|---|---|
| DINOv3 ViT-S+ fwd, no grad | 256 | 28.7M | 16.19 | – | 16.19 | 11.90 | – | 11.90 | 0.14 GiB |
| DINOv3 ViT-S+ re-encode fwd+bwd (frozen) for | 256 | 28.7M | 16.19 | 18.07 | 34.26 | 14.11 | 15.74 | 29.85 | 0.39 GiB |
| Decoder base fwd+bwd | 256 | 131.4M | 34.97 | 70.69 | 105.66 | 26.46 | 45.18 | 71.64 | 1.46 GiB |
| Decoder large fwd+bwd | 256 | 459.3M | 121.69 | 245.59 | 367.28 | 51.91 | 89.73 | 141.64 | 4.37 GiB |
| DINOv3 ViT-S+ fwd, no grad | 512 | 28.7M | 16.19 | – | 16.19 | 11.96 | – | 11.96 | 0.15 GiB |
| Decoder base fwd+bwd | 512 | 133.2M | 35.87 | 72.50 | 108.37 | 26.77 | 45.84 | 72.61 | 1.52 GiB |
| Decoder large fwd+bwd | 512 | 461.6M | 122.90 | 248.01 | 370.91 | 52.54 | 91.48 | 144.02 | 4.43 GiB |
We will add this table to the appendix. In short, the frozen encoder is DINOv3-small, and its computational cost is small compared with the decoder’s.
Compare with standard VAEs
Could you clarify this suggestion? Because standard VAEs are typically used for reconstruction, or as tokenizers for latent generative models (as with the latent baselines in Table 2), we are unsure how to set up an experiment that compares a standalone standard VAE with our method on sampling new images on ImageNet. The original sphere encoder (https://arxiv.org/abs/2602.15030) already tried the random sampling from standard VAEs in Figure 3, with sota VAEs such as FLUX, which is failed due to posterior hole. We would be happy to run experiments if you could say what setup you have in mind.
We thank the reviewer
qNYEfor the comments and suggestions, and /appreciate the time spent on reviewing our paper.For the discussion of max training angle and sampling angle, please see Q2&4 in our response to Reviewer
jEUp.We put the section of relate work in the main paper in the middle of Introduction and Preliminaries, to make the reader have background.
For the over claim, please see G1: Experiment claim in our general response at the top.
For accumulative ablation, please see Q3 in our response to Reviewer
GEM3.We added Section B.6 (page 23) on the training dynamics of Lscore.
Figure A13 (page 23) tracks Sphere2-B on Oxford Flowers 256×256 from epoch 1000, when Lscore is switched on, to epoch 2000, total 1000 epochs of training.
It plots: (1) the training loss and gradient norm of the 2-transformer-layer real and fake score models; and (2) Lscore value and the magnitude of its transport direction g.
This sentence in Section 4.4.3 (Latent Consistency) compares the encoder with the decoder, not Sphere Encoder 2 with other models. The frozen DINOv3-small+ encoder has only 29M parameters, against 161M and 487M in total for Sphere2-B and Sphere2-L (Table 2), and its weights are not updated. Its cost is therefore small relative to the decoder during training. We did not mean that the overall compute of Sphere Encoder 2 is trivial compared with other methods, and we removed the sentence.
Sorry for the confusion. The footnote refers to the checkpoints we will release with the public version of the paper for the future. We have removed the footnote. In the meantime, we added large side-by-side comparisons of uncurated ImageNet 256×256 samples from Sphere2-B with and without the FD-lite loss, at 1 step in Figure A16 (page 26) and at 4 steps in Figure A17 (page 27).
We added Section B.4 (Text-to-Image Generation on Flowers, page 21), a small-scale text-to-image experiment on Oxford Flowers 256×256. The model and all training hyperparameters stay the same as the class-conditional Sphere2-B, and only three things change:
Captions. We caption its 8,189 images with Qwen2.5-VL-7B-Instruct, 20 captions per image ranging from 5 to 40 words.
Decoder condition. We encode each caption with the SigLIP2 text tower (up to 64 tokens), project the token embeddings to the decoder width, and prepend them to the image tokens as in-context tokens, following Qwen-Image-2.1 and FLUX 3 Image Klein. The text tokens replace the class embedding, and the angle embedding is still processed through adaLN-Zero.
Score model condition. The two score models of Lscore are conditioned on the pooled SigLIP2 text embedding instead of the class embedding.
Figure A12 (page 23) shows uncurated 1-step and 4-step samples for 17 unseen, out-of-distribution prompts listed in Table A7. Already at 1 step, Sphere2-B renders colors and scenes that never appear in the training images, such as a blue water lily, a purple dandelion, a pink sunflower, and a water lily growing between stones. More details are in Section B.4 (page 21).
Following this suggestion, we added Section B.3 (Sweeping Sampling Steps, page 21), which sweeps the number of sampling steps K from 1 to 500 for three pixel-space base models on ImageNet 256×256.
Each model is pushed away from the budget it was designed for: the many-step diffusion model JiT-B (50 steps) toward few steps, and the one-step pMF-B and the few-step Sphere2-B (1 to 4 steps) toward many steps. All entries are FDr6 ↓ in this compact version of Table A6 (page21).
Three observations follow:
Each model is best near its design budget. JiT-B collapses below 8 steps and plateaus from 32 steps on, pMF-B peaks at 8 steps, and Sphere2-B peaks at 4.
Sphere2-B degrades gracefully with many steps like pMF-B. Its FDr6 rises from 8.62 at 4 steps to 10.05 at 500 and then saturates, so the encode–rotate–decode loop stays stable rather than drifting away. Same behavior is also observed in pMF-B, which rises from 10.61 at 8 steps to 13.06 at 500.
We also have a Figure A11 (page 22) to show the qualitative results at 512×512 with pMF-L and Sphere2-L.