- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Hybrid Diffusion, a latent-to-pixel diffusion framework for image generation under limited-data regimes. The method performs most of the early denoising process in the latent space of a frozen VAE, then switches at a predefined noise level to a conditional pixel-space denoiser for low-noise refinement. To reduce the mismatch between training and inference, the pixel stage is conditioned on decoded predictions produced by the latent denoiser rather than on clean VAE reconstructions. Experiments on several low-shot datasets, FFHQ subsets, and BRECAHAD show improvements over multiple pixel-space and latent-space diffusion baselines.
Critical strengths
- The paper addresses a meaningful problem and presents a simple, practical formulation. The complementary behavior of latent and pixel diffusion is particularly relevant in limited-data regimes: latent modeling reduces the difficulty of learning global structure, while low-noise pixel refinement can recover details constrained by a frozen VAE.
- The empirical evaluation is broad across limited-data settings. The paper evaluates multiple low-shot datasets, FFHQ subsets, and a higher-resolution medical dataset, and additionally demonstrates that the framework can be applied to a SiT backbone.
Critical weaknesses
- The methodological novelty is not sufficiently distinguished from prior cascaded and latent-to-pixel diffusion approaches. The general strategy of obtaining a coarse or latent representation before pixel-space refinement has substantial precedent, including DiffuseVAE [2], Latent Forcing [3], Hi-DiT [4], and PiD [5]. Moreover, the proposed model-produced conditioning is closely related in motivation to conditioning augmentation in Cascaded Diffusion Models [1], since both aim to expose the downstream stage to imperfections introduced by the preceding generation stage. The paper does not sufficiently clarify what is fundamentally new beyond adapting these ideas to limited-data generation.
- The experimental validation does not include the most direct alternatives to the proposed method. The main baselines are predominantly pure pixel-space or pure latent-space diffusion models, so the experiments demonstrate that a hybrid approach can be beneficial but do not establish that this particular hybrid design is preferable to existing latent-to-pixel or generator-refiner approaches. This concern is particularly important because the paper's central contribution is precisely the latent-to-pixel transition and cross-stage conditioning strategy. Broader side-by-side qualitative comparisons would also help verify the claimed recovery of fine-grained details.
References
[1] Ho, J.; Saharia, C.; Chan, W.; Fleet, D. J.; Norouzi, M.; and Salimans, T. 2022. Cascaded Diffusion Models for High Fidelity Image Generation. Journal of Machine Learning Research.
[2] Pandey, K.; Mukherjee, A.; Rai, P.; and Kumar, A. 2022. DiffuseVAE: Efficient, Controllable and High-Fidelity Generation from Low-Dimensional Latents. Transactions on Machine Learning Research.
[3] Baade, A.; Chan, E. R.; Sargent, K.; Chen, C.; Johnson, J.; Adeli, E.; and Fei-Fei, L. 2026. Latent Forcing: Reordering the Diffusion Trajectory for Pixel-Space Image Generation. ICML.
[4] Shen, Y.; Li, Y.; Pan, Y.; Zhang, Y.; and Yao, T. 2026. Hi-DiT: Hybrid Latent-Pixel Diffusion Transformer for Image Generation. ECCV.
[5] Lu, Y.; Wu, Q.; Wu, J. Z.; Wang, Z.; Ling, H.; Fidler, S.; and Ren, X. 2026. PiD: Fast and High-Resolution Latent Decoding with Pixel Diffusion. NeurIPS.
Questions for authors
- How does the proposed model-produced conditioning differ conceptually and empirically from conditioning augmentation in Cascaded Diffusion Models [1]? In particular, have the authors compared against a simpler baseline that directly corrupts the clean decoded condition rather than producing the condition through the latent denoiser?
- Can the authors provide comparisons against more direct latent-to-pixel or generator-refiner approaches, such as DiffuseVAE [2], Latent Forcing [3], Hi-DiT [4], or PiD [5]? Such comparisons would materially affect my assessment of the novelty and effectiveness of the proposed hybrid design.
- Can the authors provide broader side-by-side qualitative comparisons with latent and hybrid baselines? Since a central claim is that the pixel stage recovers details lost by latent diffusion, direct visual evidence across multiple datasets would help determine whether the improvement comes from the proposed refinement mechanism.
- What is the additional training cost relative to the latent baseline? The inference-time analysis is useful, but the proposed training pipeline requires additional latent prediction, VAE decoding, and pixel denoising. Reporting training time or GPU-hours and peak memory would help assess the practical trade-off.