acceptodds
Withdrawn from ICLR 2027

Equivalent Flows, Unequal Learning: Clean-Latent Prediction in Transformers

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Flow samplers consume velocity, but the neural network can predict the clean endpoint and convert it to velocity through a fixed affine readout. We study this choice with JLT, a latent Transformer in a frozen variational autoencoder (VAE) representation. For squared error, the optimal clean and velocity predictors are algebraically equivalent; a finite Transformer assigns different computation to its learned output under the two interfaces. A local Gaussian analysis identifies a known residual response supplied by the readout and isotropic target variance added by velocity prediction. Measured FLUX.2 channel spectra support this geometric distinction: 90% of target variance occupies 83 of 128 clean directions versus 109 velocity directions. Under a matched velocity objective, clean prediction improves ImageNet FID-50K from 6.56 to 2.70 at Base scale and from 2.12 to 1.47 at Large scale, with lower FID at every measured Large checkpoint. Scaling clean prediction to 951M parameters reaches FID-50K 1.19 and IS 271.96. In addition, an objective ablation at Base scale shows that direct clean regression reaches FID-50K 2.38 without time-dependent error weighting. These results show how moving known computation outside the network changes learning under algebraically equivalent flow interfaces. Code: https://anonymous.4open.science/r/JLT-0482/README.md

Primary area:
generative models
Keywords:
Flow Matching, Output Parameterization, Clean-Latent Prediction, Diffusion Transformers, Latent Generative Models, Denoising

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_USmeOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies the effectiveness of x-prediction comparing to v-prediction in Diffusion Transformer equipped with VAE latent space. The authors argue that x-pred supplies a known residual response outside the network and lets the network concentrate on structured clean targets. They support this interpretation with a Gaussian analysis, channel covariance measurements, and competitive results on ImageNet.

Critical strengths
  1. The paper presents competitive FID score on ImageNet 256x256 dataset.
Critical weaknesses

My main concern is the positioning of the paper.

If the paper aims to identify a problem (i.e. "does this equivalence survive the replacement of an ideal predictor by a finite neural network" in line 78-79) and propose a solution / novel training method (i.e. JLT), the formulation is identical to JiT (Li & He, 2026, Section 4.2, the x-prediction formulation & the time-dependent weighting) with the only difference being training the model in VAE latent space instead of pixel.

If the paper aims to provide theoretical insights on why x-pred works better than v-pred, the proposition 1 is made on an extremely simple case with a multi-variant Gaussian with no practical experimental evidence other than Figure 5, which I believe again overlaps with the message conveyed from Figure 2 in Li & He, 2026.

Either way, unless the authors providing more evidence on how this work establishes itself on top of JiT, I'm leaning towards rejection.

Li, Tianhong, and Kaiming He. "Back to basics: Let denoising generative models denoise." In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 36115-36125. 2026.

Questions for authors

Could the author provide concrete evidence on the novelty / technical insights of the paper on top of JiT?

Reviewer_MhUMOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper extends JiT’s use of x0-prediction with a velocity-space loss from pixel space to VAE latent space. The network predicts the clean latent, which is analytically converted to velocity for training and sampling. The authors use a local Gaussian model and measurements of FLUX.2 latent covariance to explain the differences between clean and velocity prediction. They compare the two across model sizes and training losses. On ImageNet-256 with FLUX.2 VAE, clean prediction reduces FID from 6.56 to 2.70 for the Base model and from 2.12 to 1.47 for the Large model. The Huge model achieves an FID of 1.19.

Critical strengths
  1. The paper shows that clean prediction remains useful in VAE latent space.
  2. The analysis of FLUX.2 latent spectra provides useful evidence about the structure of the prediction targets.
  3. The paper describe the idea and method clearly.
Critical weaknesses

I have significant concerns about this paper.

  1. The paper introduces JLT, but what is new about it? A VAE followed by a diffusion Transformer is already a standard setup for image generation. What distinguishes JLT from this existing framework?
  2. Although the paper cites JiT, it does not explicitly discuss the extent of the overlap. The writing gives the impression that this paper introduces x0x_0-prediction with vv-loss. However, JiT already discusses all nine combinations of x0x_0, ϵ\epsilon, and vv as prediction targets and loss spaces in Table 1. The main methodological difference here is that the experiments are conducted in latent space rather than pixel space. Figure 5 largely supports the same intuition discussed in JiT: clean data have a more concentrated, lower-dimensional structure than noise-containing targets. I find the current presentation misleading, as it makes the contribution appear more substantial than it is. The paper should clearly distinguish the formulation inherited from JiT from its additional analysis and experiments.
  3. Why is FLUX.2 VAE the primary choice, and why is the improvement much smaller with Qwen-VAE? Other established choices for ImageNet experiments include the Stable Diffusion VAE [1] and VA-VAE [2]. Why are these not evaluated? Without broader comparisons or an explanation of the discrepancy between FLUX.2 and Qwen, it is difficult to judge how general the findings are.

[1] Rombach et al. High-Resolution Image Synthesis with Latent Diffusion Models. CVPR 2022.

[2] Yao et al. Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models. CVPR 2025.

Questions for authors
  1. Does the improvement hold with SD-VAE and VA-VAE? Please add ablation.
  2. Why does Table 2 use vv-loss when x0x_0-loss performs better in Figure 6? Which loss works best at each model scale?
  3. Could the authors provide all nine prediction/loss combinations (x0,ϵ,vx_0,\epsilon,v), following JiT’s Table 1, to separate parameterization and loss-weighting effects?
  4. I suggest the authors to clear seperate the contribution of this paper and previous works.
Reviewer_1toxOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper trains a JiT-style latent Transformer (JLT) on frozen VAE latents to predict the clean latent instead of the velocity.

Critical strengths
  1. The presentation of the paper was clear, and the key aspects of the methodology were well articulated.
Critical weaknesses
  1. The headline comparison does not use matched CFG settings. The x-prediction numbers in Table 1 and the abstract are the best points of per-model CFG sweeps. The v-prediction numbers, by contrast, are the endpoints of the Fig. 4 training curves, and on those same curves, x-prediction only reaches 2.84 (B/1).

  2. The evaluation is too thin to support the claims. The results' FID of 1.19 is already at or below the FID of real ImageNet validation images (1.78 [1]). In this range, further decreases in FID no longer reliably reflect sample quality [2]. Yet every conclusion rests on this single metric, measured with a single seed.

  3. The empirical finding does not go beyond the existing dimensionality explanation. With FLUX.2 VAE (128 dimensions per token), x-prediction improves FID by 56.7% under the matched setting. With Qwen-Image VAE, the gain drops to 4.5%. Qwen-Image VAE has 16 latent channels instead of 32, so about 64 dimensions per token. The benefit therefore tracks per-token dimensionality, which is exactly the explanation already given by JiT [3] and by [4].

  4. Missing citations: [5–9].

[1] Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction. NeurIPS 2024.

[2] The Role of ImageNet Classes in Fréchet Inception Distance. ICLR 2023.

[3] Back to Basics: Let Denoising Generative Models Denoise. 2026.

[4] Revisiting Diffusion Model Predictions through Dimensionality. 2026.

[5] Hierarchical Text-Conditional Image Generation with CLIP Latents. arXiv 2022.

[6] Dynamic Dual-Output Diffusion Models. CVPR 2022.

[7] Imagen Video: High Definition Video Generation with Diffusion Models. arXiv 2022.

[8] Common Diffusion Noise Schedules and Sample Steps Are Flawed. WACV 2024.

[9] The Unreasonable Effectiveness of Gaussian Score Approximation for Diffusion Models and its Applications. TMLR 2024.

Questions for authors
  1. Please report the rFID of the VAE decoders used, as well as sFID [11], FD-DINOv2 [12] and precision/recall [13] for both parameterizations at all scales.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.