acceptodds
Under review as a conference paper at ICLR 2027

Learning Anisotropic Prediction Targets for Flow Models

Abstract

Flow-based generative models are trained on a hand-chosen prediction target, such as the velocity, the noise, or the clean data. This choice strongly affects training and sample quality. We introduce Anisotropic Flow Models (AnisoFlow), which replaces a fixed prediction target with a fully learnable one. AnisoFlow learns to scale the noise component of the target anisotropically, assigning a separate nonnegative weight to each direction in a jointly learned basis. A closed-form map recovers the velocity from the network's prediction, leaving the architecture, flow-matching loss, and sampler unchanged. Since recovery is exact, training optimizes the target purely to make it easier for the network to predict. We theoretically analyze the resulting prediction error and predict that the preferred noise weight in each direction scales near-linearly with data variance, in agreement with the weights learned empirically in pixel space. AnisoFlow improves FID from to over the JiT baseline on ImageNet , by learning only the prediction target, without representation alignment or architectural changes. In pixel-space text-to-image generation, AnisoFlow improves image quality and, through better-conditioned velocity recovery, largely removes patch-seam artifacts under reduced-precision inference. Code and models will be released.

Primary area:
generative models
Keywords:
diffusion, flow matching, image generation, pixel space

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_seK8OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AnisoFlow. It replaces the hand-designed prediction targets in flow models with a learnable one by using a learnable orthogonal basis where each direction gets its own noise weight while the data term stays fixed. It consistently improves ImageNet and text-to-image generation, and also stabilizes low-precision inference. Interestingly, the learned basis converges to the data PCA basis and the weights grow nearly linearly with directional variance, which matches the theoretical analysis.

Critical strengths
  1. The paper takes something (velocity vs. x vs. ε) treats as a fixed choice, the prediction target, and makes it learnable. Nothing else in the pipeline has to change, it never hurts performance, and the gains in pixel space are solid. So I think it is likely useful for other tasks too.

  2. The learned basis lines up with PCA and the weights follow the data spectrum. This regularity is worth reporting regardless of how it is explained, and it gives the community a concrete takeaway beyond FID numbers.

Critical weaknesses
  1. The main conclusion, that the preferred weight scales near-linearly with data variance, depends on several strong assumptions, like the noise and data errors being uncorrelated. None of them are checked. They seem picked to fit the observed results, so the theory does not really predict anything new.

  2. The learned weights basically converge to a simple analytic curve. But the paper never tests just using that fixed curve directly. So I cannot tell if the gains come from the anisotropic shape itself or from the learning process. This is a confusion for me.

  3. The operator only works inside each patch and is shared everywhere, so it ignores image content entirely, and the paper never explains why that is fine. Worse, on the mainstream latent-space setup it barely helps and just falls back to the default target. Please explain.

If you can convince me, I will increase my score,Question 1 is the most important, please convince me. (They seem picked to fit the observed results, so the theory does not really predict anything new. )

Questions for authors

See weaknesses,thank you.

4 comments

We thank the reviewer for the positive comments on the simplicity and generality of our learnable prediction target, the empirical improvements in pixel space, and the spectral structure of the learned weights and basis, which provides insights beyond the FID improvements.

W1: Checking the assumptions behind the near-linear scaling (e.g., uncorrelated noise and data errors)

Checking these assumptions exactly is difficult, for two reasons. First, the prediction target shapes the errors the network learns, while these errors in turn determine the preferred target. Second, AnisoFlow predicts a single combined target, so separating the noise- and data-estimation errors requires additional assumptions. Our analysis therefore treats the errors as locally fixed and explains the near-linear scaling under these conditions. We will state this clearly in the revised paper.

Within these limits, we add an error analysis of a trained AnisoFlow-H/16 model as a consistency check (Appendix A.3, Fig. 8). It estimates effective noise- and data-error scales that include their correlation, so it does not assume uncorrelated errors. The noise-error scale is nearly constant across directions (∝λi−0.09\propto \lambda_i^{-0.09}), while the data-error scale grows linearly (∝λi1.00\propto \lambda_i^{1.00}). Their ratio scales as λi1.09\lambda_i^{1.09}, close to the λi1.07\lambda_i^{1.07} of the learned weights.

W1: Whether the assumptions are picked to fit the results, and whether the theory predicts anything new

Beyond the near-linear scaling, our analysis predicts that the preferred weights stay approximately constant across timesteps. This is not obvious: the zero-skill solution, which corresponds to direction-wise EDM (Appendix A.2), scales with αt/σt\alpha_t/\sigma_t and thus changes strongly over time. The time-bucketed experiment in the original submission supports our prediction (Fig. 5): the four timestep-bucket profiles nearly coincide, and time-dependent weights do not improve FID (4.92 for shared wiw_i vs. 4.96 for time-dependent wi(b)w_i^{(b)} after 160 epochs). Our new error analysis agrees: while αt/σt\alpha_t/\sigma_t varies by 11.7×11.7\times across timesteps, the estimated preferred weights vary by only 1.19×1.19\times (Fig. 8(c)).

W2: Why learn the weights if they converge to a simple analytic curve

Our analysis suggests that the learned weights approximately follow wi≈cλiw_i \approx c\lambda_i, but does not determine cc in advance. Moreover, the coefficient depends on the model size: our smaller JiT-B/16 models learn c≈0.2c \approx 0.2, compared with c≈0.15c \approx 0.15 for JiT-H/16. Choosing cc for a new model would therefore require a costly hyperparameter sweep over full training runs, whereas learning the weights adds negligible overhead (Table 1).

Nonetheless, we are running a baseline with frozen PCA basis and fixed weights wi=0.15λiw_i = 0.15\lambda_i, using the coefficient obtained from our trained model. Note that even if this baseline performs well, it uses a coefficient obtained from our trained model rather than one selected independently. Results will be added later during the rebuttal period.

W3: Patch-wise operator shared across the image, ignoring image content

We use a shared operator across patches because the prediction target primarily aims to capture low-level statistics, such as directional data variance and prediction difficulty, rather than image content. Making the target content-dependent would introduce additional complexity, whereas our shared operator already achieves substantial improvements in pixel space with negligible overhead. We are also investigating how prediction-error statistics vary across spatial positions and patch content, and will report additional results during the rebuttal period if available.

W3: Limited gains in latent space, where AnisoFlow falls back to the default target

We do not claim FID improvements in latent space. The point of these experiments is that AnisoFlow does not simply learn the linear scaling but adapts to each representation: for SiT-REPA and RAEv2, it learns profiles close to their default velocity and data prediction targets (Fig. 7). This is why the target should be learned rather than fixed.

We hope our responses and additional experiments address the reviewer's concerns, and we kindly ask them to consider raising their score.

Thank you for your reply,

W1: The new error analysis is a useful sanity check, but it still feels mostly post hoc since it’s based on the trained model. It doesn’t yet convince me that the theory has real predictive power.

W2: The key fixed-weight baseline is still missing. Without it, I can’t tell whether the gains come from the target shape itself or from the learning process.

W3: The explanation for the shared operator sounds reasonable, but there’s no supporting experiment.

For latent models, the authors confirm that the method doesn’t improve over the default target.

So, the pixel space results of the paper are indeed good, so this is not a work lacking value. But, I think the author's response is not enough to change the core flaw of the paper. If a fixed weight baseline is submitted later and it is proven that the learning version is significantly better, the paper will be more reasonable.

We thank the reviewer for their follow-up and for acknowledging the value of our pixel-space results. We would like to clarify the theoretical contribution and report the completed fixed-weight baseline, which directly addresses the reviewer's main concern.

W1: Predictive power of the theoretical analysis

We agree that our new error analysis is a consistency check on a trained model. The prediction errors in our theory inherently depend on the trained network and the chosen target, so measuring them necessarily requires a trained model. However, our theory goes beyond explaining the observed near-linear scaling: it identifies conditions under which the timestep dependence of prediction errors cancels the explicit αt/σt\alpha_t/\sigma_t factor, favoring shared weights, unlike the strongly time-dependent zero-skill solution, which we evaluated and found much worse (4.26 vs. 2.30 FID at 160 epochs).

Importantly, this implication was already tested in our original submission (Fig. 5): four independently learned timestep-bucket profiles nearly coincide, and time-dependent weights do not improve FID (4.92 vs. 4.96). Our new analysis further supports this cancellation: estimated preferred weights vary by only 1.19×1.19\times across timesteps versus 11.7×11.7\times for αt/σt\alpha_t/\sigma_t (Fig. 8(c)). Thus, the theory identifies a separate, testable implication beyond a post-hoc explanation of spectral scaling.

W2: Fixed-weight baseline and the value of learning

We have completed the requested baseline with a frozen PCA basis and fixed wi=0.15λiw_i=0.15\lambda_i. At 600 epochs, it achieves 1.65 FID, compared with 1.90 for JiT, showing that the anisotropic target shape indeed accounts for much of the improvement. However, our full method, jointly learning weights and basis from W=IW=I, achieves 1.55 FID. This additional 0.10 improvement over the fixed baseline represents nearly 30% of the total FID reduction from JiT.

More importantly, the value of learning is not solely measured by its improvement over a carefully chosen fixed profile. The fixed baseline uses c=0.15c=0.15, obtained from our trained AnisoFlow model rather than determined independently. Our analysis does not predict cc, and the learned coefficient varies across model sizes (c≈0.2c\approx0.2 for JiT-B/16 versus 0.150.15 for JiT-H/16). For a new architecture, selecting cc would require additional tuning, potentially involving costly training sweeps, whereas AnisoFlow learns it automatically in a single training run.

Furthermore, learning provides a more general approach than fixing an analytic target shape. Our default W=IW=I initialization requires neither a precomputed PCA basis nor a manually specified weight profile. Instead, AnisoFlow learns both the weights and basis automatically, adapting to different architectures and representations with negligible overhead. Thus, the value of learning extends beyond outperforming a fixed profile: it provides an efficient, general way to discover suitable prediction targets across architectures and representations, while also achieving the best FID in our experiments.

W3: Empirical evidence for the shared operator

We conducted an additional analysis using the estimated error statistics from Appendix A.3, grouping 480,000 patches by spatial position and content. Across five spatial groupings, estimated preferred weights vary by at most 1.07×1.07\times, and by only 1.02×1.02\times for four of the five. Grouping patches by high-frequency content fraction yields 1.11×1.11\times variation across quartiles, while local patch variance produces a somewhat larger 1.42×1.42\times variation. In contrast, preferred weights vary by over three orders of magnitude across PCA directions. These results suggest that directional variation dominates spatial and content-dependent variation, supporting our choice of a simple shared operator.

We hope these additional experiments address the reviewer's central concerns. We would also appreciate clarification on what further evidence would be needed to change their assessment, and how they weigh the remaining weaknesses against the paper's empirical improvements, simplicity, and generality. We kindly ask the reviewer to reconsider their score.

Thanks for the additional experiments.

For W1, I still see the spectral analysis mainly as a consistency check, but the time-invariance prediction and the zero-skill comparison make the theoretical argument more convincing. For W2, the fixed-weight baseline answers my main question: much of the gain comes from the anisotropic profile itself, while learning provides a further improvement and avoids choosing the profile manually.

For W3, the spatial and content analysis gives reasonable support for the shared operator.

The limited gains in latent space remain a limitation, but I think the pixel-space results and the added controls make the paper substantially stronger, if the experiment is true.

Sooo, I am therefore inclined to raise my score to 3.

Reviewer_VcVJOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces AnisoFlow, which promotes learning the per-direction scales of the noise component, parameterized by a PSD operator with a learnable orthogonal basis and nonnegative weights. Its theoretical results suggest the learned noise weights correlate well with the data variances directionally, and empirically it obtains better performance in class-conditioned and text-conditioned image generation, compared to v-prediction in the latent space and x-prediction in the pixel space.

Critical strengths
  • The prediction family is clean and general. Exact recovery under PSD W\boldsymbol{W} contains x, v and ϵ\epsilon-prediction, k-diff-style scalar mixing and AsymFlow's binary mask as special cases.
  • The method is a drop-in change with at most 2% training overhead learning the weights and basis.
  • Empirically on pixel-space ImageNet256x256, AnisoFlow with pure rectified flow objective obtains a matching FID score compared to many other SOTA baselines (e.g., AsymFlow) with REPA-style alignment or DINO-based losses.
Critical weaknesses
  • I am not fully convinced that the target should be learned, which is the central claim of the paper, as the results in Table 1 can point to a simpler explanation. With PCA initialization, a frozen basis and a learned basic both reach 1.63 FID, so learning the basis does not seem to help there. The best result (1.55 FID) changes both the basis initialization (PCA to Identity) and weight initialization (0.15$\lambda_i$ to 1), but the first spectral update immediately yields a noisy PCA basis anyway, leaving that the improvement from 1.63 to 1.55 mainly comes from learning of the weights. Then, when I looked at Figure 4 where learned weights converges to a near-linear profile, a natural question pops up: why don't we add a Table 1 experiment with frozen PCA basis and frozen 0.15$\lambda_i$ weights for JiT-H/16? Another reasonable baseline is the analytic direction-wise EDM target the paper derives itself in Eq. (30). Both experiments can help answer if learning is really needed or a simpler procedure can obtain the same improvement.
  • The near-linear law wi∝λiw_i \propto \lambda_i is trivial as it follows directly from assuming Cˉi∝λi\bar C_i \propto \lambda_i and constant Aˉi\bar A_i, but these assumptions are never tested (i.e., these error statistics are never measured from a trained network). More importantly, a direction-wise EDM already yields proportionality to λi\lambda_i from second-order statistics alone. Therefore, I don't think the theory "predicts" the near-linear scaling as stated in contribution, but at most the theory explains it.
Questions for authors

Besides the critical weaknesses, which shall be prioritized, I have a couple minor concerns:

  • The k-diff fits as a special case of AnisoFlow with a learned isotropic scalar, but currently it's also absent from the experiments. It's a controlled experiment to test if "anisotropic" is needed.
  • The additional quantization experiment is a little incomplete. The reduced-precision stability result is qualitative only and is compared against with only x-prediction with no metric and no AsymFlow.
  • In latent space, there is no FID gain, but the interpretation is shallow. From Figure 7, both appear like valid slopes instead of horizontal lines that would've represented v prediction and x prediction.
3 comments

We thank the reviewer for recognizing the generality, simplicity, and low overhead of our method.

W1: Necessity of learning the target

Table 1 shows most gains come from learning directional weights: with a frozen PCA basis, FID improves from 1.90 (JiT) and 1.76 (AsymFlow) to 1.63. Learning the basis adds no gain with PCA initialization, while starting from W=IW=I further improves FID to 1.55.

Learning the weights is necessary in practice. Our analysis suggests wi≈cλiw_i \approx c\lambda_i, but leaves cc undetermined. The coefficient varies by model: JiT-B/16 learns c≈0.2c \approx 0.2, versus c≈0.15c \approx 0.15 for JiT-H/16. Choosing cc for a new model would require costly hyperparameter sweeps, whereas learning the weights adds negligible overhead (Table 1).

W1: PCA vs. identity initialization

The first spectral update produces a noisy PCA basis, but the basis continues evolving and ultimately differs from PCA (Fig. 3(b)). Both the basis dynamics and weight initialization (11 vs. 0.15λi0.15\lambda_i) therefore differ, so the 1.63-to-1.55 gain cannot be attributed to weight initialization alone. Identity initialization is worse at 160 epochs (2.30 vs. 2.25) but better at 600 (1.55 vs. 1.63). We hypothesize that different training dynamics may mitigate the overfitting reported by JiT.

W1: Frozen PCA basis with fixed wi=0.15λiw_i = 0.15\lambda_i

We are running this baseline with c=0.15c=0.15 taken from our trained model, giving the fixed-profile approach a favorable choice of coefficient. Results will be added during the rebuttal period.

W1: Analytic direction-wise EDM baseline (Eq. 30)

We trained and evaluated the analytic EDM baseline at 160 epochs and obtained 4.26 FID, much worse than JiT (2.68) and AnisoFlow (2.30; Table 1). EDM's analytic rule wi(t)=(αt/σt)λiw_i(t)=(\alpha_t/\sigma_t)\lambda_i (Appendix A.2) uses only data variance and timestep, ignoring network prediction errors.

W2: Error-scaling assumptions

As discussed in seK8 W1, AnisoFlow's combined output cannot uniquely separate noise- and data-estimation errors. We instead estimate effective error scales accounting for their correlation (Appendix A.3, Fig. 8). The noise-error scale is nearly constant (∝λi−0.09\propto \lambda_i^{-0.09}), while the data-error scale grows linearly (∝λi1.00\propto \lambda_i^{1.00}). Their ratio scales as λi1.09\lambda_i^{1.09}, close to the learned weights (λi1.07\lambda_i^{1.07}), supporting our scaling analysis.

W2: Theory beyond EDM

EDM's rule wi∗(t)=(αt/σt)λiw_i^*(t)=(\alpha_t/\sigma_t)\lambda_i is the zero-skill limit of our analysis (Appendix A.2). Our theory additionally accounts for network prediction errors and identifies when their timestep dependence cancels αt/σt\alpha_t/\sigma_t, favoring shared weights. In Fig. 5, four learned profiles nearly coincide, and time-dependent weights do not improve FID (4.92 vs. 4.96). Our new error analysis finds 1.19×1.19\times variation in estimated preferred weights versus 11.7×11.7\times for αt/σt\alpha_t/\sigma_t (Fig. 8(c)).

Since errors depend on the learned target, near-linear scaling is a conditional explanation, not an unconditional prediction. We have revised the contribution statement to reflect this and the conditions for timestep invariance.

Q1: k-Diff comparison

Published k-Diff and JiT FIDs are nearly identical (3.64 vs. 3.66 on ImageNet 256; 4.03 vs. 4.02 on ImageNet 512), suggesting little gain from learning one isotropic scalar. We therefore omit it from Table 1; published results also use smaller models than our H/16 experiments.

Q2: Quantitative low-precision results

We evaluated both T2I models under FP8 (W8A8) inference (Fig. 6(b)) on OneIG-Bench and COCO-10k:

MethodPrecisionOneIG↑FID↓pFID↓
JiT-L/16-T2I (xx-pred.)BF160.51222.024.4
FP80.50924.226.6
AnisoFlow-L/16-T2IBF160.51620.122.7
FP80.51617.519.1

Under FP8, the xx-prediction baseline degrades on all metrics, while AnisoFlow's FID improves, likely because mild quantization noise better matches sensor/compression noise in real images. We will add these results to the paper.

Q2: Quantization comparison beyond xx-prediction

Due to costly T2I training (see appendix), we could train only one matched baseline. We chose xx-prediction, the most common target in pixel-space generation, and will clarify this limitation in the paper.

Q3: Latent-space weights (Fig. 7)

We do not claim exact recovery of the default targets, only that learned weights remain close. Fig. 7 is log-log: SiT-REPA learns weights near 1 (velocity) and RAEv2 near 0 (data), with sublinear dependence on variance. Latent representations may have stronger nonlinear dependencies across directions, so near-linear pixel-space scaling assumptions need not hold. The learned targets thus adapt to each representation.

We hope our responses and additional experiments address the reviewer's concerns, and we kindly ask them to consider raising their score.

W1: Completed fixed-weight baseline

We have completed the requested baseline with a frozen PCA basis and fixed wi=0.15λiw_i=0.15\lambda_i. At 600 epochs, it achieves 1.65 FID, compared with 1.90 for JiT. Learning the weights with the same frozen PCA basis achieves 1.63 FID, while our full method, jointly learning the weights and basis from W=IW=I, achieves 1.55 FID. Thus, the simpler fixed-profile approach does not recover the full improvement of AnisoFlow: the additional 0.10 FID gain represents nearly 30% of the total improvement over JiT. Interestingly, the fixed profile performs best at 160 epochs (2.20 vs. 2.25 for learned weights with frozen PCA), but is surpassed by both learned configurations at 600 epochs.

Importantly, the value of learning is not solely measured by its improvement over a carefully chosen fixed profile. The fixed baseline uses c=0.15c=0.15, obtained from our trained AnisoFlow model rather than determined independently. Our analysis does not determine cc, and the learned coefficient varies across model sizes (c≈0.2c\approx0.2 for JiT-B/16 versus 0.150.15 for JiT-H/16). For a new architecture, selecting cc could require costly training sweeps, whereas AnisoFlow learns the weights automatically during training.

Furthermore, our default W=IW=I initialization requires neither a precomputed PCA basis nor a manually specified weight profile. The method learns both automatically and can adapt to different architectures and representations without assuming a fixed analytic target shape. Thus, learning offers a general and efficient way to discover suitable prediction targets while achieving the best FID in our experiments.

We hope this additional experiment addresses the reviewer's central concern, and we would appreciate their consideration of a higher score.

Thank you for your reply. The weaknesses and questions I raised are all resolved, and I will raise my score from 2 to 3. I appreciate the method from the perspective that it lifts the prediction target to a more general function class, and a big conceptual advantage is that it saves the manual effort of tuning the weight by learning. The methodology is conceptually elegant, but the empirical improvement against a fixed frozen PCA basis (a one-time, cheap, offline estimation) is marginal (within noise). Though the method does not work at latent diffusion, which I believe requires a separate analysis and I don't think it's merely about "stronger nonlinear dependencies across directions", the contribution to pixel space diffusion itself I think it worth an acceptance already.

Reviewer_eMQKOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper makes the prediction target of flow models learnable. It predicts Wε − x, where W is a PSD matrix with a learned basis and one weight per direction, and velocity can be recovered in closed form. x-pred, v-pred, and AsymFlow are special cases. A simple analysis says the best weight grows almost linearly with data variance, which matches what the model learns. On ImageNet 256 pixel space, FID improves from 1.90 to 1.55 over JiT-H/16. It also helps pixel-space text-to-image and 8-bit inference, but gives no gain in latent space.

Critical strengths

1.Simple and general method with clear gains. The target is a small change with exact velocity recovery, so the network, loss, and sampler stay the same, and the extra cost is under 2%. It still improves JiT-H/16 from 1.90 to 1.55 FID without REPA. This matters because it is easy to adopt and the gain is solid for pixel-space models.

2.Theory matches the experiments. The analysis predicts w ≈ cλ and one shared weight across timesteps. The learned weights follow this from different inits, and time-bucketed weights give no gain. This makes the method more than a tuning trick and helps explain why it works.

Critical weaknesses

1.Key baseline missing, small gain over prior work. There is no baseline with a fixed w = cλ in a frozen PCA basis and no learning. Since both the theory and the results point to this curve, it is unclear if learning is needed at all. The gain over AsymFlow is also small (1.57 vs 1.55). This weakens the main claim that a learnable target is the key.

  1. Weak statistical support. All FID results come from single runs with no error bars, and each AnisoFlow model gets its own CFG grid search. Gaps like 1.55 vs 1.60 vs 1.63 may be noise, and it is not clear if the JiT baseline was tuned the same way. This makes the ranking of variants and the size of the gain hard to trust.
Questions for authors

What FID do you get with a fixed w = 0.15λ in a frozen PCA basis, with no learning? Is learning needed?

Can you report FID with several seeds or error bars? Was the JiT baseline tuned with the same CFG grid search?

The learned basis ends up close to PCA, so why does W = I init beat the frozen PCA basis (1.55 vs 1.63)?

Can you measure A_i(t), C_i(t), and Γ_i(t) on a trained model to check the separable form and Γ ≈ 0?

Can you give FID or pFID numbers for 8-bit inference, not just images? The weights are very small in low-variance directions, so why do seams still go away there? How much does the σ_min = 0.04 clamp help?

3 comments

We thank the reviewer for the positive assessment of AnisoFlow's simplicity, empirical gains, and theoretical analysis.

W1/Q1: What FID does a fixed wi=0.15λiw_i = 0.15\lambda_i in a frozen PCA basis give? Is learning needed?

Our analysis predicts wi≈cλiw_i \approx c\lambda_i, but does not determine cc in advance. The coefficient also depends on the model: JiT-B/16 learns c≈0.2c \approx 0.2, while JiT-H/16 learns c≈0.15c \approx 0.15. Choosing cc for a new model would require a hyperparameter sweep over full training runs, whereas learning the weights adds negligible overhead (Table 1). We are running a baseline with frozen PCA basis and fixed weights wi=0.15λiw_i = 0.15\lambda_i. Results will be added during the rebuttal period.

W1: Gain over AsymFlow

The 1.57 FID reported for AsymFlow includes REPA, whereas our 1.55 FID does not. Under the matched no-REPA setup in Table 1, AsymFlow obtains 1.76 FID and AnisoFlow obtains 1.55 FID. We will make this distinction clearer in the paper.

W2/Q2: Can you report FID with several seeds or error bars?

Across five sampling seeds, AnisoFlow-H/16 obtains 1.552±0.0131.552 \pm 0.013 FID. Moreover, all models in every table are evaluated using the same sampling seed and initial noise. Repeating 600-epoch training runs would require thousands of GPU-hours; JiT and AsymFlow likewise report single training runs.

W2/Q2: Was the JiT baseline tuned with the same CFG grid search?

Yes. We performed the same CFG grid search for the JiT and AsymFlow baselines as for AnisoFlow, which is standard practice in ImageNet evaluation (as discussed in the AsymFlow paper), and their optimal settings matched those reported in the original papers. For all models, we searched CFG scales with a step size of 0.1 and guidance intervals with a step size of 0.02 (Appendix B.2). We will clarify this in the paper.

Q3: Why does W=IW = I initialization beat the frozen PCA basis (1.55 vs. 1.63)?

The basis learned from W=IW=I aligns only approximately with PCA, particularly differing in low- and high-variance directions (Fig. 3(b)). Its different training dynamics yield worse FID at 160 epochs (2.30 vs. 2.25), but better FID at 600 epochs (1.55 vs. 1.63; Table 1). Reduced overfitting during longer training, as discussed by JiT, is one possible explanation; we will state this as a hypothesis in the paper.

Q4: Can Ai(t)A_i(t), Ci(t)C_i(t), and Γi(t)\Gamma_i(t) be measured on a trained model?

Not uniquely: AnisoFlow predicts one combined target, so separate noise- and data-estimation errors cannot be directly identified. Instead, we analyze the prediction-error structure of a trained AnisoFlow model (Appendix A.3, Fig. 8). We estimate effective error statistics that account for the cross moment without assuming it vanishes, providing a consistency check of our theoretical analysis. The effective noise- and data-error scales follow λi−0.09\lambda_i^{-0.09} and λi1.00\lambda_i^{1.00}; their ratio scales as λi1.09\lambda_i^{1.09}, close to the learned weights (λi1.07\lambda_i^{1.07}). Estimated preferred weights vary by 1.19×1.19\times across timesteps versus 11.7×11.7\times for αt/σt\alpha_t/\sigma_t. See seK8 W1.

Q5: Can you give FID or pFID numbers for 8-bit inference?

We have evaluated both text-to-image models under FP8 (W8A8) inference, the setting of Fig. 6(b), on OneIG-Bench and COCO-10k:

MethodPrecisionOneIG↑FID↓pFID↓
JiT-L/16-T2I (xx-pred.)BF160.51222.024.4
FP80.50924.226.6
AnisoFlow-L/16-T2IBF160.51620.122.7
FP80.51617.519.1

Under FP8, the xx-prediction baseline degrades on all metrics, while AnisoFlow's FID improves, likely because mild quantization noise better matches sensor/compression noise in real images. We will add these results to the paper.

Q5: Why do seams go away in low-variance directions, where the weights are very small?

Patch seams are coarse, low-frequency errors, such as mismatched color or brightness between neighboring patches, corresponding to high-variance PCA directions. AnisoFlow learns large weights in these directions, improving velocity-recovery conditioning and reducing quantization-error amplification. Low-variance directions correspond to fine high-frequency details with near-zero data that is easy to predict, so their small weights do not result in visible seams.

Q5: How much does the σmin⁡=0.04\sigma_{\min} = 0.04 clamp help?

The σmin⁡=0.04\sigma_{\min} = 0.04 clamp plays no role in the text-to-image results. The clamp acts on the recovery denominator αtwi+σt\alpha_t w_i + \sigma_t, which is at least σt\sigma_t since wi≥0w_i \geq 0. Our 32-step sampler uses a timestep shift of 3, so the last network evaluation is at σt≈0.088\sigma_t \approx 0.088, and the clamp is never active.

We thank the reviewer again for their support and thoughtful questions. We hope our responses and additional experiments address their concerns, and we kindly ask the reviewer to consider raising their score.

w1: can you add repa to your method? I am not sure if it is consistent with the proposed method. I think the reply solved my other concerns. Thanks for the efforts.

We thank the reviewer for the follow-up. AnisoFlow is compatible with REPA, and our latent-space experiments already combine them: Aniso-SiT-XL/2-REPA (Table 4) applies our learned prediction target under the REPA alignment objective, obtaining 1.44 FID (1.40 under the JiT protocol), compared with 1.44 (1.41) for the REPA baseline.

The two methods act on different parts of training. REPA adds an auxiliary representation-alignment loss on intermediate network features, while AnisoFlow changes the prediction target to Wϵ−xW\epsilon - x and recovers the velocity in closed form. The two mechanisms are therefore compatible without architectural modifications.

For our pixel-space ImageNet results we deliberately report AnisoFlow without REPA, so that the reported gain over JiT reflects the learned prediction target alone rather than additional representation alignment.

We hope this resolves the reviewer's remaining concern, and we would appreciate their consideration of a higher score.

Comments on the paper

4 comments
~Shanchuan_Lin1OpenReview ↗

Public question

Very interesting work! Could the authors also share the unguided FID results for AnisoFlow-H/16 and explain how they compare to those of JiT-H/16? Reviewer eMQK also pointed out that the AnisoFlow models use different CFG settings from the original JiT-H/16. Therefore, reporting unguided results would provide an additional fair point of comparison and could be valuable for future research.

Thank you for your interest in our work. We might consider adding that after finishing other experiments -- we have a GPU shortage right now due to rebuttals and CVPR ddls.

Regarding the CFG settings, we'd like to note that every baseline in Table 1 has gone through the same grid search process to find the optimal guidance scale and interval. This is common practice and can also be found in prior work. So the reported comparisons are fair.

Also, we'd like to point out that unguided FID could be easy to manipulate, making the comparison less fair than guided FID. For example, adding dropout during training can greatly improve unguided FID without changing guided FID noticeably.

Update. We have run the non-guided evaluations, below are the FIDs:

  • JiT 7.05
  • AsymFlow 8.31
  • AnisoFlow 5.84

All models are based on JiT-H/16 (600 epochs, without REPA). AnisoFlow uses the fully learnable variant (guided FID 1.55). Note that the ranking of JiT and AsymFlow flips, while AnisoFlow is still the best. The flipped baseline ranking shows that unguided FID isn't a very stable metric compared to the optimal guided FID.

AuthorsOpenReview ↗

Summary of Revisions

We have uploaded an updated manuscript incorporating the additional experiments and clarifications discussed during rebuttal. Changes are highlighted in blue.

The main updates include:

  • Table 1: Added the fixed spectral-profile baseline (1.65 FID) and the direction-wise EDM baseline (4.26 FID at 160 epochs).
  • Appendix A.3, Fig. 8: Added an empirical prediction-error analysis, including directional scaling, timestep invariance, and spatial/content dependence.
  • Appendix B: Added quantitative FP8 results (Table 7), sampling-seed variability, and a clarification of CFG tuning for all baselines.
  • Main text: Clarified the theoretical assumptions, initialization effects, the AsymFlow comparison, and other points raised during discussion.

We thank the reviewers for their constructive feedback, which helped strengthen the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.