acceptodds
Under review as a conference paper at ICLR 2027

DINO Forcing Flow Models: Do not denoise what you can predict

Abstract

Co-denoising pretrained representations such as DINO can substantially improve the training speed and quality of flow matching models, but it introduces a second denoising trajectory and requires carefully designed schedules. We propose a simpler alternative: predict the pretrained representation directly, then condition the model on its own prediction. This removes the need for a second ODE and any representation-specific denoising schedules, while retaining the benefits of representation guidance. Our approach converges substantially faster and achieves better generation quality as measured by FID score. On ImageNet, it outperforms the state of the art in latent space at 2 fewer epochs than prior methods; in pixel space, it improves over comparable prior methods by more than 20%. These results support a simple principle: do not denoise what you can predict. We will release all code and models.

Primary area:
generative models
Keywords:
image generation, dino

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_WGNxOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes DINO Forcing (DF), a method for incorporating pretrained visual representations into flow-matching image generators without assigning those representations a separate noisy generative trajectory. Training generally uses two passes through the same network: 1) a look-ahead pass which predicts a representation from the noisy image; 2) a conditioned pass which receives the look-ahead pass prediction and optimizes both the image flow-matching objective and an alignment objective against frozen pretrained features. At inference, the predicted representation is carried forward from one sampling step to the next, eliminating a separate representation ODE and representation-specific noise schedule. The experiments cover class-conditional ImageNet generation in latent space and pixel space, results show that in setting of latent space without guidance, DF achieves SOTA FID score at 2x fewer epochs, while in setting with guidance, DF's FID score still lags behind 'SFD+LDiT/1' (2.35 vs. 1.06)

Critical strengths
  1. The proposed replacement of feature co-denoising with supervised feature prediction is reasonable and well motivated.
  2. The experiment verification is very clear, table 7 clearly shows that with the same augmented architecture, unsupervised self-conditioning and image-supervised self-conditioning improve generation, but pretrained-feature supervision produces a much larger improvement.
Critical weaknesses
  1. Although DF achieves best latent-space FID score in 'unguided' setting, the guided setting latent-space FID score is significantly higher than existing method 'SFD+LDiT/1', 'REED', and 'iREPA', this make the claim in abstract "outperforms the state of the art in latent space at 2× fewer epochs than prior methods" somwhat not fully supported
  2. Epoch efficiency is not the same as training computation efficiency: the paper repeatedly emphasizes faster convergence and a twofold reduction in training budget. However, DF generally performs an additional gradient-free forward pass for each training step, it would be good to further report training FLOPs.
Questions for authors
  1. In sec 5.1, is the training stability sensitive to training recipes such as "10% of the time optimization on just the 1st forward pass", "5% of the time to make the model more stable", etc.
  2. Why does guidance provide relatively little improvement in latent space, and how do feature predictions change as guidance strength increases?
2 comments

We thank the reviewer for finding our work motivating and also for their questions and comments. We have answered their concerns below as well as have updated the paper accordingly with the changes marked in green.

W1: Efficiency claims

Our claim for 2x fewer epochs over prior methods is indeed applicable only for the unguided setting. We have updated the manuscript to specifically mention that.

W2: Flop comparison

We report training FLOPs in Table 5. The FLOP counts include DINO feature extraction and the additional gradient-free forward pass. Even with this cost, DINO Forcing outperforms REG (2.61 vs. 2.70 FID) with about half its training FLOPs (101,343 vs. 197,493), and outperforms REGLUE (2.47 vs. 2.50 FID) with about 36% fewer FLOPs (126,678 vs. 198,812). We have added the FLOP comparison in the main paper in Table 8 (page 10) and also additional discussion about the training cost in Section 5.6 (also on page 10).

Table 5: Training FLOPs of SiT-XL models (unguided FID). Computational costs include DINO feature extraction and follow Table 13 of REGLUE.

MethodStepsPFLOPsFID
REG1M197,4932.70
REGLUE1M198,8122.50
DINO Forcing100K25,3364.78
DINO Forcing200K50,6713.08
DINO Forcing300K76,0072.88
DINO Forcing400K101,3432.61
DINO Forcing500K126,6782.47

Q1: Training stability

We did not observe any training instabilities (e.g., loss spikes or divergence) with any of these settings; both choices were made to improve performance and not for stability. Training on Pass 1 alone 10% of the time teaches the model to make good DINO predictions from the all-zero condition used at the first sampling step. We ablate this ratio in Appendix C ("How often should we train on Pass 1 alone?"). Conditioning on the ground-truth DINO features 5% of the time teaches the model to behave consistently once its predicted features are already close to the true ones, as happens in late sampling steps.

Q2: CFG Limitation

With guidance, the predicted DINO features become much more class-specific: a linear probe trained on real DINO features recovers the correct class with 89.1% top-1 accuracy without guidance and 96.0% with CFG (Table 4). Since these features already act as a much stronger condition than the class label, CFG, which guides through the class label, has little room to improve the image further. This is in line with prior work built on DINO features, such as Latent Forcing and RAE, which also rely on AutoGuidance. We leave further investigation, e.g., internal guidance or balancing the DINO and class conditions, to future work. We have added a discussion about the CFG failure in the paper in Appendix C (page 15) for future readers.

Table 4: Class recovery from predicted DINO features with a linear probe.

SamplingTop-1 (%)
Unguided (CFG 1.0)89.1
Guided (CFG 4.0)96.0

Thanks the authors for detailed experiment on computational cost comparison and explanations on training stability and CFG limitation. I think my concerns have been addressed and I'll keep the positive rating

Reviewer_sr3xOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

See below

Critical strengths
  1. The paper proposes DINO-Forcing, which departs from conventional co-denoising paradigms by directly predicting pretrained representations within a flow-based generative model. This paradigm offers a novel perspective on representation-guided generative modeling and has the potential to inspire further research into representation-conditioned generation.

  2. The method achieves two forward passes during training and a single forward pass during inference. This asymmetric training–inference design is particularly intriguing and warrants further investigation in the context of ultra-high-resolution image generation.

Critical weaknesses
  1. Incomplete empirical evaluation. The paper primarily compares the number of training epochs while overlooking actual training time and GPU computational cost. Since the co-denoising baseline requires only a single forward pass during training, comparing methods solely by epoch count is insufficient to substantiate the claimed convergence advantage. Several critical ablation studies are also missing, particularly those validating the necessity and individual contributions of the two-pass training mechanism (Pass-1 and Pass-2).

  2. Insufficient analysis of guidance effectiveness and generalization. The native latent-space classifier-free guidance (CFG) performs relatively poorly, leading the method to rely on AutoGuidance, yet the underlying reasons for this limitation are not systematically investigated. Furthermore, the paper does not examine potential error accumulation during autoregressive representation sampling, leaving unclear whether inaccurate early-stage predictions propagate and degrade subsequent outputs. Generalization is also insufficiently explored, as all experiments use DINOv2-B without evaluating whether the framework can be transferred to other pretrained encoders, such as CLIP or MAE.

Questions for authors
  1. Pass-1 provides conditioning information without gradient propagation. How is the quality of its predictions quantified, and how are these predictions used to supervise subsequent learning? Please provide a more detailed explanation of this mechanism.

  2. What are the underlying reasons for the poor performance of native latent-space CFG? Could the authors provide targeted ablation studies to identify the main bottlenecks?

  3. Could the authors report actual training time and GPU FLOPs for the proposed method and its baselines to enable a fair comparison of computational efficiency and convergence?

  4. During sampling, representations are generated by the model's own predictions. How do early-stage prediction errors propagate and potentially compromise subsequent conditioning? Can the authors quantify the extent of this error accumulation?

  5. Could the authors replace DINOv2-B with alternative pretrained encoders, such as CLIP or MAE, to evaluate the transferability and generality of the proposed framework across different representation spaces?

  6. If representation transfer is retained during inference but training is modified to use only a single forward pass, how much would performance deteriorate? Such an experiment would help clarify the necessity of the proposed two-pass training strategy and the respective contributions of training-time representation transfer and inference-time conditioning.

3 comments

We thank the reviewer for finding our work has a novel perspective and for their interesting feedbacks and questions. We have answered their main concerns in the comments below as well as have updated the pdf accordingly with the changes marked in green.

Q1/W1: Clarity about Pass-1

Below is the pseudocode of one Dino Forcing training step:

# x: noisy images, c: class conditions, t: timesteps
# scale: loss weight for alignment loss
# true_velocity: the ground truth velocity 
# original_dino: the original dino extracted from the clean images

## First forward pass
dino_0 = torch.zeros(x.shape[0], 256, 768)
with torch.no_grad():
    _, dino_1 = net(x, dino_0, c, t)

## Second forward pass
velocity, dino_2 = net(x, dino_1, c, t)

loss = mse(velocity, true_velocity) + scale * align_loss(dino_2, original_dino)

We use the DINO features predicted in the first forward pass as the condition for the second forward pass. The alignment loss is applied to the final DINO prediction from the second pass, comparing it with the DINO features of the clean image. In addition, 10% of the time we run only the first forward pass, with gradients, so that the model also learns to make good initial predictions. We have now added pseudocodes for both training (Algorithm-1) and sampling (Algorithm-2) in the main paper (page 4) for ease of understanding of the readers.

Q2/W2: CFG limitations

We hypothesize the limitations of CFG to the strength of the DINO conditioning. Our predicted DINO features already carry strong class information: a linear probe (from Q4) recovers the correct class with 89.1% top-1 accuracy without guidance, rising to 96.0% with CFG (Table 4). The DINO features thus act as a much stronger condition than the class label, leaving CFG, which guides through the class label, little room to improve the image further.

This is in line with prior work built on DINO features, such as Latent Forcing and RAE, which also rely on AutoGuidance. We leave further investigation, e.g., internal guidance or balancing the DINO and class conditions, to future work. We have updated the paper with a discussion of this CFG limitation in Appendix-C (page 15).

Table 4: Class recovery from predicted DINO features with a linear probe.

SamplingTop-1 (%)
Unguided (CFG 1.0)89.1
Guided (CFG 4.0)96.0

Q3/W1: Training cost

We report training FLOPs (Table 5) and wall-clock training time (Table 6) for SiT-XL models; the FLOP counts include DINO feature extraction and our second forward pass. DINO Forcing outperforms REG (2.61 vs. 2.70 FID) with about half its training FLOPs, and outperforms REGLUE (2.47 vs. 2.50 FID) with about 36% fewer FLOPs. For wall-clock time, we measured all methods on the same NVIDIA H100 hardware. DINO Forcing reaches a better FID than REG (2.47 vs. 2.70) with about 36.2% less training time (146.1 vs. 228.9 H100 GPU-hours), and a better FID than REGLUE (2.47 vs. 2.50) with about 1.6× less training time (146.1 vs. 233.6 H100 GPU-hours). We have revised the paper and have added Table-8 and Table-9 along with Section 5.6 (in page 10) to discuss the training cost of our proposed method.

Table 5: Comparing the training flops of SiT-XL size models with prior methods. The computational costs include the dino feature extraction. The FID provided is in an unguided setting.

MethodStepsPFlopsFID
REG1M197,4932.70
REGLUE1M198,8122.50
DF100K25,3364.78
DF200K50,6713.08
DF300K76,0072.88
DF400K101,3432.61
DF500K126,6782.47

Table 6: Training wall-clock time of SiT-XL size models compared with prior methods. All times were measured on NVIDIA H100 GPUs under the same setup (batch size 256) and are reported in total H100 GPU-hours. FID is reported without guidance.

MethodStepsTimeFID
REG1M228.92.70
REGLUE1M233.62.50
DF500K146.12.47

Q4/W2: Error propagation quantification

We run two experiments to quantify how error propagates and impact the generated image (Experiment 1) and its semantic content (Experiments). Each experiment perturbs the model's own DINO prediction (the self-condition) at a single sampling step KK out of 50 by adding Gaussian noise with zero mean and standard deviation σ\sigma, and then measures the effect on the final output.

Experiment 1: Effect on the generated image. We first generate 256 reference images with standard sampling. We then repeat sampling with the same seeds, but add noise to the self-condition at step KK. Finally, we compute the PSNR between each perturbed image and its reference: a higher PSNR means the perturbation changed the image less.

Table 7: PSNR (dB, mean ± std) between perturbed and reference images, over N=256 images. Gaussian noise with standard deviation σ\sigma is added to the self-condition at sampling step KK.

Step Kσ=0.1σ=0.25σ=0.5σ=1.0σ=2.0
157.2 ± 9.349.9 ± 10.242.6 ± 9.935.8 ± 8.631.7 ± 7.5
256.2 ± 9.948.9 ± 10.240.7 ± 9.734.7 ± 8.330.7 ± 7.2
553.9 ± 10.844.9 ± 10.537.3 ± 9.531.9 ± 8.429.0 ± 7.4
1050.7 ± 10.841.2 ± 10.234.2 ± 8.729.2 ± 7.326.8 ± 6.4
2054.6 ± 7.946.4 ± 7.639.3 ± 6.933.6 ± 5.930.6 ± 5.3
3566.8 ± 4.959.6 ± 5.153.2 ± 5.047.4 ± 4.843.7 ± 4.5

For small perturbations (σ≤0.25\sigma \leq 0.25), the final image stays very close to the reference (PSNR above 41dB). Larger perturbations change the image more. The effect is strongest at intermediate steps, with the lowest PSNR of 26.8 dB at K=10K=10 and σ=2.0\sigma=2.0. Perturbations at earlier or later steps are largely corrected: errors do not accumulate, because the model predicts a new self-condition from the current image at every step.

Experiment 2: Effect on semantic content (linear probing). A lower PSNR shows that the image changed, but not whether its content is still correct. We therefore also test whether the final predicted DINO features still encode the right class. We first train a linear classifier on real DINO features: 100K training features, 30 epochs, AdamW with learning rate 1e-3 and cosine decay. It reaches 89.1% top-1 accuracy on 5K held-out real features. We then generate 2,000 DINO features with our model (2 per class), perturb the self-condition at step KK as in Experiment 1, and classify the final predicted features with this probe.

Table 8: Top-1 class-recovery accuracy (%) of a linear probe on the final predicted DINO features, with Gaussian noise of standard deviation σ\sigma added to the self-condition at sampling step KK. The probe's accuracy on real DINO features is 89.1%.

Step Kσ=0.1σ=0.25σ=0.5σ=1.0σ=2.0
189.089.088.688.087.3
289.088.988.387.687.4
589.188.787.987.987.7
1088.888.688.387.787.3
2088.988.888.888.488.2
3589.189.089.188.989.2

Class recovery stays close to the 89.1% reference in every setting. Even the strongest perturbations reduce it by at most 1.8 points (87.3% at σ=2.0\sigma=2.0). As in Experiment 1, intermediate steps are the most sensitive, while early and late perturbations have almost no effect. The perturbations therefore change image details slightly but preserve semantic content.

We have added both of these experiments and the analysis in the paper in Appendix D (page 16).

Q5/W2: Other frozen representation

We thank the reviewer for this suggestion. We evaluate generalizon of the framework by replacing DINOv2-B with MAE as the frozen encoder. Everything else is unchanged. We refer to these two variants as DINO Forcing (DINOv2) and DINO Forcing (MAE).

Table 9: Effect of the frozen encoder, with SiT-L models trained for 400K steps (unguided FID).

MethodFID
SiT-L (baseline)18.80
DINO Forcing (MAE)7.97
DINO Forcing (DINOv2)3.04

With MAE features, DINO Forcing reduces the FID of the SiT-L baseline from 18.80 to 7.97, a 2.4× improvement. This shows that the gains come from the framework itself and are not tied to DINOv2. However, the choice of encoder still matters: DINOv2 gives the best results (3.04 FID). This is consistent with prior work such as REPA, which also finds that semantic-oriented representations like DINOv2 help generation more than reconstruction-oriented ones like MAE. We have added the MAE experiments in Table 7 and the relevant analysis in Section 5.5 (page 10).

Q6/W1: Single forward pass

Our training uses two forward passes: Pass 1 predicts DINO features from an all-zero condition, and Pass 2 is conditioned on this predicted DINO to learn the velocity. The model is thus trained on its own predictions.

We train a variant model with a single forward pass, leaving inference unchanged. The condition is the ground-truth DINO features for 90% of samples and zeros for the remaining 10%, which keeps sampling from zeros possible

Table 10: Effect of removing the second forward pass during training (unguided FID at 100K training steps).

TrainingStepsFID
Two passes (DINO Forcing)100K5.70
Single pass (ground-truth DINO)100K198.87

Removing the second pass breaks generation entirely (198.87 vs. 5.70 FID). Trained only on perfect DINO features, the model cannot use its own imperfect predictions at inference. The two-pass strategy is necessary.

Reviewer_c92EOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Previous work has shown that grounding image generative models to pretrained SSL representations (either by aligning the features or co-denoising) improves the training speed of such image generators, making them converge much faster. This work proposes predicting the pretrained representations directly instead of codenoising or aligning, and using the prediction as self-conditioning for each step of the denoising process. This results in improved training speed for ImageNet pixel and latent space models.

Critical strengths
  • The idea to predict SSL representations from the noisy images and use them as self-conditioning instead of two ODEs, one for the image and one for the representations, is straightforward, simple and significantly improves training speed for both latent and image space diffusion models on imagenet.
Critical weaknesses
  • It seems like the model predicts a rough "estimate" of the dino features and refines with every new prediction, using the previous state and the current noisy image, and the generative quality depends to the quality of the predicted representations. From the current formulation, when using fewer inference steps (e.g. 5 instead of 50) the model refines the representations less (there is no equivalence to making "bigger" but fewer steps since the output is just a prediction without any ODE for the representations). This implies that the method would not transfer well to the more practical few step regime, and is effective mostly when doing expensive many step generation.
Questions for authors
  • previous methods like REPA used representations only at training time so inference time was unaffected. How significant is the extra computational cost that is required to predict the representations at inference time? What is the difference in the time it takes to generate 1 image with 50 steps?
  • See "Critical Weakness": How does this model perform in few-step (5-10-25) generation compared to other related latent and pixel space models?
  • From the results, the model seems to have less of an edge against pixel space models like JiT. Another difference of those models to the common latent space ones (SiT, DiT, REPA, etc) is that they do x-prediction instead of v-prediction which seems to be more stable for generation. A model that would use 2 x-prediction ODEs for image and representations, respectively, seems to be very similar to DF, as both would predict the clean representations, with their only difference being that the former would condition on the noisy representations instead of the clean predictions. How important is conditioning on the clean representations given the flexibility that the alternative option of the 2 ODEs with x-prediction models would provide (ability to have different time schedules, reducing the number of timesteps for only the easier of the two predictions, either image or representations etc)?
3 comments

We thank the reviewer for finding our work simple and for their insightful comments. We have answered their concerns below and have updated the PDF accordingly, with the changes marked in green.

Q1: Computation cost compared to REPA

At inference, DINO Forcing differs from REPA only by two extra linear projections per forward pass: one mapping the predicted DINO features into the model, and one mapping the output back to DINO space. Everything else is identical, with a single forward pass per step. As Table 2 shows, generating one image with 50 Euler steps costs only 0.2% more FLOPs and 0.19 ms (0.9%) more time than REPA. For clarity, we have also added Algorithm-2 in the main paper (page 4) describing our sampling pseudocode.

Table 2: Inference cost of generating one image with SiT-L and a 50-step Euler ODE sampler.

MethodGFLOPsTime per image (ms)
REPA8,27120.96
DINO Forcing8,29121.15

Q2/W1: Few step regime

We have added Table 11 in the paper in Appendix D (page 16) to compare the few step generation results of Dino Forcing. Table 1 shows the same table below and compares DF with SiT-L and REPA (all L-sized) from 10 to 50 sampling steps. DF achieves the best FID at every step count, so predicting DINO representations does not hurt few-step generation.

Table 1: Few step sampling scores. All models are evaluated using Euler ODE solver.

Sampling StepsSiT-LREPAOurs
1032.6424.0522.23
1525.9417.4510.88
2023.3015.036.92
2522.0813.755.15
5020.0211.813.04

Q3: Codenoising vs DF in Pixel space setting

We thank the reviewer for this insightful question.

Difference. Besides conditioning on clean predictions instead of noisy representations, DINO Forcing also trains the representations differently. A two-ODE model in pixel space setting uses a timestep weighted flow-matching (MSE) loss for representation learning. DINO Forcing instead uses a cosine similarity loss against the DINO features of the clean image, similar to REPA.

Deterministic. DINO features are a deterministic function of the image: each image has exactly one DINO feature map and not a distribution of them. A second ODE therefore adds no new information, and one forward pass is enough to predict the DINO features from the noisy image. This makes Dino Forcing a much attractive choice than 2-ODE based methods.

Cost. Separate schedules require the model to be accurate for every combination of image and representation noise levels, and during inference the schedule must be hand-tuned. Latent Forcing, for example, finds that the representation must be denoised to a low noise level before image generation starts. Dino Forcing needs no such schedule: it simply predicts the DINO features at each step and uses them as the condition for the next one with no schedule involved.

Performance. Latent Forcing is the closest existing method to the reviewer's alternative, since it co-denoises pixels and DINO features with a hand-crafted schedule. At comparable model size and training budget, Dino Forcing improves over it from 7.20 to 5.53 FID (Table 3).

Table 3: Pixel-space comparison.

MethodFID
JiT-L17.49
Latent Forcing7.20
DINO Forcing5.53

Thank you for the clarification regarding inference compute requirements.

For few-step sampling, it seems that the performance difference compared to baselines is significantly smaller, reinforcing my question: what if the refinement of the representations requires more iterations than the denoising in pixel/latent space? Wouldn't two independent schedules enable that flexibility? This is why i see the 2-ODEs or 2-prediction-schedules as a strength instead of weakness, but it is possible i am misunderstanding something related to that. I would appreciate it if the authors could provide their view on this topic.

We thank the reviewer for their interesting question.

In order to see the influence of the direct prediction of the representation in a few-step setup, we compare in Table A two settings with a fixed budget of sampling steps: (a) Latent forcing setting with 2-ODE and more sampling steps on the representation than on the image; (b) Dino Forcing with a single ODE on the image and all sampling steps focused on image denoising but with self-predicted representation conditioning.

Table A: Few step sampling comparison between Latent Forcing and Dino Forcing (pixel space generation). Scores reported are FID without guidance. Dino-Forcing performs better at each budget. Latent Forcing has been evaluated using the best schedule strategy the authors proposed.

Total Sampling StepFID Dino ForcingFID Latent Forcing
532.0380.83
1019.5830.27
258.8511.54

As reported in Table A, at a fixed number of steps, spending all the NFEs on the image (conditioned on predicted representations), as DF does, performs better.

In the standard sampling setup (50 steps), Latent Forcing reports in Figure 3 of their paper that the optimal ODE schedules denoise the representation faster than the pixels. In this sense, DF can be viewed as taking this principle to its extreme. It provides an estimate of the clean representation at every step, conditioned on the current pixel state. Rather than handcrafting a denoising schedule for the representation, DF lets the model learn the appropriate level of representation refinement at each step: remaining blurry when the pixels are highly ambiguous and becoming sharp as soon as the image content is sufficiently determined (see Figure 15).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.