acceptodds
Under review as a conference paper at ICLR 2027

Self-Drift: One-Step Generative Flows Learn from Their Own Representations

Abstract

One-step flow models promise low-latency generation by compressing transport into a single network evaluation, but trajectory supervision alone can leave a mismatch between predicted endpoints and the data distribution. Existing distribution-correction methods commonly obtain their comparison geometry from an external teacher, encoder, or discriminator, so the feedback comes from a model separate from the flow being optimized. We introduce Self-Drift, a joint trajectory–distribution training scheme that closes this loop inside the flow itself. Our dual-horizon trajectory formulation pairs local transport learning with long-horizon prediction to a fixed clean endpoint, enabling the flow to compare predicted and real endpoints through its own representation and class-conditional feature memories. A frozen-lens semi-gradient routes a detached attraction–repulsion target through the endpoint, providing distribution feedback across the trajectory while preserving path-wise transport learning. Real-data-anchored, condition-number-capped whitening stabilizes the evolving feature space. Self-Drift learns this correction geometry in the same from-scratch process. The correction is self-contained and training-only: it needs no separately trained distribution-correction model and leaves the original one-step inference path unchanged. Self-Drift achieves a leading FID of 1.36 at NFE=1 among the listed from-scratch diffusion/flow methods on class-conditional ImageNet .

Primary area:
generative models
Keywords:
Generative Models, One Step, Own Representations

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_KhhYOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Self-Drift, which improves one-step image generation using the generator’s own internal features to compare generated and real images. Alongside trajectory learning, it adds a training signal that moves generated features toward real ones and discourages excessive clustering. It requires no separate pretrained feature encoder, and generation still takes one network evaluation.

Experiments on ImageNet 256×256 in VAE latent space show an FID improvement from 2.46 to 1.36 for the XL/2 model. Ablations examine the effects of feature selection, gradient handling, and feature normalization.

Critical strengths

Substantial performance improvements without extra inference cost. Adding Self-Drift reduces the XL/2 model’s FID from 2.46 to 1.36 compared with the authors’ trajectory-only baseline. Improvements are also reported across all four model sizes, providing evidence that the benefit is consistent rather than limited to one configuration. These gains are consequential because they preserve one-step generation and require no separate pretrained feature encoder for distribution correction.

Critical weaknesses
  1. The need for the new trajectory formulation is not fully established. Its endpoint-learning objective closely resembles consistency training, but there is no direct comparison with a standard consistency model or iMF using the same self-feature drift loss. The short-step ablation removes endpoint learning entirely, so it does not establish an advantage over these alternatives. This leaves the contribution of the new trajectory design unclear.

  2. The quality trade-off needs stronger evaluation. In Table 2a, Self-Drift improves FID from 3.57 to 2.74 but reduces Inception Score from 259.3 to 230.1. This does not by itself imply worse diversity, but it makes the overall quality improvement less clear. For example, this may be due to a suboptimal choice of CFG scale. Additional measures of sample quality and coverage would help establish whether the method improves distribution matching beyond FID on ImageNet.

Questions for authors

The following questions seek clarification and additional evidence:

  1. How does the proposed trajectory formulation compare with standard consistency training or iMF using the same Self-Drift loss? What benefit comes specifically from the dual-horizon design?
  2. Does the IS drop persist when CFG is tuned separately for the baseline and Self-Drift? How do their FID–IS trade-offs compare across CFG scales?
  3. What are the measured training time and peak memory costs relative to trajectory-only training? Do the improvements remain under matched training-compute budgets?
  4. Have you evaluated Self-Drift in pixel space? If so, how does it compare with pixel-space methods such as pMF under comparable model sizes and training budgets?
7 comments

We thank the reviewer for the expert and concrete questions; Q2 led us to find an evaluation inconsistency, which we correct in the General Response together with Tables G1–G4.

W1 / Q1. Need for the dual-horizon formulation. SD does not require the dual-horizon flow (it improves all three objectives in Table G2), but the combination gives the best FID; we therefore present the dual-horizon flow as the trajectory interface designed for SD, not as a standalone improvement over iMF. Table G2 applies the identical SD loss under the same schedule: iMF (our re-run; JVP, auxiliary v-head and one extra forward pass at r=0r=0 per SD query) goes from 3.62 to 2.89; a CT-style target (ours with the ground-truth step) from 3.74 to 2.89; ours from 3.69 to 2.74. Without SD our objective is slightly behind iMF; with SD it leads both other objectives by 0.15, beyond the sampling-seed variation (≤0.05), while needing no JVP, auxiliary v-head or extra forward pass. We did not train a standard consistency model with its own parameterization; the CT-style row isolates the consistency target within our network.

Relation to CT and MeanFlow/iMF. Our long-horizon loss (Eq. 6) is a discrete self-consistency on the remaining displacement t uθt\,u_\theta: the jump from ztz_t must equal a short step followed by the jump from the resulting state. It is the MeanFlow identity restricted to the data endpoint (r=0r=0) and discretized by a backward finite difference (App. B.2), so it belongs to the CT/MeanFlow family. Three design choices tailor it to SD: (1) the destination is always the data endpoint, so every sampled state yields a clean endpoint without an extra forward pass, whereas iMF conditions on intervals (r,t)(r,t); (2) the intermediate state is reached with the model's own local step δ sg[vθ]\delta\,\mathrm{sg}[v_\theta] (same network, task condition h=0h=0) rather than CT's ground-truth interpolation; the CT-style row isolates this choice (2.74 vs. 2.89); (3) there is no JVP and no time input.

Why is the trajectory-only flow weaker than iMF? Partly because of schedule length and partly by design. Under the same 480-epoch schedule at B/2 it is close to iMF (3.69 vs. 3.62); we have matched-schedule evidence only at B/2, and the larger gaps in Table 2b compare with iMF's 640/800-epoch results. By design, our objective is the trajectory interface for SD rather than a standalone generator: in our implementation it is 3.4× faster per iteration than iMF with 48% less peak memory (Table G3), and it gives the best FID once SD is added (Table G2).

W2 / Q2. IS drop and CFG. Yes, the reviewer's hypothesis is correct: the trajectory-only row of Table 2 used ω=10/11, whereas SD and all our other entries used ω=0.85. At matched ω, SD lowers FID by 0.95 (ω=0.85) and 0.74 (ω=10/11) with IS within 5.9 and 0.5 points, and at M/L/XL by 1.10–1.20 with IS within 1.2 points (Table G1), so SD reaches lower FID at comparable IS (e.g., at ω=10/11: 2.83 / 258.8 vs. 3.57 / 259.3). When each model is taken at its better scale (10/11 for the baseline, 0.85 for SD), as in the paper's 0.83, the IS drop remains, because SD reaches its best FID at the lower guidance scale. Precision and recall (Table G4) show that the gain comes mainly from coverage: recall 0.518 → 0.564 at unchanged precision (0.794 → 0.795). We chose ω=0.85 by a sweep in steps of 0.05; the revision will include the full sweep.

Q3. Training time, memory and matched compute. Drift-active iterations cost 1.22× trajectory-only iterations (analytical: 1.26×) and raise peak memory from 28.0 to 34.1 GiB; memory-update iterations cost 1.17× (Table G3). Even with SD, our flow is 2.8× faster per iteration than iMF without it in our implementation (0.22 vs. 0.61 s) and uses less memory (34.1 vs. 53.6 GiB). In this setting, extra compute does not explain the gain: 160 SD epochs (≈195 trajectory-only-equivalent) reach 2.91, whereas 240 further trajectory-only epochs leave FID unchanged (3.69; Table G4).

Q4. Pixel space and pMF. We have not evaluated SD in pixel space and cannot compare with pMF within the rebuttal period; we will state this as a limitation. Nothing in the method is specific to the VAE latent space, since the lens is a prefix of the generator. However, pixel space is where drifting methods rely on the strongest external encoders (Drifting Models use a stronger MAE and ConvNeXt-V2 features there), so it remains to be shown that SD removes this dependence in pixel space.

Thank you for the additional experiments and for identifying the CFG mismatch in the original table. The new comparisons help clarify how much of the gain comes from Self-Drift and how much comes from the trajectory formulation. I have two follow-up questions:

  1. CFG tuning. For the main comparison, I would report each model’s best FID after tuning its guidance scale independently under the same evaluation protocol and with a comparable search budget. The fixed-ω\omega results are useful as a control, but a shared scale may favor one model. Could you report the selected ω\omega, best FID, and corresponding IS for both the trajectory-only baseline and Self-Drift, along with the full sweep?

  2. iMF checkpoint. Could you clarify how the iMF-B/2 checkpoints used for the 3.62 and 2.89 FIDs in Table G2 were initialized, trained, and selected? I understand the rerun uses a 480-epoch schedule, whereas the original iMF paper reports 3.39 after 640 epochs. Were the CFG scale and guidance interval tuned independently for iMF with and without Self-Drift, and which settings produced the reported numbers?

We thank the reviewer for the follow-up questions.

1. CFG tuned independently. We swept ω for both B/2 models on one grid, with the evaluation protocol of Table G1 (FID-50K / IS at NFE=1, epoch-480 checkpoints, same reference statistics and sampling seeds). The grid uses 0.05 steps from 0.75 to 0.90, plus 10/11 and 12/13 around the baseline's optimum, all within the training range of ω; FID rises at both ends for both models, so both optima lie inside it. It includes the 0.05-step grid mentioned in our first response.

Table R3-1. ω sweep (B/2, FID / IS; best FID per model in bold).

ωTrajectory-onlySD
0.754.44 / 204.33.45 / 199.1
0.803.97 / 219.53.01 / 214.3
0.853.69 / 236.02.74 / 230.1
0.903.62 / 253.12.74 / 247.7
10/113.57 / 259.32.83 / 258.8
12/133.65 / 260.62.88 / 260.8
  • Selected ω, best FID and IS. Trajectory-only: ω=10/11, 3.57 / 259.3. SD: ω=0.85, 2.74 / 230.1. With each model tuned independently, SD improves FID by 0.83, far beyond the sampling-seed variation (≤0.05).
  • FID–IS trade-off. At every ω, SD lowers FID by 0.74–0.99 with IS within 5.9 points. For every trajectory-only setting, some SD setting has both lower FID and higher IS; e.g., SD at ω=12/13 (2.88 / 260.8) vs. the baseline's best (3.57 / 259.3). At ω=0.90, SD keeps FID 2.74 with IS 247.7.

This sweep covers B/2 (Table 2a). The revision will report independently tuned results as the main comparison for all sizes, include the full sweeps, and keep fixed ω as a control.

2. iMF checkpoints (Table G2). First, a correction to the G2 caption: iMF without SD did not use the equivalent of our ω=0.85, and "17/3" was a conversion error. In iMF's convention (ω=1: no guidance; guidance coefficient 1−1/ω, which plays the role of our ω), the settings were:

  • Without SD: the official iMF setting, ω=8 (coefficient 0.875) on the interval [t_min, t_max]=[0.4, 0.65]; sweeping ω on our checkpoint confirmed ω=8 as the best scale (it is also the upper end of iMF's training range [1, 8]).
  • With SD: the same interval; sweeping ω selected ω=20/3 (coefficient 0.85). Thus the scale was tuned independently for both runs, while both kept the official interval; a joint search over the interval was beyond our budget.

Initialization and training. Both runs start from the same iMF-B/2 checkpoint, trained from scratch for 240 epochs with the official iMF recipe (including its guidance-conditioning ranges); it reaches FID 3.85, close to the 3.82 that iMF reports at 240 epochs (3.39 at 640; Tab. 1c). Each run then continues to epoch 480 with the continuation settings shared by all G2 rows (App. C.1); iMF + SD adds the SD loss with the same hyperparameters as ours, not re-tuned. Both keep iMF's JVP objective and auxiliary v-head. Selection: both numbers come from the final epoch-480 checkpoint.

Table R3-2. Each model at its selected guidance (B/2, epoch 480, FID / IS; iMF settings in iMF's convention, coefficient after the arrow).

Objectivew/o SDw/ SD
iMF (our re-run)3.62 / 251.7 (ω=8 → 0.875)2.89 / 225.7 (ω=20/3 → 0.85)
Dual-horizon (ours)3.57 / 259.3 (ω=10/11)2.74 / 230.1 (ω=0.85)

SD improves iMF by 0.73 and our objective by 0.83. Without SD the two objectives are close (3.62 vs. 3.57); with SD ours is 0.15 lower. Like ours, iMF selects a weaker guidance once SD is added.

Reviewer_4AkzOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Self-Drift, a one-step generative flow framework that combines dual-horizon trajectory learning with distribution correction. Unlike existing approaches relying on external pretrained models, Self-Drift uses the flow's own intermediate representations to construct class-conditional attraction–repulsion feedback. It achieves an FID of 1.36 at NFE=1 on ImageNet 256×256 without an additional correction model or inference-time overhead.

Critical strengths
  • Self-contained framework: Distribution correction uses the flow's own representations without requiring an external pretrained teacher or encoder.
  • Effective design: The dual-horizon formulation and frozen-lens semi-gradient provide a practical way to combine trajectory learning with distribution correction.
  • Strong empirical results: Achieves a competitive FID of 1.36 at NFE=1, outperforming the listed from-scratch one-step diffusion/flow baselines.
Critical weaknesses

Weaknesses

W1. Unclear reliability of self-representation during early training.

Self-Drift assumes that the flow's evolving features provide meaningful geometry for distribution correction.

  • During early training, it is unclear whether real–real, generated–generated, and real–generated feature differences provide meaningful correction directions rather than arbitrary representation differences.
  • Fig. 3 evaluates drift-direction alignment at a 240-epoch trajectory-only checkpoint, not during early from-scratch training.

Suggested: Analyze drift-direction reliability at different training stages, particularly when Self-Drift is first activated.

W2. Insufficient justification for early online correction over external or staged alternatives.

The benefit of using online self-representation from early training remains unclear.

  • The two-stage continuation achieves FID 2.74, close to the from-scratch result of 2.66, although training configurations differ.
  • While pretrained MAE-Drift and Frozen-self are evaluated, it remains unclear why online self-features should provide more effective correction than pretrained external features, especially early in training.
  • An external-to-self transition could potentially combine reliable initial guidance with later online adaptation.

Suggested: Compare early online, delayed online, and external-to-self correction under matched training budgets, or clarify why early online learning is beneficial.

W3. Insufficient analysis of distribution quality and training dynamics.

Self-Drift improves FID but decreases IS (e.g., 259.3 → 230.1 for B/2).

  • It is unclear whether attraction–repulsion improves sample fidelity, mode coverage, or both.
  • Final FID/IS results do not explain how external versus self-representation correction affects generation throughout training.

Suggested: Report training curves of FID, generative precision, and recall for trajectory-only, MAE-Drift, and Self-Drift, using an independent evaluation representation.

W4. Limited evaluation beyond ImageNet.

All experiments are restricted to class-conditional ImageNet 256×256.

  • Generalization to larger and more diverse image distributions remains unexplored.
  • It is unclear how class-conditional feature memories would extend to less structured or open-domain datasets.

Suggested: Additional evaluation on another dataset, such as a LAION-2B subset, would strengthen the paper. Given the rebuttal timeline, preliminary results or a discussion of these limitations would be reasonable.

Expectations for Rebuttal

I believe additional evidence is needed to clarify the reliability and necessity of the proposed online self-representation correction. At minimum, I would like the authors to provide experimental evidence or convincing explanations addressing:

  • Early-stage reliability: Whether self-representation yields meaningful drift directions during early training (W1).
  • Training strategy: Why early online correction is preferable to delayed or external-to-self alternatives (W2).
  • Distribution quality: How Self-Drift affects precision, recall, and generation quality throughout training compared with external correction (W3).

I understand that additional large-scale dataset experiments (W4) may be impractical within the rebuttal period.

My current assessment is borderline accept. I would maintain this assessment if the authors adequately address these concerns through additional experiments or well-supported explanations. Otherwise, I would consider lowering my rating.

Questions for authors

Please refer to the Weaknesses section for my questions and concerns.

3 comments

We thank the reviewer for the clearly stated requests. We answer W1–W3 with new experiments and discuss W4. Tables G1–G4 and our corrections, including the Table 1 entry 2.66 noted in your W2, are in the General Response.

W1. Is the self-drift direction meaningful early in training? Yes: it is above chance already at activation and throughout training. We applied the Fig. 3 probe (Sec. 4.3.3) to checkpoints of a from-scratch B/2 run with drift applied every other epoch after epoch 10.

Table R2-1. Alignment during from-scratch training. Fraction (%) of the 1,000 classes whose one-step SD displacement aligns positively with the class's real-minus-generated direction in Inception space (layer 9, ρ=0.02; mean ± std over 5 probe seeds). FID-8K uses the 8,000 probe samples (online weights).

Epoch10 (activation)204080160240480Traj.-only 240 (Fig. 3, re-measured)
Own-class banks58.3±1.578.1±1.265.9±0.963.6±0.774.6±1.362.8±0.870.0±1.176.1±0.6
Mismatched-class banks55.4±2.276.8±1.163.8±0.954.1±1.466.5±1.855.9±1.662.7±1.269.5±1.8
Random direction47.3±2.044.9±1.745.6±1.845.3±1.743.6±1.446.3±1.746.3±1.744.9±1.0
FID-8K90.544.918.811.58.67.57.08.6

At activation, before any drift update and at FID-8K 90.5, the SD direction is already above the random control (58.3% vs. 47.3%; gap 11.0 ± 2.0 pp across probe seeds), although one step does not yet measurably lower FID-8K (−0.01 vs. +0.14 for a random step). At epoch 20, alignment reaches 78.1%, above the 240-epoch trajectory-only checkpoint of Fig. 3 (76.1%), and one step lowers FID-8K by 1.2 (not at every checkpoint, e.g. +0.04 at epoch 80). Across all 47 checkpoints from epoch 20 to 480 (every 10 epochs, one probe seed), alignment stays above 56% and random below 48%. It is not monotonic and varies between checkpoints by more than the probe noise, but never approaches chance. Mismatched-class banks are weaker at every epoch, by 1–3 pp up to epoch 40 and 7–10 pp from epoch 80 on, so the class-specific signal grows with training.

W2. Early online vs. delayed or external-to-online correction. Early online correction matches or outperforms every alternative we tested (Table R2-2). All variants continue from the same 240-epoch trajectory-only checkpoint to epoch 480 (B/2, ω=0.85), matching the epoch budget; W1 covers early from-scratch behavior.

Table R2-2. Activation timing and lens source (FID / IS at epoch 480).

StrategyLens in epochs 240–360 → 360–480FID / IS
Trajectory-onlynone → none3.69 / 236.0
Delayed onlinenone → online2.88 / 233.9
External → onlineMAE → online2.72 / 235.6
External only (MAE-Drift)MAE → MAE2.84 / 248.2
Frozen snapshotfrozen → frozen3.19 / 236.9
Online from epoch 240 (default)online → online2.74 / 230.1

(i) Starting earlier, and thus correcting for longer, is better (2.74 from epoch 240 vs. 2.88 from epoch 360). Together with W1, we see no benefit from waiting. (ii) A pretrained warm start brings no measurable benefit: MAE → online reaches FID 2.72 (IS 235.6), within sampling-seed variation of 2.74. (iii) The second-phase lens matters: switching from MAE to the online lens outperforms keeping MAE (2.72 vs. 2.84), and a frozen snapshot is clearly worse (3.19), presumably because only the online lens adapts as the generator's errors change.

W3. Distribution quality and dynamics. (a) Most of the IS drop came from comparing two guidance scales (correction in the General Response): at matched ω, SD lowers FID at every scale, with IS within 5.9 points at B/2 and 1.2 points at M/L/XL (Table G1). (b) In Inception space, the gain comes mainly from coverage: at epoch 480, recall rises from 0.518 to 0.564 while precision stays within 0.002 of trajectory-only at every epoch. The FID gap is already 0.81 at epoch 320 (80 epochs after activation) and 0.95 at epoch 480. The pretrained MAE lens instead trades precision (0.783) for recall (0.577) and ends at higher FID (2.84; Table G4).

W4. Beyond ImageNet. A LAION-scale study is beyond the rebuttal period; we will list it as a limitation. Part of the drift signal does not depend on class keys: mismatched-class banks stay well above random throughout training (Table R2-1), although class keys add 7–10 pp later on. Class-agnostic banks are untested; for open-domain data, banks could be keyed by clusters of text embeddings or by nearest neighbors in the lens space.

Reviewer_4AkzOpenReview ↗

Follow up on the Author Response

Thank you for the additional experiments. The new results alleviate some of my concerns about early-stage drift reliability and distribution quality (W1 and W3), and partially address my concern about the benefit of early online correction in from-scratch training (W2). The remaining concerns are minor enough that I will maintain my rating of 3.

We thank you for your follow-up and for your continued interest in our work! We truly appreciate your time, constructive comments, and support for our work. We will ensure that all discussed details are thoroughly updated in the revised manuscript.

Best regards, The Authors

Reviewer_hvQEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents Self-Drift, a one-step generative framework that combines MeanFlow-style trajectory learning with distribution correction from Drifting Models. Instead of relying on an external pretrained encoder, the method uses its own intermediate features to compute attraction-repulsion signals between real and generated samples. The proposed method achieves an FID of 1.36 on ImageNet 256×256 with one function evaluation.

Critical strengths

The method achieves strong one-step generation results, improving over iMF from 1.72 to 1.36 FID.

Using the model's own features for distribution correction is an interesting idea that removes the need for an external pretrained encoder.

The paper includes useful ablations to support its main design choices.

Critical weaknesses

Limited novelty: The method mainly combines MeanFlow-style training with the drifting loss introduced in Drifting Models. Using the model's own features is a useful extension, but the overall approach is still quite similar to existing methods.

Training complexity: The proposed method requires additional feature extraction, memory banks, and whitening during training. It would be helpful to report the extra computational cost compared to standard MeanFlow.

Limited comparisons: Although the paper includes an MAE-based drifting baseline, a more direct comparison with Drifting Models and W-Flow under similar training settings would make the improvements more convincing.

Limited evaluation: The experiments are restricted to class-conditional ImageNet. It would be interesting to see whether the method also works for more challenging tasks such as text-to-image generation.

Questions for authors

see weaknesses

3 comments

We thank the reviewer for noting the strong one-step results and the value of removing the external encoder. Tables G1–G4 are in the General Response.

W1. Novelty. Our contribution is to make the generator's own online features usable as the drift geometry; to our knowledge, existing drifting methods do not do this, and a naive combination fails. We agree that the attraction–repulsion field, its multiscale normalization and the detached target follow Drifting Models.

  1. Removing the external feature encoder is an open problem for drifting. Drifting Models report that they "were unable to make our method work on ImageNet without a feature encoder", and W-Flow lists its reliance on a pretrained feature encoder (Drifting's released MAE) as a limitation. SD learns the generator and its drift geometry in the same from-scratch run, with no pretrained feature encoder, teacher or discriminator.
  2. Naively using the generator's own features fails. If the drift loss also updates the lens parameters, it can be reduced by moving the lens instead of the samples, and training collapses (FID 193.68, Table 2a). The frozen-lens semi-gradient (Eq. 15) blocks drift gradients into the lens and makes a shared online lens usable. A frozen lens snapshot is clearly worse (3.19 vs. 2.74), and removing real-anchored whitening raises FID to 2.92.
  3. The online lens needs a clean endpoint at every training state. An iMF network needs an extra forward pass in addition to its JVP and auxiliary v-head; with the same drift loss it reaches 2.89, as does a CT-style target. Our JVP-free objective yields the endpoint as a by-product of the trajectory loss and gives the best FID in our runs (2.74, 0.15 ahead of both, beyond the ≤0.05 sampling-seed variation; Table G2).
  4. Outcome. With no pretrained feature encoder, the online lens reaches lower FID than Drifting's pretrained latent MAE plugged into the same framework (2.74 vs. 2.84 at matched ω; higher precision, lower recall; Table G4). After an MAE warm start, switching to the online lens also beats keeping MAE (2.72 vs. 2.84; Table R2-2, response to R2).

W2. Training cost. The overhead is moderate (Table G3): a drift-active SD iteration costs 1.22× a trajectory-only iteration (0.22 vs. 0.18 s; 34.1 vs. 28.0 GiB) and is still 2.8× faster than an iMF iteration without SD in our implementation (0.61 s, 53.6 GiB), since our trajectory objective needs no JVP. Feature extraction reuses a prefix of the same network (9 of 12 blocks for B/2) and adds no learned parameters; the memory banks take 409.5 MiB; inference is unchanged. In this setting, extra compute does not explain the gain: 160 SD epochs (≈195 trajectory-only-equivalent) reach 2.91, whereas 240 further trajectory-only epochs leave FID unchanged (3.69; Table G4).

W3. Comparison with Drifting Models and W-Flow. Retraining both methods in our setting is not feasible within the rebuttal period, so we provide two comparisons. (i) Within our framework, replacing our online lens with Drifting's released MAE gives 2.84 vs. 2.74 (MAE-Drift, Table G4). (ii) Published results (Table R1-1; their "B/2" has 133M parameters, ours 87M):

Table R1-1. One-step results by generator size (FID / IS; ours from Table G1, Drifting Models and W-Flow as published).

MethodParamsFID / ISPretrained feature encoder
SD B/287M2.74 / 230.1no
Drifting B/2133M1.75 / 263.2yes
SD M/2172M1.78 / 262.4no
Drifting L/2463M1.54 / 258.9yes
SD L/2406M1.47 / 282.2no
W-Flow B/2 / L/2 / XL/2133M / 463M / 679M1.52 / 271.8, 1.35 / 272.5, 1.29 / 265.4yes
SD XL/2608M1.36 / 291.8no

Without an external encoder, SD is clearly behind Drifting Models at B/2 (2.74 at 87M vs. 1.75 at 133M), close at M/2 vs. their B/2 (1.78 vs. 1.75), and better at L/2 (1.47 vs. 1.54). W-Flow has lower FID at every scale, using a Sinkhorn objective and a pretrained encoder; we state this gap in the conclusion. Computing such objectives in the generator's own representation is an interesting direction.

W4. Text-to-image. A text-to-image study is beyond our compute within the rebuttal period; we will list it as a limitation. The lens is an intermediate generator layer, which text-conditioned transformers also have. Part of the drift signal is not class-specific: mismatched-class banks still align positively for 67.6–69.8% of classes (random: 44.9%; Fig. 3), although class keys add 7–10 pp later in training (Table R2-1). For text conditioning, banks could be keyed by clusters of text embeddings.

Dear Reviewer,

The discussion deadline is approaching. Please review the authors’ responses and clearly indicate whether they adequately address your concerns. Kindly update your score if appropriate.

Best regards,

AC

Thank you again for your review. In brief, our answers to your four points:

  • W1, novelty. We agree that the attraction–repulsion field, its normalization and the detached target follow Drifting Models. Our contribution is making the generator's own online features usable as the drift geometry, which removes the external encoder Drifting Models report they could not do without on ImageNet. Naive self-features collapse training (FID 193.68); the frozen-lens semi-gradient prevents this, and in the same framework the online lens gives lower FID than Drifting's MAE (2.74 vs. 2.84).
  • W2, cost (Table G3). A drift iteration costs 1.22× a trajectory-only iteration (28.0 → 34.1 GiB) and is still 2.8× faster than an iMF iteration; no learned parameters are added and inference is unchanged.
  • W3, Drifting / W-Flow (Table R1-1). Without a pretrained encoder, SD reaches lower FID than Drifting Models at L/2 (1.47 vs. 1.54); W-Flow, which uses a pretrained encoder, is slightly lower, as we state in the paper. Table R1-1 gives the full size-aligned comparison.
  • W4, text-to-image. Beyond the rebuttal period; listed as a limitation.

We would be glad to clarify anything before the discussion closes.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Response (to all reviewers)

We thank all reviewers for their constructive reviews. We report two corrections and the new results shared by all responses (Tables G1–G4). SD denotes Self-Drift; numbers are FID-50K / IS at NFE=1 on ImageNet 256×256; re-evaluating a checkpoint with different sampling seeds changes FID by ≤0.05 ("sampling-seed variation").

Corrections. (1) Prompted by R3-Q2, we found that the B/2 trajectory-only row of Table 2 (3.57 / 259.3) was evaluated at ω=10/11 by mistake; all our other entries use ω=0.85. The 240-epoch starting checkpoint (3.63; Sec. 4.3, App. C.1) was also evaluated at ω=10/11 (3.69 at ω=0.85). (2) For B/2, Table 1 mistakenly lists a from-scratch measurement (2.66; the run probed in Table R2-1) instead of the Table 2b result (2.74; 240 trajectory-only + 240 SD epochs, App. C.1). We apologize for these errors.

Table G1. Matched ω (FID / IS).

Modelωw/o SDw/ SD
B/20.853.69 / 236.02.74 / 230.1
B/210/113.57 / 259.32.83 / 258.8
M/20.852.98 / 263.11.78 / 262.4
L/20.852.64 / 283.41.47 / 282.2
XL/20.852.46 / 292.91.36 / 291.8

At matched ω, SD lowers FID at every scale (by 0.74–1.20) with IS within 5.9 points at B/2 and 1.2 at M/L/XL, so most of the IS drop in Table 2 came from comparing two guidance scales. At fixed ω=0.85 the B/2 gain is 0.95; the paper's 0.83 compared each model at its better scale.

Table G2. Same SD loss on three trajectory objectives (B/2, FID; each continues from its own 240-epoch trajectory-only checkpoint to epoch 480 with identical settings and SD hyperparameters (not re-tuned); ω=0.85, i.e. 17/3 on iMF's scale).

ObjectiveRequiresw/o SDw/ SD
iMF (our re-run)JVP, aux. v-head, 1 extra forward (r=0r=0) per SD query3.622.89
CT-style (ground-truth step)none3.742.89
Dual-horizon (ours)none3.692.74

SD improves all three objectives. Without SD our objective is slightly behind iMF; with SD it is 0.15 ahead of both (beyond sampling-seed variation), and the CT-style row isolates the model's own local step. The reported iMF-B/2 (3.39) uses 640 epochs and its own schedule; our re-run measures SD's effect under identical conditions, not iMF's best performance.

Table G3. Training cost (B/2, batch 1024, bf16, our implementation on one 8×H100 node; SD uses the same query batch for both objectives).

Runs / iterPeak GiB / GPU
Ours, trajectory-only0.18 (1.00×)28.0
Ours + SD, memory update only0.21 (1.17×)28.4
Ours + SD, drift active0.22 (1.22×)34.1
iMF0.61 (3.39×)53.6
iMF + SD (incl. the r=0r=0 forward)0.65 (3.61×)59.6

The overhead matches our analytical estimate (App. C.2: 1.17× / 1.26×); the banks take 409.5 MiB. Even with SD, our flow is 2.8× faster per iteration than iMF without it. Extra compute does not explain the gain: 160 SD epochs (≈195 trajectory-only-equivalent) reach 2.91, whereas 240 further trajectory-only epochs leave FID at 3.69 (Table G4).

Table G4. Dynamics and precision / recall (B/2, ω=0.85, continuations from the shared 240-epoch checkpoint; FID / precision / recall, improved P/R with k=3 in Inception space).

VariantEpoch 320Epoch 400Epoch 480
Trajectory-only3.84 / 0.801 / 0.5203.78 / 0.798 / 0.5093.69 / 0.794 / 0.518
MAE-Drift3.02 / 0.787 / 0.5673.03 / 0.780 / 0.5782.84 / 0.783 / 0.577
SD3.03 / 0.803 / 0.5362.91 / 0.797 / 0.5512.74 / 0.795 / 0.564

SD mainly improves coverage: recall rises (0.518 → 0.564 at epoch 480) while precision stays within 0.002 of trajectory-only. MAE-Drift trades precision for recall and ends at higher FID (2.84 vs. 2.74). Continuations restart the optimizer without warm-up, which likely causes the temporary rise of trajectory-only.

Revision. We will correct Tables 1–2 and the related text, report ω in every table, add Tables G1–G4, R2-1 and R2-2, the CFG sweep and the DMD comparison at matched ω, discuss consistency training, iMF, Drifting Models and W-Flow, remove the FID–IS trade-off sentence in Sec. 4.1, and add limitations (single dataset, class-conditional banks, pixel space, text-to-image).

Area_Chair_b5UCOpenReview ↗

Reminder: Discussion Deadline — November 18

Dear Reviewers,

Thank you for your valuable time and contributions to ICLR. This is a friendly reminder to review the authors' responses and engage in discussions with them. The discussion deadline is November 18.

Thank you for your continued efforts!

ACs

Summary of the discussion

We sincerely thank the AC and all reviewers for their time and constructive feedback. To help the final assessment, we briefly summarize where the discussion stands. Unless noted, numbers are FID-50K at NFE=1 for B/2, and table IDs refer to our responses.

In short. The central question raised in the discussion was whether the gains hold under a fair choice of guidance. With guidance tuned separately for each model, the B/2 gain remains 0.83 (3.57 → 2.74; R3-1), and at a fixed ω the gain holds at all four model sizes (G1). The main point that remains open is novelty relative to Drifting Models.

What we corrected. While answering the reviewers, we found and corrected two evaluation errors at B/2: the trajectory-only entries (the Table 2 row and the 240-epoch starting checkpoint) had been evaluated at ω=10/11 rather than 0.85, and the Table 1 entry should read 2.74 instead of 2.66. Neither affects the SD results at M/L/XL/2, including 1.36.

What the new evidence shows. The new experiments address the reviewers' main questions as follows:

  • IS drop and guidance (R2, R3). Beyond the 0.83 gain at tuned guidance, every baseline setting on the grid is matched by an SD setting with lower FID and equal or higher IS (R3-1). At ω=0.85, SD also lowers FID at all four sizes, with IS within 1.2 at M/L/XL (G1).
  • Fidelity vs. coverage (R2, R3). The gain comes mainly from coverage: recall rises from 0.518 to 0.564 while precision is unchanged (G4).
  • Other trajectory objectives (R3). The same SD loss also improves iMF, by 0.73 (ours: 0.83), and ours + SD reaches the lowest FID of the three objectives (2.74 vs. 2.89; G2, R3-2), without a JVP. We therefore position the dual-horizon flow as the trajectory interface for SD rather than as a standalone improvement over iMF.
  • Cost (R1, R3). A drift iteration takes 1.22× the time of a trajectory-only iteration and 28.0 → 34.1 GiB of memory, while inference is unchanged (G3).
  • Self-features as the lens (R2; MAE comparison: R1). The drift direction is already above a random control at activation (58.3% vs. 47.3%) and stays above it through epoch 480 (R2-1). In the same framework, the online lens reaches 2.74, compared with 2.84 for Drifting's MAE, 2.88 with delayed activation and 3.19 with a frozen lens; switching from MAE to the online lens gives 2.72 (R2-2).

What remains open. Novelty relative to Drifting Models is ultimately a judgment. The attraction–repulsion field, its normalization and the detached target follow Drifting Models. What is new is making the generator's own online features usable as the drift geometry, which removes the external encoder that Drifting Models and W-Flow rely on. This is not a direct plug-in: using the generator's features naively collapses training (FID 193.68), and the frozen-lens semi-gradient is what prevents it. Finally, text-to-image, other datasets and pixel space will be listed as limitations; the revision will add all new tables and use tuned guidance as the main comparison.

We thank the reviewers again for helping us improve the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.