- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Self-Drift, which improves one-step image generation using the generator’s own internal features to compare generated and real images. Alongside trajectory learning, it adds a training signal that moves generated features toward real ones and discourages excessive clustering. It requires no separate pretrained feature encoder, and generation still takes one network evaluation.
Experiments on ImageNet 256×256 in VAE latent space show an FID improvement from 2.46 to 1.36 for the XL/2 model. Ablations examine the effects of feature selection, gradient handling, and feature normalization.
Critical strengths
Substantial performance improvements without extra inference cost. Adding Self-Drift reduces the XL/2 model’s FID from 2.46 to 1.36 compared with the authors’ trajectory-only baseline. Improvements are also reported across all four model sizes, providing evidence that the benefit is consistent rather than limited to one configuration. These gains are consequential because they preserve one-step generation and require no separate pretrained feature encoder for distribution correction.
Critical weaknesses
-
The need for the new trajectory formulation is not fully established. Its endpoint-learning objective closely resembles consistency training, but there is no direct comparison with a standard consistency model or iMF using the same self-feature drift loss. The short-step ablation removes endpoint learning entirely, so it does not establish an advantage over these alternatives. This leaves the contribution of the new trajectory design unclear.
-
The quality trade-off needs stronger evaluation. In Table 2a, Self-Drift improves FID from 3.57 to 2.74 but reduces Inception Score from 259.3 to 230.1. This does not by itself imply worse diversity, but it makes the overall quality improvement less clear. For example, this may be due to a suboptimal choice of CFG scale. Additional measures of sample quality and coverage would help establish whether the method improves distribution matching beyond FID on ImageNet.
Questions for authors
The following questions seek clarification and additional evidence:
- How does the proposed trajectory formulation compare with standard consistency training or iMF using the same Self-Drift loss? What benefit comes specifically from the dual-horizon design?
- Does the IS drop persist when CFG is tuned separately for the baseline and Self-Drift? How do their FID–IS trade-offs compare across CFG scales?
- What are the measured training time and peak memory costs relative to trajectory-only training? Do the improvements remain under matched training-compute budgets?
- Have you evaluated Self-Drift in pixel space? If so, how does it compare with pixel-space methods such as pMF under comparable model sizes and training budgets?
7 comments
Thank you for the additional experiments and for identifying the CFG mismatch in the original table. The new comparisons help clarify how much of the gain comes from Self-Drift and how much comes from the trajectory formulation. I have two follow-up questions:
-
CFG tuning. For the main comparison, I would report each model’s best FID after tuning its guidance scale independently under the same evaluation protocol and with a comparable search budget. The fixed- results are useful as a control, but a shared scale may favor one model. Could you report the selected , best FID, and corresponding IS for both the trajectory-only baseline and Self-Drift, along with the full sweep?
-
iMF checkpoint. Could you clarify how the iMF-B/2 checkpoints used for the 3.62 and 2.89 FIDs in Table G2 were initialized, trained, and selected? I understand the rerun uses a 480-epoch schedule, whereas the original iMF paper reports 3.39 after 640 epochs. Were the CFG scale and guidance interval tuned independently for iMF with and without Self-Drift, and which settings produced the reported numbers?
We thank the reviewer for the follow-up questions.
1. CFG tuned independently. We swept ω for both B/2 models on one grid, with the evaluation protocol of Table G1 (FID-50K / IS at NFE=1, epoch-480 checkpoints, same reference statistics and sampling seeds). The grid uses 0.05 steps from 0.75 to 0.90, plus 10/11 and 12/13 around the baseline's optimum, all within the training range of ω; FID rises at both ends for both models, so both optima lie inside it. It includes the 0.05-step grid mentioned in our first response.
Table R3-1. ω sweep (B/2, FID / IS; best FID per model in bold).
| ω | Trajectory-only | SD |
|---|---|---|
| 0.75 | 4.44 / 204.3 | 3.45 / 199.1 |
| 0.80 | 3.97 / 219.5 | 3.01 / 214.3 |
| 0.85 | 3.69 / 236.0 | 2.74 / 230.1 |
| 0.90 | 3.62 / 253.1 | 2.74 / 247.7 |
| 10/11 | 3.57 / 259.3 | 2.83 / 258.8 |
| 12/13 | 3.65 / 260.6 | 2.88 / 260.8 |
- Selected ω, best FID and IS. Trajectory-only: ω=10/11, 3.57 / 259.3. SD: ω=0.85, 2.74 / 230.1. With each model tuned independently, SD improves FID by 0.83, far beyond the sampling-seed variation (≤0.05).
- FID–IS trade-off. At every ω, SD lowers FID by 0.74–0.99 with IS within 5.9 points. For every trajectory-only setting, some SD setting has both lower FID and higher IS; e.g., SD at ω=12/13 (2.88 / 260.8) vs. the baseline's best (3.57 / 259.3). At ω=0.90, SD keeps FID 2.74 with IS 247.7.
This sweep covers B/2 (Table 2a). The revision will report independently tuned results as the main comparison for all sizes, include the full sweeps, and keep fixed ω as a control.
2. iMF checkpoints (Table G2). First, a correction to the G2 caption: iMF without SD did not use the equivalent of our ω=0.85, and "17/3" was a conversion error. In iMF's convention (ω=1: no guidance; guidance coefficient 1−1/ω, which plays the role of our ω), the settings were:
- Without SD: the official iMF setting, ω=8 (coefficient 0.875) on the interval [t_min, t_max]=[0.4, 0.65]; sweeping ω on our checkpoint confirmed ω=8 as the best scale (it is also the upper end of iMF's training range [1, 8]).
- With SD: the same interval; sweeping ω selected ω=20/3 (coefficient 0.85). Thus the scale was tuned independently for both runs, while both kept the official interval; a joint search over the interval was beyond our budget.
Initialization and training. Both runs start from the same iMF-B/2 checkpoint, trained from scratch for 240 epochs with the official iMF recipe (including its guidance-conditioning ranges); it reaches FID 3.85, close to the 3.82 that iMF reports at 240 epochs (3.39 at 640; Tab. 1c). Each run then continues to epoch 480 with the continuation settings shared by all G2 rows (App. C.1); iMF + SD adds the SD loss with the same hyperparameters as ours, not re-tuned. Both keep iMF's JVP objective and auxiliary v-head. Selection: both numbers come from the final epoch-480 checkpoint.
Table R3-2. Each model at its selected guidance (B/2, epoch 480, FID / IS; iMF settings in iMF's convention, coefficient after the arrow).
| Objective | w/o SD | w/ SD |
|---|---|---|
| iMF (our re-run) | 3.62 / 251.7 (ω=8 → 0.875) | 2.89 / 225.7 (ω=20/3 → 0.85) |
| Dual-horizon (ours) | 3.57 / 259.3 (ω=10/11) | 2.74 / 230.1 (ω=0.85) |
SD improves iMF by 0.73 and our objective by 0.83. Without SD the two objectives are close (3.62 vs. 3.57); with SD ours is 0.15 lower. Like ours, iMF selects a weaker guidance once SD is added.
We thank the reviewer for the expert and concrete questions; Q2 led us to find an evaluation inconsistency, which we correct in the General Response together with Tables G1–G4.
W1 / Q1. Need for the dual-horizon formulation. SD does not require the dual-horizon flow (it improves all three objectives in Table G2), but the combination gives the best FID; we therefore present the dual-horizon flow as the trajectory interface designed for SD, not as a standalone improvement over iMF. Table G2 applies the identical SD loss under the same schedule: iMF (our re-run; JVP, auxiliary v-head and one extra forward pass at r=0 per SD query) goes from 3.62 to 2.89; a CT-style target (ours with the ground-truth step) from 3.74 to 2.89; ours from 3.69 to 2.74. Without SD our objective is slightly behind iMF; with SD it leads both other objectives by 0.15, beyond the sampling-seed variation (≤0.05), while needing no JVP, auxiliary v-head or extra forward pass. We did not train a standard consistency model with its own parameterization; the CT-style row isolates the consistency target within our network.
Relation to CT and MeanFlow/iMF. Our long-horizon loss (Eq. 6) is a discrete self-consistency on the remaining displacement tuθ: the jump from zt must equal a short step followed by the jump from the resulting state. It is the MeanFlow identity restricted to the data endpoint (r=0) and discretized by a backward finite difference (App. B.2), so it belongs to the CT/MeanFlow family. Three design choices tailor it to SD: (1) the destination is always the data endpoint, so every sampled state yields a clean endpoint without an extra forward pass, whereas iMF conditions on intervals (r,t); (2) the intermediate state is reached with the model's own local step δsg[vθ] (same network, task condition h=0) rather than CT's ground-truth interpolation; the CT-style row isolates this choice (2.74 vs. 2.89); (3) there is no JVP and no time input.
Why is the trajectory-only flow weaker than iMF? Partly because of schedule length and partly by design. Under the same 480-epoch schedule at B/2 it is close to iMF (3.69 vs. 3.62); we have matched-schedule evidence only at B/2, and the larger gaps in Table 2b compare with iMF's 640/800-epoch results. By design, our objective is the trajectory interface for SD rather than a standalone generator: in our implementation it is 3.4× faster per iteration than iMF with 48% less peak memory (Table G3), and it gives the best FID once SD is added (Table G2).
W2 / Q2. IS drop and CFG. Yes, the reviewer's hypothesis is correct: the trajectory-only row of Table 2 used ω=10/11, whereas SD and all our other entries used ω=0.85. At matched ω, SD lowers FID by 0.95 (ω=0.85) and 0.74 (ω=10/11) with IS within 5.9 and 0.5 points, and at M/L/XL by 1.10–1.20 with IS within 1.2 points (Table G1), so SD reaches lower FID at comparable IS (e.g., at ω=10/11: 2.83 / 258.8 vs. 3.57 / 259.3). When each model is taken at its better scale (10/11 for the baseline, 0.85 for SD), as in the paper's 0.83, the IS drop remains, because SD reaches its best FID at the lower guidance scale. Precision and recall (Table G4) show that the gain comes mainly from coverage: recall 0.518 → 0.564 at unchanged precision (0.794 → 0.795). We chose ω=0.85 by a sweep in steps of 0.05; the revision will include the full sweep.
Q3. Training time, memory and matched compute. Drift-active iterations cost 1.22× trajectory-only iterations (analytical: 1.26×) and raise peak memory from 28.0 to 34.1 GiB; memory-update iterations cost 1.17× (Table G3). Even with SD, our flow is 2.8× faster per iteration than iMF without it in our implementation (0.22 vs. 0.61 s) and uses less memory (34.1 vs. 53.6 GiB). In this setting, extra compute does not explain the gain: 160 SD epochs (≈195 trajectory-only-equivalent) reach 2.91, whereas 240 further trajectory-only epochs leave FID unchanged (3.69; Table G4).
Q4. Pixel space and pMF. We have not evaluated SD in pixel space and cannot compare with pMF within the rebuttal period; we will state this as a limitation. Nothing in the method is specific to the VAE latent space, since the lens is a prefix of the generator. However, pixel space is where drifting methods rely on the strongest external encoders (Drifting Models use a stronger MAE and ConvNeXt-V2 features there), so it remains to be shown that SD removes this dependence in pixel space.