acceptodds
Withdrawn from ICLR 2027

Past-Self Guidance: Internalizing Temporal Contrast in Flow Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Training produces a history of predictors, but generation usually uses only its final model. At sampling time, adding the component of a current-past prediction difference perpendicular to the current prediction improves samples but not regression loss. We introduce Past-Self Guidance (PSG), which adds the full prediction difference to the flow-matching target. Both predictions use the same noisy input, one from the model's recent exponential moving average and one from an earlier ordinary-training checkpoint. Sampling needs only the trained model. On ImageNet 256 x 256, one setting improves FID-50k by 34-55% across four SiT scales at 400k updates, without classifier-free guidance (CFG). At XL/2, FID falls from 16.89 to 7.70 at similar recall. PSG also has lower FID than ordinary training that uses about twice as much compute. Applied only in the last 10% of a 2M-update run, PSG reaches CFG-free FID-50k 5.73, against 10.10 for ordinary training. It needs that run's earlier checkpoints and about 6% more training compute. Activated at 1M updates, it reaches FID-50k 1.99 with CFG at 2M. It combines with REPA and improves diffusion and unconditional generation. Ablations and sampling interventions show that the gain comes mainly from the direction of this difference and disappears when the difference is taken from another sample.

Primary area:
generative models
Keywords:
flow matching, generative models, diffusion models, training-time guidance, temporal prediction contrast

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_dkBMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Past-Self Guidance (PSG), a training scheme that improves flow-based generative models by incorporating an AutoGuidance-inspired temporal contrast into the flow-matching objective. Specifically, PSG perturbs the standard flow-matching velocity target with the difference between the predictions of a recent teacher and a past teacher, and trains the student model to regress toward this modified target. The recent teacher is either the student's exponential moving average (EMA) or a periodically refreshed copy of it, while the past teacher is selected from checkpoints of ordinary flow-matching training at a fixed fraction of the current training step. By internalizing this temporal guidance during training, PSG requires no additional computation at inference time. Experiments demonstrate substantial improvements in generation quality, as measured by FID, across multiple model scales (SiT-S/B/L/XL), with additional improvements demonstrated for diffusion-based DiT models.

Critical strengths

Simplicity and effectiveness. The proposed PSG method is conceptually simple and easy to integrate into existing flow-matching training pipelines, requiring only a modification to the regression target without architectural changes or additional inference-time computation. Despite its simplicity, PSG achieves substantial improvements in generation quality, reducing FID by 34–55% across SiT model scales. Particularly compelling is its effectiveness when applied only during the final 10% of training, yielding a significant FID improvement with approximately 6% additional training compute.

Extensive experimental validation and insightful analyses. The paper presents a comprehensive empirical evaluation across multiple model scales (SiT-S/B/L/XL), generative objectives (flow matching and diffusion), and generation settings (conditional and unconditional). The authors further demonstrate compatibility with existing techniques such as REPA and CFG. Beyond performance comparisons, the paper provides insightful ablation studies investigating the underlying mechanism of PSG. In particular, the authors show that most of the improvement comes from the angular component of the historical prediction difference and that the correction is sample-specific. These findings provide interesting empirical evidence that optimization history contains useful information for improving generative modeling.

Critical weaknesses

Insufficient comparison with closely related guidance-distillation methods. Although the paper demonstrates substantial improvements over vanilla flow matching, it remains unclear whether PSG offers a superior fidelity–diversity tradeoff compared to existing guidance-distillation approaches, particularly CFG-based training contrast. The authors provide a limited comparison in Appendix B.4, where conditional guidance achieves significantly better FID at the expense of recall. However, a more systematic comparison across guidance strengths, ideally examining FID–recall Pareto frontiers under matched computational budgets, would be valuable for establishing the distinct advantages of history-based guidance.

Limited understanding of the underlying mechanism. While the paper provides insightful ablation studies demonstrating the importance of the angular component and sample-specific nature of historical corrections, the underlying reasons for their effectiveness remain unclear. In particular, it is not well understood why extrapolating along historical prediction differences improves generation quality, or what useful information these differences capture about the learned velocity field. The stationary-point analysis establishes a connection to sampling-time guidance but does not fully explain the observed performance improvements.

Questions for authors
  • Comparison against CFG-based guidance distillation. In Appendix B.4, conditional guidance achieves substantially lower FID than PSG, albeit with lower recall. Could the authors provide a more systematic comparison across guidance strengths, examining the FID–recall Pareto frontiers of PSG and conditional-guidance training under comparable computational budgets? Does PSG offer a meaningful advantage at matched fidelity or diversity?
  • Complementarity with conditional guidance. Could the authors investigate whether temporal guidance provides complementary benefits when combined with CFG-based training contrast? Such an experiment would help establish whether historical prediction differences contain useful information beyond existing guidance signals.
  • Mechanism underlying historical guidance. Although the ablations demonstrate the importance of the angular component of historical prediction differences, it remains unclear why this direction improves generation. Could the authors provide further evidence about what information this direction captures? For example, does it systematically correct prediction errors, or does it primarily alter the fidelity–diversity tradeoff?
  • Generality beyond ImageNet. While PSG demonstrates consistent improvements across model scales and generative objectives, the experiments primarily focus on ImageNet 256×256. Do the authors have evidence suggesting that PSG generalizes to other datasets or more challenging generation settings, such as text-to-image synthesis?
Reviewer_Mq42OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Past-Self Guidance (PSG), a training-time guidance method for flow-based generative models. The key idea is to construct a temporal contrast between the predictions of a recent EMA model and an earlier checkpoint, and incorporate this difference into the standard flow-matching regression target. In this way, the model internalizes the correction during training without requiring an additional teacher model at inference time.

Critical strengths
  1. A simple and practical approach with consistent improvements over vanilla training. The proposed method is straightforward to implement and does not require additional model evaluations at inference time. The improvements are consistent across four SiT model scales.

  2. Extensive ablation studies and empirical analysis. The paper provides a comprehensive investigation of the temporal correction mechanism. The authors demonstrate that PSG can be combined with REPA.

Critical weaknesses
  1. Limited conceptual novelty and insufficient understanding of the proposed guidance mechanism.

My primary concern is that the proposed method appears closely related to guidance distillation, particularly the training-time internalization of self-guidance. I do not find the analysis in Section 3 sufficient to explain why the proposed correction should improve the underlying generative distribution. In particular, the analysis establishes an algebraic relationship between the modified training objective and guidance-based extrapolation, rather than explaining why this extrapolation produces a better approximation of the target distribution.

This distinction is important because guidance methods can substantially improve sample quality metrics without necessarily improving distributional fidelity. The proposed objective may therefore be better understood as a form of training-time guidance internalization rather than a fundamentally improved flow-matching objective.

  1. The improvements may partially reflect a guidance-induced fidelity–diversity trade-off.

A related concern is whether PSG genuinely improves the learned generative distribution or primarily changes the fidelity–diversity operating point, similarly to conventional CFG distillation techniques. For example, according to Tables 18 and 19, increasing the PSG weight from w=0.5w=0.5 to w=0.75w=0.75 for SiT-XL/2 at 400K updates reduces the CFG-free FID from 7.70 to 3.94. However, the recall simultaneously decreases from 0.614 to 0.535, while sFID deteriorates from 4.56 to 6.30. These observations suggest that at least part of the improvement may arise from favoring sample fidelity over distributional coverage.

Interestingly, stronger PSG also reduces the benefits of additional inference-time CFG. For instance, applying CFG to the w=0.75w=0.75 model worsens its FID. This behavior is consistent with the interpretation that the model has already internalized a guidance-like correction during training. I believe the paper should more carefully distinguish genuine improvements in distribution modeling from guidance-induced changes in the fidelity–coverage trade-off. In particular, comparisons against existing guidance methods at matched recall or precision would be more informative than comparing FID values alone.

  1. Limited competitiveness against strong baselines, particularly REPA.

Although the proposed method substantially improves vanilla SiT, I am less convinced by its competitiveness against stronger training approaches. For instance, the original REPA paper reports FID 1.96 and IS 264.0 after 1M training updates on ImageNet 256×256. In comparison, PSG achieves FID 1.99 and IS 246.4. Moreover, PSG introduces additional teacher forward passes during training, which should also be considered when evaluating training efficiency.

Besides, the relatively lower Inception Score in the final system-level comparison also raises questions about the overall sample quality. Although IS alone is insufficient to establish poor distributional coverage, it would be useful to investigate why the substantial improvements in FID do not translate into equally competitive IS results.

Questions for authors

See weaknesses

Reviewer_vXQdOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces Past-Self Guidance (PSG), a training-time modification to flow matching that incorporates the prediction difference between a recent exponential moving average (EMA) teacher and an earlier ordinary-training checkpoint into the velocity target. Unlike sampling-time guidance methods, PSG internalizes this temporal contrast into a single predictor, requiring no additional model evaluations at inference. Experiments on ImageNet 256×256 show substantial FID improvements across four SiT model scales. The paper also investigates integration with REPA and CFG.

Critical strengths
  1. PSG modifies only the flow-matching training objective while preserving the original model architecture and sampling procedure. This makes the method straightforward to integrate into existing pipelines, and the reported results suggest that it can improve generation quality without introducing additional inference-time model evaluations.
  2. The paper provides a relatively comprehensive set of ablations examining the temporal prediction difference, its directional components, teacher configurations, and input assignments. These experiments offer useful empirical evidence regarding the factors associated with PSG’s performance gains.
Critical weaknesses
  1. Equation (2) characterizes the stationary prediction as a correction away from the past model. However, the population-optimal flow-matching predictor (\mu ) is already associated with the target data distribution under the paper's idealized formulation. It is therefore unclear why moving away from this predictor should improve FID. Although Section 5 investigates several factors associated with the observed gains, the analyses remain largely diagnostic and do not provide sufficient insight into why the temporal correction improves the generated distribution. The central mechanism behind PSG's effectiveness remains insufficiently understood.
  2. The proposed temporal prediction difference appears to be a relatively general auxiliary training target that could potentially be applied to other learning frameworks with a history of model predictions. The paper does not sufficiently explain what properties of flow matching make it particularly suitable for this approach, why the proposed correction is effective in this setting, or what advantages arise specifically from its interaction with flow-matching training dynamics. Consequently, the method's distinctive contribution beyond a general history-based target modification remains unclear.
  3. The evaluation relies primarily on quantitative metrics such as FID, without presenting qualitative visualizations of generated ImageNet samples to illustrate the reported improvements.
  4. The motivation, theoretical interpretation, and empirical analysis are not always connected clearly, making the paper hard to follow.
Questions for authors
  1. Can the authors clarify the assumptions underlying the stationary interpretation in Eq. (2)? For a parameterized neural network, a stationary point of the stop-gradient objective generally implies a Jacobian-weighted orthogonality condition, rather than necessarily implying the stated pointwise equality. Is Eq. (2) intended as an idealized function-space optimum, and if so, can the corresponding assumptions be stated explicitly?
  2. Can the authors provide qualitative visualizations of the generated samples?
  3. Can the authors provide a deeper explanation of why PSG improves generation quality?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.