- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Temporal-Difference (TD) Guidance, a method for improving diffusion guidance by addressing the mismatch between value prediction accuracy and guidance-gradient accuracy. TD Guidance uses temporal consistency across adjacent noise levels through TD learning. The paper includes theoretical guarantees and experiments on synthetic benchmarks, CIFAR-10, and molecular generation demonstrate improvements over direct regression in guidance accuracy and generation quality.
Critical strengths
- The proposed TD guidance framework is relatively straightforward to implement and does not need to retrain the diffusion model.
- The paper provides a systematic theoretical analysis and provides useful insights into why temporal consistency can benefit guidance learning.
Critical weaknesses
- The empirical evaluation primarily compares TD guidance against direct regression. Given that the main motivation of the paper is to improve guidance-gradient accuracy, it would be valuable to include comparisons with existing gradient-aware guidance methods.
- For CIFAR-10 results, caption on Figure 1 writes "Selected exploratory CIFAR-10 generation results, not a fresh-seed confirmation.". The result is on five training seeds and looks promising, but the caption also mentioned the result is "unadjusted for method selection".
- As mentioned in the limitation section, the theoretical assumptions and the practical algorithm are different, which is a gap as acknowledged.
Questions for authors
Following weakness.
- Could author include more baseline?
- Could author explain the CIFAR-10 results, I understand TD learning improves upon direct regression, but I was confused on the caption.
- The molecular generation is very interesting, for image experiments, is there possibility to do a reward guidance like aesthetic scores, CLIP similarity or learned human preference score with T2I models?
4 comments
Response to Reviewer kSKc (2/2)
W3. Gap between assumptions and algorithm. In the revision, we have added a paragraph to articulate the bridge between our theory and the practical implementation. Algorithm 1 is precisely the semi-gradient stochastic-approximation scheme solving the empirical TD variational problem in section 3.3. Under some setting (fixed linear features, an undelayed target, boundary weight , and discount ), Algorithm 1 strictly coincides with the iteration analyzed in Theorem 5 in Appendix C. Theorem 5 proves that this iteration converges directly to the empirical solution characterized by Theorem 2. Consequently, the core guarantees of Theorems 1 and 3—which rigorously bound the terminal KL divergence—apply to the fixed point targeted by our algorithm.
Clarification on the discount factor (): In our theory, a positive discount () is required to ensure coercivity (making the problem mathematically well-posed). Crucially, because the actual guidance signal during sampling relies entirely on the spatial gradient , multiplying the target by a purely time-dependent scalar has mathematically zero effect on the resulting guidance direction. In practice, since the deep learning implementation remains highly stable in the unweighted limit (), the discount can be safely omitted without altering the final guidance applied to the generator.
Standard Implementation Practices: Our theory identifies which optimal fixed point to target. Standard empirical choices—dropping the theoretical discount, using nonlinear networks, EMA targets, and gradient clipping—are simply the engineering mechanisms required to optimize this mathematically principled objective stably in high dimensions.
Q3. Reward guidance for images. We have implemented CIFAR-10 reward guidance using a CLIP text–image similarity reward (). To rigorously prevent reward hacking, guidance is driven by CLIP ViT-B/32, while evaluation uses a held-out CLIP ViT-L/14. Under a pre-registered operating point (global FID unguided + 5), fresh-seed evaluation shows TD outperforms Direct on 6/6 seed × prompt pairs. Furthermore, TD matches or exceeds the training-free DPS baseline while requiring less than half the sampling compute (DPS costs 2.3× more per image).
| Prompt | TD(Mean) | Direct(Mean) | DPS |
|---|---|---|---|
| "a photo of a red car" | 0.1918 | 0.1879 | 0.1903 |
| "a photo of a bird flying" | 0.1847 | 0.1829 | infeasible* |
| *DPS exceeds the strict FID degradation budget even at the lowest guidance scale. |
Because DPS requires backpropagating through the denoiser at every step, it incurs 2.3× the per-image sampling compute of TD (or Direct).
Thank you
Thank you very much for the detailed rebuttal and additional experiments, the additional baseline helps the empirical results. However, I believe the gap between theory and practice remains. I will maintain my original score.
Dear Reviewer kSKc,
Thank you for your follow-up and for noting that the additional baseline strengthens our empirical results.
Regarding the remaining gap between theory and practice, our theoretical analysis is intentionally formulated over linear and convex function classes to isolate the intrinsic structure of diffusion guidance—specifically, why value accuracy alone fails to control the guidance field and how the TD energy norm addresses this issue. Directly analyzing deep neural networks and EMA targets would entangle this question with non-convex optimization challenges. Separating tractable linear guarantees from neural implementations is a common practice in TD learning, where classical convergence theory is established for linear approximation (Tsitsiklis & Van Roy, 1997) while practical deep RL algorithms use neural networks with slowly updated targets (e.g., DDPG, Lillicrap et al., 2016).
Within this scope, Theorem 5 directly connects our theoretical results to Algorithm 1: specializing Algorithm 1 to linear features and an undelayed target recovers the exact stochastic-approximation iteration analyzed there, along with its energy-norm and downstream KL bounds (Theorems 1–3). The neural network and EMA target function as standard empirical mechanisms to approximate this same fixed point in high dimensions.
To help us address your concern concretely, could you clarify which specific aspect of the theory–practice gap remains your primary reservation? Knowing this would allow us to respond directly and refine the scope discussion in the revision accordingly.
Thank you again for your time and feedback.
Response to Reviewer kSKc (1/2)
We thank the reviewer for the positive assessment of our implementability and theoretical analysis.
W1/Q1. Baselines beyond direct regression. We evaluated three gradient-aware methods: (a) direct regression + input-gradient (Sobolev) penalty; (b) Doob matching (Chang et al., 2026); (c) two-stage regression (CDG-ML/MCL; Tang & Xu, 2024; Guo et al., 2026).
2D Benchmarks
Log-guidance MSE (×10⁻², mean ± SE over the 5 table seeds; lower is better):
CIFAR-10 Evaluation
W2/Q2. CIFAR-10 caption and fresh-seed confirmation. The original submission reported results on the same fixed seeds used for hyperparameter tuning. We evaluated the selected TD configuration on ten entirely new training seeds under a pre-registered protocol.
The pre-registered primary endpoint (macro conditional KID×10³, TD − Direct, averaged over seven matched-control targets) is −4.61 (95% CI [−6.17,−3.05]), consistently favoring TD on 10/10 seeds. Furthermore, TD achieves lower FID at all seven control levels, with every 95% CI strictly excluding zero.