acceptodds
Under review as a conference paper at ICLR 2027

Temporal-Difference Guidance for Diffusion Models

Abstract

Learned diffusion guidance aims to sample from a tilted target distribution using gradient-based corrections derived from a noise-dependent value function. Standard regression-based guidance, however, trains this function through value prediction alone: small value error need not imply accurate gradients, creating a mismatch between the training objective and the quantity that drives sampling. We introduce temporal-difference (TD) guidance, a simple, easily implementable modification of existing guidance training that bootstraps across adjacent noise levels. By exploiting the temporal structure of the diffusion process, TD guidance addresses this value–gradient mismatch. We establish a finite-sample oracle inequality in a Sobolev norm that jointly controls value and spatial-gradient errors, balancing approximation, discretization, and statistical errors. Across image and molecular generation benchmarks, TD guidance achieves better quality–control and reward–validity trade-offs than direct regression at comparable conditioning strength.

Primary area:
generative models
Keywords:
diffusion models, temporal-difference learning, conditional generation, reward guidance, oracle inequality

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_kSKcOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Temporal-Difference (TD) Guidance, a method for improving diffusion guidance by addressing the mismatch between value prediction accuracy and guidance-gradient accuracy. TD Guidance uses temporal consistency across adjacent noise levels through TD learning. The paper includes theoretical guarantees and experiments on synthetic benchmarks, CIFAR-10, and molecular generation demonstrate improvements over direct regression in guidance accuracy and generation quality.

Critical strengths
  1. The proposed TD guidance framework is relatively straightforward to implement and does not need to retrain the diffusion model.
  2. The paper provides a systematic theoretical analysis and provides useful insights into why temporal consistency can benefit guidance learning.
Critical weaknesses
  1. The empirical evaluation primarily compares TD guidance against direct regression. Given that the main motivation of the paper is to improve guidance-gradient accuracy, it would be valuable to include comparisons with existing gradient-aware guidance methods.
  2. For CIFAR-10 results, caption on Figure 1 writes "Selected exploratory CIFAR-10 generation results, not a fresh-seed confirmation.". The result is on five training seeds and looks promising, but the caption also mentioned the result is "unadjusted for method selection".
  3. As mentioned in the limitation section, the theoretical assumptions and the practical algorithm are different, which is a gap as acknowledged.
Questions for authors

Following weakness.

  1. Could author include more baseline?
  2. Could author explain the CIFAR-10 results, I understand TD learning improves upon direct regression, but I was confused on the caption.
  3. The molecular generation is very interesting, for image experiments, is there possibility to do a reward guidance like aesthetic scores, CLIP similarity or learned human preference score with T2I models?
4 comments

Response to Reviewer kSKc (1/2)

We thank the reviewer for the positive assessment of our implementability and theoretical analysis.

W1/Q1. Baselines beyond direct regression. We evaluated three gradient-aware methods: (a) direct regression + input-gradient (Sobolev) penalty; (b) Doob matching (Chang et al., 2026); (c) two-stage regression (CDG-ML/MCL; Tang & Xu, 2024; Guo et al., 2026).

2D Benchmarks

  • Setup: All baselines received an identical tuning budget (a 5-point grid on 3 disjoint discovery seeds). Direct and TD both include a clean-endpoint anchor (a squared/CE loss fitting the network to QQ at the clean end), whereas Chang et al. and CDG, as originally specified, do not. We therefore also ran each with Direct’s anchor added, and report, per task, the better of the two variants.

Log-guidance MSE (×10⁻², mean ± SE over the 5 table seeds; lower is better):

MethodRingBistableTwo moonsSpiral
Direct4.98 ± 0.244.29 ± 0.5318.41 ± 2.9914.03 ± 0.72
Direct + grad. penalty4.15 ± 0.114.15 ± 0.5419.02 ± 1.3911.63 ± 0.20
Chang et al. (2026)5.38 ± 0.225.15 ± 0.3320.86 ± 4.3712.61 ± 0.20
CDG-ML3.98 ± 0.185.00 ± 0.4222.69 ± 4.3514.71 ± 0.12
CDG-MCL10.71 ± 0.5710.61 ± 0.9930.97 ± 0.5618.64 ± 1.01
TD (submitted config)3.61 ± 0.183.37 ± 0.2713.04 ± 3.0811.47 ± 0.28
  • Results: TD achieves the lowest mean guidance error across all four tasks.

CIFAR-10 Evaluation

  • Setup: Baselines (a) and (b) were trained using identical architectures and optimization budgets (200 epochs). Hyperparameters were selected via a pre-specified rule on 3 seeds. The frozen configurations were then compared on 5 fresh training seeds against Direct across 7 matched-control levels (2,000 samples each). Method (c) was omitted as it requires reverse-SDE simulation and a second fitted network, and proved to be the least stable method in 2D.
Method − Directmacro cKID×10³ [95% CI]Seeds favourableGlobal FID [95% CI]
Direct + grad. penalty (γ=0.1\gamma=0.1)+1.56 [−0.63, +3.75]1/5+0.83 [−0.04, +1.70]
Chang et al. (λ=0.1\lambda=0.1)−1.52 [−2.31, −0.73]5/5−0.61 [−0.95, −0.27]
TD−3.81 [−5.54, −2.09]5/5−1.82 [−2.77, −0.87]
  • Results: Chang et al. improves upon Direct, but TD's improvement is approximately 2.5× larger (and outperforms Chang et al. on 5/5 seeds in a post-hoc comparison).
  • Computational Cost: Due to the double-backward pass, baselines (a) and (b) cost 2.3–2.5× more per training update than Direct. In contrast, TD requires only one additional forward pass, resulting in an overhead of just 1.10–1.18×.

W2/Q2. CIFAR-10 caption and fresh-seed confirmation. The original submission reported results on the same fixed seeds used for hyperparameter tuning. We evaluated the selected TD configuration on ten entirely new training seeds under a pre-registered protocol.

The pre-registered primary endpoint (macro conditional KID×10³, TD − Direct, averaged over seven matched-control targets) is −4.61-4.61 (95% CI [−6.17,−3.05][-6.17, -3.05]), consistently favoring TD on 10/10 seeds. Furthermore, TD achieves lower FID at all seven control levels, with every 95% CI strictly excluding zero.

Response to Reviewer kSKc (2/2)

W3. Gap between assumptions and algorithm. In the revision, we have added a paragraph to articulate the bridge between our theory and the practical implementation. Algorithm 1 is precisely the semi-gradient stochastic-approximation scheme solving the empirical TD variational problem in section 3.3. Under some setting (fixed linear features, an undelayed target, boundary weight α=β\alpha=\beta, and discount γ=e−λΔ\gamma=e^{-\lambda\Delta}), Algorithm 1 strictly coincides with the iteration analyzed in Theorem 5 in Appendix C. Theorem 5 proves that this iteration converges directly to the empirical solution characterized by Theorem 2. Consequently, the core guarantees of Theorems 1 and 3—which rigorously bound the terminal KL divergence—apply to the fixed point targeted by our algorithm.

Clarification on the discount factor (λ\lambda): In our theory, a positive discount (λ>0\lambda > 0) is required to ensure coercivity (making the problem mathematically well-posed). Crucially, because the actual guidance signal during sampling relies entirely on the spatial gradient ∇xlog⁡ft\nabla_x \log f_t, multiplying the target by a purely time-dependent scalar e−λte^{-\lambda t} has mathematically zero effect on the resulting guidance direction. In practice, since the deep learning implementation remains highly stable in the unweighted limit (λ→0\lambda \to 0), the discount can be safely omitted without altering the final guidance applied to the generator.

Standard Implementation Practices: Our theory identifies which optimal fixed point to target. Standard empirical choices—dropping the theoretical discount, using nonlinear networks, EMA targets, and gradient clipping—are simply the engineering mechanisms required to optimize this mathematically principled objective stably in high dimensions.

Q3. Reward guidance for images. We have implemented CIFAR-10 reward guidance using a CLIP text–image similarity reward (Q=erQ=e^r). To rigorously prevent reward hacking, guidance is driven by CLIP ViT-B/32, while evaluation uses a held-out CLIP ViT-L/14. Under a pre-registered operating point (global FID ≤\le unguided + 5), fresh-seed evaluation shows TD outperforms Direct on 6/6 seed × prompt pairs. Furthermore, TD matches or exceeds the training-free DPS baseline while requiring less than half the sampling compute (DPS costs 2.3× more per image).

PromptTD(Mean)Direct(Mean)DPS
"a photo of a red car"0.19180.18790.1903
"a photo of a bird flying"0.18470.1829infeasible*
*DPS exceeds the strict FID degradation budget even at the lowest guidance scale.

Because DPS requires backpropagating through the denoiser at every step, it incurs 2.3× the per-image sampling compute of TD (or Direct).

Reviewer_kSKcOpenReview ↗

Thank you

Thank you very much for the detailed rebuttal and additional experiments, the additional baseline helps the empirical results. However, I believe the gap between theory and practice remains. I will maintain my original score.

Dear Reviewer kSKc,

Thank you for your follow-up and for noting that the additional baseline strengthens our empirical results.

Regarding the remaining gap between theory and practice, our theoretical analysis is intentionally formulated over linear and convex function classes to isolate the intrinsic structure of diffusion guidance—specifically, why value accuracy alone fails to control the guidance field and how the TD energy norm addresses this issue. Directly analyzing deep neural networks and EMA targets would entangle this question with non-convex optimization challenges. Separating tractable linear guarantees from neural implementations is a common practice in TD learning, where classical convergence theory is established for linear approximation (Tsitsiklis & Van Roy, 1997) while practical deep RL algorithms use neural networks with slowly updated targets (e.g., DDPG, Lillicrap et al., 2016).

Within this scope, Theorem 5 directly connects our theoretical results to Algorithm 1: specializing Algorithm 1 to linear features and an undelayed target recovers the exact stochastic-approximation iteration analyzed there, along with its energy-norm and downstream KL bounds (Theorems 1–3). The neural network and EMA target function as standard empirical mechanisms to approximate this same fixed point in high dimensions.

To help us address your concern concretely, could you clarify which specific aspect of the theory–practice gap remains your primary reservation? Knowing this would allow us to respond directly and refine the scope discussion in the revision accordingly.

Thank you again for your time and feedback.

Reviewer_qZExOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose a method to better steer a pretrained model to sample from a tilted target distribution. The central claim (and they also give a practical example where this is true) of this paper is that small value error does not imply accurate gradients, which is what is used in sampling. Based on that, they show theoritocaly that using the energy norm of the error instead of the L2 norm controls better the guidance error and the terminal KL divergence because it account both errors in value functions and on gradients. The proposed guidance algorithm is a temporal-difference approache that enforces local consistency between value predictions at adjacent noise levels using an EMA target network. They experiment on 2D benchmarks (closed form references), CIFAR-10 (classifier guidance) showing improved FID score compared to the classifier guidance (guiding using a classifier trained on noisy data) and COMPAS (reward guidance).

Critical strengths

The problem is well motivated and proposition 1 gives an example where value accuracy alone does not ensure tilted sampling.

They show that the guidance error bounds the terminal KL and is bounded by the error energy norm which controls both the value and spatial-gradient errors used in guidance.

Critical weaknesses

The paper does not provide a transition from the theoretical part to the practical guidance algorithm. It misses a connection of the proposed algorithm and the theory presented in the paper. In particular, It is not clear if formal guaranties apply in practice to the proposed method, how Theorems 1-2 apply to the proposed method, how the TD variational solution is related to the proposed method.

In the experimental evaluation is limited. They do not test the theoretical setting and the proposed method is compared only against direct regression. Image evaluation is limited on low-resolution generation on CIFAR-10. It is not clear if the gain comes from the total loss, TD consistency or the boundary loss, reporting each term separately would help to understand the effect of each component.

Questions for authors
  • How Theorem 1-2 apply to the proposed method?
  • Does the proposed method use a discount factor λ>0\lambda >0? If no, why its is not needed? In Variational formulation paragraph they mention that a discounted factor λ>0\lambda >0 is needed in the target u∗u^* in order to have a well-posed variational problem (line 172).
  • Can the authors explain in Table1 why TD’s mean value error is higher on three out the four synthetic benchmarks compared to direct regression? The theory controls both value and gradient errors, so how do you explain the Log-guidance MSE to be lower on all four tasks for TD, and not the Value MSE?
  • Which is the effect of each term in the proposed objective? The gain comes from the total objective, TD consistency term or the boundary loss? Reporting each term separately would help to understand the effect of each term.
  • On Figure 1 you mention that you use ‘not a fresh-seed confirmation’. Does the result hold on new seeds? Can you report the scores for guidance scale 1 and 0 (no guidance) for a better comparison?
  • What is TD’s computational cost compared to to direct regression for both training and guidance?
  • How the proposed method performs compared to training‑free guidance baselines?
5 comments

Response to Reviewer qZEx (1/2)

We thank the reviewer for the detailed and constructive questions, particularly regarding the theoretical mechanisms.

W1 (theoretical setting) / Q1. How do Theorems 1-2 apply to the proposed method? In the revision, we have added a paragraph to bridge our theory and practical implementation. Algorithm 1 is exactly the semi-gradient stochastic-approximation scheme solving the empirical TD variational problem in section 3.3. Under some setting (fixed linear features, an undelayed target, boundary weight α=β\alpha=\beta, and discount γ=e−λΔ\gamma=e^{-\lambda\Delta}), Algorithm 1 strictly coincides with the iteration analyzed in Theorem 5 (Appendix C). Theorem 5 proves that this iteration converges directly to the empirical solution characterized by Theorem 2, allowing the core guarantees of Theorems 1 and 3 (bounding the terminal KL divergence) to apply to our algorithm's fixed point.

Q2. Is λ>0\lambda > 0 used? Why? In our theory, a strictly positive discount (λ>0\lambda > 0) is required to ensure coercivity (making the problem mathematically well-posed). Crucially, because the actual guidance signal during sampling relies entirely on the spatial gradient ∇xlog⁡ft\nabla_x \log f_t, multiplying the target by a purely time-dependent scalar e−λte^{-\lambda t} has mathematically zero effect on the resulting guidance direction. In practice, we found that the deep learning implementation remains highly stable in the unweighted limit (λ→0\lambda \to 0), so the discount can be safely omitted.

Q3. Why is TD's value MSE higher on three out of four benchmarks compared to direct regression? This observation directly highlights the core thesis of Proposition 1: low value error does not mathematically guarantee low gradient error. While Direct minimizes L2L^2 value error (hence a lower value MSE), TD targets the energy-norm, which controls both. Because sampling quality relies entirely on the spatial gradient (∇xlog⁡ft\nabla_x \log f_t), gradient accuracy dictates performance. As shown in Table 1, despite a negligible increase in value error (∼10−4\sim 10^{-4}), TD substantially reduces log-guidance error (∼10−2\sim 10^{-2}) across all tasks—which is our direct optimization target.

W1 (loss terms) / Q4. Effect of each term in the proposed objective. TD's objective has two strictly necessary terms: a clean-endpoint (boundary) loss and a TD term. To isolate the TD term, we compared against Direct regression, which shares the exact same boundary loss.

  • Setup: We ablated the TD loss weight λTD\lambda_{\rm TD} across 5 seeds at 7 matched-control levels (2,000 samples each). Values are macro cKID×10³ differences (Variant − Direct):
Variantvs Direct [95% CI] (seeds fav.)vs submitted TD [95% CI]
λTD=0.1\lambda_{\rm TD}=0.1−4.05 [−6.82, −1.28] (5/5)−0.38 [−1.83, +1.07]
λTD=0.2\lambda_{\rm TD}=0.2−3.31 [−7.00, +0.37] (4/5)+0.44 [−1.27, +2.14]
λTD=0.75\lambda_{\rm TD}=0.75−4.04 [−7.28, −0.79] (5/5)−0.28 [−2.47, +1.90]
λTD=1.5\lambda_{\rm TD}=1.5−4.21 [−5.89, −2.53] (5/5)−0.47 [−1.59, +0.65]
  • Weight Robustness: Performance is insensitive to λTD\lambda_{\rm TD} across a 15× range; no variant differs significantly from the submitted configuration (λTD=0.373\lambda_{\rm TD}=0.373).
  • Term Ablations (Omitted from table): Single-term variants cannot be included above because they fail to reach the target matched-control range (0.85–0.95). Eliminating the TD term caps maximum control at 0.43. Eliminating the boundary loss drops control to chance levels (~0.10, erasing guidance entirely), perfectly aligning with our theory: absent a boundary condition, any constant solves the TD equations.

Q5. Fresh-seed confirmation and scores for guidance scale 1 and 0. We evaluated the selected TD configuration on ten entirely new training seeds under a pre-registered protocol.

The pre-registered primary endpoint (macro conditional KID×10³, TD − Direct, averaged over seven matched-control targets) is −4.61-4.61 (95% CI [−6.17,−3.05][-6.17, -3.05]), consistently favoring TD on 10/10 seeds. Furthermore, TD achieves lower FID at all seven control levels, with every 95% CI strictly excluding zero.

  • Scale 0 (No Guidance): By construction, both methods reduce to the exact same frozen unconditional base generator, yielding mathematically identical outputs (global FID = 46.24±0.3046.24 \pm 0.30, external control = 0.098±0.0020.098 \pm 0.002, matching chance level 0.100.10).
  • Scale 1 (Weak Guidance): We transparently observe that TD's superiority is localized to the strong-guidance regime. At scale 1 (weak guidance, control ≈0.46\approx 0.46), Direct performs marginally better, with TD showing a minor global FID increase of +0.51+0.51 ([+0.27,+0.74][+0.27, +0.74]) and a slight KID penalty over 10 fresh seeds.

Response to Reviewer qZEx (2/2)

Q6. Computational cost. Because TD only requires one additional forward pass while avoiding costly double-backpropagation, it incurs a minimal training overhead of just 1.10× to 1.18× the compute of Direct per update, while maintaining identical sampling costs.

W1 (baselines) / Q7. Training-free and gradient-aware baselines. We evaluated three gradient-aware methods: (a) direct regression + input-gradient (Sobolev) penalty; (b) Doob matching (Chang et al., 2026); (c) two-stage regression (CDG-ML/MCL; Tang & Xu, 2024; Guo et al., 2026).

2D Benchmarks

  • Setup: All baselines received an identical tuning budget (a 5-point grid on 3 disjoint discovery seeds). Direct and TD both include a clean-endpoint anchor (a squared/CE loss fitting the network to QQ at the clean end), whereas Chang et al. and CDG, as originally specified, do not. We therefore also ran each with Direct’s anchor added, and report, per task, the better of the two variants.

Log-guidance MSE (×10⁻², mean ± SE over the 5 table seeds; lower is better):

MethodRingBistableTwo moonsSpiral
Direct4.98 ± 0.244.29 ± 0.5318.41 ± 2.9914.03 ± 0.72
Direct + grad. penalty4.15 ± 0.114.15 ± 0.5419.02 ± 1.3911.63 ± 0.20
Chang et al. (2026)5.38 ± 0.225.15 ± 0.3320.86 ± 4.3712.61 ± 0.20
CDG-ML3.98 ± 0.185.00 ± 0.4222.69 ± 4.3514.71 ± 0.12
CDG-MCL10.71 ± 0.5710.61 ± 0.9930.97 ± 0.5618.64 ± 1.01
TD (submitted config)3.61 ± 0.183.37 ± 0.2713.04 ± 3.0811.47 ± 0.28
  • Results: TD achieves the lowest mean guidance error across all four tasks.

CIFAR-10 Evaluation

  • Setup: Baselines (a) and (b) were trained using identical architectures and optimization budgets (200 epochs). Hyperparameters were selected via a pre-specified rule on 3 seeds. The frozen configurations were then compared on 5 fresh training seeds against Direct across 7 matched-control levels (2,000 samples each). Method (c) was omitted as it requires reverse-SDE simulation and a second fitted network, and proved to be the least stable method in 2D.
Method − Directmacro cKID×10³ [95% CI]Seeds favourableGlobal FID [95% CI]
Direct + grad. penalty (γ=0.1\gamma=0.1)+1.56 [−0.63, +3.75]1/5+0.83 [−0.04, +1.70]
Chang et al. (λ=0.1\lambda=0.1)−1.52 [−2.31, −0.73]5/5−0.61 [−0.95, −0.27]
TD−3.81 [−5.54, −2.09]5/5−1.82 [−2.77, −0.87]
  • Results: Chang et al. improves upon Direct, but TD's improvement is approximately 2.5× larger (and outperforms Chang et al. on 5/5 seeds in a post-hoc comparison). Neither TD nor Chang et al. improves global KID (both CIs include zero).

We evaluated TD against the training-free DPS baseline on the CIFAR-10 CLIP-reward task. To rigorously prevent reward hacking, guidance targets CLIP ViT-B/32, while evaluation relies on a held-out CLIP ViT-L/14 under a pre-registered strict quality-preservation budget (global FID ≤\le unguided + 5).

  • Performance (at pre-registered budget): TD outperforms Direct on 6/6 fresh seed × prompt pairs and matches or strictly exceeds DPS:
PromptTD (mean of 3 fresh seeds)Direct (mean)DPS
"a photo of a red car"0.19180.18790.1903
"a photo of a bird flying..."0.18470.1829infeasible*
*DPS exceeds the strict FID degradation budget even at the lowest guidance scale.
  • Computational Cost: Because DPS requires backpropagating through the denoiser at every sampling step, it incurs 2.3× the per-image sampling compute of TD (or Direct), allowing TD to achieve superior or comparable performance at less than half the sampling cost.

W1 (evaluation beyond CIFAR-10). Text Generation (ELF continuous diffusion language model). We guided ELF toward specific topics (Sports/Health) using a noisy topic classifier trained from KK labeled documents.

  • Label-Efficiency Gain: At high noise (t=0.1t=0.1, where t=0t=0 is pure noise), TD trained with 250 labels matches the performance of Direct trained with 2,000 labels. On fresh seeds with pre-registered criteria, the Sports-vs-rest AUC at t=0.1t=0.1 is: Direct (K=250K=250) 0.795 ± 0.003; TD (Δt=0.05\Delta t=0.05, K=250K=250) 0.839 ± 0.003; Direct (K=2000K=2000) 0.834 ± 0.003. This yields an approximate 8× label-efficiency gain for the noisy classifier, which successfully replicates on the Health topic (TD 0.785 vs Direct 0.735 at K=250K=250).

Thank you for your responses and the additional experiments. However, the theory still does not support the empirical results, so I will maintain my score.

Reviewer_ZqkzOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Temporal-Difference Guidance (TD Guidance), which learns a diffusion guidance value function through TD bootstrapping across adjacent noise levels to address the mismatch between value prediction error and gradient error in conventional regression-based methods. Theoretically, the authors establish finite-sample error bounds that jointly control value and gradient errors and further relate them to the KL divergence of the generated distribution. Experiments on synthetic benchmarks, CIFAR-10, and COMPAS demonstrate the effectiveness of the proposed method.

Critical strengths
  1. The paper addresses the mismatch between value prediction accuracy and gradient accuracy in diffusion guidance. Since the sampling process relies on the spatial gradient of the value function, this is a theoretically and practically relevant problem.

  2. The authors establish theoretical guarantees for jointly controlling value and gradient errors through a TD variational framework and further derive an upper bound on the KL divergence of the generated distribution, providing a comprehensive theoretical foundation.

  3. The paper introduces TD learning into diffusion guidance training without requiring explicit gradient supervision, making the method relatively straightforward to integrate into existing frameworks.

Critical weaknesses
  1. Insufficient comparisons with existing gradient-aware guidance methods. The main experiments primarily compare TD Guidance with direct regression, without direct comparisons against other gradient-aware guidance methods. The current results are insufficient to establish the advantages of TD Guidance over existing approaches in terms of generation performance and computational efficiency.

  2. Limited experimental complexity and scale. The image generation experiments are mainly conducted on CIFAR-10, leaving the effectiveness of TD Guidance on more complex, high-resolution image generation tasks unexplored.

Questions for authors
  1. Could the authors provide additional comparisons with existing gradient-aware guidance methods in terms of generation quality and training costs?

  2. Could the authors include a high-resolution image generation experiment to further evaluate the effectiveness and scalability of TD Guidance in more challenging generation settings?

2 comments

3. Response to Reviewer Zqkz

We thank the reviewer for recognizing the relevance of the value–gradient mismatch and our theoretical foundation.

W1/Q1. Comparison with gradient-aware guidance methods (quality and training cost). We evaluated three gradient-aware methods: (a) direct regression + input-gradient (Sobolev) penalty; (b) Doob matching (Chang et al., 2026); (c) two-stage regression (CDG-ML/MCL; Tang & Xu, 2024; Guo et al., 2026).

2D Benchmarks

  • Setup: All baselines received an identical tuning budget (a 5-point grid on 3 disjoint discovery seeds). Direct and TD both include a clean-endpoint anchor (a squared/CE loss fitting the network to QQ at the clean end), whereas Chang et al. and CDG, as originally specified, do not. We therefore also ran each with Direct’s anchor added, and report, per task, the better of the two variants.

Log-guidance MSE (×10⁻², mean ± SE over the 5 table seeds; lower is better):

MethodRingBistableTwo moonsSpiral
Direct4.98 ± 0.244.29 ± 0.5318.41 ± 2.9914.03 ± 0.72
Direct + grad. penalty4.15 ± 0.114.15 ± 0.5419.02 ± 1.3911.63 ± 0.20
Chang et al. (2026)5.38 ± 0.225.15 ± 0.3320.86 ± 4.3712.61 ± 0.20
CDG-ML3.98 ± 0.185.00 ± 0.4222.69 ± 4.3514.71 ± 0.12
CDG-MCL10.71 ± 0.5710.61 ± 0.9930.97 ± 0.5618.64 ± 1.01
TD (submitted config)3.61 ± 0.183.37 ± 0.2713.04 ± 3.0811.47 ± 0.28
  • Results: TD achieves the lowest mean guidance error across all four tasks.

CIFAR-10 Evaluation

  • Setup: Baselines (a) and (b) were trained using identical architectures and optimization budgets (200 epochs). Hyperparameters were selected via a pre-specified rule on 3 seeds. The frozen configurations were then compared on 5 fresh training seeds against Direct across 7 matched-control levels (2,000 samples each). Method (c) was omitted as it requires reverse-SDE simulation and a second fitted network, and proved to be the least stable method in 2D.
Method − Directmacro cKID×10³ [95% CI]Seeds favourableGlobal FID [95% CI]
Direct + grad. penalty (γ=0.1\gamma=0.1)+1.56 [−0.63, +3.75]1/5+0.83 [−0.04, +1.70]
Chang et al. (λ=0.1\lambda=0.1)−1.52 [−2.31, −0.73]5/5−0.61 [−0.95, −0.27]
TD−3.81 [−5.54, −2.09]5/5−1.82 [−2.77, −0.87]
  • Results: Chang et al. improves upon Direct, but TD's improvement is approximately 2.5× larger (and outperforms Chang et al. on 5/5 seeds in a post-hoc comparison).
  • Computational Cost: Due to the double-backward pass, baselines (a) and (b) cost 2.3–2.5× more per training update than Direct. In contrast, TD requires only one additional forward pass, resulting in an overhead of just 1.10–1.18×.

W2/Q2. Scalability and modality extensions (ImageNet and Text). To address the request for more complex tasks, we have extended our evaluation to both high-resolution images and continuous diffusion language models.

Text Generation (ELF continuous diffusion language model). We guided ELF toward specific topics (Sports/Health) using a noisy topic classifier trained from KK labeled documents.

  • Label-Efficiency Gain: At high noise (t=0.1t=0.1, where t=0t=0 is pure noise), TD trained with 250 labels matches the performance of Direct trained with 2,000 labels. On fresh seeds with pre-registered criteria, the Sports-vs-rest AUC at t=0.1t=0.1 is: Direct (K=250K=250) 0.795 ± 0.003; TD (Δt=0.05\Delta t=0.05, K=250K=250) 0.839 ± 0.003; Direct (K=2000K=2000) 0.834 ± 0.003. This yields an approximate 8× label-efficiency gain for the noisy classifier, which successfully replicates on the Health topic (TD 0.785 vs Direct 0.735 at K=250K=250).

ImageNet 256×256. We fine-tuned the official ADM noisy classifier using direct regression and an added TD term, evaluated at matched external control.

  • Fine-tuning evaluation: Across 10 training seeds, both fine-tuning methods yielded only marginal improvements over the base classifier, and their generation quality was statistically indistinguishable from each other (mean KID×10³ at matched control: 16.0 for Direct vs. 15.9 for TD; difference +0.1±0.32+0.1\pm0.32). We do not claim an ImageNet improvement. Our hypothesis for this result is performance saturation: an already extensively trained classifier leaves little room for the noisy-time objective to meaningfully alter the gradient field.

In summary, TD is now robustly validated across a highly diverse set of settings: CIFAR-10, 2D benchmarks, text classification, and the COMPAS molecular-design task.

Thank you for the detailed response and additional experiments. The new results address several of my concerns and strengthen the empirical evaluation. I will maintain my original rating of 3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.