acceptodds
Under review as a conference paper at ICLR 2027

Integration Matters: Rollout-Based Training for Constrained Diffusion Models

Abstract

Constrained generative models aim to produce samples that satisfy complex feasibility constraints while remaining faithful to the underlying data distribution. Existing methods typically enforce constraints through either training-time optimization or sampling-time correction. Training-time optimization approaches optimize over states induced by the training distribution, which can differ substantially from those encountered during sampling. Sampling-time correction methods instead modify the sampling process at inference, introducing distribution shift and requiring costly tuning, particularly in few-step sampling regimes. We propose IMCD (Integration Matters for Constrained Diffusion), a fine-tuning framework that integrates constraint guidance into training through online rollouts. IMCD aligns training with sampling by differentiating through the fixed noise schedule used to numerically integrate the denoising process. This exposes the model to constraint violations that arise along the denoising trajectory and aligns diffusion learning with the sampling process. Experiments across multiple tasks show that our method improves constraint satisfaction while maintaining competitive sampling quality compared to prior methods.

Primary area:
generative models
Keywords:
Diffusion models, Constrained generation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_hpZdOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes IMCD, a rollout-based fine-tuning framework for constrained diffusion models. The authors identify a mismatch between forward-noised states used during training and the states encountered during reverse-time sampling, which may limit the effectiveness of existing constraint optimization methods. To address this issue, IMCD performs full denoising rollouts during training and backpropagates the terminal constraint loss through the entire sampling trajectory. The framework further incorporates learnable constraint guidance and an EDM denoising objective to improve constraint satisfaction while preserving generation quality. Experiments on bouncing-ball simulation and traffic trajectory prediction demonstrate promising results in reducing constraint violations.

Critical strengths

The paper identifies a meaningful mismatch between training and sampling distributions in constrained diffusion models.

Critical weaknesses
  1. The current evaluation is largely restricted to structured trajectory generation, leaving the generalizability of the proposed framework insufficiently explored. Although the paper evaluates both bouncing-ball simulation and real-world traffic data, both tasks involve relatively low-dimensional geometric trajectories with explicitly defined and differentiable constraints, such as distances, overlaps, and boundary violations. In contrast, constraints in more complex generative domains, such as realistic image generation, may involve semantic consistency, object relationships, or structural requirements that are difficult to express through differentiable loss functions. While the rollout-based training formulation is potentially general, the current experiments provide limited evidence of its applicability beyond geometric trajectory generation.

  2. Backpropagation through the full denoising rollout introduces substantial computational overhead, while the efficiency and necessity of this training strategy remain insufficiently investigated. The constraint objective is evaluated only on the terminal generated sample, requiring a complete denoising rollout and backpropagation through all sampling steps for each optimization update. Each step also involves additional pretrained-denoiser evaluations and constraint-gradient computations. Although gradient checkpointing helps reduce memory consumption, the overall training cost may still be substantial, particularly for higher-dimensional generation or longer sampling trajectories.

Questions for authors

See weaknesses.

1 comment

We thank the reviewer for recognizing the training-sampling mismatch we identify in constrained diffusion models and for the thoughtful feedback.

1. Generalization concern.

We emphasize that our work specifically targets hard-constrained generation, where the goal is to generate samples within a feasible set, rather than maximize a reward or preference score. In this setting, constraint violations constitute hard failures rather than gradual degradations in sample quality. Collision avoidance and drivable-area compliance in autonomous driving are representative examples. This problem formulation is also relevant to other domains, including human motion synthesis and robotic trajectory planning, where satisfying physical feasibility and collision-avoidance constraints is essential. Recent works have similarly explored constrained diffusion for these applications [1, 2].

Importantly, the geometric nature of our constraints does not make the evaluated problems low-dimensional or straightforward. Each bouncing-ball sample has 4000 dimensions, comparable to a 64×6464\times 64 image. More importantly, our setting imposes strict feasibility requirements involving pairwise interactions across agents and timesteps. Unlike reward-based alignment in image generation, where incremental improvements in a reward score constitute progress, our objective is to drive constraint violations toward zero: even a single collision or off-road event can invalidate an otherwise plausible sample. This makes satisfying the constraints while preserving the underlying generative distribution a challenging problem, as also evidenced by the limitations of existing sampling-time baselines, including MPGD, FK steering, and D-Flow.

2. Backpropagation concern.

We refer the reviewer to our general response for a detailed discussion of training and inference costs. In particular, IMCD's peak training memory is no greater than that of DPOK and AM, despite backpropagation through the full denoising rollout. At inference, IMCD uses standard reverse denoising without additional optimization. Empirically, it achieves better constraint satisfaction while preserving distributional fidelity, demonstrating the effectiveness of this training strategy.

[1] Christopher JK, Baek S, Fioretto F. Constrained synthesis with projected diffusion models. Advances in Neural Information Processing Systems. 2024 Dec 16;37:89307-33.

[2] Liang J, Christopher JK, Koenig S, Fioretto F. Simultaneous multi-robot motion planning with projected diffusion models. arXiv preprint arXiv:2502.03607. 2025 Feb 5.

Reviewer_1pGSOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes IMCD (Integration Matters for Constrained Diffusion). The paper start from a gap between the theoretical sampling dynamics and the actual discretized SDE in earlier work. With numerical integration errors, even if the training set consists entirely of feasible samples, samples generated by the discretized SDE can still violate the constraints. The paper's answer is rollout-based training: the model is trained on on-policy trajectories rolled out on the discretized SDE. An EDM denoising loss is added to avoid deviating from the origin data. The paper evaluates on Bouncing Balls and Traffic Scene Trajectory Prediction. IMCD reduces constraint violations, while the generated trajectories remain consistent with the original data

Critical strengths
  1. In the experiments, IMCD clearly beats the other baselines.
  2. The method is explained clearly. The rollout constraint loss idea is very insightful.
Critical weaknesses
  1. In the proof of Proposition 4.2 (Appendix B.1), the authors say to "treat γϕ(xt,t)\gamma_\phi(x_t,t) as constant", but that contradicts the state-dependent parameterization proposed in the paper. Also, even if we accept that γ\gamma does not depend on xtx_t: take α(t)=t4>0\alpha(t)=t^4>0 and β(t)=−2\beta(t)=-2, then γ(t)=t2→0\gamma(t)=t^2\to 0, which disagrees with the claim in line 746 that γ→∞\gamma\to\infty as t→0t\to 0.

  2. The paper proposes a learnable γϕ\gamma_\phi, but I didn't find an ablation on whether learning γϕ\gamma_\phi is actually more effective than using a fixed one.

If the author can address the weaknesses I raised, I would be willing to raise my score.

Questions for authors
  1. BPTT over the whole rollout is expensive. I wonder whether the authors have tried training without full BPTT. Methods based on reward gradients, such as Adjoint Sampling [1], or methods based on reward-weighted or reward-guided regression losses, such as DiOpt [2], DiffusionNFT [3], and Reinforce Adjoint Matching (RAM) [4], seem to me to offer possible ways to avoid this cost. Of course, this is just out of personal curiosity.

[1] Havens, Aaron, et al. "Adjoint Sampling: Highly Scalable Diffusion Samplers via Adjoint Matching." arXiv preprint arXiv:2504.11713 (2025).

[2] Ding, Shutong, et al. "Diffusion-based learning framework for Constrained Nonconvex Optimization with Weighted Bootstrapped Refinement." arXiv preprint arXiv:2502.10330 (2025).

[3] Zheng, Kaiwen, et al. "DiffusionNFT: Online Diffusion Reinforcement with Forward Process." arXiv preprint arXiv:2509.16117 (2025).

[4] Bergmeister, Andreas, et al. "Reinforce Adjoint Matching: Scaling RL Post-Training of Diffusion and Flow-Matching Models." arXiv preprint arXiv:2605.10759 (2026).

5 comments

We thank the reviewer for recognizing its strong empirical performance over existing baselines. We are also glad that the reviewer finds our method clearly presented and the rollout constraint loss particularly insightful.

1. Treat γϕ\gamma _\phi as constant.

We thank the reviewer for pointing this out. The phrase was inaccurate since γϕ(xt;t)\gamma_{\phi}(\mathbf{x}_t; t) can be treated as a constant only if it is independent of xt\mathbf{x}_t, which is not the case in our parameterization. Differentiating the log of Eq.(7) with respect to xt\mathbf{x}_t yields the constraint score of Eq.(6) plus an additional term −Eγ-E _{\gamma}, where Eγ=ℓΩ(D(xt;t);t)∇γϕ(xt;t)E _{\gamma} = \ell^{\Omega}(D(\mathbf{x}_t; t); t) \nabla \gamma _{\phi}(\mathbf{x}_t; t). This is what our original proof omitted. This term vanishes exactly on states whose denoised estimate is feasible, since ℓΩ(D(xt;t);t)=0\ell^{\Omega} (D(\mathbf{x}_t; t); t) = 0 there. To assess its effect elsewhere, we measured it along sampling trajectories empirically: as t→0t\to 0, the mean norm of EγE _{\gamma} decreases faster than that of the guidance term γϕ(xt;t)⋅∂1ℓΩ(D(xt;t);t)\gamma _{\phi}(\mathbf{x}_t; t)\cdot \partial_1 \ell^{\Omega}(D(\mathbf{x}_t; t); t) and vanishes to 00. We will revise Proposition 4.2 to state this vanishing condition explicitly: under it, Eq. (6) is the score of Eq. (7) up to EγE _{\gamma}, which vanishes as t→0t\to 0, so the convergence result of Eq. (8) is unaffected.

Divergence of γϕ\gamma _{\phi} . The reviewer is also correct that our original condition, αϕ>0\alpha _{\phi} > 0, is too loose. The intended role of αϕ\alpha _{\phi} is to provide a state- and noise-dependent scaling, so that γϕ\gamma _{\phi} is of order tβt^{\beta}. This requires αϕ\alpha _{\phi} to be bounded away from zero. Empirically, the learned αϕ\alpha _{\phi} remains bounded away from zero throughout sampling in figure 3 and 4. We will revise the parameterization of αϕ\alpha _{\phi} by introducing a positive lower bound αmin⁡\alpha _{\min}, ensuring that γϕ\gamma _{\phi} retains the desired order behaviour in tt.

2. Fixed schedule

We thank the reviewer for this ablation suggestion. As shown in Table 3, existing constraint-guidance schedules typically use a fixed coefficient and exponent. In section 4.1, we mentioned that such schedules require manual tuning, as the appropriate scaling depends on the sampling regime, noise level, and constraint strength. In contrast, our scaling network learns both the coefficient and exponent as functions of the xt\mathbf{x}_t and noise level tt, enabling state- and noise-adaptive guidance without a separate search. To investigate the benefit of this adaptivity, we conduct an additional ablation by varying the fixed exponent, which has the dominant influence of the scaling. The table below generally exhibit worse performance than IMCD, particularly in distributional quality, suggesting that adaptively learning the scaling is important for balancing constraint satisfaction and distribution preservation.

Methodβ\betaBoundary rate (%)Overlap rate (%)r-ELBO (×10−2\times 10^{-2})F2F (×10−1\times 10^{-1})MCD (×10−1\times 10^{-1})MED (×10−1\times 10^{-1})
ablation−2.53.710.00−922.22832.51.81263662.1
ablation−1.80.020.03−24.36.013.40.5
ablation0.00.030.17−1268.627.116.25.3
IMCD (Ours)–0.010.01−22.93.41.20.6

Training without full BPTT. We thank the reviewer for these pointers and agree that avoiding backpropagation through the full rollout is an appealing direction. We have already discussed Adjoint Sampling in our related work and will incorporate the additional references to DiOpt, DiffusionNFT, and RAM.

Adjoint Sampling targets an energy without a pretrained data model (as discussed in our related work), while DiOpt addresses learning-to-optimize, seeking feasible near-optimal solutions to constrained programs rather than constrained generation from a data distribution. Thus, its optimization objective differs from ours.

Regression-based methods such as DiffusionNFT and RAM, apply to pretrained models and train on forward-noised terminal samples using scalar rewards. In particular, RAM establishes that, at the optimum of a KL-regularized objective with a terminal reward, the optimal process tilts only the terminal distribution while preserving the forward noising process. Our setting differs in that IMCD applies constraint corrections at every denoising step (Eq. 6), so its intermediate states need not follow the forward noising distribution conditioned on the terminal sample. We therefore optimize the constraint loss through the actual sampling rollout. This allows gradients from the terminal violation to propagate through the entire trajectory, accounting for the cumulative effects and interactions of corrections across denoising steps.

Moreover, as reported in our general response, despite backpropagating through the full denoising rollout, IMCD's peak training memory is no greater than that of other baselines that avoid backpropagation through the rollout. Overall, this makes full rollout optimization practical in our setting.

I thank the authors for their patient responses to my questions. I believe these responses have largely resolved my doubts and concerns, and they have convinced me that the proposed method is indeed supported by both theoretical foundations and empirical evidence. Based on these considerations, I am inclined to raise my score, provided that the authors first revise the paper accordingly so that I can see these changes directly in the paper.

Reviewer_RCpDOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a training method for constrained diffusion models. Some previous approaches have focused on regularizers in addition to the standard diffusion models loss (causing denoiser outputs to match with the constraint), and approximate inference-time corrections to take the constraints into account. It is assumed that the constraint is specified by a constraing violation function which minimum values correspond to the data matching with the constraint. The paper argues that training-based methods suffer from a train-test mismatch: Due to differences in the data distribution at inference and at training, the denoiser outputs violate the constraints significantly more than during training. The paper also argues something similar for inference-time correction methods: the approximate corrections may push the model outside the training distribution during sampling, making the learned denoiser unrealiable. The proposed solution is two-fold: 1) add an adapter to a pretrained denoiser which incorporates an inference-time guidance term, scaled by a learned scaling network, and also input information about the guidance term to the core denoiser network itself 2) rollout-based training, evaluating the constraint loss at the end of a sampling and backpropagating through the entire inference process. This results in improved constraint violation metrics and physical consistency metrics in a bouncing-ball simulation task and good balance between constraint satisfaction and generative quality traffic scene trajectory prediction.

Critical strengths
  • The method addresses real issues with some popular approaches to constrained diffusion, and is well-motivated.
  • The paper is well-written, and easy to follow.
  • The method provides clear improvements over the baselines especially in the bouncing ball task, and considers a reasonably wide range of baselines.
Critical weaknesses
  • It seems that core methodological improvements in the paper are close to ideas in existing literature on constrained diffusion models: 1) the rollout-based loss is similar to the "Sample estimation" variant of PIDM [1] (where the constraint is evaluated at samples and the network is updated with backprop through the DDIM sampling loop), and 2) the idea of a scaling network that adjusts a constraint-gradient term was proposed in [2], although [2] does not include the constraint gradient as input to the network. That said, this is done in [3], although in an inverse-problem application context. These works should be cited and the novelty claims should be positioned accordingly (PIDM is cited but I do not think their rollout-based training is noted in the paper).
  • The regular backpropagation through the sampling chain is expensive and might not scale well with the size of the network / dimensionality of the problem / amount of inference steps used, especially w.r.t. memory. The paper would benefit from a clearer analysis on what kinds of problems is the proposed method an improvement over other methods, either via experiments or theoretical arguments. I would, e.g., assume that adjoint matching or RL methods could be more scalable rollout-based training rollout-methods due to smaller memory requirements.
  • (smaller point) I am not entirely convinced that the training-test mismatch in methods like PIDM is simply due to approximation or discretization errors: One issue is that the regulariser causes a consistent bias in the score function / denoiser, relative to the optimal denoiser under the regular diffusion loss, and the standard theoretical reasons for expecting the diffusion training and inference distributions to match disappear. This is discussed at least in [2].
  • It seems that matching inference-time compute for the different methods (or at least increasing the compute for the baselines until they match with IMCD) would make for a fairer comparison between different methods.

[1] Bastek, Jan-Hendrik, WaiChing Sun, and Dennis Kochmann. "Physics-informed diffusion models." International Conference on Learning Representations. Vol. 2025. 2025.

[2] Yeom-Song, Victor M., et al. "Softly Constrained Denoisers for Diffusion Models Applied to Partial Differential Equations." arXiv preprint arXiv:2512.14980 (2025).

[3] Denker, Alexander, et al. "DEFT: Efficient fine-tuning of diffusion models by learning the generalised $ h $-transform." Advances in Neural Information Processing Systems 37 (2024): 19636-19682.

Questions for authors
  • What is the r-ELBO metric? How is it reweighted?
  • What is the data dimensionality in the two benchmarks? What about the size of the networks?
  • Could you provide a table for the training time for the different benchmark methods? This is not critical, but would be useful to have in the paper.
  • I don't think this comment in the related works section is correct: "However, these methods are designed to optimizer reward or energy objectives directly, rather than leverage a pretrained model as the base distribution" (seemingly referring to adjoint matching and DPOK) -> adjoint matching and DPOK explicitly leverage a pretrained model. This relates to the question on what is the theoretical improvement over traditional RL methods or adjoint matching.
5 comments

We thank the reviewer for taking the time to review our paper and for the constructive assessment. Here are our responses to the weakness and questions:

1. Relation to PIDM, SCD and DEFT.

We thank the reviewer and will cite and discuss all three works. While each shares a component with IMCD, our approach differs in how constraint corrections are constructed and optimized. Motivated by the training–sampling mismatch identified in section 3.1, IMCD learns corrections on the model's own reverse-time rollouts, rather than relying on intermediate denoiser estimates from forward-noised states, and directly optimizes the feasibility of generated samples. The scaling network is designed to diverge as t→0t\to 0, strengthening constraint enforcement near the data manifold. Meanwhile, the EDM denoising loss anchors fine-tuning to the data distribution, helping preserve the pretrained model's distributional fidelity while improving constraint satisfaction. This achieves near-zero constraint violations while preserving physical plausibility close to the pretrained model (Tables 1-2), with Table 4 isolating the contribution of rollout-based training.

PIDM. PIDM-SE obtains its estimates from forward-noised states via a two-step DDIM jump (Appendix A.3–A.4), and the authors report no notable improvement from adding intermediate timesteps. This relates to the mismatch identified: forward-noised states remain close to feasible data samples at low noise, providing little constraint signal precisely where reverse-time sampling can still produce violations. This motivates evaluating constraints along the entire reverse-time trajectory, rather than relying on estimates obtained from forward-noised states. IMCD does so by learning corrections on states generated by the model's own sampler and optimizing the feasibility of its terminal samples.

SCD. SCD defines its corrected denoiser in Eq. (9) by adding a constraint-gradient correction evaluated at pre-correction output DθorigD_{\theta}^{\mathrm{orig}}. However, its denoising objective is applied to the corrected output DθSCDD_{\theta}^{\mathrm{SCD}}, which at convergence approximates E[x0∣xt]\mathbb{E}[\mathbf{x}_0\mid \mathbf{x}_t]. Thus, the constraint gradient is evaluated at an estimate shifted from the posterior mean by the correction itself, potentially biasing the guidance direction. IMCD avoids this issue by evaluating the constraint gradient at the frozen pretrained denoiser estimate, which approximates posterior mean.

SCD and DEFT. Both learn their corrections through denoising-based objectives on forward-noised data, whereas IMCD trains its state- and time-dependent scaling using the rollout constraint loss. Moreover, unlike these methods, IMCD constrains the scaling to diverge as t→0t\to0, strengthening constraint enforcement near the data manifold.

2. Training, inference compared to AM and RL methods.

Regarding training and inference costs, we refer the reviewer to our general response. In summary, IMCD's peak training memory is no greater than that of DPOK and AM, despite backpropagation through full denoising rollout. At inference, it uses standard reverse denoising without additional optimization. Empirically, IMCD achieves better constraint satisfaction while preserving distributional fidelity.

Theoretically, IMCD differs primarily in its regularization objective, rather than simply its gradient estimator. While AM and DPOK anchor the fine-tuned model to the pretrained model, IMCD anchors it to the data distribution through the EDM denoising loss. This distinction matters when the pretrained model itself generates infeasible samples: a KL anchor penalizes corrections to these violations, whereas the EDM loss encourages data-consistent denoising. Proposition C.1 formalizes this trade-off. IMCD is therefore particularly suitable when the pretrained model exhibits constraint violations, and many samples must be generated under fixed constraints.

3. Mismatch and Bias concerns

We thank the reviewer for raising these two related concerns. First, as discussed in section 3.1, forward-noised feasible data remains near-feasible at low noise, so ℓΩ(D(xt;t))≈0\ell^\Omega(D(\mathbf{x}_t; t))\approx 0 and provides little constraint supervision. In contrast, reverse sampling can still reach infeasible low-noise states (Fig. 1). This mismatch exists even with pretrained model, without a regularizer.

Second, methods such as PIDM evaluate constraints on one-step denoiser estimates rather than actual generated samples. At high noise, the posterior mean can itself be infeasible, so penalizing its constraint violation shifts the optimal denoiser away from the posterior mean, which is discussed in SCD. IMCD avoids these issues by evaluating constraints on terminal samples from rollouts of the deployed sampler, directly optimizing feasibility under the model-induced sampling distribution rather than relying on intermediate denoiser estimates.

4. Matching inference cost ablation

We conducted additional experiments evaluating EDM, MBM, PIDM (FT), DPOK, and AM under a matched 100-NFE budget on bouncing ball task, corresponding to IMCD's inference-time compute. Increasing the inference budget does not substantially change the performance of these baselines or close the gap with IMCD, which show that IMCD's advantage persists under matched inference-time compute, providing a more favorable balance between constraint satisfaction, distributional fidelity, and trajectory quality.

MethodStepsNFEBoundary (%)Overlap (%)r-ELBO (×10−2)(\times 10^{-2})F2F (×10−1\times 10^{-1})MCD (×10−1)\times 10^{-1})MED (×10−1)\times 10^{-1})
EDM1001006.1838.75−21.73.10.80.4
MBM1001000.960.55−22.222.00.877.6
PIDM (FT)1001006.6528.91−21.63.00.70.3
DPOK1001002.581.50−53.52.94.30.3
AM1001000.584.60−39.43.33.90.5
MPGD1001000.000.00−27.03.80.63.2
MBM++501000.000.00−22.810.30.841.1
FK-steering (16 iters)508006.1316.72−21.73.10.80.4
D-Flow (30 iters)5015005.2035.92−21.73.11.10.4
IMCD (Ours)501000.010.01−22.93.41.20.6

5. Questions

Q1: r-ELBO definition. We apologize for the omission. r-ELBO is the negative EDM denoising loss (Eq. 11) evaluated on held-out test data, We use the same noise distribution and weighting for all methods, with each method's learned denoiser. Higher values indicate better distributional fidelity. We will add this definition to Appendix E.2.

Q2: Dimensionality and model size. In the bouncing-ball benchmark, each sample contains 10 balls over 100 timesteps, with four features (position xx, position yy, velocity vxv_x and velocity vyv_y) per ball, resulting in 4000 dimensions. In the traffic benchmark, the model jointly predicts 30 future timesteps with three features (position xx, position yy, and heading) for each of the AA vehicles in a scene conditioned on 10 observed timesteps, and vehicle count AA varies across scenes. This results in 90A dimensions. The corresponding networks contain 20.3M total weights for bouncing ball task and 20.6M total weights for traffic scene task, respectively, while IMCD updates only 0.60M weights in both tasks (approximately 3%) through the LoRA adapters, gradient embedding, and scaling network. Consequently, gradients and optimizer states are needed for only a small fraction of the model weights. We will add these details to Appendices E.6 and F.4.

Q4: Related work. We thank the reviewer for pointing this out. We agree that the distinction in our original related work discussion was unclear, and we will revise it to distinguish these methods from AS and ASBS.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General response

We sincerely thank the reviewers for their positive and encouraging feedback. We grateful that the reviewers recognized the importance of training-sampling distribution mismatch in constrained diffusion, the motivation and clarity of our proposed IMCD framework, and the insight behind incorporating rollout constraint losses into fine-tuning. We are also encouraged by the recognition of IMCD's consistent improvements over existing baselines, and the breadth of our experimental comparisons. We greatly appreciate the reviewers' thoughtful evaluation and construction feedback. Reviewers raised questions about the computational cost of our method, both during training and at inference. We address both here.

Training cost

  1. Training budget. We agree that rollout backpropagation increases IMCD's training cost compared with forward-process methods. Nevertheless, we fine-tune all models under the same compute budget. For the bouncing-ball benchmark, Table 7 reports the number of optimization iterations completed within this budget on a single NVIDIA L40S for 72 hours. Appendix E.6 provides the corresponding training details. For traffic scene benchmark, Appendix F.4 provides the training details and fine-tuning uses two L40S GPUs. Under this matched budget, IMCD completes fewer optimization iterations than some fine-tuning methods because each iteration backpropagates through the rollout. This additional computation is incurred only during fine-tuning and is amortized over subsequent deployment.

  2. Cost of back propagation through the rollout. Backpropagating through the full rollout provides IMCD with a gradient of the terminal constraint objective with respect to the sampler's trainable components. We compare the per-iteration memory and computation of IMCD, Adjoint Matching (AM) and RL baseline DPOK, all of which simulate a full sampling trajectory during training while AM and DPOK avoid backpropagation. Let TT denote the rollout steps during fine-tuning, FF the memory required to store one intermediate state xt\mathbf{x}_t, AA the activation memory of one network forward pass, and CC the FLOPs of one forward pass.

Memory:

  • IMCD: With per-step gradient checkpointing, IMCD stores the rollout states TFTF and recomputes one step's activations AA at a time during the backward pass, avoiding the (TA) activation memory required by naive backpropagation. This gives a peak rollout-dependent memory of TF+ATF+A.
  • AM: additionally maintains the lean adjoint states along the trajectory, giving approximately 2TF+A2TF+A for the stored state and adjoint quantities.
  • DPOK: stores the sampled trajectory and computes policy gradients without differentiating through the sampling process, which also requires TF+ATF+A.

Computation: All three methods incur the cost of simulating a full rollout TCTC. Their additional per-step computations differ:

  • IMCD (finetuned with LoRA): a forward evaluation of the frozen pretrained denoiser for constraint guidance (C), activation recomputation during the backward pass (C), and backpropagation through the trainable correction (C). This results in 4TC4TC in overall.
  • AM (finetuned with EMA): to solve the lean adjoint ODE, it requires a forward of the base model (C) and an input-VJP through it (C); for the matching loss, forwards of the fine-tuned model (C) and the base model (C), and a full fine-tuning backward pass (2C). This results in 7TC7TC in overall.
  • DPOK (finetuned with LoRA): a forward evaluation of the current policy (C), a reference-model evaluation for the KL term (C), and a policy-gradient update that does not backpropagate through the rollout (C). This results in 4TC4TC in overall.

Therefore, backpropagation through the rollout does not necessarily cost more than avoiding it: IMCD matches DPOK and requires less memory and compute than AM.

Inference cost

Backpropagation through the rollout occurs only during training: at inference, IMCD samples with a standard reverse process. Table 8 reports NFE and wall-clock time for all methods. IMCD uses 2 NFE per step, one pretrained evaluation for the guidance and one for the constraint-aware prediction, giving 100 NFE and 1281 ms per sample at 50 steps. Several comparisons are already at matched or greater compute. MBM++ uses the same 100 NFE (1191 ms) but with inferior performance. In table 6, MPGD at the matched 100 steps still shows substantially higher energy deviation and reaches comparable plausibility only at 150 steps, with 1.7× our wall-clock time (2238 ms). FK-Steering (800 NFE) and D-Flow (1500 NFE) use considerably more compute while retaining violations. Thus, IMCD incurs higher per-step cost than the pretrained model, but achieves substantially lower inference cost than forward-process fine-tuning and sampling-time guidance methods at comparable sample quality.

We will also revise the paper and include the additional results done during rebuttal period in the updated version.

We have revised the paper based on the reviewers' feedback and revised parts are highlighted in blue.

Methods: We revised the scaling function discussion in section 4.1, and proposition 4.2 with the vanishing condition.

Experiments: We added the data dimensionality for both experiments and linked the main manuscript to the ablations and analysis in the appendix.

Related work: We added the closely related methods PIDM (both variants), SCD, DEFT, DiffusionNFT, RAM, DiOpt. We also clarified our discussion of AM, DPOK, Adjoint Sampling and ASBS.

Appendix:

  • We added model parameter counts (E.6 F.4) and the r-ELBO definition (E.2).
  • We added two new studies: a comparison under a matched inference budget (E.8) and learned versus fixed guidance scaling (E.9).
  • We updated the proof of proposition 4.2 and added an empirical verification of the vanishing condition (Appendix G).
  • We added an analysis of training-time memory and compute for backpropagation through the rollout (Appendix H).

We sincerely thank Reviewer (1pGS) for their follow-up and for engaging with our rebuttal. As the discussion period is approaching end, we would appreciate hearing from the remaining reviewers (RCpD, hpZd). We believe we have addressed the concerns raised in the reviews through the additional experiments and analysis and hope our clarifications resolve remaining questions. We would be grateful if the reviewers could reconsider their scores accordingly. Thank you for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.