- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes ABBS, which treats diffusion guidance as a series of optimization problems at different noise levels. It combines approximate proximal steps for the prior and target conditions, mainly to avoid explicit step-size selection. The authors also attempt to derive an error bound and test the method on image guidance and motion editing.
Critical strengths
- The proposed guidance framework can be applied to image restoration, style guidance, and motion editing across different pretrained models (Tab 2-3).
Critical weaknesses
- One of my major concerns is that ABBS is very close to DiffPIR (cited in line 76, only once). Using the same notation and squared measurement loss, ABBS's Eq. 8-12b with the weight from Sec 3.2 gives
DiffPIR's preprint Eq. 12b gives
Setting makes the overlap clear. The remaining difference between these subproblems is ABBS's factor . Without a DiffPIR baseline and separate tests of these changes, benefit of ABBS is very unclear.
- Another major concern is the claim in Sec 4.1, stating not as a step size. It is a tradeoff weight in Eq. 9, but it also controls the update size. For an exact solution of Eq. 12b, substituting from Sec 3.2 gives
Therefore actually acts as implicit step size, even though the main motivation of the paper is avoiding step size selection.
-
In Tab 3 and 6, ABBS-Z starts with a better backbone (FID 0.081 vs 0.301 for MotionLCM). Its better final FID (0.177 vs 0.309) may therefore come from the backbone rather than the proposed method. There needs to be comparisons for the guidance methods using the same backbone and control module.
-
Remark 2 claims that the approximation is accurate (with low noise) but Eq. B.7 does not establish this. Because its denominator can approach zero even when is small. The paper also acknowledges that the required condition cannot be verified (line 308). The authors should therefore justify that, this condition holds within the guidance window, and the resulting bound is small.
Questions for authors
-
[W1] Can the authors clarify what ABBS adds to DiffPIR through a direct comparison and separate tests of the penalty schedule and sampling update?
-
[W2] How does tuning avoid step-size selection when acts as an implicit step size?
-
[W3] Can the authors compare the guidance methods using the same backbone and control module, in order to isolate ABBS's contribution in Tables 3 and 6?
-
[W4] Can the authors justify holds throughout the guidance window and that the error bound in Eq. B.7 is small enough to support Remark 2?
3 comments
Response to Reviewer N8Nz (2/2): W2–W4
W2. Role of
You are right. The exact solution of Eq. 12b is an implicit gradient step with step size , and sets the overall scale of this step size. The statement " is not a step size" in Section 4.1 was inaccurate on this point, so we rewrote it. We also withdrew "remove the step size from the design" in the Abstract and Section 1.
Since is the strength of guidance and sets the trade-off between validity and fidelity (Figure 2), we choose it for each task (Table 4). The revised paper therefore makes a narrower claim (Section 3.2). ABBS does not remove the need to choose the strength of guidance, but it removes two design choices of explicit gradient-step guidance: (i) an upper bound on the step size from the smoothness of the loss, and (ii) a hand-designed schedule of the step size over .
For (i), since , every with , in particular the exact solution of Eq. 12b, satisfies for every . In contrast, for an explicit gradient step with the same step size, the standard guarantee that decreases requires , and is unknown for losses that involve a pretrained network (Section 1).
(ii) comes from Proposition 1, not from the implicit step. If we set the weight of Problem 9 to for all , the dependence of the step size on is determined by (). Thus, the schedule over is induced by the sampler through , rather than hand-designed. The end of the guidance window remains a task-level hyperparameter, and we now report its value in Appendix D.
Consistently, V1 in Table 8 (W1), which replaces by , lies far below the trade-off curve of ABBS.
The argument above concerns the exact solution of Eq. 12b, as does your derivation. In the implementation, we compute the prox inexactly by Adam iterations, and the results change little when we increase from 30 to 80 (Appendix D.3).
W3. Comparison with the same backbone and control module
We agree that Table 3 is an end-to-end comparison of systems with different backbones and representations, not a controlled comparison of guidance algorithms, and the revised Section 4.2 states this. Table 3 uses bold only for the editing metrics, and we do not claim that ABBS- has a better FID.
The controlled evidence for ABBS in motion is Table 7, which compares the results with and without ABBS under the same backbone and the same control module. On both backbones, adding ABBS greatly reduces the average error (0.268 to 0.004 m and 0.253 to 0.019 m), and on the latent-space backbone it slightly increases the FID (0.146 to 0.177). There, the FID without guidance is already smaller than that of MotionLCM (0.309), so, as you point out, the difference in FID comes from the backbone.
A controlled comparison with the motion-specific guidance of the baselines would require reimplementing it on the same backbone with the same control module, and we did not include this experiment. We therefore base the contribution of ABBS itself on Table 7 and read Table 3 as a comparison of complete systems.
W4. Validity of
As you point out, the statement in Remark 2 that the bound is tight at low noise levels does not follow from Eq. (B.7), so we removed it.
By the second-order Tweedie formula, , where is the Jacobian of . If has a direction with an eigenvalue close to 0, then is approximately or larger. Since for every sampler, (A2) need not hold near . The first term of the bound is times an upper bound on the distance that moves (added to Remark 2), so the bound is small compared with this distance only when . Thus, we do not claim that (A2) can be verified or that the bound is uniformly small over all guided steps, and the revised Remark 2 states this.
Algorithm 1 does not require checking (A2). The assumptions (A1) and (A2) are used only to read one reverse step as an inexact proximal step and to bound its residual. The dependence of on is determined by the noise schedule and the sampler, whether or not (A2) holds.
Changes to the paper
- We added a new Remark 3, Appendix D.3, and Table 8 (W1).
- We withdrew " is not a step size" and "remove the step size from the design," and we rewrote the Abstract and Sections 1, 3.2, and 4.1 (W2).
- We stated in Section 4.2 that Table 3 compares complete systems (W3).
- We rewrote Remark 2 (W4).
Thank you again for the careful review. We hope that the revision addresses your concerns.
Follow-up on our responses
Dear Reviewer N8Nz,
Thank you again for your thoughtful and constructive feedback.
As the discussion period approaches its end, we would appreciate any feedback on whether our responses and the revised manuscript have adequately addressed your concerns. We apologize for following up only two days after posting our responses, and we appreciate your time.
Best regards,
The Authors
## Response to Reviewer N8Nz (1/2): W1
Thank you for the detailed and technical review. Following your comments, we revised the paper on the relation to DiffPIR (new Remark 3), the statements on the step size (Sections 3.2 and 4.1), and Remark 2.
W1. Relation to DiffPIR
Thank you for making the comparison explicit. As you point out, the two subproblems differ only in the weight of the quadratic term, and the weight in ABBS is 1/ct times that in DiffPIR. When the re-noising is included, the two updates agree to first order ((a) below).
DiffPIR is designed and evaluated mainly for image restoration, where the data subproblem is often solved in closed form. In contrast, ABBS targets training-free guidance, where the loss involves a pretrained network or a decoder and the subproblem has no closed form. In this setting, ABBS provides the following two points. First, we derived the scale νt at which one reverse step is read as a proximal step on the prior, for the samplers of the form of Eq. 5, extending the first-order relation in App. A.1 of DiffPIR, and we gave a bound on the residual (Proposition 1; (b) below). Second, ABBS solves the subproblem iteratively, instead of approximating it by the single gradient step of Eq. 13 of DiffPIR ((c) below). We examined both in Appendix D.3 ((d) below).
(a) DiffPIR also works at the scale of Proposition 1, to first order. The re-noising of DiffPIR (Eq. 15 of DiffPIR) is the case ωt2=(1−αˉt−1)ζ of Eq. 5, for which ct=1−(σˉt−1/σˉt)1−ζ. By the identity in Proposition 1 and Eq. 8, the two updates can be written exactly as follows. ABBS: xˉt−1=proxft/ρt(vˉt−1),DiffPIR: xˉt−1=vˉt−1+ct(proxft/(ctρt)(vˉt−1)−vˉt−1). The factor 1/ct in the weight of ABBS appears in DiffPIR as the coefficient ct in the update. To first order, both updates are vˉt−1−(νt/λ)∇ft(vˉt−1), and this holds for every ζ. The two updates differ only in directions where the curvature of fS is comparable to or larger than λ/σˉt2.
(b) Difference in viewpoint. DiffPIR plugs a denoiser into HQS and returns to the sampler by re-noising. ABBS builds the method on reading one reverse step itself as the proximity operator of the smoothed prior. For DDPM, App. A.1 of DiffPIR shows this reading as a first-order approximation with the same scale. Proposition 1 extends it, with a bound on the residual, to all samplers of Eq. 5.
(c) What ABBS adds. In training-free guidance, the main case is that the loss involves a pretrained network or a decoder, and the data subproblem has no closed form. In this case, DiffPIR approximates the subproblem by one gradient step (Eq. 13 of DiffPIR). Under the same sampler, this corresponds to ABBS whose inner solver is one gradient descent step with step size 1/ρt, whereas ABBS solves the subproblem with K iterations. When the subproblem is solved in closed form, the only difference is where ct enters, so we do not claim that ABBS improves on DiffPIR in that case.
(d) Experiments (new Appendix D.3; excerpt of Table 8 below). On Task (iv), we compared the components in your question separately (all with λ=5×10−2).
Table 8 (excerpt). "Corr." is the factor on ut−x^0∣t in the update.
Note. V3 is not a tuned gradient-step baseline: it keeps the scale of ABBS to isolate the effect of the single gradient step of Eq. 13 of DiffPIR. MPGD, which has the same form with its own step sizes (Ye et al., 2024, Thm. 3.2), is compared in Table 2, and ABBS improves on it in both scores.