- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
Summary
- This paper propose a looser constraint of the MCG manifold () [Improving Diffusion Models for Inverse Problems using Manifold Constraints] and constraint the updated latent to a safe region that is a tube around MCG manifold. This relaxation removes the linear operator limits of MCG, makes the algorithm appliable to a broader family of forward operators.
- The authors provide thorough empirical study to validate the performance of proposed approch.
Critical strengths
Strength
- The overall idea of relaxing MCG constraint to non-linear operators looks novel and interesting. The proposed approach is supported by theoretical analysis.
- The empirical validation is thorough and solid. Both ImageNet and FFHQ dataset are included, both pixel space and pixel space are included. PSNR, LPIPS, SSIM and rFID are all included and 7 operators are included.
- The Armijo frequency provides a better complexity-performance trade-off than vanilla increasing the parameter of DPS.
Critical weaknesses
Weakness
- The proposed approach introduces quite some new parameters, which non trivially increase the complexity of the proposed approach. I am not sure whether all parameters are covered in Table 4 and Table 5. For example, seems to be determined by Eq.11, but how is in Eq.11 determined? Sec 3.2 introduces those parameters step by step, maybe adding a overall description and collection of all the parameters can help the readers understand the methods better.
- The proposed approach can be hard to implement. According to Algorithm 1, the proposed approach written into sudo code is several time longer than DPS baseline (See algoritm 1 of [ICLR 22 Diffusion Posterior Sampling for General Noisy Inverse Problems]). This makes readers wonder whether the performance gain really comes from a robust neat improvement, or comes from additional complexity and nudging of parameters.
- The ablation study in Table 2 is using non-linear deblurring as operator, which is kind of unusual. Can the authors provide results with more commonly used linear operators such as super-resolution? As baselines, DPS and PSLD seems to totally fail for this task (with PSNR 17dB). According to ablation studies, Armijo step seems to contribute most to performance, and the other components are not so important?
- In complexity Table 6, what is the sampling step for vanilla DPS?
- As the proposed approach follows the manifold assumption of MCG, it would be great if it could add MCG as baseline for linear operators. Besides, another possible assumption is the spherical manifold assumption, which is adopted by DSG [Guidance with Spherical Gaussian Constraint for Conditional Diffusion] and DMAP [Rethinking Diffusion Posterior Sampling: From Conditional Score Estimator to Maximizing a Posterior]. It would be better if the authors could discuss and compare to DSG and DMAP.
- rFID using 100 images might not be credible enough. Please consider using 1000 or more image if rFID is a must.
Questions for authors
Questions
- How is in Algorithm 1 determined? They are not in Table 3, 4.
- See weakness
Justification
- Overall I appreciate the strong and thorough validation on diverse tasks and dataset and recommand acceptance. However, many aspects of this paper remains unclear and should be improved. The proposed approach in Algorithm 1 is too complex compared with DPS baseline, with 5 more hyper-parameters. It becomes really hard to figure out whether the proposed approach's empirical advantage comes from the method itself or additional complexity. In ablation studies, the baseline method also fails to work (17 dB) and the contribution of component beyond Armijo step is unclear. I think this paper has too many new components and contributions, perhaps it would be better to reduce to 1-2 new part that is really effective.
2 comments
Part2, Q3-Q6
Q3. Ablations with linear operators and component contributions
Thank you for suggesting ablations with linear operators. We clarified the shared-initial-step protocol and expanded Table 2 with DPS Gaussian deblurring and HDR reconstruction. The results show task-dependent contributions from Armijo and geometric control, with their combination achieving the best results across the evaluated ablation settings.
-
Why the host performs poorly in the ablations. Within each task, we fix the initial guidance step size across variants so that each variant adjusts the direction and step size through its own mechanism. The unmodified host collapses at this shared step size because it lacks adaptive direction adjustment and step-size control. This protocol isolates the effects of the added components. Table 1 separately reports comparisons using the respective guidance settings of the host and Host+CAT.
-
Contributions of Armijo and geometric control. The expanded ablations show that Armijo provides substantial gains in deblurring, while second-order tubular control substantially improves HDR reconstruction even without backtracking. Their relative contributions vary across tasks, and combining them gives clearly stronger overall results. These ablations aim to isolate the contribution of each component and demonstrate the benefit of combining them across different tasks, with less emphasis on comparing their relative gains, which may depend on the task.
-
Linear operators and super-resolution. Following your suggestion, we added DPS Gaussian-deblurring ablations, a common linear task, alongside the PSLD motion-deblurring ablation. We selected Gaussian deblurring because it more clearly distinguishes the effects of the components. The differences between variants on super-resolution are small, making it less informative for this component analysis. Appendix D.2, Table 7, includes Host+Armijo versus Host+CAT on super-resolution for both DPS and PSLD. We hope these results and clarifications address your concern.
Revision locations: Sec. 4.3, Table 2, page 9 (clarified protocol, additional ablations, and discussion of component contributions). Appendix D.2, Table 7, page 25 (Armijo–CAT comparisons including super-resolution). Revisions are highlighted in blue.
Q4. Sampling steps for vanilla DPS
Thank you for requesting this clarification. The vanilla DPS baseline uses sampling steps. We specify the step counts in the table and added DPS runs with , , and for comparison.
Revision locations: Appendix D.4, Table 9, page 27 (originally Table 6 in the reviewed manuscript). Revisions are highlighted in blue.
Q5. Comparisons with MCG and DSG
Thank you for suggesting these additional comparisons. We added comparisons with MCG and DSG on linear inverse problems, where CAT achieves better perceptual quality in the evaluated settings.
-
Additional baselines and tasks. We compare DPS+CAT with MCG on random-mask inpainting, and with DSG on random-mask inpainting, Gaussian deblurring, and 4× super-resolution. We chose the three operators for which its original paper recommends parameter settings, and use the respective papers' recommended settings for both baselines. All comparisons follow the FFHQ evaluation protocol with 100 images and report PSNR, SSIM, LPIPS, and DISTS.
-
Perceptual quality. DPS+CAT achieves better perceptual quality than MCG and DSG in the evaluated comparisons, as reflected by lower LPIPS and DISTS.
Revision locations: Appendix D.1, pages 23–24. Revisions are highlighted in blue.
Q6. Reliability of rFID with 100 evaluation images
Thank you for raising this concern about the reliability of rFID with 100 images. We replaced FID with DISTS in the FFHQ and ImageNet inverse-problem evaluations to better assess reconstruction perceptual quality.
-
Evaluation focus. We agree that FID estimated from 100 images cannot reliably establish distribution-level quality. Our conclusions primarily concern the perceptual fidelity of individual reconstructions, and FID was previously included only as an auxiliary metric. This change does not affect our main conclusions.
-
Updated metrics. DISTS measures perceptual similarity to each reference image, complementing LPIPS without distribution-level estimation. We report DISTS and LPIPS alongside PSNR and SSIM to assess both perceptual fidelity and reconstruction accuracy.
-
Results. The results show substantial perceptual improvements over competing methods across many evaluated settings. As a plugin, CAT also broadly improves both reconstruction accuracy and perceptual fidelity relative to the original host methods.
Revision locations: Sec. 4.1, pages 5-6 (evaluation metrics and main results); Table 1, page 6; Sec. 4.3, Table 2, page 9; Appendix D.4, Table 9, page 27. These tables all report DISTS. Revisions are highlighted in blue.
Part 1, Q1-Q2
Q1. Hyperparameter settings and overall method summary
Thank you for pointing out the need for a clearer account of the additional parameters. We added a “Method summary” paragraph at the end of Sec. 3.2 and expanded the reproducibility appendix to collect all the hyperparameter settings.
Armijo parameters. We fix the sufficient-decrease coefficient to c=10−4 and the backtracking shrinkage factor to β=0.5 throughout the reported inverse-problem experiments, without task-specific tuning. Each search tests at most Jmax=3 candidate scales. The correction scale γt is determined by the Armijo acceptance test, with the most recently accepted scale reused between scheduled searches. Thus, γt is computed during sampling rather than independently tuned.
Overall parameter summary. After the definitions and derivations, the new “Method summary” paragraph in Sec. 3.2 brings together the two components of CAT and their additional hyperparameters. The tubular-budget coefficient ρ is the main geometric hyperparameter requiring tuning and can be reused across multiple tasks sharing the same prior. The search period p controls the frequency of Armijo evaluations and the efficiency–quality trade-off. The remaining Armijo settings, c, β, and Jmax, are fixed as described above. This summary provides a single place to understand the roles and settings of the additional hyperparameters.
Expanded reproducibility details. Appendix C now includes detailed settings for all hyperparameters used in our experiments.
Revision locations: Sec. 3.2, “Method summary,” page 5; Appendix C, pages 20–23. Revisions are highlighted in blue.
Q2. Implementation complexity and parameter tuning
Thank you for raising this concern about implementation complexity and parameter tuning. We reorganized the pseudocode to make the two components of CAT and their integration into a host sampler clearer, and summarized the additional hyperparameters in Sec. 3.2.
Two explicit components. As described in the “Method summary,” CAT consists of a second-order curvature-aware tubular constraint that regulates geometric departure and Armijo backtracking that calibrates the correction scale. Algorithms 2 and 3 now describe these components separately. Algorithm 1 then presents their integration into the host sampling loop through two procedure calls, showing how CAT adjusts the measurement-guidance update after the host prior transition. This organization makes the overall procedure and the implementation of each component easier to follow.
Limited additional tuning. The tubular-budget coefficient ρ is the main additional hyperparameter requiring tuning and can be reused across multiple tasks sharing the same prior. We follow common Armijo backtracking settings with c=10−4 and β=0.5, and fix Jmax=3. We use these values throughout the reported inverse-problem experiments without task-specific adjustment. The search period p can simply be set to 1 to apply Armijo at every step, or increased to reduce computational cost. The scale γt is selected automatically by the acceptance test. The “Method summary” paragraph in Sec. 3.2 summarizes these parameter choices. Thus, the main tuning burden introduced by CAT is ρ, which can be reused across tasks sharing the same prior, while the other parameters can be fixed with simple choices.
Revision locations: Sec. 3.2, “Method summary,” page 5; Appendix C, Algorithms 1–3, page 21, and Tables 3–5, pages 22–23. Revisions are highlighted in blue.