acceptodds
Withdrawn from ICLR 2027

High-Order Diffusion Solvers for Linear Inverse Problems

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Diffusion models (DMs) are effective generative priors but typically require hundreds to thousands of sampling steps. Recent high-order solvers accelerate unconditional generation by exploiting the semilinear structure of diffusion ordinary differential equations (ODEs) and stochastic differential equations (SDEs), integrating linear terms exactly and reducing sampling to 5 to 20 steps while maintaining generation quality. DMs have also become effective generative priors for inverse problems, where measurement conditioning introduces an intractable measurement matching score. Efficient diffusion inverse solvers typically approximate the measurement matching score in various forms. However, to our knowledge, none of these methods can effectively exploit the semilinear structure of the corresponding differential equations to derive conditional high-order solvers. To address this issue, we identify a class of inverse solvers with structured measurement matching scores that incorporate the model prediction while retaining semilinearity. However, the corresponding integral equations contain dense matrices in the linear terms, making direct computation inefficient. We utilize the singular value decomposition of the forward operator to diagonalize the dense matrices in the spectral domain, enabling efficient computation of the linear terms and high-order approximation of the remaining nonlinear terms. We further introduce guidance strength and stochasticity to control measurement influence and sampling noise, respectively. Experiments on CelebA and ImageNet demonstrate that our method achieves improved reconstruction quality over baselines across linear inverse problems with 5 to 10 sampling steps.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Inverse problems, diffusion models, high-order numerical solvers

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_rJwDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes high-order diffusion solvers for linear inverse problems, building on the posterior-mean corrections of DiffPIR and DDNM. The authors formulate conditional diffusion ODEs and SDEs that retain a semilinear structure and use the singular value decomposition of the forward operator to efficiently compute operator-dependent coefficients in the spectral domain. Existing numerical approximation strategies from DPM-Solver++ and UniPC are then applied to the remaining nonlinear model prediction, with guidance strength and stochasticity controlling measurement influence and sampling noise. Experiments on CelebA and ImageNet evaluate random inpainting, compressed sensing, and motion deblurring, reporting improved reconstruction quality over several baselines at 5–10 neural function evaluations (NFEs).

Critical strengths
  1. The paper is well-written and clearly organized. The motivation, mathematical formulation, and numerical implementation are presented coherently, making the proposed approach relatively easy to follow.

  2. The paper provides a systematic formulation connecting existing diffusion inverse solvers with high-order numerical integration. In particular, identifying the shared structure of DiffPIR/DDNM measurement corrections and exploiting spectral diagonalization to efficiently handle operator-dependent coefficients is technically useful.

  3. The experiments demonstrate substantial improvements in the few-step regime, particularly for random inpainting. The evaluation covers multiple datasets and inverse tasks, and the appendices provide additional comparisons, numerical-order ablations, guidance and stochasticity studies, and runtime analyses with closed-form baseline implementations.

Critical weaknesses
  1. The technical novelty appears limited. The method builds on the posterior-mean corrections of DiffPIR/DDNM and adopts numerical approximation strategies from DPM-Solver++ and UniPC. Guidance scaling and controllable stochasticity are also established components. Although combining these techniques into a spectral formulation for conditional diffusion is useful, the paper does not clearly establish a distinctive numerical principle beyond adapting and combining existing methods. The overall contribution therefore appears relatively incremental.

  2. The source of the large reconstruction gains remains insufficiently isolated. In Table 1, DiffPIR improves from 17.53 dB to 26.10 dB with the proposed first-order DPM-1 variant, while UniPC-2 further improves this to 27.13 dB. Table 14 also shows that the same UniPC-2 solver at NFE = 5 and τ=1\tau=1 improves from 18.37 to 27.13 dB when γ0\gamma_0 increases from 1 to 4. These results suggest that the reformulation and guidance settings contribute substantially to the reported baseline gap, but their effects are not adequately separated from those of higher-order integration.

  3. The experimental comparisons are restricted to very small sampling budgets. While equal-NFE comparisons at 5 and 10 steps are appropriate for evaluating few-step performance, existing methods such as DDRM, DDNM, and DiffPIR typically operate at 20–100 NFEs, as acknowledged in the introduction. The paper does not compare the proposed solvers against these baselines at their commonly used sampling budgets. Consequently, it remains unclear whether the proposed methods can achieve comparable reconstruction quality with substantially fewer evaluations. Such comparisons would provide stronger evidence of their practical acceleration benefits.

  4. The hyperparameter-selection protocol is insufficiently specified. Appendix A reports parameter values and their sources, but does not describe the validation data, selection criterion, search ranges, or tuning budgets. The proposed methods use task-specific guidance settings, whereas most baseline settings are inherited from earlier work. Given the strong guidance sensitivity shown in Appendix F.3, it is unclear whether the reported advantage persists when baseline parameters and timestep schedules receive comparable tuning for the evaluated NFE budgets.

  5. The statistical reporting requires clarification. The quantitative tables report PSNR, SSIM, and LPIPS with ±\pm values, but do not specify whether these represent standard deviations, standard errors, or variability across repeated sampling runs. Although Section 4.1 states that 100 test images are used, the aggregation procedure and number of stochastic sampling repetitions are not clearly described.

Questions for authors
  1. Could the authors further isolate the contributions of the conditional reformulation, guidance strength, and numerical approximation order to the reported reconstruction gains? The existing ablations are informative, but additional matched controls would help establish how much of the improvement is specifically attributable to higher-order integration.

  2. Could the authors explain why the performance improvement observed for random inpainting is substantially larger than for compressed sensing?

  3. Could the authors additionally evaluate the baselines at their commonly used sampling budgets (e.g., 20–100 NFEs) and compare their reconstruction quality with the proposed 5–10 NFE results? Reporting the corresponding total inference runtimes would help establish whether the proposed solvers achieve comparable reconstruction quality at lower computational cost.

  4. How were hyperparameters selected for each method, including the validation split, selection metric, search ranges, and tuning budgets? Were baseline parameters and timestep schedules adjusted for each NFE budget?

Reviewer_yihHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a high-order solver for solving linear inverse problems with pretrained diffusion models in a training-free manner, aiming to achieve high accuracy within a small number of steps, specifically 5 to 10 steps. Conventional high-order solvers such as DPM-Solver and UniPC have been used to accelerate unconditional generation, but applying these high-order solvers to inverse problem settings has been difficult. Building on DDNM and DiffPIR, the proposed method uses the SVD of the measurement operator to handle the conditional diffusion equations in the spectral domain, and then applies high-order solvers to them. Although the experiments are limited, they confirm the effectiveness of this method.

Critical strengths
  • In the general context of inverse problems, the application of high-order solvers is not very straightforward. However, this method successfully applies high-order solvers in a reasonable formulation, at least for DDNM-type and DiffPIR-type methods.
Critical weaknesses

I believe that the current experimental evaluation does not sufficiently confirm the essential contribution of the proposed method. SVD-based solvers for linear inverse problems and high-order diffusion solvers have already been widely studied. In this paper, the authors need to verify more carefully that combining them yields substantial improvements over existing methods. In particular, I am concerned about the following points.

  1. In Table 13, simply changing the guidance strength leads to a substantial performance improvement. However, there is no comparison in which the same guidance schedule is applied to the conventional DDNM or DiffPIR. It is unclear whether the current performance gain comes from the new solver or merely from adjusting the guidance.
  2. It is also questionable that DDRM is not compared in motion deblurring. The authors cite implementation constraints related to the SVD as the reason, but DDRM should also be able to compute efficiently if it uses the FFT in the same way as the proposed method. I am not convinced that an important baseline should be excluded for this reason.
  3. It is also unclear how much the high-order integration itself contributes to the performance improvement. I think it is necessary to fix the guidance and stochasticity, compare the conventional DDIM-type update with DPM-1, DPM-2, and UniPC-2, and evaluate the effect of using higher-order schemes independently.
  4. The comparisons are biased toward the limited setting of 5 to 10 NFEs. The method should be compared against existing approaches such as DDRM, DAPS, and DAPS++ in terms of the trade-off between reconstruction quality and runtime, without fixing the NFE. Demonstrating superiority only at a step count so small that existing methods fail to work sufficiently does not amount to showing a substantial improvement.
  5. I also do not think the evaluation tasks are sufficient. Standard linear inverse problems such as super-resolution and Gaussian deblurring are not included, and the strength of the measurement noise is fixed. It is necessary to verify whether the improvements are consistently obtained under a wider range of conditions.

For these reasons, based on the current experimental results alone, I cannot determine whether the improvement achieved by the proposed method could also be attained through a proper implementation and hyperparameter tuning of existing methods. In particular, since many strong methods already exist in this field, I believe that a performance gain in a limited setting alone does not sufficiently demonstrate the substantial contribution of this work.

Questions for authors
  • I understand that it is generally not desirable to ask authors to expand the scope of their experiments during the review process. However, I believe that the current experimental scope does not demonstrate the essential contribution of this method. Please consider evaluating the method under different measurement noise levels and with different operators. I do not think this method needs to outperform existing methods in every scenario. For the sake of contribution to this field, I am interested in understanding under what conditions this method performs well.
  • Could you discuss the essential differences from prior work on conditional high-order solvers?
Reviewer_iQNwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper develops training-free, few-step solvers for linear inverse problems using pretrained diffusion priors. It identifies a shared affine dependence on the clean-data prediction in measurement corrections derived from DDNM and DiffPIR. Spectral representations diagonalize the associated coefficients, enabling integral formulations in which higher-order approximations are applied to the neural prediction. The implementation combines DPM-Solver-style and UniPC-style updates with tunable guidance and stochasticity. Experiments on CelebA and ImageNet cover noisy inpainting, compressed sensing, and motion deblurring at 5–10 neural function evaluations (NFEs).

Critical strengths
  1. An implementable, training-free acceleration mechanism. The shared measurement-correction structure and spectral coefficient handling provide a concrete way to adapt existing high-order prediction strategies to measurement-conditioned reconstruction without retraining. The coordinate-wise and FFT-based implementations make this approach computationally practical for the tested structured operators.

  2. Encouraging low-NFE results and useful implementation controls. The reported inpainting improvements are substantial, and the evaluation includes two datasets, multiple inverse tasks, solver-order comparisons, and guidance/stochasticity ablations. The DiffPIR-CF and DDS-CF controls are particularly useful for separating the cost of iterative data-consistency computations from changes to the sampling procedure.

Critical weaknesses
  1. Insufficient differentiation from existing guided high-order solvers. Sections 2.2–3.1 provide a partial unification of DDNM and DiffPIR, but the incremental contribution remains insufficiently isolated. A more systematic comparison should separate conditional-score approximation, operator representation, and numerical integration. The authors should explain and test what is gained by integrating the time-dependent spectral coefficients separately rather than approximating this composite predictor. This distinction is important for establishing the methodological novelty beyond adapting established components.

  2. Restricted validation of applicability and quality–cost advantages. Structural requirements, decomposition/setup costs, and applicability when only matrix–vector products are available should be clarified. The comparisons remain concentrated at 5–10 NFEs. Quality–NFE and end-to-end runtime curves across the baselines’ effective operating ranges are needed to establish acceleration at matched reconstruction quality, rather than only superiority under a fixed few-step budget.

  3. Incomplete convergence justification for the evaluated algorithm. Appendix E provides ODE local/global error statements, but does not establish a corresponding strong or weak convergence result for the stochastic discretization used in the main experiments. A consistent stability analysis and controlled empirical order tests are needed. Numerical accuracy for the chosen approximate dynamics should also be distinguished from accuracy relative to the true posterior.

  4. Limited statistical validation and insufficient attribution of few-step gains. The 100-image-per-dataset evaluation provides initial evidence, but image selection, repeated-seed variability, confidence intervals, and separation of tuning from testing are insufficiently documented. FID computation and sample-size sensitivity also require clarification. A broader test set and discussion or controlled comparisons with applicable flow-matching inverse solvers would strengthen generality, provided differences in priors and training are distinguished from improvements in the integrator.

Questions for authors

See Critical Weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.