- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper presents Likelihood Score Approximation (LSA), a method for using diffusion models as priors in two common settings: (1) when the degradation operator is known, and (2) when the degradation is unknown but paired data are available. The authors build on the decomposition of the posterior score into the prior score and the gradient of the log-likelihood at each diffusion step. Since the prior score is readily available from a pretrained diffusion model, they propose amortizing likelihood score estimation by training a network on paired data to predict it from the current diffusion state and the degraded measurement. When the degradation operator is known, paired data are generated using the prior model. The paper evaluates LSA against a range of inverse-problem and restoration baselines across multiple speech and image tasks.
Critical strengths
- Method: The paper unifies two common inverse-problem settings within a single framework. To my knowledge, using pretrained diffusion models for image restoration with unknown degradation operators and limited paired data is novel and addresses an important limitation of existing methods. The proposed approach handles both settings elegantly and is conceptually simple, which I consider a strength.
- Clarity: The paper is clearly written and easy to follow. Most relevant information is provided in the main text or appendix.
- Experiments: The experimental evaluation is comprehensive, covering a variety of inverse-problem tasks on both speech and image data.
Critical weaknesses
- Evaluation: My main concern is the evaluation protocol.
- I do not believe that samples are sufficient to reliably demonstrate improvements on the reported metrics, particularly FID. Although this protocol follows DAPS, I consider it inadequate and do not think it should be retained. I would recommend evaluating on at least samples.
- The baseline comparisons are too sparse to establish the method’s strengths convincingly. I would prefer consistent comparisons across all tasks against three strong, recent baselines for each data modality and inverse-problem setting (i.e, known and unknown degradation operators). Although the comparison with ControlNet is important, I believe it is better treated as an ablation than as a separate baseline. Furthermore, the advantages of LSA over RED-diff and DMAP+CSE, which share similarities with the proposed approach, are not convincingly demonstrated by a single comparison with each method. I am also concerned that applying restart-based refinement only to LSA gives it an unfair advantage, since this component is orthogonal to the proposed method.
- The appendix reports an extensive hyperparameter search (p. 20, line 1029), raising concerns about the method’s sensitivity to hyperparameter choices and the effort required to identify a suitable configuration. It is also unclear which hyperparameters were tuned for the baselines and which configurations were ultimately used.
- Related work: The paper omits a substantial body of related work on inverse problems and image restoration. In addition, DiffPhase, StreamFlow, and AnyEnhance are included in the comparisons without corresponding citations. See, for instance, the related work in [1] which should also be included as a reference.
- Method: Although I appreciate the proposed approach, a minor limitation is its reduced generality. A desirable property of inverse problem methods, such as DPS, is that they can be applied across tasks without task-specific training or validation-based model selection.
[1] Idan Achituve, Hai Victor Habi, Amir Rosenfeld, Arnon Netzer, Idit Diamant, Ethan Fetaya Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:420-443, 2025.
Questions for authors
Please refer to the Weaknesses section.
4 comments
W3. Hyperparameter selection and its sensitivity
We clarified the hyperparameter search protocol and report the selected configurations (Appendix C, Tables 6–7).
To investigate hyperparameter sensitivity, we first conducted an ablation of inference refinements across all seven image tasks (Figure 6). We then selected two representative tasks with different sensitivity profiles, super-resolution and motion deblurring, for a more detailed analysis (Appendix C.1, Figures. 7–8). Keeping the trained LSA networks fixed, we varied total NFE, restart time , and the guidance weight around the configurations selected in validation. These experiments quantify how restoration quality changes when inference hyperparameters deviate from their optimized values.
Importantly, all baselines that we train ourselves, including ControlNet, LoRA, and direct posterior models, undergo the same validation-based hyperparameter search. For external pretrained baselines, we use the authors' original inference configurations whenever available. These protocols are detailed in Appendix B.4.
W4. Related work and citations
We thank the reviewer for pointing out the missing references. We corrected a minor naming typo for Stream.FM and updated the bibliographic references for DiffPhase, Stream.FM, and AnyEnhance.
We have also added several relevant citations of works on inverse problems.
W5. Task-specific adaptation
We agree that training-free methods such as DPS have the important advantage of requiring no task-specific model adaptation. LSA offers a different trade-off: it trades initial training and validation costs, for reduced computation during inference. Moreover, our method can be applied to a wide range of real-world problems where the degradation operator is unavailable.
We therefore view LSA as complementary to training-free posterior samplers and not as a universal replacement. We explicitly acknowledge task-specific training as a limitation in Section 6.
I thank the authors for their clarifications and additional experiments, which have strengthened the paper and addressed my concerns. In light of the rebuttal, I am increasing my score by one point.
Big thanks!
We thank the reviewer for their positive feedback and careful reconsideration of our work. We greatly appreciate the constructive comments, which helped us strengthen the experimental evaluation and improve the manuscript!
We thank the reviewer for the critical assessment of our work. We appreciate the reviewer's recognition of the method's simplicity, novelty in the low-data setting, and broad applicability. We understand the concerns regarding the experimental evaluation and have made a substantial effort to address them. The feedback was particularly valuable, as the additional experiments and analyses motivated by your comments have significantly strengthened the paper. We hope the revised results provide a more convincing and transparent assessment of LSA. We encourage the reviewer to consult the revised manuscript for further details.
W1. Larger evaluation set
As requested, we extended all main ImageNet-256 results and all additional evaluations to 1,000 test images, including the updated Table 2 and Figure 5. The original DAPS benchmark evaluates only the first 100 images due to the substantial computational cost of posterior-sampling methods. Fortunately, the authors also provide an extended set of exactly 1,000 images, which we used with the same degradation protocols.
This set closely matches the size recommended by the reviewer and is publicly available, ensuring that our expanded evaluation remains directly reproducible.
The original 100-image evaluation results is used only for the comparison with ControlNet and RePS (Appendix D, Table 10), whose publicly available implementation was not accessible to us for a larger-scale evaluation, and also for a few ablations within the paper.
The ImageNet-512 and speech evaluations already used 1,000 test images and 886 long audio files respectively.
W2. Baselines and restart refinement
We appreciate the request for broader baseline coverage and have made a substantial effort to extend the comparisons across tasks and training regimes.
Known-operator image restoration.
We expanded the ImageNet-256 comparison to six baselines across all seven tasks (Table 2), including DAPS, RED-diff, DMAP, and several recent 2026 methods (DDiff, CLAMP, and PC-DAPS). All methods were evaluated on the same 1,000-image benchmark using consistent degradation operators and measurement noise. Notably, LSA still holds leading results in perceptual quality metrics, while requiring only 0.8–2.7 seconds per image on an A100.
Low-data image restoration.
We understand the request for three recent baselines for each data modality and inverse-problem setting. However, established benchmarks for learning unknown degradations from only 128 paired images remain limited. We therefore utilize ControlNet and LoRA as a representative low-data adaptation baselines and additionally include DEFT using its official implementation on the three supported tasks (Figure 5, Table 11). All methods use the same 128 training pairs. We also explicitly indicate that DEFT is evaluated without restart refinement.
For ImageNet-512, we follow the existing benchmark on low-data 8× super-resolution. This benchmark currently covers only one restoration task in the low-data setting, starting from 25 training pairs. Here, we compare against all three available CSE-based configurations: CSE, DPS+CSE, and DMAP+CSE, wherever results are provided (Figure 3, Table 13).
Speech restoration.
A similar difficulty arises for speech tasks, where published methods often use different datasets, degradation settings, and evaluation protocols. This is particularly challenging for the newer EARS-v2 benchmark. Nevertheless, we extended the full-data comparisons with an additional available baseline (Table 1). Importantly, our primary goal in the full-data regime is to verify that LSA remains competitive beyond limited supervision. We therefore include direct posterior training as the closest controlled comparison, using the same training data and backbone.
Restart refinement.
We agree that restart refinement is orthogonal to the proposed method. To isolate its contribution, we added LSA results without restarts throughout the main comparisons (Tables 1–2, Figure 3). Furthermore, Figure 6 presents a dedicated ablation across the seven ImageNet-256 tasks, examining the impact of removing restarts, both with and without matching the total NFE, as well as the contribution of guidance scaling.
The results show that removing restarts generally degrades LPIPS and FID, with similar trends under matched-NFE comparisons, showing that the improvements are not solely due to additional network evaluations.
Moreover, we justify the use of restarts by the fact that our model is very lightweight during inference; additional restart-based refinement does not add a significant amount of absolute inference time, whereas training-free methods take significantly more time per image on an NVIDIA A100 GPU.