- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes an error-conditioned diffusion model for solving ill-posed inverse problems. At each diffusion step, a denoiser first predicts a clean estimate, computes its residual, and then uses this residual as additional conditioning in a second evaluation of the same denoising network. Both evaluations are trained using a supervised denoising objective, avoiding differentiation through the forward operator during inference. The method is evaluated on six inverse problems, including compressible flows, MRI, CT, and inverse scattering. The authors report improvements over conditional diffusion and several plug-and-play sampler solvers in reconstruction accuracy and inference time.
Critical strengths
-
The paper presents consistent empirical improvements across six inverse problems. Importantly, the comparison against conditional diffusion uses the same backbone neural network and an equal number of network evaluations, making the reported gains fair.
-
Additionally, the zero-residual ablation provides useful evidence that improvements arise from residual conditioning rather than simply introducing a second denoising evaluation. The analysis of corrections in the operator's null space is also an interesting attempt to characterize the learned mechanism.
Critical weaknesses
-
The central idea of feeding measurement or physics residuals into a learned reconstruction network is well established in recurrent inference machines, unrolled reconstruction methods, and error-conditioned neural solvers. The main methodological difference here is incorporating this mechanism into a two-pass diffusion denoiser with shared parameters. While this is a reasonable design, I find the conceptual advance relatively modest. The paper primarily demonstrates that an existing residual-conditioning principle can improve conditional diffusion, rather than establishing a substantially new approach to inverse inference.
-
The method requires paired training data and operator-specific training, which limits practicability, as each inverse problem requires training a diffusion model. The experiments demonstrate some robustness within a predefined measurement family, but do not establish transfer to fundamentally different forward operators. This limits the generality of the contribution relative to reusable diffusion priors, as it requires retraining for each inverse problem, leading to high computational costs.
Questions for authors
How does the proposed method perform on unseen acquisition geometries or forward operators? Is retraining necessary?
3 comments
Response to authors
Thank you for your answers and clarifications. Regarding generalization, I appreciate the experiments in Appendix A.8 showing that a single model can handle different view counts and noise levels. However, I would be interested in understanding whether the proposed method can transfer to unseen acquisition geometries within the same operator family. For instance, could a model trained on one set of projection angles be evaluated on a different angular configuration without retraining? Since the method explicitly incorporates the forward operator and its residual during inference, I think this would be an interesting experiment to better understand its generalization capabilities. Additionally, I think the experimental evaluation could be strengthened by including more plug-and-play diffusion baselines, such as DiffPIR and DDNM, among other established solvers. This comparison would help better assess the advantages of the proposed method relative to existing approaches. Thank you again for addressing my concerns.
Thank you for the additional feedback.
Transfer to unseen angular configurations. Without retraining, our model transfers well to intermediate view counts it was not trained on, but not to angle configurations of a different kind, such as shifted, irregular or limited-arc angles (new App. A.8.5). Because the conditioning is computed in image space from the filtered and unfiltered backprojections, a trained model accepts any set of projection angles without changes to the architecture.
Nonetheless, with residual feedback, the model is more robust to these shifts than conditional diffusion. First, the model of App. A.8 is trained on 4, 16, 64, 256 and 1024 evenly spaced views. At the held-out counts in between, it has the lowest NRMSE at 32, 128 and 512 views and is as accurate as DPS at 8 views. In the tables below, bold indicates the best NRMSE and italics the second best in each column.
| Method | 8 views | 32 views | 128 views | 512 views |
|---|---|---|---|---|
| DPS | 0.232 | 0.185 | 0.154 | 0.132 |
| Conditional diffusion | 0.281 | 0.187 | 0.148 | 0.128 |
| Ours | 0.232 | 0.173 | 0.141 | 0.117 |
Second, we evaluate the same model on unseen angle configurations, a 4-view set shifted by 10°, 8 irregularly spaced angles and 16 views over a 90° arc. On these configurations, DPS is more accurate than both amortized models (ours and conditional diffusion). Relative to evenly spaced angles at the same view count, the NRMSE of ours increases 26–35% less than that of conditional diffusion. On the irregular 8-view set, for example, the NRMSE of conditional diffusion rises from 0.281 at 8 evenly spaced views to 0.535, and ours from 0.232 to 0.397. This suggests that recomputing the residual with the test-time operator compensates for part of the shift. Third, after we fine-tune the same model for 200,000 steps on random angle sets, it has a lower NRMSE than DPS on all three configurations. The paired 95% confidence interval excludes zero on the 90° arc and includes zero on the other two. It also remains more accurate than conditional diffusion fine-tuned the same way.
| Method | Shifted, 4 views | Irregular, 8 views | 90° arc, 16 views |
|---|---|---|---|
| DPS | 0.286 | 0.273 | 0.239 |
| Conditional diffusion, fixed angles | 0.411 | 0.535 | 0.599 |
| Ours, fixed angles | 0.359 | 0.397 | 0.458 |
| Conditional diffusion, random angles | 0.325 | 0.314 | 0.269 |
| Ours, random angles | 0.282 | 0.269 | 0.228 |
These results are consistent with the scope of our method. As §7 states, the family of forward operators must be fixed before training, and App. F now states which angle sets a model trained on evenly spaced angles transfers to. Our general response discusses this scope in more detail ("Problem setting"). Within the family of geometries seen in training, our method is at least as accurate as DPS and about 10× faster. For geometries outside that family, a plug-and-play prior or training on a wider family is the better choice.
DiffPIR and DDNM. Following your suggestion, we added both to Table 3 on the three imaging tasks. They use the same unconditional prior as DPS, DAPS and PnP-DM and were tuned with the same protocol (App. D.2). Our method has a lower NRMSE than both on every task, and the paired 95% confidence intervals exclude zero (App. A.9). DDNM runs at about our speed, and DiffPIR about 5× slower. Both methods require the proximal map or pseudo-inverse of a linear operator. On Kelvin–Helmholtz and Rayleigh–Taylor the radiographs measure the partial density of the heavy fluid, ρ(1−Y), a product of two state channels, so the operator is not linear. On compressible Euler the operator is linear, but the projection step of both methods cannot include the physics term that the other plug-and-play samplers use on the flows (App. D.2).
| Method | CS-MRI | CT | Scattering |
|---|---|---|---|
| DPS | 0.281 | 0.292 | 0.325 |
| DiffPIR | 0.293 | 0.304 | 0.348 |
| DDNM | 0.347 | 0.298 | 0.394 |
| Ours | 0.257 | 0.234 | 0.333 |
Conceptual advance relative to RIMs, unrolled networks and error-conditioned solvers. We agree that feeding a residual to a network is an established principle, and we cite it as such. RIMs, unrolled networks and ENS produce a deterministic point estimate, and our contribution is to use the residual inside a posterior sampler. Please see "Concurrent work and point-estimation methods" in the general response for more detail. §6.3 also shows that the method uses the residual differently from a learned data-consistency step. 85% of the residual-driven correction lies in the null space of A. A data-consistency step cannot make such a correction, because its update A⊤(Ax−y) always lies in the row space of A.
Operator-specific training and unseen operators. Like conditional diffusion, our method is designed for problems where the same family of measurements recurs and the requirement for accuracy justifies a one-time training run, rather than for transfer across operator families. A new operator family therefore needs a new model, while new view counts and dose levels within a family do not (App. A.8). Please see "Problem setting: one model per measurement family" in the general response for more detail and for the settings this covers.