acceptodds
Withdrawn from ICLR 2027

Learning to Trust Krylov Refinement in Second-Order Diffusion Inverse Problems

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Second-order diffusion inverse solvers exploit local covariance information to improve posterior conditioning beyond mean-only or isotropic approximations. However, this covariance is often obtained from derivatives of a pretrained denoiser whose Jacobian is not explicitly constrained as a covariance operator. The resulting second-order information can therefore be useful yet imperfect. We find that this creates a safety–quality tradeoff in the inner Krylov solver: shallow conjugate gradient (CG) can leave useful refinement unexploited, whereas deeper CG often improves reconstruction but can occasionally amplify covariance errors into catastrophic image-space corrections. We introduce KRC (Krylov Reliability Controller), a lightweight controller that determines which Krylov proposals should be trusted while preserving useful deeper exploration. A 7.8K-parameter two-head MLP uses only causal scalar solver/proposal statistics to predict the value of the current proposal and the recoverable value of deeper iterations. Additionally, a separate trusted correction allows Krylov exploration to continue without injecting unreliable proposals into the diffusion trajectory. Experiments on random-mask single-pixel imaging (SPI), with zero-shot transfer to cake-cutting Hadamard SPI, super-resolution, and deblurring, demonstrate improved reconstruction quality and reliability over fixed-depth and residual-based Krylov control without operator-specific retraining.

Primary area:
generative models
Keywords:
diffusion models, inverse problems, Krylov methods, conjugate gradient, posterior conditioning, reliability estimation, single-pixel imaging

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_DnC4OpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the reliability of inner Krylov refinement in covariance-aware diffusion inverse problems. The authors observe that deeper conjugate-gradient refinement often improves reconstruction quality, while a smaller measurement-space residual does not necessarily yield a more reliable image-space correction when the Jacobian-derived covariance action is imperfect. They propose KRC, a lightweight two-head MLP that predicts the immediate value of the current CG proposal and the value potentially recoverable through deeper iterations using 23 causal scalar diagnostics. A trusted-correction mechanism shadows risky proposals while allowing the internal CG recursion to continue. The controller is trained on random-mask single-pixel imaging and evaluated on random-mask and Hadamard SPI, Gaussian deblurring, and super-resolution.

Critical strengths
  1. The paper identifies an important practical mismatch between numerical convergence of the inner linear solve and reliability of the transported image-space correction. The population-level trajectory study provides substantial empirical evidence that deeper CG can be beneficial while occasionally producing severe reconstruction failures.

  2. The matched control experiments are carefully constructed. Fixed-depth CG, residual-based stopping, and the hand-designed controls use the same prior, sensing operator, covariance action, damping, and DDIM schedule. Reporting fifth-percentile degradation, worst-case degradation, and collapse counts in addition to average PSNR, SSIM, and LPIPS makes the reliability analysis substantially more informative.

  3. The controller is lightweight at inference and uses only causal scalar diagnostics. It requires neither raw images nor learned image-feature tensors, future CG states, or ground-truth quality during deployment. Its transfer from random-mask SPI to Hadamard SPI, super-resolution, and deblurring without operator-specific retraining or recalibration is encouraging.

Critical weaknesses
  1. The posterior interpretation of the controller is not fully supported by its supervision and evaluation. KRC learns ground-truth PSNR changes and selects trusted corrections using a reconstruction-quality proxy rather than a criterion derived from posterior accuracy. A proposal may therefore be preferred because it improves PSNR even if it deviates further from the nominal Gaussian-conditioned update. This is reasonable for reconstruction-oriented control, but the terms "posterior reliability" and "posterior acceptance" suggest a stronger probabilistic interpretation than the results establish. The paper should clarify this distinction or provide additional evidence connecting the selected corrections to posterior accuracy.

  2. The future-value target is local to one inner CG trajectory, whereas the ultimate objective is the final reverse-diffusion reconstruction. The target measures the best value available among deeper CG proposals at the current timestep, but accepting a different proposal changes all subsequent DDIM states. The appendix acknowledges that this local oracle headroom is not a final-reconstruction oracle, yet the resulting mismatch is not quantified. A held-out rollout analysis would help determine whether locally preferred proposals and continuation decisions actually improve final reconstruction quality.

  3. The contributions of the controller's individual components are not adequately isolated. The paper compares KRC against fixed-depth, residual-based, and hand-crafted controls, but does not report immediate-head-only, future-head-only, no-shadowing, or no-re-trust variants. This is particularly important because the future-value head is reported to be pessimistic on the rare harmful-but-recoverable cases, which directly motivate continued Krylov exploration. Moreover, the final controller combines the learned predictions with several calibrated and fixed policy choices, including unconditional early trust, multiple thresholds, minimum stopping depth, patience, and cumulative re-trust criteria. Without component ablations or a simpler predictor using the same 23 features, it is difficult to determine which elements are responsible for the reported improvements.

  4. The PSNR-based definition of reliability and the catastrophic threshold require further justification. Both prediction targets are based on ground-truth PSNR, although the paper also evaluates SSIM and LPIPS and discusses visible reconstruction artifacts. It is unclear whether the same proposals would be considered reliable under these alternative metrics. Furthermore, the -3 dB criterion is used to define catastrophic proposals for analysis and loss weighting, and the same numerical threshold is used to report final reconstruction collapses relative to FIXED-CG2. Although a 3 dB PSNR drop approximately corresponds to doubling the MSE, the sensitivity of the conclusions to this threshold is not examined.

  5. Robustness to measurement noise remains insufficiently evaluated. Appendix A specifies zero added measurement noise for the random-mask SPI protocol used to collect the controller's training trajectories, while the additive measurement-noise settings for Hadamard SPI, Gaussian deblurring, and super-resolution are not clearly reported. Since KRC relies on measurement-consistency statistics, CG residuals, and correction-growth features, it is unclear whether these reliability signals remain effective under noisy observations. Evaluating both the existing controller trained on noiseless trajectories and a controller trained on noisy trajectories, alongside matched fixed-depth and hand-crafted controls, would help distinguish zero-shot noise robustness from the effectiveness of noise-aware training.

A minor presentation issue is that Figure 1(a) and Figure 1(c) appear to contain broken mathematical glyphs for several diffusion states.

Questions for authors
  1. Do the authors intend "posterior acceptance" as a claim about posterior accuracy, or simply as acceptance of an approximate conditioning correction? If the latter, would "reconstruction reliability" or "trusted conditioning update" be more precise?

  2. On held-out cases, how often do locally preferred CG proposals or continuation decisions improve the final DDIM reconstruction? Can the authors quantify the disagreement between local PSNR-based targets and final reconstruction outcomes?

  3. Can the authors report immediate-only, future-only, no-shadowing, and no-re-trust ablations, together with a simpler predictor using the same 23 features? In particular, what measurable benefit does the future head provide on harmful-but-recoverable proposals?

  4. How sensitive are the reported catastrophe statistics and reconstruction results to the -3 dB definition and the calibrated control thresholds? Do PSNR-based trust decisions agree with SSIM- or LPIPS-based outcomes?

  5. Appendix A specifies zero added measurement noise for random-mask SPI, while the measurement-noise settings for Hadamard SPI, Gaussian deblurring, and ×4/×8 super-resolution are not explicitly reported. Could the authors clarify the additive measurement-noise model and level for each experiment? More importantly, could they evaluate KRC under several controlled nonzero measurement-noise levels, comparing (i) the existing controller trained on noiseless SPI trajectories and (ii) a controller trained on noisy SPI trajectories? Evaluating both on random-mask SPI and at least one transfer task, alongside matched fixed-depth CG and hand-crafted controls using identical noisy measurements, would help determine whether the learned reliability signals remain effective under measurement noise and whether noise-aware training is necessary.

Reviewer_k4cNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a learned Krylov Reliability Controller (KRC) for covariance-aware diffusion inverse problems. The method predicts the reliability of intermediate CG corrections and the potential benefit of further iterations, allowing the solver to explore deeper Krylov proposals without necessarily accepting unreliable updates. Experiments on single-pixel imaging, super-resolution, and deblurring demonstrate improvements in reconstruction quality and stability.

Critical strengths
  1. The paper addresses an interesting and practically relevant problem in covariance-aware diffusion reconstruction. In particular, I liked the observation that reducing the CG residual does not necessarily lead to better image-space corrections.

  2. I found the idea of separating Krylov exploration from correction acceptance intuitive and well-motivated. I also appreciate the comparisons against fixed-depth CG and hand-crafted stopping strategies, as well as the zero-shot transfer experiments, which provide some encouraging evidence of the proposed controller's effectiveness and generalizability. The zero-shot transfer experiments are also promising, particularly considering that the controller uses only scalar solver statistics and does not require retraining for each measurement operator.

Critical weaknesses
  1. Although the empirical results are promising, I found it difficult to determine which components of the proposed controller are primarily responsible for the observed improvements. The method combines immediate-value prediction, future-recoverability estimation, shadow exploration, trusted correction storage, and recovery-based acceptance. However, the ablations mainly compare the full method against fixed-depth CG and hand-crafted stopping rules. I could not find experiments isolating the contribution of the two prediction heads or the proposed shadow exploration mechanism. Since these components constitute the central methodological contribution, I believe more targeted ablations are necessary to establish their individual importance.

  2. Although the experimental results are encouraging, I find the evaluation somewhat limited given that reliability is a central contribution of the paper. The main experiments use only 18 test images, and the controller is trained under noiseless measurement conditions. While the zero-shot transfer across different operators is promising, I would like to see whether these improvements hold under different measurement noise levels and on a larger, more diverse set of inverse problems. It would also be interesting to understand how well the learned controller generalizes to other pretrained diffusion models.

Questions for authors
  1. The proposed controller addresses instabilities arising from covariance-aware CG refinement. However, I could not find comparisons against conventional CG-based data-fidelity methods, such as DDS (Chung et al., ICLR 2024), which do not require estimating the denoiser-derived covariance. Given that the covariance approximation itself can introduce instability and substantial computational overhead, could the authors clarify whether covariance-aware refinement provides a meaningful advantage over standard regularized data-fidelity updates under matched computational budgets? In particular, would a simpler CG-based correction achieve comparable reconstruction quality and reliability without requiring the proposed learned controller?

  2. The experiments appear to assume zero measurement noise, at least for the SPI setting (Appendix A). Could the authors clarify the motivation for this assumption and whether it also applies to the super-resolution and deblurring experiments? More importantly, how does the proposed controller perform under different measurement noise levels? Given that the method relies on estimated covariance information and learned CG proposal reliability, I believe evaluating its robustness to measurement noise is particularly important.

  3. I found the severe artifacts exhibited by DPS and ΠGDM in Fig. 4 somewhat surprising, particularly given the relatively standard inverse problems considered. Could the authors clarify the exact implementation and hyperparameter configurations used for these baselines, including their guidance strengths, sampling schedules, measurement-noise assumptions, and forward operators? In particular, Section 4.1 states that ΠGDM retains its original DDPM1000 configuration, whereas the original ΠGDM paper employs DDIM-based sampling with 100 sampling steps. Were the baseline hyperparameters appropriately tuned for each inverse problem?

Reviewer_ptEtOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies covariance-aware diffusion solvers for linear inverse problems, where updating the posterior mean requires an inner conjugate-gradient (CG) solve using a covariance derived from the denoiser Jacobian. The authors show that running CG deeper usually improves the reconstruction but occasionally causes catastrophic corrections in image space, since convergence in measurement space does not guarantee a reliable transported update. To address this problem, they propose KRC, a lightweight two-head MLP that uses causal scalar solver statistics to predict both the immediate value of the current CG proposal and how much value can still be recovered at deeper iterations.

Critical strengths
  1. The studies in Fig. 2, Fig. 6, and Appendix B show that measurement-space convergence does not guarantee a reliable image-space correction. I believe this insight is useful for MMPS-style solvers, independently of the proposed controller.

  2. Separating Krylov exploration from posterior acceptance through shadowing is well motivated by the observed recovery of some catastrophic proposals, as shown in Appendix B.5. In addition, the controller is lightweight and reuses cached covariance actions, so it adds no denoiser evaluations.

  3. The formulation, which first denoises and then conditions on the measurements, is well motivated for linear inverse imaging problems, and the tables show improvements by large margins.

Critical weaknesses
  1. The gains in Tables 3 and 4 may come from simply avoiding deep refinement rather than from learned reliability cues. Please provide an ablation study.

  2. The symmetrized Jacobian is not guaranteed to be positive semidefinite (PSD), all experiments are noiseless, and MMPS, the closest prior method, is never compared directly. I would like to see a discussion of these points.

  3. Several external baselines seem untuned for the shared sensing setup, with nearly flat performance across sampling ratios, so the headline gain over external methods largely reflects unfair comparisons. In addition, the results come from a small evaluation set with a single seed and no confidence intervals, which makes the comparison less reliable.

  4. All experiments use a single resolution, a single prior, and a limited set of tasks; results at another resolution (e.g., 64×64 or 512×512) would strengthen the paper. In addition, the controller components should be ablated.

  5. The paper does not discuss two concurrent works: EPS [1], which shows that conditioning before denoising keeps the learned covariance out of the linear solve (suggesting that KRC's failure mode stems from its denoise-then-condition ordering), and Meng [2], which analyzes the exact posterior–prior score difference that KRC's correction approximates.

[1] Abbas Mammadov, Ozgur Kara, Kaan Oktay, Iskander Azangulov, Adil Kaan Akan, Hyungjin Chung, James Matthew Rehg, Yee Whye Teh. Exact Posterior Score Estimation for Solving Linear Inverse Problems. arXiv:2606.17048, 2026. https://arxiv.org/abs/2606.17048 [2] Xiangming Meng. Posterior Information Dynamics of Diffusion Models for Linear Inverse Problems. arXiv:2608.21709, 2026. https://arxiv.org/abs/2608.21709

Questions for authors
  1. As τ→0\tau \to 0, Eq. (5) becomes a Σt−1\Sigma_t^{-1}-weighted projection of μt\mu_t onto {x:Ax=y}\{x : Ax = y\}. It therefore reduces to DDNM when Σt∝I\Sigma_t \propto I, which matches the zero-noise limit discussed in [2]. Could you explain this relation and add a DDNM-style baseline under the same schedule? In addition, could you check whether the catastrophic energy concentrates in the null-space component (I−A†A)Δ(I - A^\dagger A)\Delta?

  2. Following up on (1), since τ\tau in Eq. (5) plays the role of the noise variance σy2\sigma_y^2, the method targets a posterior with a time-varying noise level even in the noiseless experiments. How sensitive are the reconstruction quality and the catastrophe rate to τ\tau?

  3. Several controller features are absolute norms that scale with image dimension (Appendix C.2). How do the frozen normalization and calibration behave at other resolutions, and how would you perform normalization and standardization in this formulation?

  4. Could you report results with a single head, without shadowing, without re-trust, and with a correction-growth-only threshold (cf. Table 5)? How much does performance vary across controller training seeds?

[2] Xiangming Meng. Posterior Information Dynamics of Diffusion Models for Linear Inverse Problems. arXiv:2608.21709, 2026. https://arxiv.org/abs/2608.21709

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.