Learning to Trust Krylov Refinement in Second-Order Diffusion Inverse Problems

Second-order diffusion inverse solvers exploit local covariance information to improve posterior conditioning beyond mean-only or isotropic approximations. However, this covariance is often obtained from derivatives of a pretrained denoiser whose Jacobian is not explicitly constrained as a covariance operator. The resulting second-order information can therefore be useful yet imperfect. We find that this creates a safety–quality tradeoff in the inner Krylov solver: shallow conjugate gradient (CG) can leave useful refinement unexploited, whereas deeper CG often improves reconstruction but can occasionally amplify covariance errors into catastrophic image-space corrections. We introduce KRC (Krylov Reliability Controller), a lightweight controller that determines which Krylov proposals should be trusted while preserving useful deeper exploration. A 7.8K-parameter two-head MLP uses only causal scalar solver/proposal statistics to predict the value of the current proposal and the recoverable value of deeper iterations. Additionally, a separate trusted correction allows Krylov exploration to continue without injecting unreliable proposals into the diffusion trajectory. Experiments on random-mask single-pixel imaging (SPI), with zero-shot transfer to cake-cutting Hadamard SPI, super-resolution, and deblurring, demonstrate improved reconstruction quality and reliability over fixed-depth and residual-based Krylov control without operator-specific retraining.