- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This article proposes a "reconstructed perception" distribution matching method to address the challenge of estimating unknown forward operators (such as blur kernels, PSF fields, etc.) from unpaired data in unsupervised image inverse problems. The existing methods based on pure degraded end distribution matching often find it difficult to constrain high-frequency or low signal-to-noise ratio information that is strongly suppressed by operators, resulting in operators that appear similar in the forward model performing differently in inverse operations (reconstruction). Therefore, based on the original degraded end matching (measure matching), this article innovatively introduces the reconstruction matching loss of the clean end. Real time reconstruction of real degraded images using differentiable alternating direction multipliers/PnP solvers (such as DPIR), and requiring the reconstructed image distribution to approximate a clean image distribution. Using conditional flow matching (CFM) combined with ensemble KL (IKL) divergence to measure distribution differences; Addressing the issue of high memory consumption in iterative solvers through truncated backpropagation; We have designed an adaptive gradient modulus balance heuristic strategy to prevent the reconstruction gradient from dominating the optimization process due to iterative amplification. In tasks such as uniform blurring, spatial variation PSF calibration, blind super-resolution, and real camera lens calibration, this method is significantly superior to traditional only degraded end matching methods, significantly narrowing the gap with the reconstruction effect of real operators
Critical strengths
-
Deep theoretical insight and motivation (Theorem 1): The paper rigorously proves theoretically that pure degraded end matching attenuates information extremely quickly on low signal-to-noise ratio components ($\ Theta (\ rho ^ 2) \ Theta (\ rho) $), accurately hitting the fundamental pain point of existing non paired operator estimation methods, and providing strong theoretical support.
-
Clever end-to-end differentiable reconstruction adaptation: By embedding the classic PnP IR algorithm (DPIR) into the training loop and combining truncated backpropagation with dynamic gradient normalization, the expensive iterative differentiation of VRAM and multi loss scale imbalance problems have been successfully solved. The engineering implementation is elegant and has strong universality.
Critical weaknesses
1.Although the author discussed the differences between biased loss and unbiased loss in the appendix, the overall framework still heavily relies on fixed pre trained denoising (prior in DPIR). If the pre trained denoising device itself has inherent artifacts or biases, it may mislead the operator estimation.
- Due to the need to run an iterative solver (DPIR) with an internal deep neural network in each iteration for the reconstruction branch, its optimization time significantly increases (about 5.7 to 6.6 times that of pure degraded end matching), which limits its scalability in ultra large models or ultra high definition image scenes.
3.In practical operation, the forward noise and the required noise level ($\ sigma $) of the denoising device are treated as fixed hyperparameter inputs. However, in complex real-world blind image restoration scenarios, the noise is often unknown and spatially variable, and the robustness evaluation of the method for noise parameter misestimation is not sufficient.
- In order to save video memory, the author adopted a truncation strategy of backpropagation through only the last $K0 {bp} K0 {bp}=1 $in the experiment). This strong truncation may not fully capture long-term dependencies in multi-step iterative solvers, weakening the theoretical convergence guarantee.
5.In the core practical application of "real shot calibration" (Section 5.4), due to the lack of Ground Truth, relying only on visual contrast and subjective qualitative analysis, there is a lack of quantitative indicators to support it in real and complex scenes.
Questions for authors
-
Regarding solver steps and truncation: In the experiment, the total number of iterations for DPIR was $K=8 $, while the number of return steps was $K {bp}=1 $. Has the author tested the extent of improvement in operator estimation accuracy and the boundary of video memory overhead by increasing $K_ {bp} $?
-
Regarding the sensitivity of noise hyperparameters: In real-world tasks, if the preset denoising level $\ sigma $deviates from the actual data (e.g. by 50%), to what extent will the convergence stability of the algorithm and the final estimated operator degrade?