- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper presents a reinforcement learning algorithm to choose the best per-local-region estimate among a set of optimization-based reconstruction algorithms relying on different explicit or implicit priors (deep image prior, total variation, etc.). The learned policy receives information about the error of each algorithm on a held-out set, the number of iterations, and the current best reconstruction, and is trained on a synthetic dataset following the 'dead leaves' generative model. The resulting algorithm is evaluated in a series of 2D imaging inverse problems (MRI, CT, inpainting, deblurring, demosaicing), showing a mild performance improvement (around .3 to .5 dB) when compared with the best reconstruction by a single reconstruction algorithm.
Critical strengths
- The idea of using reinforcement learning in the context of optimization-based image reconstruction algorithms is new (to the best of my knowledge) and interesting
- I also find the idea of having a space-dependent prior innovative as well (although space-adaptive priors have existed for a while, I haven't seen work in this form)
Critical weaknesses
-
The results lag significantly behind the state-of-the-art in imaging inverse problems, which hinders the usage of the proposed algorithm in any practical setup:
- The proposed method only improves .3 dB over TV or DIP, which are widely surpassed by plug-and-play, diffusion, or end-to-end algorithms (e.g., in the 2019 FastMRI challenge, 7 years ago, the best U-Net at the time was already many dB better than TV).
- The reconstruction time is very slow (multiple seconds per 256^2 image) compared with the state-of-the-art: end-to-end networks (e.g., DRUNet, SwinIR, etc) take around 5-20 ms per image, which is 100x faster.
-
There is an important confusion regarding the literature of 'unsupervised methods' (lines 058 to 073 of page 2): the authors mix up self-supervised learning losses (splitting losses, equivariant imaging, SURE, etc.) that can be used to train an end-to-end network over a large dataset of measurement data (associated with multiple images), with methods relying on an 'unsupervised' prior (DIP, TV, etc), which use this before fit a single image at the time, and do not benefit from having a larger dataset of images. The statement "This abundance indeed turns reconstruction into a design problem: which unsupervised scheme should be used, and where?" is thus highly misleading. The paper relies on the latter methods, which are known to perform significantly worse in practice than the former ones.
Minor comments:
- "0.0 GFLOP for TV" is probably missing a decimal entry
- The images in Figure 3 seem to have significant aliasing; please display them in a larger format and with better interpolation
Questions for authors
- Can you compare the method with proper 'self-supervised' methods that train on a dataset of measurements instead of fitting one image at a time?
- If you assume that this 'raw measurement dataset' doesn't exist (which is hard to believe, since in practice we can always take more raw measurements), can you compare it with a supervised model trained on the dead leaves synthetic data?