acceptodds
Under review as a conference paper at ICLR 2027

SPIRA: A Spatially-Variant Reinforcement-Learning Agent for Unsupervised Imaging Inverse Problems

19 views

Abstract

Imaging inverse problems arise ubiquitously in many applications, from medical imaging to consumer photography. In many scenarios, the lack of ground truth images prevents the development of data-driven supervised learning reconstruction approaches. Thus, unsupervised learning methods have gathered significant attention in recent years by developing different reconstruction loss functions, image priors, or neural architectural designs aimed at approximating a supervised loss function. However, the best unsupervised scheme varies not only across inverse problems (changes in the sensing matrix, imaging modality, noise distribution, etc.) but across spatial regions. Thus, we propose a SPatIally-variant Reinforcement-learning Agent (SPIRA). We build a library of unsupervised reconstruction experts, each fitted only to the measurements, and train a policy that assigns one expert to each spatial region at each step of the reconstruction. The policy is trained entirely on computer-generated data, where the reference image is available at no cost, from a per-region reward that combines the improvement of the composite with the regret against the best available expert for that region; an auxiliary head predicts each expert's regional error and shapes the action logits. At inference time, the policy provides an instance-specific reconstruction pipeline that adapts to different image content and inverse problem physics. The proposed approach was validated in different imaging modalities, including image restoration (inpainting, demosaicing, and deblurring), compressive sensing, sparse-view CT, and single-coil MRI, improving over any single unsupervised expert applied throughout while requiring one-third to one-half of the expert iterations of greedy expert selection.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Reinforcement Learning, Unsupervised imaging, inverse problems

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_6pZVOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents a reinforcement learning algorithm to choose the best per-local-region estimate among a set of optimization-based reconstruction algorithms relying on different explicit or implicit priors (deep image prior, total variation, etc.). The learned policy receives information about the error of each algorithm on a held-out set, the number of iterations, and the current best reconstruction, and is trained on a synthetic dataset following the 'dead leaves' generative model. The resulting algorithm is evaluated in a series of 2D imaging inverse problems (MRI, CT, inpainting, deblurring, demosaicing), showing a mild performance improvement (around .3 to .5 dB) when compared with the best reconstruction by a single reconstruction algorithm.

Critical strengths
  • The idea of using reinforcement learning in the context of optimization-based image reconstruction algorithms is new (to the best of my knowledge) and interesting
  • I also find the idea of having a space-dependent prior innovative as well (although space-adaptive priors have existed for a while, I haven't seen work in this form)
Critical weaknesses
  • The results lag significantly behind the state-of-the-art in imaging inverse problems, which hinders the usage of the proposed algorithm in any practical setup:

    • The proposed method only improves .3 dB over TV or DIP, which are widely surpassed by plug-and-play, diffusion, or end-to-end algorithms (e.g., in the 2019 FastMRI challenge, 7 years ago, the best U-Net at the time was already many dB better than TV).
    • The reconstruction time is very slow (multiple seconds per 256^2 image) compared with the state-of-the-art: end-to-end networks (e.g., DRUNet, SwinIR, etc) take around 5-20 ms per image, which is 100x faster.
  • There is an important confusion regarding the literature of 'unsupervised methods' (lines 058 to 073 of page 2): the authors mix up self-supervised learning losses (splitting losses, equivariant imaging, SURE, etc.) that can be used to train an end-to-end network over a large dataset of measurement data (associated with multiple images), with methods relying on an 'unsupervised' prior (DIP, TV, etc), which use this before fit a single image at the time, and do not benefit from having a larger dataset of images. The statement "This abundance indeed turns reconstruction into a design problem: which unsupervised scheme should be used, and where?" is thus highly misleading. The paper relies on the latter methods, which are known to perform significantly worse in practice than the former ones.

Minor comments:

  • "0.0 GFLOP for TV" is probably missing a decimal entry
  • The images in Figure 3 seem to have significant aliasing; please display them in a larger format and with better interpolation
Questions for authors
  • Can you compare the method with proper 'self-supervised' methods that train on a dataset of measurements instead of fitting one image at a time?
  • If you assume that this 'raw measurement dataset' doesn't exist (which is hard to believe, since in practice we can always take more raw measurements), can you compare it with a supervised model trained on the dead leaves synthetic data?
Reviewer_j9xvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a policy for selecting experts (e.g., priors like total variation, deep image prior, etc.) for reconstructing different regions of the image. The motivation for this idea is that different parts of the image will have different best-fit priors (e.g., TV is good for high contrast / sharp regions, SIREN is better for smoother regions). Experimental results compare SPIRA against using a single global prior on tasks like deblurring, in-painting, CT reconstruction, etc.

Critical strengths
  • The idea to use a spatially-varying prior is sensible, since different parts of the image (intuitively) will have different best-fit priors.
  • The introduction and basic methods of the paper are well-presented and the motivation for the underlying idea is clear.
Critical weaknesses
  • Much of the context required to understand the paper and the experiments is not explained in the main body of the paper. Some are explained in the appendix; however, the main method and experiments should be understandable solely from the main text. For instance: what are the formal definitions for GG and gg? What is the "unsupervised polish"? What does "winning" a cell mean?
  • The experiment design and results are also difficult to interpret. On the design side: the iteration budgets for all of the methods in Table 1 are quite different (e.g. 300 iterations vs 2100 iterations), I'd suggest allocating a similar budget for fair comparison. On the results side: the metrics reported in Table 2 seem to suggest that using spatially-varying experts yields very marginal improvements over a global prior (i.e., PSNR for "Best fixed" vs "Oracle").
Questions for authors
  • I would rework the experiment design to better evaluate SPIRA. As it stands, the experiment raises more questions than it answers; e.g., the greedy approach seems to generally perform better than the proposed SPIRA method, why not use that? It might be worth reporting the worst fixed prior for each dataset in Table 2 to indicate the best / worst performance range (hopefully the MOE approach lands consistently somewhere in the middle). Essentially, what is described in the "What the policy selects" section is intuitively what I'd expect the model to do, so it's confusing that a global prior outperforms SPIRA. Additional investigation / justification for why this is the case may be helpful.
  • A formal explanation / definition of the unsupervised polish and other terms should be included in the main body of the text.
  • Some intuition should also be given about the specific set of regularizers that were chosen; e.g., why are four different variants of DIP needed?
Reviewer_7syGOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an unsupervised imaging framework for inverse problems using a spatially-variant reinforcement learning. The authors construct an expert library including several solvers like DIP, TV, SIREN and train an RL policy to adaptively utilize these experts. Specifically, this policy uses a per-region reward measuring the improvement of the best expert for that region. Experiments on multiple tasks, including natural image restoration, CT and MRI, demonstrate the comparable performance of the proposed method compared to existing methods.

Critical strengths
  1. The motivation that different unsupervised reconstruction algorithms have different inductive biases is reasonable.
  2. The experiments cover multiple inverse problems, demonstrating the generalizability of the proposed framework.
Critical weaknesses
  1. Although this paper proposes a multi-stage RL strategy. It is unclear why RL is needed in the first place. The authors should justify more on the motivation for using RL.
  2. There is no clear ablation study on the proposed RL strategy, making the contribution of this paper unclear.
  3. The authors claim that their algorithm is unsupervised, but the ground truth is used during training.
  4. There is no related work section included in the paper.
  5. The overall reconstruction performance does not outperform the single expert in most cases.
Questions for authors

Please refer to the weakness mentioned above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.