acceptodds
Under review as a conference paper at ICLR 2027

RegionAgent: Two-Stage Preference Optimization for Regional Image Restoration

Abstract

Images under non-uniform coupled degradations require diverse restoration capabilities for different degraded regions. Existing methods employ globally uniform operations, potentially overlooking regional differences and causing incomplete restoration or even damage to image content. Moreover, a degraded region may require successive restoration actions, and earlier actions may alter residual degradation characteristics or introduce artifacts, potentially hindering subsequent restoration. To address these challenges, we propose RegionAgent, an adaptive restoration framework that trains a Vision-Language Model (VLM) through two-stage preference learning to generate a region-specific restoration action at each iteration. Specifically, we first train the VLM to select an action suited to the current region through step-level Direct Preference Optimization (DPO), using preferences derived from immediate restoration feedback to supervise only the differing component in each action pair. To account for the effects of current actions on subsequent restoration, we then apply trajectory-level DPO, using preferences derived from complete trajectory scores to supervise the sole differing action in each trajectory pair. Furthermore, as conventional image-pair datasets do not directly provide preferences over regional restoration decisions, we construct a regional restoration preference dataset comprising 10K action pairs and 20K trajectory pairs for two-stage VLM training. Experiments across multiple datasets demonstrate strong restoration performance under both mixed and single degradations. Ablation studies further validate the effectiveness of two-stage preference learning in improving restoration quality.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Image Restoration, Restoration Agents, Vision-Language Models, Direct Preference Optimization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ESMcOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
3(+1)
Read review
Paper summary

RegionAgent uses a VLM to choose where and how to refine a degraded image. The policy is trained first on preferences over individual actions and then on preferences based on final trajectory scores. The authors construct a regional restoration preference dataset and evaluate the method on mixed- and single-degradation benchmarks.

Critical strengths
  1. Regional refinement is well motivated, and the PSNR results are strong, particularly on Group C: 20.90 dB versus 19.85 dB for the best reported baseline. The comparison with RAR also shows a trade-off between PSNR and LPIPS.

  2. I found the B/C and E/F comparisons in Table 3(a) useful. They keep the preference data and update budget fixed, and both support supervising only the differing component or action. These comparisons give clear evidence for the proposed masking choices.

Critical weaknesses
  1. I am not yet sure how much the learned region selection adds. The shared restorer is fine-tuned on MiO, but the main tables do not include a Base-only result. A comparison with simple heatmap-based selection, using the same restoration components and refinement budget, would help establish the contribution of the VLM’s region choices.

2 .The trajectory stage improves the reported scores, but the full model also receives more preference data and updates than the step-only model. This makes the gain harder to attribute specifically to trajectory feedback. I would like to understand when this feedback changes the policy’s decisions and how those changes improve the final restoration.

Questions for authors
  1. Could you add Base-only results? I would also like to see the full policy compared with always selecting the highest-scoring heatmap window, using the same restoration components and refinement budget.

  2. Can you show cases where step-level and trajectory-level feedback favor different actions? Comparing the final results of the step-only and full policies in these cases would help explain what the second stage learns.

  3. Appendix D.1 states that baseline results are taken from prior work. Were RegionAgent’s results computed using the same test images, preprocessing and metric settings? Please clarify any differences.

Reviewer_HBdBOpenReview ↗
Rating
3(+1)
Confidence
3(−1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents RegionAgent, an iterative framework for images with spatially varying and coupled degradations. It uses a VLM to select degraded regions, configure restoration experts, and determine when to stop, while a structure-guided fusion module integrates local corrections. A two-stage preference optimization scheme improves both immediate regional decisions and long-term restoration trajectories. Experiments show promising results on mixed- and single-degradation benchmarks, making this a coherent and practically relevant approach to adaptive image restoration.

Critical strengths
  1. The paper addresses a practically relevant problem. Different regions of a real-world image may exhibit heterogeneous and coupled degradations, for which uniform image-level restoration is suboptimal.
  2. The paper is technically detailed, with substantial appendices covering the localizer, expert training, action space, preference construction, optimization settings, and efficiency measurement.
  3. The authors provide transparent end-to-end efficiency measurements, including both latency and peak GPU memory, which is particularly valuable for evaluating a multi-stage VLM-based restoration system.
  4. The supervision-construction pipeline is a meaningful contribution in its own right, as it converts conventional degraded‒clean image pairs into structured regional action and trajectory preferences.
Critical weaknesses
  1. The ablation experiments need to be more comprehensive. Although Table 3 separately analyzes preference learning and local fusion, it remains unclear whether the performance gains mainly come from DPO, the model itself, the combination of experts, or post processing.
  2. The paper describes replacing one action and then continuing to execute the subsequent actions from the original trajectory. However, once the first action changes, the subsequent visual states will also change, meaning that the original follow-up actions may no longer be appropriate for the new states. As a result, the trajectories being compared may involve action mismatches under counterfactual states. Since the subsequent actions are not replanned based on the new states but are simply inherited from the original trajectory, the reviewer is not convinced that the resulting preferences truly reflect a better long-term policy.
  3. The paper provides many case studies, but the reviewer noticed that analysis or examples of failure cases are not included within the paper.
Questions for authors

See Weaknesses. The reviewer's rating of 2 is mainly based on these. When with satisfactory rebuttal, the reviewer is happy to adjust the score.

Reviewer_j3QiOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes RegionAgent, a region-aware image restoration framework that uses a VLM to select regions, restoration experts, and fusion strengths. The VLM is trained through two-stage preference optimization, combining step-level DPO for immediate restoration decisions and trajectory-level DPO for longer-term outcomes. The method achieves competitive results on both mixed- and single-degradation benchmarks.

Critical strengths
  1. The paper addresses a practical and meaningful problem, as different image regions may suffer from different degradations and require different restoration strategies. The proposed regional restoration framework and two-stage preference optimization are reasonably motivated and well designed.
  2. The experimental results are strong across multiple benchmarks, with relatively comprehensive ablation studies supporting the effectiveness of the proposed DPO strategy and fusion module.
Critical weaknesses
  1. The framework mainly integrates existing techniques, including region localization, LoRA expert composition, and preference optimization. Moreover, the contribution of the learned VLM policy is not sufficiently justified through comparisons with simpler region selection strategies under the same restoration backbone and inference budget.
  2. The Localizer plays a critical role in the framework, but its localization accuracy and individual contribution are not quantitatively evaluated. Additionally, while RegionAgent achieves strong PSNR and SSIM results, its weaker LPIPS and CLIP-IQA scores compared with RAR on several benchmarks suggest a trade-off between distortion and perceptual quality that deserves further discussion.
Questions for authors
  1. Could the authors provide quantitative evaluations of the Failure Localizer and compare it with simpler region proposal strategies or oracle regions?
  2. How does the proposed VLM policy compare with a simple greedy strategy that always selects the region with the highest predicted degradation score?
  3. For trajectory-level DPO, subsequent actions are kept fixed when constructing preference pairs. How well does this reflect the actual inference process, where subsequent decisions are dynamically updated?
  4. Could the authors explain why the proposed method performs worse than RAR on some perceptual metrics despite its improvements in PSNR and SSIM?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.