acceptodds
Under review as a conference paper at ICLR 2027

GRASP: Grounded Restoration with Agent-Refined Semantic Prompts for Real-World Image Super-Resolution

Abstract

Real-world image super-resolution (Real-ISR) requires preserving structures supported by a low-quality (LQ) input while synthesizing realistic fine details not directly observable in it. However, existing generative Real-ISR methods often leave the complementary roles of visual and semantic conditions implicit, limiting control over structural fidelity and detail realism. To address this challenge, we propose Grounded Restoration with Agent-Refined Semantic Prompts (GRASP), a Real-ISR framework that explicitly assigns complementary roles to local visual evidence, global image context, and semantic prompts. Specifically, we introduce a Decoupled Prior Injection mechanism that separates local visual evidence from global image context. Within this mechanism, a diptych provides spatially aligned local evidence to anchor observable structures and guide local detail synthesis, while a lightweight Vision-Context Bridge (VCB) supplies global image context to stabilize the scene-level layout. We further propose an inference-time agent-driven prompting strategy that iteratively refines semantic prompts to guide realistic detail synthesis in ambiguous regions. Experiments show that GRASP achieves strong perceptual performance across multiple Real-ISR benchmarks.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Real-world image super-resolution, Diffusion Transformers, Visual in-context completion, Semantic prompt refinement, Visual conditioning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_4yG5OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GRASP, a diffusion transformer-based framework for real-world image super-resolution that balances structural fidelity and realistic detail synthesis through decoupled visual and semantic conditioning. The method introduces a Decoupled Prior Injection mechanism, combining diptych-based visual in-context completion for local structure preservation with a Vision-Context Bridge (VCB) for global image guidance. An inference-time agent iteratively refines semantic prompts, followed by Best-of-K selection. Experiments on five Real-ISR benchmarks demonstrate competitive perceptual performance, particularly on no-reference quality metrics. Ablation studies, qualitative comparisons, and a human preference study further evaluate the proposed components.

Critical strengths
  1. This paper introduces a structured conditioning mechanism that separates local visual evidence, global image context, and semantic guidance into distinct information pathways. This modular design makes the roles of different conditioning sources explicit and reduces the need for additional local feature extraction networks. It provides a flexible and interpretable way to incorporate heterogeneous information into a pretrained diffusion transformer.

  2. The separation between model adaptation and inference-time semantic control is appealing because it allows the generation behavior to be adjusted through prompts rather than repeated fine-tuning. Although the iterative inference process introduces substantial computational overhead, the parameter-efficient training design and flexible semantic control remain valuable technical characteristics.

Critical weaknesses
  1. The paper presents agent-driven prompt refinement as a major contribution. However, the complete inference pipeline requires approximately 750.7 seconds per image, compared with only 7.7 seconds without the agent. Although the experiments demonstrate improvements across refinement rounds, they do not sufficiently establish that the agent outperforms simple repeated sampling and Best-of-K selection under the same computational budget. This raises concerns about the necessity and practical value of the proposed mechanism.

  2. Introducing the agent improves no-reference quality metrics across three paired benchmarks but consistently worsens LPIPS, DISTS, and FID. While this does not necessarily indicate that the model generates incorrect details, it suggests that the paper has not sufficiently established a connection between improved perceptual quality and faithful detail restoration supported by the input evidence. This is a significant limitation for a method whose central objective is Grounded Restoration.

Questions for authors
  1. The proposed inference-time strategy combines iterative agent-driven prompt refinement with Best-of-K candidate selection. Although the paper provides round-wise results, i still wonder how much of the improvement comes from semantic reasoning rather than repeated generation and candidate selection.

  2. Table 1 shows that agent-driven prompt refinement consistently improves no-reference perceptual metrics while worsening LPIPS, DISTS, and FID compared with static GRASP across all three paired benchmarks. Could the authors clarify how the refined semantic prompts are constrained to avoid introducing unsupported or hallucinated details? In particular, what evidence demonstrates that the additional details are consistent with the original low-quality input rather than merely improving perceptual quality scores?

  3. The proposed framework separates local visual evidence and global image context through diptych completion and the Vision-Context Bridge. However, Table 2 shows that the global-only variant achieves higher MUSIQ and LIQE scores than the full visual GRASP model. Could the authors clarify how these results support the claimed complementarity between local and global conditions?

3 comments

Thank you for the detailed rebuttal and additional experiments. After carefully reading all the reviews and the authors’ rebuttal, I have decided to maintain my original score.

Thank you for the additional experiments. They have alleviated some of my concerns. My remaining concerns are the incremental value of closed-loop refinement and the strength of the fidelity claims. Matching the number of candidates does not match the computational budget. Moreover, the open-loop prompt control reported in your response to another reviewer recovers approximately 70–77% of the improvement at substantially lower runtime. These results support prompt adaptation, but provide a less compelling justification for the additional cost of sequential feedback.

The OCR improvement is encouraging. However, fixed visual conditioning and low-frequency consistency do not guarantee that synthesized high-frequency details are supported by the input. The thin-structure improvement is small without uncertainty estimates, and the 30-image audit supports only a limited non-inferiority conclusion under the specified margin. The consistent deterioration in LPIPS, DISTS, and FID therefore remains relevant; attributing it to imperfect references does not establish that the added details are faithful.

Thank the authors for the additional experiments and clarifications. The new comparison substantially addresses my concern about the incremental value of closed-loop feedback and the strength of the fidelity claims. Hence I decide to raise my rating from 2 to 3 , and I hope these experiments will be included in the final revision.

Reviewer_gfPkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

GRASP formulates real-world image super-resolution as diptych completion using FLUX.1-Fill-dev. An upsampled low-quality image provides local context, a Vision-Context Bridge supplies global CLIP features, and text prompts guide detail synthesis. Training updates the bridge and LoRA adapters while freezing the backbone weights. At inference, GPT-5.4 iteratively revises an initial LLaVA caption using the input, the previous SR result, and four no-reference quality scores. The default pipeline generates ten candidates and selects one using those metrics. Agent refinement improves the reported no-reference scores, but worsens LPIPS, DISTS, and FID on all three paired benchmarks relative to the static-prompt version.

Critical strengths
  1. The diptych is a useful way to reuse the pretrained Fill model for SR without a separate local conditioning encoder. In Table 2, adding adapted global context improves all four reported scores over the local-only variant. Figure 9 complements this result by showing that global-only conditioning can produce plausible outputs that depart substantially from the input.

  2. The paper goes beyond reporting selected outputs. Table 9 includes raw refinement rounds, Table 11 tests an open-weight Qwen backend, and Table 12 compares static, raw-round-10, and selected outputs in a blinded human audit. Participants prefer the selected outputs for naturalness, while the reported unsupported-change rates are similar across settings. These are useful checks on the agent component.

Critical weaknesses
  1. The main concern is that the same four quality metrics provide agent feedback, select candidates, and measure the gains. Even the raw-round results remain subject to this overlap. The human audit offers independent evidence, but covers only 30 images, and the confidence-interval calculation is not described. Reference-based results also need more weight in the interpretation: DRealSR LPIPS increases from 0.3662 to 0.3863 after refinement. The authors' explanation that some HR targets are blurry is plausible; however, low no-reference scores for those targets do not establish that newly synthesized details are faithful to the scene. I would like stronger evidence separating improved appearance from unsupported content changes.

  2. The current ablations do not establish how much iterative visual feedback contributes beyond generating more candidates. Missing comparisons include ten static-prompt samples with different seeds and ten prompt variants generated without intermediate SR feedback, both using the same selector. Such controls matter given the cost: on the reported 512 × 512 RealSR timing test with an H800, the full pipeline takes 750.7 seconds per image versus 7.718 seconds for the static version. For context, RealLR200 MUSIQ increases from 69.5311 to 70.2334. The human preference results suggest this gain should not be dismissed, but the paper still needs to show why the sequential agent is preferable to simpler uses of the generation budget.

Questions for authors
  1. With ten SR generations and the same selector, how does closed-loop refinement compare with static-prompt sampling across seeds and prompt variants generated without SR feedback? Please report quality and runtime for these controls.

  2. Do the gains hold on an additional quality metric used in neither feedback nor selection, and on a targeted fidelity evaluation of text or thin structures? For Table 12, how were images sampled, and how do the confidence intervals handle shared participants and images across judgments?

  3. Could you reconcile the initial scores in Figures 5–6 with Tables 9–10? For example, the figures show initial MUSIQ/MANIQA values of 69.821/0.5674, versus 69.5311/0.5651 in the tables. Which baseline was used for the refinement trajectories and reported gains?

1 comment
Reviewer_gfPkOpenReview ↗

Official Comment by Reviewer gfPk

Thanks for the detailed response. My major concerns are well addressed. I will keep my original score.

Reviewer_5mR4OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

GRASP adapts FLUX Fill to real-world image super-resolution through diptych completion, combining a low-quality reference panel with global CLIP features and semantic prompts. At inference, GPT-5.4 iteratively revises the prompts, and a no-reference metric selector chooses among ten generated candidates. Experiments include benchmark comparisons, component ablations, and human evaluations.

Critical strengths

The static model achieves competitive results, including the lowest FID among the compared baselines on all three paired benchmarks. The human preference and fidelity assessments provide useful evidence beyond automatic quality metrics.

Critical weaknesses
  • I am not convinced that SR needs to be reformulated as diptych completion. The input and target already have spatial correspondence, so asking the model to reconstruct the scene in another panel seems unnecessarily indirect. It also enlarges the canvas. What is gained by doing this? Table 2 does not settle the question: it reports no-reference scores on RealLR200, without showing whether this formulation gives better reconstruction or lower cost than directly using the aligned LQ input.

  • The agent experiments are missing a basic control: keep the original prompt, sample ten times with different seeds, and apply the same selector. The current comparison gives the agent ten candidates but the static baseline only one. The raw-round improvements provide some evidence for prompt refinement, but they do not answer whether it beats ordinary sampling at the same candidate budget. This matters given the cost, nine GPT-5.4 calls and 750.7 seconds per image, versus 7.718 seconds for the static model. I would also like to see whether one prompt rewrite, a smaller agent, or a single call to an image enhancement model could achieve comparable results.

Questions for authors
  • What advantage does diptych completion offer over directly using the spatially aligned LQ input? Is there a matched comparison of reconstruction quality and computational cost?

  • What happens if the original LLaVA prompt is kept fixed and ten outputs are generated with different seeds, then ranked by the same selector? Does iterative prompt refinement still provide a clear gain?

  • Are nine GPT-5.4 calls necessary? How do quality and total runtime compare with rewriting the prompt once, using a smaller agent, or making a single call to an image enhancement model?

Comments on the paper

1 comment
Area_Chair_vHCYOpenReview ↗

Author-Reviewer Discussion

Dear Reviewers,

Thank you for your time and thoughtful reviews. With the author–reviewer discussion period ending on November 18 (AoE), please review the authors' rebuttal and share your updated assessment. Specifically, let us know whether your main concerns have been resolved, and if not, which points remain open.

Your input before the deadline is greatly appreciated!

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.