- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The authors proposed AutoRestore, which focuses on alleviating the problems in iterative image restoration. The method combines VLM-based assessment with instruction-guided spatial modulation and history-grounded restoration. The proposed regional instructions and historical KV cache enable targeted refinement while preserving visual evidence across iterations Comprehensive evaluation, including real-world and synthetic benchmarks, ablation studies, efficiency comparisons, and a user study, shows competitive fidelity-perceptual quality trade-offs and better performance compared to existing related methods.
Critical strengths
- The proposed spatial modulation links semantic assessment and the restoration process. It allows VLM to influence spatially varying restoration behavior. This is a useful design for handling spatially variant degradation and restoration requirements within an image.
- The historical cache alleviates a practical limitation in iterative restoration.
- Comprehensive evaluation experiments are conducted, covering real-world and synthetic benchmarks, separate degradation settings, full-reference and no-reference metrics, and a user study.
Critical weaknesses
-
The current experiments do not sufficiently isolate the contribution of instruction-guided spatial control. Table 1(b) evaluates residual-scale prediction, but does not establish whether local instructions produce spatially accurate corrections or preserve untargeted regions. Comparisons with scene descriptions, global restoration instructions, and spatially uniform modulation are missing.
-
The ablation studies of historical memory do not separate original-input anchoring from accumulated intermediate evidence. The improvement may largely arise from retaining the original image. Without an original-plus-current-reference baseline, the necessity of storing all intermediate results remains unclear. The paper also does not examine whether incorrect details in cached results can reinforce errors in later rounds.
-
The contribution of adaptive iteration and stopping is not sufficiently isolated. Table 1(a) shows that the full model maintains similar PSNR and slightly higher LPIPS than the single-round variants while improving several no-reference metrics. Although Fig. 4 provides qualitative examples, comparisons with fixed-round and fixed-instruction variants are needed to distinguish the benefits of adaptive decisions from those of additional restoration rounds. Fig. 12 provides a pairwise comparison prompt, but the maximum iteration count and the rules for translating its judgments into continuation, termination, and output selection remain insufficiently specified.
-
The impact of the training–inference instruction gap is unquantified. Although the intermediate results used for training are generated with LQ-only instructions, the instructions used for subsequent optimization remain GT-assisted. The effect of this discrepancy on uncertainty estimation and restoration quality is not analyzed. It is unclear whether the validation instructions in Table 1(b) are GT-assisted or LQ-only. Evaluating performance with LQ-only instructions would help clarify the impact of this gap.
Questions for authors
Several experiments and evaluations would help to improve this paper, the authors could have a small-scale validation.
- Evaluate spatial controllability by varying regional instructions, and observe the changes on performance. It would help isolate the contribution of instruction-guided spatial modulation.
- Compare the impact of current-only, original-plus-current, and full-history conditioning. This would clarify whether retaining intermediate results provides benefits beyond preserving the original input.
- A fixed-instruction variant, with per-round quality and runtime measurements, would help distinguish the contributions of instruction updates and stopping decisions.
3 comments
[Part 2] Official Comment by Authors
W.4 Training–Inference Instruction Gap
We use GT-assisted restoration instructions during training to teach the restoration model the correct correspondence between image semantics and restoration actions. This is consistent with existing image restoration methods that use LLaVA to generate descriptions from GT images when preparing training data.
To mitigate the instruction gap, we discourage the VLM from describing overly specific details that would be difficult to infer from the LQ image alone. During inference, we actively encourage the VLM to make reasonable inferences about the underlying clean image content, guided by the visual cues and scene context available in the LQ image.
GT images are not provided to the VLM or restoration model during inference. We will explicitly clarify the instruction source used for validation in Table 1(b). We acknowledge that the above design rationale alone does not quantify the remaining gap, and we will add comparisons involving LQ-only training instructions to assess its effects on both uncertainty estimation and final restoration quality.
The authors are recommended to conduct small experiments and report the corresponding results of global and regional instructions on the same locally degraded images.
[Part 1] Official Comment by Authors
We thank the reviewer for the thoughtful suggestions on isolating the contributions of our design choices. Below, we clarify the roles of spatial instructions, reference caching, and adaptive iteration, and describe additional experiments to examine spatial controllability, historical conditioning, and the training–inference instruction gap.
W.1 Validation of Instruction-Guided Spatial Control
We thank the reviewer for highlighting this limitation. We agree that evaluating residual-scale prediction in Table 1(b) alone does not sufficiently establish whether regional instructions produce spatially accurate corrections while preserving untargeted regions.
In the revised manuscript, we will add comparisons using scene descriptions, global restoration instructions, regional restoration instructions, and spatially uniform modulation. For a fixed degraded input, we will vary the regional instructions and examine the corresponding uncertainty maps and restoration results. We will evaluate changes in both targeted and untargeted regions to better isolate the contribution of instruction-guided spatial control.
W.2 Original-Input Anchoring and Reference Caching
We thank the reviewer for raising this point. Our original design retains the original LQ input together with intermediate restoration results to preserve reference information across rounds. However, we observe that subsequent rounds typically introduce smaller changes than the first restoration round, and retaining only the original LQ input and the most recent restoration result achieves comparable quality to retaining the full history. We will therefore adopt this simpler reference strategy and revise the relevant sections to distinguish the role of original-input anchoring from that of accumulated intermediate references. In the manuscript, Table 1(a) shows the importance of retaining reference information for fidelity preservation during multi-round restoration. In addition, we benchmarked the inference time and GPU memory usage of full joint attention, reference caching, and LQ-only caching. For subsequent rounds, the caching mechanism reduces inference time from 6.24 s to 2.50 s, approximately a 60% reduction compared with full joint attention.
We will revise the relevant sections to distinguish original-input anchoring from accumulated historical evidence and avoid implying that retaining all intermediate results is necessary. We will also supplement the conditioning comparisons described in Q2.
W.3 Instruction Updates and Adaptive Stopping
We thank the reviewer for identifying this important issue. We have examined fixed-instruction variants and observed that updating instructions can help correct errors in scenes containing complex details, such as text.
For example, when the initial degradation is severe, the VLM may be unable to identify the exact text and provide only a generic instruction such as “recover the text.” Reusing this instruction in subsequent rounds can leave the text incorrectly restored. When instructions are regenerated, the VLM can jointly examine the original LQ input and the intermediate restoration result. In the examples we examined, this enabled it to provide an updated instruction specifying the correct text, which subsequently guided the restoration model to correct the output. We will include these visual comparisons in the revised manuscript.
Our stopping strategy combines the VLM’s adaptive decisions with explicit iteration bounds: a minimum of two and a maximum of four restoration rounds. The observed iteration distributions are reported below.
These datasets share the same GT images but use real-world and synthetic degradations, respectively. Their iteration distributions and average numbers of rounds are similar. We also observe that severely degraded samples do not consistently require more rounds. We believe this may be because severe degradation can erase substantial image detail, leaving limited information that can be reliably recovered.
We will include these results and clarify how the VLM’s judgments determine continuation, termination, and output selection. We will also supplement the controlled per-round comparisons described in Q3.