- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper propose Failure-Aware Image Restoration (FAIR) to address the “lastmile” problem in image restoration, which is to correct localized restoration failures using region-specific guidance. The method constructs a dataset of approximately 67K annotated failure regions, categorizes failures into four types, and incorporates textual instructions, reference images, or geometric contours to guide restoration. Based on FLUX.2-klein-9B, FAIR employs a two-stage training curriculum and a region-weighted reconstruction objective. Extensive experiments show that the proposed approach resolves common failure cases and improves perceptual quality, bridging the gap between technical accuracy and practical usability.
Critical strengths
-
The problem framing is useful and the system is complete. Localized failures in otherwise good restorations are a real deployment issue, and the paper delivers an end-to-end pipeline (annotation, region control, multi-modal guidance, single-step inference) rather than a single component.
-
The paper does many of experiments. An equal-information comparison with all conditions removed (Table 10), per-tier breakdowns (Table 9, 11), a cascade baseline (Table 12), run-to-run variance (Table 17), and robustness to smaller condition models and mismatched references (Tables 15, 16).
Critical weaknesses
- Limited methodological novelty.
Most of the proposed framework's components closely resemble existing methods:
- Region-specific restoration and mask-based control: InstructRestore [1] already introduces region-customized restoration using textual instructions, region masks, and spatially differentiated enhancement.
- Reference-guided restoration: ReFIR [2] uses retrieved high-quality reference images to mitigate hallucinations and improve restoration fidelity.
- Region-adaptive generative priors and one-step restoration: CODSR [3] introduces region-adaptive prior activation, text-matching guidance, and a one-step diffusion framework.
- Adaptive reference conditioning: AdaRefSR [4] explicitly regulates reference information based on its relevance to the degraded input.
- Pretrained generative backbone with adversarial fine-tuning: HYPIR [5] already employs a pretrained diffusion model with adversarial training, without requiring a diffusion loss.
Although FAIR combines these components with failure-type prediction and guidance selection, the individual techniques are largely established. More importantly, the paper does not adequately demonstrate whether its failure-aware selection mechanism offers substantial improvements over simpler, failure-type-agnostic guidance strategies.
- Excessive system complexity and questionable practical value.
FAIR combines multiple large pretrained models, including Qwen3-VL, SAM3, DINOv3, CLIP, and FLUX.2-klein-9B, together with a large reference database and a multi-stage condition-generation pipeline. While each component serves a purpose, the overall system appears heavily dependent on assembling existing models rather than introducing a compact and efficient restoration mechanism.
This complexity raises concerns about practical deployment, particularly in terms of computational cost, memory requirements, implementation overhead, and inference latency. Although the paper emphasizes single-step restoration, this does not necessarily translate into an efficient end-to-end pipeline when failure detection, segmentation, and reference retrieval are included.
- Unclear contribution of the proposed method versus the substantially larger pretrained backbone.
FAIR employs FLUX.2-klein-9B as its core restoration backbone, substantially larger than the approximately 0.87B SD2.1-based backbones and 2.6B SDXL-based backbone used by several competing methods. Furthermore, FAIR incorporates additional large models for text encoding, failure detection, segmentation, and reference retrieval, making its overall system considerably larger.
This raises a fundamental question: Are the reported restoration improvements primarily attributable to the proposed failure-aware framework, or to the substantially stronger pretrained generative backbone and larger overall system?
Although the LQ-only ablation demonstrates additional improvements from region and reference guidance, it does not isolate the effect of backbone capacity. The comparisons therefore do not sufficiently control for model size or computational budget.
Moreover, despite this considerable model capacity, FAIR still underperforms AdaRefSR on two important full-reference restoration metrics in Table 1: PSNR (20.91 vs. 21.23) and SSIM (0.558 vs. 0.561). These results further weaken the claim of consistently superior restoration quality.
References
[1] Liu, S., et al. "InstructRestore: Region-Customized Image Restoration with Human Instructions." NeurIPS, 2025.
[2] Guo, H., et al. "ReFIR: Grounding Large Restoration Models with Retrieval Augmentation." NeurIPS, 2024.
[3] Chen, H., et al. "Bridging Fidelity-Reality with Controllable One-Step Diffusion for Image Super-Resolution." CVPR, 2026.
[4] Wang, Y., et al. "Trust but Verify: Adaptive Conditioning for Reference-Based Diffusion Super-Resolution via Implicit Reference Correlation Modeling." ICLR, 2026.
[5] Lin, X., et al. "Harnessing Diffusion-Yielded Score Priors for Image Restoration." ACM Transactions on Graphics, 2025.
Questions for authors
-
Given the substantial overlap with existing methods such as InstructRestore, ReFIR, CODSR, AdaRefSR, and HYPIR, what is the key technical innovation that distinguishes FAIR beyond integrating existing components?
-
Considering that FAIR integrates multiple large pretrained models (Qwen3-VL, SAM3, DINOv3/CLIP, and FLUX.2-klein-9B), what are the total parameter count, GPU memory requirements, and end-to-end inference latency, including failure detection, segmentation, and reference retrieval? How do these costs compare with existing restoration methods, and are the performance improvements sufficient to justify the additional complexity?
-
To what extent are FAIR's improvements attributable to its substantially larger FLUX.2-klein-9B backbone rather than the proposed failure-aware mechanism? Could the authors provide backbone-matched comparisons or evaluate FAIR with a smaller restoration backbone?
-
Despite using a larger backbone, FAIR achieves lower PSNR and SSIM than AdaRefSR in Table 1. How do the authors interpret this results, particularly in relation to the increased computational cost?
7 comments
Official Comment by Authors
Q3. Backbone capacity
With model capacity fixed, failure-aware guidance improves restoration. The last three rows compare inference settings with the same trained FAIR checkpoint and the same FLUX.2-klein-9B [6] backbone. The first uses only the degraded image and the generic prompt “Restore,” with no region mask or visual reference. The second predicts the region mask and repair instruction from the degraded image and uses a crop from that image to retrieve the visual reference. The third still predicts the region mask from the degraded image, but uses the annotated instruction and a clean ground-truth crop for reference retrieval.
| Configuration | Training and inference setting | FID ↓ | In-region LPIPS ↓ |
|---|---|---|---|
| Stage-1 restoration (Table 2a) | Same 9B backbone, trained for general restoration | 80.38 | — |
| Final FAIR, generic instruction (Table 10) | Final checkpoint, LQ input and “Restore” prompt | 71.79 | 0.0353 |
| Final FAIR, fully LQ-driven (response to 529s) | Same checkpoint, region, text, and retrieval query derived from LQ | 67.97 | 0.0351 |
| Final FAIR, annotation-assisted (Table 1) | Same checkpoint, LQ region prediction, annotated text, and GT-based retrieval query | 63.28 | 0.0319 |
Compared with the generic prompt, the fully LQ-driven setting reduces FID from 71.79 to 67.97, in-region LPIPS from 0.0353 to 0.0351, and out-of-region LPIPS from 0.2512 to 0.2486, while MUSIQ rises from 71.04 to 71.21. With the checkpoint fixed, the gain comes from the change in guidance.
FAIR targets last-mile restoration. As shown in Q1, existing restorers and the tested restorer-editor cascade can leave local failures unresolved, such as blurred seals or incorrect license-plate characters. If a smaller backbone struggles to restore the overall image, that comparison cannot answer whether FAIR improves local failures in otherwise successful restorations. All ablations in the paper hold the FLUX.2-klein-9B backbone fixed. This comparison measures the benefit of failure-aware guidance at fixed model capacity.
Q4. Fidelity relative to AdaRefSR
PSNR and SSIM cannot fully measure perceptual quality in generative restoration. Both reward agreement with one GT image, so a plausible reconstruction of ambiguous details can score lower [10]. We include both for completeness. We assess whether local repairs look better and preserve their surroundings using regional perceptual distances, visual comparisons, and blinded human judgments, following the human-centered approach of Yu et al. [11].
| Synthetic benchmark (Table 1) | PSNR (dB) ↑ | SSIM ↑ | In-region LPIPS ↓ | Out-of-region LPIPS ↓ | FID ↓ |
|---|---|---|---|---|---|
| AdaRefSR | 21.23 | 0.561 | 0.0368 | 0.277 | 81.93 |
| FAIR | 20.91 | 0.558 | 0.0319 | 0.243 | 63.28 |
FAIR lowers both regional LPIPS scores and FID. With the same references, it restores a coherent face in the pink-hat example and recovers the “CAFE” lettering, while AdaRefSR leaves these local failures unresolved (Figs. 11–12). FAIR's pairwise win rates over AdaRefSR, derived from blinded rankings, are 94.6% for overall quality, 93.2% for target correction, and 93.4% for preservation outside the target (Table 13). Each type of metric captures only part of restoration quality and can rank methods differently from human observers. Regional perceptual metrics, visual comparisons, and the blinded study together show a clear overall advantage for FAIR over AdaRefSR in the cases tested.
References
[10] The Perception-Distortion Tradeoff. Blau Y, Michaeli T. CVPR 2018.
[11] Evaluation of Visual Processing Should Be Human-Centered, Not Metric-Centered. Yu F, Hu J, You Z, et al. NeurIPS 2026.
Thank you for the detailed rebuttal and additional experiments. These results address some of my concerns. However, several important issues remain.
1. Methodological Novelty. The rebuttal clarifies FAIR's distinction from existing methods, but does not fully establish the contribution of failure-aware guidance selection. Table 11 demonstrates the benefit of visual guidance, rather than the necessity of failure-type-specific selection. Furthermore, the new ablation responding to Reviewer 529s shows that removing explicit region control while retaining the instructions and references produces nearly identical results (in-region LPIPS: 0.0316 vs. 0.0319).
2. Benchmark and Evaluation Fairness. The authors clarify in their response to Reviewer 529s that Table 1 uses GT crops for reference retrieval and annotated instructions. Although the reference-based baselines receive the same references, this setting does not represent fully automatic restoration. Under the fully LQ-driven setting, FID increases from 63.28 to 67.97, and in-region LPIPS from 0.0319 to 0.0351. This distinction should be emphasized when interpreting the reported improvements.
3. System Complexity and Backbone Capacity. The complete pipeline remains substantially more complicated than most competing methods. The fixed-backbone ablations demonstrate that additional guidance improves FAIR, but do not resolve whether the advantages over competing methods arise primarily from the larger pretrained backbone. The lower PSNR and SSIM relative to AdaRefSR in Table 1 further highlight the importance of evaluating the performance and complexity trade-off.
Overall, my primary concerns regarding methodological novelty, benchmark fairness, and backbone capacity remain insufficiently resolved. I therefore maintain my original rating of 2 (Weak Rejection).
Official Comment by Authors
Thank you for your careful review and for highlighting the importance of local restoration failures. Your feedback helped us clarify FAIR's contribution, inference costs, and evaluation.
Q1. Contribution and failure-aware selection
We thank you and Reviewer pVYz for recognizing the value of studying local restoration failures. FAIR's core contribution is to identify which local restoration failures need an external reference and use that distinction to build the training data and design a single-step restoration model. We collected approximately 67,000 failure regions from existing restorers' outputs (Sec. 3.1). At inference, FAIR predicts the failure type, selects a reference crop, contour, or text instruction, and applies that guidance to the corresponding region in a single restoration pass (Secs. 3.2 and 4.3).
ReFIR [1] retrieves reference images, and AdaRefSR [2] regulates their use. InstructRestore [3] uses instructions to locate regions and adjust enhancement strength. CODSR [4] adapts generative priors by region, and HYPIR [5] is a single-step restoration baseline. FAIR addresses which regions need a reference in the first place. Table 2 evaluates the training design. Table 11 shows that omitting visual guidance raises in-region LPIPS by 68% for OCR/ID and 1.6% for Subjective Feeling, demonstrating that the benefit varies by failure type.
In Fig. 7, several baselines leave the seal blurred or generate an unrelated emblem. Fig. 9 shows FAIR applying distinct references to two regions in one pass and restricting a reference-driven change to the selected object. The HYPIR-to-FLUX.2-klein-9B [6] cascade in Fig. 15 loses local cues in the first pass and copies incorrect plate characters in the second. Simply chaining the two models left last-mile failures unresolved in this test. Relative to the cascade, FAIR lowers in-region LPIPS from 0.0482 to 0.0330 and raises PSNR from 17.28 to 20.93 dB (Table 12). In the blinded study, FAIR ranks in the top two for overall quality, target correction, and preservation outside the target in 79.5%, 77.5%, and 78.5% of evaluations, respectively (Table 3). Its pairwise overall-quality win rates against the cited methods, including ReFIR+SeeSR [7] and ReFIR+SUPIR [8], range from 84.5% to 99.4% (Table 13).
Q2. End-to-end cost
FAIR restores in one pass. Users can supply the region, instruction, and reference to bypass the automatic guidance stages.
Detection uses merged LoRA and a static KV cache. At one GPU query per image, automatic guidance adds 1.08 s to the 3.16 s measured on WR-SR [9]. The stage-summed estimate is 4.24 s/image, about 26% from guidance. Restoration peaks at 41 GB.
GPU retrieval uses DINOv3-L/16, FAISS (4.1M tiles, search_k = 64), CLIP-B/32 (top eight), SAM3, and Qwen3-VL-4B. Per query, search takes 3 ms, loading 0.11 s, recropping and verification 0.06 s each, and embedding and reranking 0.02 s.
On WR-SR [9], FAIR trades speed for better LPIPS and SSIM (Table 8). With identical references, FAIR repairs a face and recovers "CAFE" lettering missed by AdaRefSR (Figs. 11-12). Blinded pairwise win rates are 94.6%, 93.2%, and 93.4% for overall quality, target repair, and preservation outside the target (Table 13). These gains justify the optional cost for users who prioritize local repair. We will report the timing assumptions and trade-off.
References
[1] ReFIR: Grounding Large Restoration Models with Retrieval Augmentation. Guo H, Dai T, Ouyang Z, et al. NeurIPS 2024.
[2] Trust but Verify: Adaptive Conditioning for Reference-Based Diffusion Super-Resolution via Implicit Reference Correlation Modeling. Wang Y, Wan Y, Zheng S, et al. ICLR 2026.
[3] InstructRestore: Region-Customized Image Restoration with Human Instructions. Liu S, Ma J, Sun L, et al. NeurIPS 2025.
[4] Bridging Fidelity-Reality with Controllable One-Step Diffusion for Image Super-Resolution. Chen H, Chen J, Pan J, Dong J. CVPR 2026.
[5] Harnessing Diffusion-Yielded Score Priors for Image Restoration. Lin X, Yu F, Hu J, et al. ACM Transactions on Graphics, 2025.
[6] FLUX.2 [klein] 9B Base. Black Forest Labs. Hugging Face 2026.
[7] SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution. Wu R, Yang T, Sun L, et al. CVPR 2024.
[8] Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild. Yu F, Gu J, Li Z, et al. CVPR 2024.
[9] Robust Reference-Based Super-Resolution via C2-Matching. Jiang Y, Chan KCK, Wang X, et al. CVPR 2021.