acceptodds
Under review as a conference paper at ICLR 2027

Query-to-Competence: Usage-Time Continual Adaptation for Real-World Image Restoration

Abstract

Real-world image restoration must handle compound and continually changing degradations. We consider a deployment setting that combines an updateable self-hosted model, referred to as the Student, with a hosted image-editing service, referred to as the Teacher. In a service receiving many complex restoration requests, the Teacher can resolve difficult cases, but repeated calls incur provider cost. Our goal is to progressively reduce hosted-service dependence by turning quality-triggered usage-time Teacher calls into reusable Student competence. We propose Query-to-Competence (Q2C), a usage-time adaptive restoration system for this purpose. Q2C constructs a Structured Restoration Brief (SRB) describing the degradations, restoration intent, and content-preservation constraints. The Student processes each request first, and the Teacher is queried only when the Student output fails a fixed quality criterion. Curated Teacher outputs are accumulated for Replay-Distill Adaptation (RDA), while Guarded Student Promotion (GSP) determines whether an updated candidate enters deployment. Across seven actual usage sessions, the deployed Student handles an increasing fraction of requests without Teacher fallback. On the fixed SDCR-200 evaluation set, Student-only coverage increases from 24.0% to 81.5%, showing that selected Teacher feedback can be retained as reusable Student competence.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Real-world image restoration, usage-time adaptation, continual learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_EZJJOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes to couple a self-hosted student with a server-side teacher with the goal of gradually reducing reliance on the teacher by distillation. More precisely, for each request a VLM perception module writes a structured restoration brief, then the student restores the image. A VLM jusdge scores the result and depending on the score it's either passed to the teacher or not. The student is adapted on such data using LoRA.

Critical strengths
  • The system is straight-forward and practical, with an overall well structured manuscript
Critical weaknesses
  • The paper is overall of low novelty. All components are existing of-the-self components used on what is a quite standard data mining/filtering technique. Even at a system level there is a vast corpus of works that study and tackle the problem either in the same or very similar way. Some examples: [1] Cache & Distil: Optimising API Calls to Large Language Models, 2023, [2] Cache me if you Can: an Online Cost-aware Teacher-Student framework to Reduce the Calls to Large Language Models, 2023, [3] Online Cascade Learning for Efficient Inference over Streams, 2024, etc. While the examples I've shared are primarily for LLM, there is no conceptual difference between the two at a method level. Similar systems were build for image system too (e.g. DiffServe, AMS, JITNet, or the already cited RealRestorer, which is very similar as a system).

  • Following on the above, the SRB-guided routing is also incrementally different from GGT-100K.

  • Agentic restoration with perception, judge and fallback are largely not covered (e,g: AgenticIR (ICLR 2025), HybridAgent (CVPR 2026), JarvisIR (CVPR 2025), EvoIR-Agent, etc). The paper should cite and compare with them, mentioning how they differ.

  • Multiple SOTA comparisons and baselines are missing. No test on the GGT-100K dataset, no comparison with RealRetorer etc. Moreover, no dedicated restoration models are compared (just image editors). For a more convincing comparison, the apper should cover the follow class of methods: Composite-degration restoration, All-in-one restoration (e.g. InstructIR, PromptIR, AdaIR, MoCE-IR, etc), Generative real-world restoration (e.g: LucidFlux, etc) and agentic restoration (e.g: AgenticIR)

-No human evaluation and no real ground truth used for eval (they are metrics against GPT-Image-2 outputs I believe).

  • The setup appears partially flawed: (1) the session boundary coincides with the update triger, hence curation accepts esentially everything. (2) there is a selection bias toward the teacher and (3) a self-preference bias due to how the judges are defined.

  • The continual-learning doesn't appear to offer notabel perforamnce gains. Moreover the GSP evidence is quite thin.

  • Very limited analysis

Questions for authors

Please see the section above as it covers the main points i would like to ask about. Some additional questions:

  • Do the results hold under a non-OpenAI Judge, and in a human study?
  • What is the Judge's agreement with humans at the 0.7 threshold?
  • How does it compare with a GGT-100K-trained Qwen-Image-Edit, RealRestorer, and OneRestore/InstructIR on a real-GT benchmark (e.g. CDD-11) and on RealIR-Bench?
  • How does Q2C compare against (a) offline fine-tuning on the same pairs and (b) random or uncertainty-based querying at the same Teacher budget?
Reviewer_iW5aOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces Query-to-Competence (Q2C), a method to train a student model for image restoration tasks with a fallback ability to a teacher model. This fallback enables the student to learn from the teacher and improve itself. The continual improvement leads to a smaller portion of samples triggering the fallback to the teacher model. The method implements multiple steps towards this goal, starting from a Structured Restoration Brief (SRB) which describes the degradations, restoration intent, and content-preservation constraints. The SRB provides a more explicit approach to the ill-posed problem of image restoration. Furthermore, Replay-Distill Adaptation (RDA) is introduced as a means to knowledge distillation from the previous student model to mitigate forgetting under a changing set of degradations, while the teacher creates a target for the student using the SRB when the fallback is triggered and a Judge decides whether to store the teacher output or to discard it. Lastly, Guarded Student Promotion (GSP) is implemented to determine whether a student candidate is promoted into deployment. The Restoration-Preservation Score (RPS) is introduced which drives the routing procedure between student and teacher models. In the experiments, at a routing threshold of 0.70.7 RPS the Student-only coverage increases from 24%24\% to 81.5%81.5\% on a fixed SDCR-200 evaluation set.

Critical strengths
  • The paper addresses a real-world problem of decreasing dependence on models which could be expensive, hosted, and from third-parties.
  • The categorization of the approach into routing, curation, retention and promotion gives a clear view of the approach and makes the paper easy to follow.
  • The gain in student-only coverage is substantial under the paper's evaluation protocol, showing that the method has potential to be applied in real-world scenarios.
Critical weaknesses
  • The method introduces a mixture of different components that already exist. The contribution is therefore the combination of these existing components such as LoRA finetuning, self-distillation, experience replay, and champion-challenger gating for deployment, for the image restoration objective.
  • The RPS, and the Judge model, create a circular system where the judge chooses which output from the teacher is used for training the student, and the student is trained to mimic the teacher output. The judge is then used again to measure the success of the student using the RPS, without using a set of samples with available ground truth clean samples for content preservation nor using human evaluation for a qualitative measure.
  • Some missing baseline such as:
    • an upper bound with joint finetuning of the LoRAs.
    • randomly falling back to the teacher model with the same budget regardless of the judge or the RPS threshold.
    • the difference between sequential finetuning and RDA is relatively small (3/2003/200 images), while using experience replay only gives mixed performance in comparison to finetuning, an experiment investigating the forgetting mitigating strategy would strengthen the motivation for using RDA.
    • an ablation over the curation threshold τcur\tau_{cur} of the target produced by the teacher, the paper only mentions that 1,3981,398 pairs were calculated by the teacher compared to the full 4,2544,254, how many of these pairs were retained in the buffer and how many were dropped? Furthermore, τcur\tau_{cur} value is not stated in the paper.
Questions for authors
  • How do the authors differentiate the student-only coverage gain between better restoration and learning to imitate the features that the judge favors?
  • Can the authors report results on a benchmark with ground-truth clean images?
  • How does the final Student compare to an upper bound that fine-tunes a single LoRA jointly on all curated teacher pairs at once?
  • How does the method compare to a baseline that falls back to the teacher on randomly selected requests with the same budget, independent of the Judge and the RPS threshold?
  • Given the small difference between sequential fine-tuning and full RDA (3/2003/200 images in AC@0.70.7) and the mixed results of replay alone, can the authors provide a direct analysis of forgetting, for example per-domain SDCR-200 coverage at each stage?
  • Again given the small differences and the mixed results in table 2, are these results consistent across multiple runs or seeds?
  • What value of τcur\tau_{cur} was used? How many of the 1,398 Teacher outputs were accepted into the buffer, and how many were discarded? How sensitive are the results to the choice of τcur\tau_{cur}?
Reviewer_34LfOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

Q2C uses a locally deployed Student to process restoration requests and queries a hosted Teacher when the Student fails a fixed quality threshold. Selected Teacher outputs are used for subsequent LoRA adaptation. Historical replay and previous-Student distillation support retention, while a shadow-plus-history evaluation determines whether a candidate replaces the deployed model. The paper evaluates seven usage sessions and a fixed, source-disjoint set of 200 images.

Critical strengths
  1. Several parts of the evaluation protocol are handled carefully. The data are split by source-image identity, SDCR-200 remains fixed and excluded from adaptation and promotion, and the appendix explains how cached Teacher outputs are made available under the selective policy.
  2. The rejected S6 update is a useful example of the promotion mechanism operating on an actual training trajectory. The paper also clearly acknowledges that Teacher-target fidelity is not fidelity to clean ground truth and that its cost accounting excludes some system components.
Critical weaknesses
  1. The main competence measure still relies on the same Judge-based scoring framework used throughout adaptation. RPS governs routing, curation, promotion, and the principal evaluation. Source-disjoint test images prevent direct data leakage, but do not remove systematic errors shared by these decisions. Improvements in no-reference metrics are useful supporting evidence, yet they do not directly verify identity, text, or local structural preservation. Likewise, PSNR against Teacher outputs measures agreement with the Teacher. The increase from 24.0% to 81.5% AC therefore establishes improved performance under the fixed Judge criterion more directly than it establishes independently verified restoration quality.
  2. The comparisons do not isolate the benefit of selective querying and the full continual-learning design. In Table 2, Sequential adaptation already reaches 80.0% AC, compared with 81.5% for Full RDA. Most of the improvement over initialization thus comes from Teacher-supervised adaptation. There is no equal-query-budget comparison with random sampling, a fixed non-adaptive selection policy, or offline distillation. The RDA ablation also emphasizes the final endpoint rather than stage-wise performance on historical domains, making the retention benefit difficult to assess.
Questions for authors
  1. Which models and versions implement Perception and the Judge? Can the AC improvement be checked through blinded human evaluation or an independent evaluator, with particular attention to text, identity, and geometry preservation?
  2. Under the same Teacher-query budget and training budget, how does Q2C compare with random or fixed non-adaptive request selection?
  3. Could you provide stage-wise, per-domain results for the RDA variants? Do all variants use the same promotion procedure and the same extended training schedule after rejection?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.