acceptodds
Under review as a conference paper at ICLR 2027

TaskIR: Task-Driven Image Restoration via Degradation Adaptation and Task Feedback

Abstract

Task-driven image restoration aims to improve both image quality and downstream task performance. However, existing methods predominantly focus on single degradation type and struggle to handle the diverse degradations encountered in real-world scenarios. Different degradations impose distinct restoration demands, and insufficient restoration may leave residual degradations and artifacts that impair object boundaries and semantic cues, thereby compromising downstream task performance. To address these challenges, we propose TaskIR, a two-stage task-driven unified image restoration framework that integrates degradation-adaptive restoration with task feedback refinement. In Stage I, a Degradation Representation Module (DRM) extracts degradation representations, enabling a Degradation-Guided Transformer Block (DGTB) to dynamically modulate feature transformations for adaptive restoration. In Stage II, a Task-to-Restoration Feedback Generation module (TRFG) transforms heterogeneous task features into restoration feedback by modeling task-representation discrepancies associated with the current restoration. Subsequently, a Selective Task Feedback Refinement module (STFR) assesses feedback relevance and selectively refines intermediate restoration features to mitigate interference with well-restored content. Extensive experiments demonstrate that TaskIR achieves competitive restoration quality and downstream task performance across diverse degradations and tasks.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Task-Driven Image Restoration, Degradation Adaptation, Task Feedback

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_BuokOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes TaskIR, a two-stage task-driven image restoration framework for handling multiple degradations while improving downstream vision performance. In Stage I, a Degradation Representation Module extracts global and spatial degradation information to modulate a shared restoration network. In Stage II, features from classification, semantic segmentation, and object detection networks are converted into task feedback. A prototype-based feedback weighting module and a selective refinement module then use these signals to further modify the restoration features. Experiments are conducted on ImageNet-1K, Cityscapes, and PASCAL VOC2012 under eight synthetically generated degradation types, evaluating both restoration quality and downstream task performance.

Critical strengths
  1. The paper studies an important trade-off in task-driven restoration. It aims to improve both restoration quality and downstream task performance under multiple degradation types, rather than optimizing either objective in isolation.
  2. The experimental evaluation is relatively comprehensive. The paper evaluates image classification, semantic segmentation, and object detection across three datasets and eight degradation types. It also reports stage-wise comparisons and ablations of the task-feedback modules, allowing the effects of task adaptation on restoration and downstream performance to be examined.
Critical weaknesses
  1. The inference cost is substantial. The full TaskIR pipeline appears to require running three additional downstream networks, i.e., ResNet-50, SegFormer, and YOLOv8, to generate task feedback during inference. This substantially increases the parameter count and FLOPs compared with the Stage-I restoration network. The paper does not clarify whether all three task networks must be executed when only one downstream task is required, nor does it report the total latency and memory cost of the complete three-task inference setting.
  2. The interaction and potential conflict among different task feedback signals are not sufficiently analyzed. The method combines feedback from classification, segmentation, and detection through prototype matching, Sinkhorn-based relevance estimation, and selective refinement. However, the paper does not directly analyze whether these task objectives provide complementary or conflicting restoration directions. It also does not show whether improving one task can degrade another task or the basic restoration quality.
  3. The evaluation is restricted to synthetic degradations. All experiments are conducted by applying eight predefined synthetic degradations to clean images from ImageNet-1K, Cityscapes, and PASCAL VOC2012. Although the benchmark covers multiple degradation types and tasks, it does not evaluate real camera noise, natural blur, real haze, low illumination, or naturally occurring compound degradations. Therefore, the experiments characterize the method under controlled synthetic degradations, while its behavior on real-world degradations remains unclear. Comparisons on real-degradation benchmarks or naturally captured degraded images would be important for assessing the practical claims.
  4. The comparison does not include sufficiently direct task-driven restoration baselines. Most compared methods are general all-in-one restoration methods rather than methods that explicitly use downstream task losses, task features, or task feedback to guide restoration. The paper should include representative task-driven restoration baselines under the same multi-degradation and downstream-task settings.
Questions for authors
  1. Does inference require running ResNet-50, SegFormer, and YOLOv8 simultaneously, even when only one downstream task is needed? Please report the total latency, memory usage, and FLOPs of the complete three-task inference setting.
  2. How do the classification, segmentation, and detection feedback signals interact during training and inference? Please report their task-specific weights and analyze whether improving one task affects the other tasks or the basic restoration quality.
  3. Can the authors evaluate TaskIR on real-degradation benchmarks or naturally captured degraded images, in addition to the current synthetic degradation setting?
  4. Can the authors compare TaskIR with representative task-driven restoration methods that use downstream task losses, task features, or task feedback under the same multi-degradation and downstream-task protocols?
Reviewer_SmJAOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TaskIR, a two-stage task-driven all-in-one image restoration framework that integrates degradation-adaptive restoration with downstream task feedback. In Stage I, a Degradation Representation Module (DRM) extracts global and spatial degradation information to guide feature modulation through Degradation-Guided Transformer Blocks (DGTB). In Stage II, Task-to-Restoration Feedback Generation (TRFG) transforms heterogeneous task features into restoration feedback, while Selective Task Feedback Refinement (STFR) adaptively incorporates this feedback into the restoration process.

Critical strengths
  1. The paper addresses a relevant problem by integrating multi-degradation image restoration with classification, semantic segmentation, and object detection. 
  2. The two-stage optimization strategy separates degradation-adaptive restoration from task feedback refinement, providing a reasonable approach to balancing image fidelity and downstream task performance. 
  3. The proposed TRFG and STFR modules provide a structured mechanism for adapting heterogeneous task representations and regulating their contributions to restoration features.
Critical weaknesses
  1. How does TaskIR differ methodologically from existing multi-degradation, task-driven restoration methods, particularly approaches that already incorporate recognition-oriented restoration constraints? 
  2. How are the EMA-based task discrepancy targets constructed and normalized across different feature levels? What evidence supports the accuracy of the predicted task-representation discrepancies? 
  3. What specific advantage does the Sinkhorn-based prototype matching provide over conventional attention or cosine-similarity-based task feedback fusion? 
  4. How does STFR identify regions that should retain their current restoration features? Does the learned keep/update gate correlate with actual improvements or deterioration in local restoration quality? 
  5. How are the three downstream feature networks used during task-specific training and inference? Are feedback signals from all three networks required when optimizing or evaluating a single downstream task? 
  6. How are the ImageNet-1K training and validation subsets sampled, and how representative are the reported classification results of performance on the full validation set? 
  7. Can the authors clarify the computational overhead attributable to task feedback and its practical benefits compared with directly incorporating downstream task losses into the restoration network?
Questions for authors

Please see the weaknesses above.

Reviewer_hRn3OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

TaskIR combines degradation-conditioned restoration with a second stage that uses frozen downstream-network features to refine restoration features. The method learns task-representation corrections and combines prototype-based feedback weighting with gated refinement, evaluated on eight synthetic degradations across classification, segmentation, and detection.

Critical strengths

The experiments evaluate restoration and recognition jointly across three tasks, and the shared task-supervised retraining protocol in Appendix B.5 is a useful control. The preservation objective also makes the trade-off between task adaptation and reconstruction quality explicit.

Critical weaknesses
  1. The contribution relative to existing multi-degradation task-driven restoration is insufficiently established (Secs. 1-2; Tables 1-2). Yang et al., CVPR 2023, already study recognition-driven restoration across multiple degradations in one model, yet this cited work is neither substantively contrasted with TaskIR nor included in the comparisons. Retraining general restoration methods with task losses is useful, but does not establish the advance over this directly relevant problem formulation.

  2. The evidence does not isolate the proposed feedback mechanisms (Sec. 3.2; Table 3). Removing TRFG changes Top-1 accuracy from 73.00 to 72.99 and mIoU from 74.69 to 74.64, while restoration metrics are essentially unchanged. The coarse module-removal ablations do not test whether gap prediction, Sinkhorn prototype matching, or keep/update scoring improves on simpler feature projection, averaging, and residual gating under the same losses. This leaves the necessity of the central architectural choices unclear, particularly when Table 2 shows only small gains over the strongest task-supervised baselines.

Questions for authors
  1. What is the substantive advance over the already cited VRD-IR (Yang et al., CVPR 2023), and is a comparable evaluation available?
  2. Do existing ablations separate gap supervision, prototype transport, and conditional keep/update scoring from simpler feedback alternatives with the same training objectives?
  3. In Sec. 3.2.2, are the three feedback signals different task networks or different feature levels of the current task? Please reconcile this with the single-task scheduling in Appendix B.4 and task-specific inference paths in Table 10.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.