SRTAR: Super-Resolution Injection with Task-Aware Refinement for Challenging Object Segmentation

Efficient extraction of high-quality features from input images is critical for accurate challenging object segmentation. Real-world inputs often contain low-resolution (LR) contents, in this circumstance, even employing transformer architectures providing powerful feature modeling, achieving satisfactory performance is limited by the lack of high-frequency components crucial for the task within LR content. Image super-resolution (SR) is one of the promising solutions. However, due to the ill-posed property of SR, typical SR methods struggle to restore task-relevant high-frequency contents, which may dilute the advantage of utilizing the SR method. Therefore, we propose Super-Resolution Injection with Task-Aware Refinement (SRTAR) that effectively guides the generation of SR images beneficial to achieving satisfactory segmentation performance when processing LR images. The critical component is task-aware refinement (TAR) that enables the SR branch to acquire task-relevant discriminative knowledge from segmentation tasks and provides detail-rich features for segmentation refinement. Moreover, we propose a mixed-quality patch stack and a trilateral fusion loss to enhance the efficacy of TAR by addressing potential problems when employing the TAR. Extensive experiments demonstrate that our SRTAR achieves outstanding performance by generating SR images useful for specific task across 8 challenging object segmentation tasks.