- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper argues that the conventional single-output formulation of multimodal image fusion is fundamentally limited: compressing complementary infrared and visible information into one fused image forces intensity, gradient, and structure objectives to compete in a shared space, and the pseudo-RGB replication used to feed downstream networks only adds channels without clear functional roles. The proposed Task-Agnostic Representations (TAR) instead learn a K-channel output in which each channel is supervised by a dedicated objective (intensity, gradient, and structure), using a dense U-Net backbone, with no downstream supervision. A lightweight fusion head, trained with the backbone frozen, maps TAR back to a conventional fused image for image-level evaluation. The representation is architecture-agnostic: only the output head needs to be swapped to plug into CNN- and Transformer-based backbones, and downstream task performance improves in both cases. They achieve excellent performance on several image fusion benchmarks.
Critical strengths
-
The central idea is simple yet effective: give each objective its own channel instead of packing everything into one fused image. This removes the forced trade-off between the two modalities and the redundancy of pseudo-RGB inputs, and it works in practice. The representation needs no downstream supervision and transfers across architectures; the authors only replace the output head of two unrelated backbones, one CNN and one Transformer, and both improve on detection and segmentation. Task-aware fusion methods, in contrast, are tied to a specific task and its labels. That this approach avoids such coupling makes the generality genuinely useful.
-
The experiments are consistently in the paper's favor. TAR is first or second on every image-level metric over three fusion benchmarks, and it brings clear gains to downstream detection and segmentation, leading every object category on the segmentation benchmark. The ablations agree: the full representation beats single-channel replication and the best single-role variant, and the trend across channel configurations holds up. The improvement comes from explicit role assignment, not from adding channels.
-
The design also keeps the method easy to use and to compare. The representation follows the standard three-channel input format, so off-the-shelf detectors and segmentation module can take it directly with no architectural changes; a lightweight head can map it back to a conventional fused image for image-level evaluation. The results therefore remain comparable with the large body of fusion literature, while the same representation serves downstream models.
Critical weaknesses
-
The role of the fusion head and the two-stage protocol is left underexplored. Image-level metrics are all computed on the fusion-head output, yet there is no ablation isolating the head design, for example, sensitivity to λs or head capacity.
-
The downstream comparison is somewhat one-sided: Table 3 compares TAR only against general-purpose fusion methods, while task-aware fusion approaches specifically designed for detection and segmentation are not included. Given that one of the paper's central claims is improving downstream perception without task-specific supervision, a direct comparison with such task-supervised alternatives would better clarify the positioning and relative advantage of the contribution.
-
The ablation on the number of channels K is insufficiently informative. In the supplementary material, different K configurations are reported only as ranking tables, without the underlying downstream metric values, making it difficult to judge how large the performance differences between different channel settings actually are.
-
The three-channel representation Z is what gets fed into the downstream networks, yet every image-level metric in the tables is computed on the fused image produced by the fusion head. This raises a question: does the head output actually tell us anything about the quality of Z itself? Since Z is the representation that detection and segmentation actually consume, why not evaluate image-level quality directly on Z?
-
The paper positions itself against "task-aware fusion," but existing methods like UniFusion and GIFNet already describe themselves as unified or task-agnostic, and feeding fused outputs to downstream networks is hardly new. Given that, it would help to spell out in the related work what "task-agnostic" means here beyond these methods, for instance, those frameworks share one fusion paradigm across tasks, whereas TAR provides downstream networks with semantically role-specific input channels. It would also be worth clarifying whether the three channels are semantically meaningful to downstream models or mainly offer better input statistics. Making this distinction explicit would sharpen the contribution's boundaries.
Questions for authors
-
Image-level metrics are all computed from the fusion-head output, while downstream models consume the three-channel representation Z directly. To what extent does the head output reflect the quality of Z, and was image-level quality considered when designing Z?
-
Table 3 compares the proposed method only against general-purpose fusion methods. Which task-aware fusion baselines were considered, and how does the method compare against them on M3FD and FMB?
-
The ablation on the number of channels K is reported as rankings without concrete values. Could you provide the actual mAP/mIoU for each K configuration to quantify the performance differences?