- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes TriReward-Fusion, a framework for task-oriented infrared–visible image fusion (IVF) that jointly optimizes three tasks: image fusion, semantic segmentation, and object detection. The central idea is to project the fused image and the predictions of the two downstream tasks into a shared BLIP-based vision–language “semantic state space,” and from it derive three learnable reward signals — perceptual preservation (Rf), semantic completeness (Rs), and detection reliability (Rd). The reward heads are first calibrated (with the task network frozen) against task-derived targets: a ranking loss between a fused image and a synthetically degraded version for Rf; a soft-IoU plus boundary-F target for Rs; and a bipartite-matched detection-quality target for Rd, together with a vision–language alignment loss to BLIP category text prototypes. In a second stage the calibrated reward model is frozen and used as a differentiable critic that feeds reward-guidance gradients back to the fusion network, using adaptive softmax reward-weight prediction and reward-conditioned feature modulation. A segmentation–detection consistency loss (soft box–mask agreement plus a Jensen-Shannon term on category presence) is added. The authors additionally annotate person/bicycle/car bounding boxes on MSRS to enable three-task training. Validation covers fusion quality on MSRS, M3FD, and LLVIP, segmentation on MSRS, and detection on M3FD, against twelve recent fusion methods, plus component ablations.
Critical strengths
- Unified, well-motivated formulation with a sensible staged-critic design. The three-task framing (fusion + segmentation + detection) sharing a vision–language semantic space is a reasonable direction, and the two-stage calibration — fitting reward heads to task-derived targets while the task network is frozen, then freezing the reward model before using it as a critic — shows explicit awareness of the co-adaptation / reward-hacking risk that plagues learned-reward training.
- Comparatively broad evaluation. Three fusion datasets, downstream segmentation and detection, twelve fusion baselines, component ablations, and a cross-dataset protocol (train on MSRS only; evaluate on M3FD and LLVIP) together make a stronger generalization case than is typical in this literature.
Critical weaknesses
- Limited novelty and unclear delta over the closest prior work. The core ingredients — VLM/text-guided fusion (Text-IF, Text-DiFuse, OmniFuse, IF-FILM, SAGE-SAM), learnable fusion objectives, and multi-task fusion-in-the-loop training — are all established. The closest work, Bai et al., “Task-driven image fusion with learnable fusion loss” (CVPR 2025), is cited but never compared or experimentally separated, even though it embodies essentially the same idea (learn a fusion objective/critic to drive the fusion network). The paper does not articulate a crisp technical delta. Relatedly, the “reward learning” framing is misleading: there is no reinforcement learning here — no policy, no sequential decision process, no return — only a supervised learned critic used as a differentiable auxiliary loss, which inflates the apparent contribution.
- Strength of the empirical evidence does not support the claims. (a) Table 1 SSIM values ≥ 1.00 are impossible for standard SSIM and must be corrected, which casts doubt on the metric pipeline. (b) The claimed gains are small and untested — e.g., downstream mIoU 76.32 vs. 74.58 (OmniFuse) and mAP50 84.11 vs. 82.85 — with no multiple runs, error bars, or significance tests. (c) The fusion-quality improvement rests on no-reference metrics that are easily gamed, an awkward basis for a method whose selling point is avoiding reward hacking. (d) The new MSRS detection annotations are used only for training while detection is evaluated on M3FD, so the value and quality of that annotation effort (no inter-annotator agreement reported) are unclear. (e) Reproducing twelve baselines, including diffusion-based ones, on a single RTX 2080 Ti is a strong claim that cannot be verified without released code/configs.
Questions for authors
- How were the SSIM values obtained such that several exceed 1.00 (IGNet 1.01/1.02, PIAFusion 1.00)? Please give the exact definition and implementation used for SSIM (and for MI and VIF) and reconcile the values with standard bounds.
- How does TriReward-Fusion differ technically and empirically from Bai et al. (2025, “Task-driven image fusion with learnable fusion loss”)? Please provide a direct comparison.
- Because the reward heads are calibrated against targets derived from the task network’s outputs and then frozen, as the network improves during stage 2 the critic may become stale. How do you ensure the frozen critic remains a valid quality estimator? Please provide a “reward vs. true quality” curve over training.
- Rf is trained only by relative ranking against synthetic degradations of the model’s own output. What evidence shows Rf tracks an external notion of fusion quality rather than simply preferring outputs close to the initial ones?
- Please report the full metric set for the ablation, add an ablation that replaces BLIP with a non-vision-language feature (to isolate the VLM contribution), and add an ablation of the two-stage calibration vs. joint training.
- Are the reported improvements statistically significant? Please report results over multiple seeds/runs.
- Why is detection evaluated only on M3FD when the new annotations are on MSRS? If the detection reward is trained on MSRS categories (person/bicycle/car), how does it transfer to M3FD’s target set?
- What is the added training and inference cost of BLIP plus the reward heads plus two-stage training, relative to the “w/o Tri-reward” variant and to the baselines?