- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes GaIF, a content-adaptive image fusion framework designed to address the limitations of coarse-grained modality integration. The core component, Patch-Conditioned Modality Routing (PCMR), combines bidirectional cross-attention with token-wise and channel-wise gating to dynamically integrate complementary features. Additionally, a Confidence-Weighted Compositor (CWC) and an Intra-Sequence Mixer (TSM) are introduced to improve feature integration and spatial coherence. Experiments across multiple image fusion tasks demonstrate improvements over several existing methods, with ablation studies supporting the effectiveness of the proposed components.
Critical strengths
The paper evaluates the proposed method across multiple fusion tasks and datasets, demonstrating promising cross-task generalization.
The quantitative results and component ablations provide empirical evidence of the effectiveness of the proposed architecture. The inclusion of downstream detection experiments further supports its practical relevance.
Critical weaknesses
-
Limited technical novelty and insufficient differentiation from existing methods. The proposed framework appears to be an incremental combination of established techniques rather than a fundamentally new fusion paradigm.The primary contribution of PCMR is to extend the gating operation to the patch–channel level through an MLP. However, the paper does not sufficiently explain why this particular combination constitutes a substantial technical advancement over existing adaptive fusion mechanisms.
-
Insufficient motivation and limited mechanistic justification. The motivation for replacing scalar gating with patch–channel routing is primarily conceptual. The paper lacks convincing failure-case analyses or targeted visual evidence demonstrating the specific limitations of existing gating approaches. Although Figure 5 illustrates different gating granularities, it does not sufficiently establish why fine-grained routing is necessary or how the learned channel-specific gates correspond to meaningful cross-modal information selection.
Questions for authors
What is the fundamental technical distinction between PCMR and existing cross-attention-based adaptive fusion methods, particularly AITFuse and CrossFuse, beyond increasing the granularity of the gating weights?
Can the authors provide representative failure cases of conventional gating methods, together with learned routing maps, to demonstrate why patch–channel-specific fusion is necessary?
Since bidirectional cross-attention accounts for the largest improvement in Table 4, can the authors further isolate the contribution of PCMR using parameter-matched comparisons with alternative adaptive gating mechanisms?
7 comments
Dear Reviewers,
The authors have posted their rebuttal responses. I would kindly encourage you to review the authors' responses and continue the discussion by providing follow-up comments where appropriate. In particular, please indicate whether the authors' responses have adequately addressed your concerns and update your assessment if necessary.
Thank you for your time and continued efforts!
Your AC
Q1
All three methods adapt fusion to the input. Their distinction lies in how cross-modal interaction determines the mixture of the two streams.
| Method | Role of cross-modal interaction in fusion |
|---|---|
| CrossFuse [1] | Uses cross-attention weights for content-aware gating. |
| AITFuse [2] | Uses an adaptive interactive Transformer to model token- and channel-level interactions. |
| PCMR | Retains both streams through bidirectional cross-attention (BiCA). A separate MLP reads the concatenated infrared and visible tokens at each patch and predicts one coefficient per channel. That coefficient weights the infrared-stream representation and its complement weights the visible-stream representation. |
Thus, PCMR's distinction extends beyond gate resolution. Its gate acts after context exchange and explicitly sets the relative contributions of the two contextualized streams. We will clarify this operation in the related-work and method sections. Attention weights determine which tokens contribute to a token representation; PCMR's coefficients determine the mixture of the resulting two streams at each patch-channel coordinate. These operations are sequential but coupled, since the gate depends on cross-attended features.
Appendix D.3, Table 9 compares the complete models under the same LLVIP/TNO protocol.
| Method | Dataset | EI ↑ | VIF ↑ | SCD ↑ | AG ↑ |
|---|---|---|---|---|---|
| CrossFuse | LLVIP | 59.84 | 0.692 | 1.487 | 6.17 |
| AITFuse | LLVIP | 61.88 | 0.748 | 1.558 | 6.47 |
| GaIF | LLVIP | 64.31 | 0.800 | 1.630 | 6.88 |
| CrossFuse | TNO | 54.27 | 0.996 | 1.638 | 5.06 |
| AITFuse | TNO | 56.27 | 1.013 | 1.663 | 5.34 |
| GaIF | TNO | 58.62 | 1.020 | 1.680 | 5.55 |
GaIF scores higher on all four metrics on both datasets. This comparison measures whole-model performance. Within GaIF, Table 4 tests the incremental benefit of adding PCMR after BiCA (Q3 below), and Appendix C, Table 5 tests routing granularity (Q2 below).
Q2
Appendix C, Table 5 compares three gate designs on LLVIP. Even when the gate already varies across patches, allowing the mixture to vary across channels within each patch improves all four reported metrics.
| Gate design | EI ↑ | VIF ↑ | SCD ↑ | AG ↑ |
|---|---|---|---|---|
| Scalar gate | 60.20 | 0.71 | 1.57 | 6.52 |
| Spatial-only gate | 62.07 | 0.75 | 1.60 | 6.73 |
| PCMR | 64.31 | 0.80 | 1.63 | 6.88 |
Figure 3 shows fusion outputs, and Appendix G explains channel averaging. To show where the gate designs differ on an individual image, we will add an existing evaluation pair to Appendix G. We will show both inputs, the scalar-gating and GaIF outputs, and matching crops of the region where their outputs differ. Alongside them, we will show the channel-averaged PCMR gate and compositor map from the trained model's forward pass. The caption will specify the channel averaging and identify the compositor values as learned mixing coefficients without uncertainty calibration.
Q3
BiCA gives the largest single EI gain in the incremental ablation (52.14 to 58.71). Appendix F, Table 13 compares PCMR on and off with the other components fixed. The rows below show both BiCA settings with TSM and CWC disabled.
| BiCA | PCMR | TSM | CWC | EI ↑ | SCD ↑ | AG ↑ |
|---|---|---|---|---|---|---|
| Off | Off | Off | Off | 52.14 | 1.42 | 5.61 |
| Off | On | Off | Off | 54.60 | 1.47 | 5.95 |
| On | Off | Off | Off | 58.71 | 1.51 | 6.10 |
| On | On | Off | Off | 61.83 | 1.57 | 6.44 |
With BiCA present, adding PCMR raises EI/SCD/AG from 58.71/1.51/6.10 to 61.83/1.57/6.44, gains of 3.12/0.06/0.34. Thus, PCMR improves the evaluated model even after the two streams have exchanged information. Table 5 also shows gains over both coarser gates. The comparison with spatial-only gating is particularly relevant: it tests the benefit of allowing channels within the same patch to use different input-dependent modality mixtures, although gate capacity is not matched. These comparisons control which modules are present, but gate parameter counts differ. We will state explicitly that the measured gain applies to the PCMR design as implemented. Isolating how much comes from routing granularity alone would require a parameter-matched control.
References
[1] H. Li and X.-J. Wu, “CrossFuse: A Novel Cross Attention Mechanism Based Infrared and Visible Image Fusion Approach,” Information Fusion, 2024.
[2] Z. Wang, F. Yang, J. Sun, J. Xu, F. Yang, and X. Yan, “AITFuse: Infrared and Visible Image Fusion via Adaptive Interactive Transformer Learning,” Knowledge-Based Systems, vol. 299, Art. no. 111949, 2024.
W1
We respectfully disagree that PCMR is merely a redundant combination of existing modules. Bidirectional cross-attention exchanges information between the two modality streams; PCMR then uses those cross-attended representations to make an explicit modality-mixing decision for every patch and feature channel. The full factorial ablation in Table 13 measures PCMR’s contribution while holding the other components fixed:
With BiCA, TSM, and CWC held fixed, adding PCMR improves EI/SCD/AG by 2.46/0.05/0.34 without BiCA, and by 3.12/0.06/0.34 with BiCA. The result with BiCA is especially relevant to the concern that PCMR merely duplicates cross-modal interaction: PCMR adds a measurable gain after BiCA is already present.
The gate-resolution ablation in Appendix C, Table 5 provides complementary evidence. On LLVIP, PCMR improves all four reported metrics over both scalar and spatial-only gating:
Compared with spatial-only gating, PCMR gains 2.24 EI, 0.05 VIF, 0.03 SCD, and 0.15 AG. These results support PCMR as a distinct, empirically tested routing mechanism. We will make the Table 13 component contrasts and Table 5 gate-resolution results more prominent in the rebuttal and revised manuscript. Since the gate variants are not strictly parameter-matched, we will also add a parameter-matched alternative-gating control and scope our claims to the evidence.
W2
We agree that the manuscript should make the routing behavior easier to inspect. The submission already provides quantitative evidence: Figure 3 compares qualitative fusion outputs, Appendix C, Table 5 evaluates scalar, spatial-only, and patch-channel gates, and Table 13 reports the full component ablation. For example, the LLVIP gate-resolution results are:
These results test the effect of routing granularity, but we agree they do not replace visual examples showing where routing differs in an individual scene. Appendix G currently discusses how to interpret the channel-averaged PCMR gate, but does not display a sample-specific gate map.
In the revision, we will add a compact set of actual inference visualizations to Appendix G. Each example will show the infrared and visible inputs, the scalar-gating and GaIF fused outputs, and the PCMR gate and compositor maps. We will select cases where the outputs reveal a clear difference in preserved thermal targets or visible details, and include a crop to make that difference easier to inspect. The gate map will be identified as the channel-averaged PCMR output, and the compositor map will be described as a learned mixing coefficient, not calibrated uncertainty. This will complement the existing ablations with direct visual evidence, without treating selected examples as a substitute for quantitative evaluation.