- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Hybrid Space Attention Refiner (HySAR), a plug-and-play cross-modal refinement framework designed to improve multimodal fusion performance, computational efficiency, and robustness to missing modalities. The framework consists of three components: Multi-scale Channel Information Learning Attention (MILA), which extracts multi-scale Euclidean spatial-channel information; Dual Hyperbolic Fusion Attention (DUHA), which combines Poincaré-aware and Proper-Velocity manifold representations to capture hierarchical dependencies; and Hybrid-Space Multi-Cross Fusion (HFuse), which integrates complementary geometric information across modalities. The authors evaluate HySAR on ten multimodal benchmarks covering medical survival prediction, medical image classification, semantic segmentation, and image-text classification. Experiments across multiple fusion architectures and backbone families report consistent performance improvements over existing refiners, together with favorable computational trade-offs and improved performance under missing-modality settings.
Critical strengths
Strength 1: The proposed MILA module uses depth-wise, group-wise, and dilated convolutions to capture spatial information at different scales. It combines these features through channel attention, providing an efficient way to preserve fine-grained information with relatively low computational cost.
Strength 2: The proposed DUHA module combines Poincaré and Proper-Velocity representations to model hierarchical relationships. HFuse further integrates these hyperbolic features with Euclidean information across modalities. This design allows HySAR to benefit from both geometric spaces while remaining compatible with different multimodal fusion architectures and supporting missing-modality scenarios.
Strength 3: The paper evaluates HySAR on ten multimodal benchmarks using different fusion architectures and backbones. The results show consistent improvements over existing refiners in prediction performance, with favorable computational efficiency and missing-modality robustness.
Critical weaknesses
Weakness 1: The IMF formulation appears to cancel out the MGHC and MDDHC outputs, yet Table 3(a) reports gains from these branches. This makes it unclear whether the formulation matches the implementation used in the experiments.
Weakness 2: All evaluated benchmarks use only two modalities, so the benefit of HFuse’s softmax-based weighting across complementary modalities remains untested. The reported improvements therefore do not establish the contribution of this particular mechanism.
Weakness 3: When only one modality is available, HFuse uses the DUHA response but leaves out the corresponding MILA response. The paper provides limited justification for this choice and does not compare it with retaining both responses during single-modality inference.
Questions for authors
Question 1: In Section 2.1, the IMF operation \mathcal I(a,b,c)=(a+b+c)+(a-b-c) simplifies to , so the MGHC and MDDHC outputs would cancel out as written. However, Table 3(a) reports gains from these branches. Could the authors provide the exact IMF formulation used in the implementation and explain how these branches contribute to the reported improvements?
Question 2: According to Table 6, all ten benchmarks contain only two modalities. In this setting, the softmax in Equation (13) operates over only one complementary modality, making its weight always equal to one. Could the authors clarify how the adaptive weighting mechanism contributes to the reported improvements? Would experiments involving three or more modalities help demonstrate its effectiveness compared with simpler fusion strategies?
Question 3: Algorithm 1 computes both MILA and DUHA for each available modality, but Algorithm 4 only uses DUHA when a single modality remains. Could the authors provide an ablation study comparing the current DUHA-only design with a variant that combines both responses under single-modality inference?