- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes MambaVF, an efficient unified video fusion framework based on State Space Models (SSMs), to address the substantial computational overhead and limited efficiency of existing video fusion methods that rely on explicit optical flow estimation and feature warping. MambaVF introduces two key components: (1) a Mutual State Fusion (MuFuse) module that enables complementary information exchange between different video sources through reciprocal state interaction; and (2) a Spatio-Temporal Bidirectional (STB) scanning mechanism that combines spatial-priority and temporal-priority bidirectional scanning strategies to model spatio-temporal dependencies. These designs enable multi-frame information aggregation without explicit optical flow estimation.
The authors evaluate MambaVF on four video fusion tasks in VF-Bench, including multi-exposure fusion (MEF), multi-focus fusion (MFF), infrared-visible fusion (IVF), and medical video fusion (MVF). Experimental results demonstrate that MambaVF significantly reduces computational overhead while maintaining competitive fusion performance.
Critical strengths
-
The paper leverages Mamba-based state space models for spatio-temporal modeling and incorporates reciprocal cross-source state interaction through MuFuse and bidirectional spatio-temporal scanning through STB. This enables video fusion without explicit optical flow estimation or feature warping, effectively reducing the computational overhead associated with conventional video fusion methods. Compared with UniVF, MambaVF reduces the number of parameters by approximately 94.3% and FLOPs by 86.6%, while achieving a 2.1× inference speedup, demonstrating its computational efficiency.
-
The paper conducts extensive experiments on four video fusion tasks in VF-Bench, including multi-exposure, multi-focus, infrared-visible, and medical video fusion, and provides both quantitative and qualitative comparisons with various existing methods. In addition, ablation studies are conducted to evaluate the effectiveness of the key components. The experimental results demonstrate that MambaVF achieves competitive performance across different fusion tasks, indicating the generalizability and practical value of the proposed framework.
Critical weaknesses
-
In the Introduction, the authors point out that conventional optical flow-based video fusion methods are susceptible to optical flow estimation errors in challenging scenarios involving fast motion, occlusions, and illumination changes, which may further lead to blurring, ghosting artifacts, or temporal inconsistencies in the fused results. However, the current experiments mainly focus on overall fusion performance comparisons on VF-Bench, without a systematic analysis of these challenging scenarios where optical flow estimation is prone to failure. I suggest that the authors conduct additional controlled experiments under varying motion magnitudes, occlusion levels, and illumination changes, comparing MambaVF with conventional flow-guided methods in terms of fusion quality, temporal consistency, and performance degradation. Such experiments would help verify whether the proposed flow-free modeling strategy can effectively mitigate the degradation in fusion quality caused by inaccurate optical flow estimation.
-
Although the ablation studies demonstrate the effectiveness of the MuFuse and STB modules, the paper does not clearly establish their unique advantages over existing Mamba-based spatio-temporal modeling approaches and cross-source feature fusion designs. I suggest that the authors include additional comparative experiments with relevant existing methods to further substantiate the necessity and novelty of the proposed designs.
-
The paper emphasizes that MambaVF achieves efficient video fusion through state space modeling and demonstrates substantial computational advantages in Table 5. However, the current experiments only report FLOPs and inference latency under a specific evaluation configuration, without analyzing computational efficiency across different input temporal window lengths and spatial resolutions. I suggest that the authors provide additional comparisons of inference latency, FLOPs, and GPU memory consumption under varying input sizes to further validate the scalability of the proposed method.
Questions for authors
Please refer to the concerns and suggestions raised in the Weaknesses section.
Thank you for recognizing our efficient flow-free design, the extensive comparisons and ablations across four fusion tasks, and the framework's competitive performance, generalizability, and practical value.
Q1:
Can controlled analyses under varying motion, occlusion, and illumination substantiate the flow-free design?R1: We added challenging-scenario evaluations for multi-exposure, multi-focus, and infrared-visible video fusion. Within each task, we score the source videos for motion magnitude, occlusion, and low illumination, and select approximately the highest-scoring 20% under each criterion. Using the same subsets for all methods, we assess image similarity with SSIM and temporal consistency with MS2R. MambaVF achieves the highest SSIM and lowest MS2R in every evaluated subset, supporting the effectiveness of MuFuse's state interaction and complementary STB paths in aggregating spatial, temporal, and source information without explicit optical-flow alignment. Section 4.2 summarizes these findings; full details are provided in Appendix C, Table 8.
Q2:
What distinguishes MuFuse and STB from existing Mamba-based modeling and fusion designs?R2:
We clarified the methodological differences in Section 3.2 and Appendix B.1:
We also added encoder replacement experiments using designs adapted from GLSMamba and Vamba. MambaVF achieves the best results across all metrics in this comparison (Appendix B.2, Table 7).
Table 5 in Section 4.4 further shows that STB outperforms scan-count-matched variants, while removing MuFuse’s state interaction degrades fusion quality and temporal consistency.
Q3:
How do latency, FLOPs, and GPU memory scale with spatial resolution and temporal window length?R3: We added comparisons of latency, FLOPs, and peak GPU memory at spatial resolutions of 64×64, 128×128, 256×256, and 512×512 (Appendix D.2, Table 9), and at 128×128 with the internal fusion window extended to T=3,5,7,9 (Appendix D.3, Table 10). MambaVF has the lowest FLOPs and second-lowest latency at every tested setting, remaining faster than UniVF. Its spatial FLOPs grow approximately in proportion to pixel count. For longer temporal windows, latency grows approximately in proportion to window length, while FLOPs grow more slowly than UniVF's. We discuss efficiency in Section 4.3 and provide detailed evaluation settings and comparisons in Appendix D.