- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper presents FREST, a frequency-based framework for infrared and visible video fusion designed to improve robustness to image degradations and temporal inconsistencies.
The proposed approach consists of two main components: degradation-robust spatial-frequency modeling and frequency-aware spatiotemporal modeling. The first component uses adaptive frequency enhancement and spatial-frequency feature interaction to reduce degradation-related interference while retaining important visual details. It further incorporates frequency-guided cross-modal fusion to combine complementary information from infrared and visible videos. The second component employs 3D Fourier transforms to capture spatiotemporal frequency information, reducing unwanted temporal fluctuations while preserving meaningful motion.
Experiments on the M3SVD and HDO datasets show that FREST achieves state-of-the-art results in fusion quality and temporal consistency across various degradation scenarios, including blur, haze, rain, and infrared noise.
Critical strengths
-
The paper presents a unified frequency-oriented framework for addressing both spatial degradations and temporal inconsistencies in infrared and visible video fusion. The overall motivation is reasonable and the proposed architecture is well structured.
-
The authors introduce several dedicated modules for spatial-frequency enhancement, cross-modal fusion, and temporal consistency modeling. The experimental results across multiple degradation settings, together with the ablation studies, support the effectiveness of the proposed framework.
Critical weaknesses
-
Unclear core technical contribution. Although the paper addresses spatial degradations and temporal inconsistencies in infrared-visible video fusion, it is not sufficiently clear which specific limitations of existing methods the proposed framework aims to overcome. Furthermore, the framework incorporates multiple frequency-based modules, but the key technical insights and distinctions from existing approaches are not clearly highlighted.
-
Insufficient justification for the architectural design. The framework introduces multiple specialized modules, but the necessity of each component and the advantages over simpler alternatives are not fully established.
Questions for authors
Could the authors further clarify the key technical differences between the proposed frequency-based modules and existing spatial-frequency modeling approaches? In particular, what are the main methodological contributions beyond combining established techniques?
2 comments
Thank you for the response. My main concern remains novelty: the decoder and losses follow VideoFusion, FSAS/DFFN follow FFTformer, and FTCM closely resembles a 3D extension of FFC's spectral transform. The claimed "explicit" exploitation of spectral differences is not reflected in any specific mechanism, so the work reads as a combination of existing components.
Response to Reviewer 3Lxp
R1: We would like to clarify that existing methods primarily address degradations by learning a hard mapping from degraded inputs to clean representations. However, degradation-induced variations and genuine scene dynamics are often spatially entangled (Fig. 1 in the original manuscript). These methods lack an explicit mechanism to distinguish abnormal degradation patterns from valid scene motion, which limits their ability to handle complex degradations while preserving meaningful dynamic information. The two most closely related works are VideoFusion and FTPFusion. VideoFusion employs bi-temporal co-attention to aggregate cross-frame information, while FTPFusion decomposes features into high- and low-frequency components to enhance temporal feature representations and robustness. Although FTPFusion incorporates frequency-domain processing, it primarily uses frequency decomposition for feature modeling rather than explicitly exploiting the spectral differences between degradation-induced disturbances and genuine scene dynamics.
Our key insight is that frequency-domain representations provide complementary cues for distinguishing these ambiguous signals (Fig. 1 in the original manuscript). Based on this insight, we develop two specialized modules to address spatial and temporal degradations from different perspectives. Specifically, SFCEM jointly models amplitude and phase information to selectively enhance informative spatial structures while reducing degradation-related interference. FTCM, rather than directly aggregating temporal features, performs joint spatiotemporal frequency modeling to capture the spectral characteristics of cross-frame variations, enabling the network to better preserve genuine scene dynamics while suppressing degradation-induced fluctuations.
R2: Thanks for this suggestion. Actually, our ablation studies have already demonstrated the effectiveness of the proposed modules, as removing any component leads to performance degradation. From a design perspective, FTCM exploits the distinct spectral responses of temporal degradations and valid scene dynamics to suppress degradation-induced fluctuations while preserving meaningful motion information. In contrast, SFCEM leverages frequency-domain characteristics to selectively enhance informative spatial structures and suppress degradation-related interference, thereby improving the preservation of complementary infrared and visible details. To further demonstrate these design necessity experimentally, we separately replace all six FTCM blocks and all ten SFCEM blocks with computationally comparable 3D convolutions while keeping the remaining architecture unchanged. The quantitative results on M3SVD and HDO are presented in Tables 1a and 1b.
Table 1a. Fusion quality and temporal consistency for the M3SVD dataset.
Table 1b. Fusion quality and temporal consistency for the HDO dataset.
As shown in Tables 1a and 1b, replacing FTCM or SFCEM with conventional 3D convolutions consistently degrades fusion quality and temporal consistency on both datasets, indicating that simply relying on conventional spatiotemporal convolutions is insufficient. These results further validate the necessity of the proposed frequency-domain modules for effectively modeling spatial and temporal information in infrared and visible video fusion.