- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes GranFlow, a two-stage Rectified Flow framework for infrared-visible image fusion under degraded inputs. In Stage I, wavelet-based subband statistics and CNN features are used to condition modality-specific encoding and dynamically gate the fusion of infrared and visible latent features. In Stage II, a conditional DiT refines the fused latent representation through Rectified Flow, using a sample-dependent latent displacement scale as an additional conditioning signal. The model is trained with synthetic noise, blur, stripe, and haze degradations and is evaluated on standard fusion benchmarks, compound degradation settings, additional datasets with naturally occurring or real-world degradations, and downstream detection or segmentation tasks. The main claim is that matching conditioning granularity to degradation characteristics improves degradation-robust fusion.
Critical strengths
- The paper evaluates degradation-robust fusion beyond the clean-input setting. It considers single degradations, compound degradations, and degradation-free inputs across multiple datasets. It also evaluates the fused outputs on object detection and semantic segmentation, providing task-level validation in addition to conventional fusion metrics.
- The framework separates two different forms of conditioning within a single fusion pipeline. The Stage-I wavelet branch extracts modality-specific subband information and produces spatial-channel fusion gates, while the Stage-II Rectified Flow branch refines the fused latent using a sample-dependent displacement scale. This separation gives the proposed architecture a clear functional organization, and the reported comparisons with RF-Fusion and the module ablations indicate that the complete model can improve performance under the tested settings.
Critical weaknesses
- The central motivation is not sufficiently convincing. The paper treats differences in frequency-domain and statistical characteristics across degradations as the main motivation. However, this observation is not by itself a compelling research gap, since prior fusion methods have already exploited degradation-specific frequency information, such as ControlFusion. The Introduction should more carefully analyze where existing degradation-robust fusion methods fail and why their conditioning strategies are inadequate. The broad schematic in Figure 1(c) does not provide enough support for this argument.
- The proposed degradation formulation has a limited scope, and its real-world robustness is not demonstrated. The design is mainly motivated by noise, blur, stripe artifacts, and haze, but it is unclear whether the same frequency-statistical formulation applies to common conditions such as low-light and over-exposure. Most controlled experiments use synthetic degradations, while the real-world experiments do not provide degradation-specific analysis. In particular, Figure 9 only reports object detection results on LLVIP and does not show that the method can effectively handle low-light degradation.
- The methodological contribution appears largely incremental. The proposed framework combines several established components, including wavelet decomposition, statistical and CNN descriptors, conditional modulation, gated fusion, VAE-based latent modeling, DiT, Rectified Flow, and scale conditioning. Although some components are redesigned for infrared-visible fusion, the paper does not clearly identify a methodological principle that goes beyond the combination of existing techniques.
- The paper does not directly verify its central adaptive-fusion mechanism. The main claim is that the model adapts the relative contributions of the infrared and visible modalities according to the degradation type and its characteristics. However, the paper does not visualize or quantitatively analyze the learned modality gates, subband weights, or scale-dependent modulation.
Questions for authors
- Which specific failure modes of existing degradation-robust fusion methods are not addressed by prior frequency-aware or degradation-conditioned approaches? Please clarify why the proposed granularity-matched conditioning is necessary.
- What evidence supports applying the proposed frequency-statistical formulation to degradations beyond noise, blur, stripe artifacts, and haze, such as low-light and overexposure? Since Figure 9 reports only detection results on LLVIP, what evidence supports the claim that GranFlow can handle low-light degradation?
- Which component or learning principle of GranFlow goes beyond the combination of wavelet decomposition, conditional modulation, gated fusion, VAE, DiT, Rectified Flow, and scale conditioning? Please clarify the specific methodological contribution that distinguishes GranFlow from existing frequency-aware, degradation-aware, and Rectified Flow-based fusion methods.
- Can the authors provide direct visualization or quantitative analysis of the learned modality gates, subband weights, and scale-dependent modulation under different degradation types?
7 comments
Response to Reviewer rs2e (Part.2)
Extreme noise and blur exceed the training strengths. The infrared-noise test differs from the training stripe setting: training uses independent Gaussian noise at – together with noise shared within columns, while the new test uses grayscale Gaussian noise at . This tests a change in both infrared-noise structure and strength. The local tests restrict noise or blur to part of the image. Across the five unseen settings, GranFlow has the best average result on all six metrics in Table 6. Its MS-SSIM ranks second under extreme blur and noisy infrared input. Across the two local tests, it has the best average SCD, MSE, PSNR, and MS-SSIM, and the second-best QNCIE. Figure 11 shows less noise in the building region and clearer outlines in the blurred shop-sign region. Nearby scene content also remains visible.
W3/Q3 — Specific Innovations within VAE- and DiT-Based Frameworks
Each module in GranFlow serves a distinct and complementary role within a coordinated restoration and fusion process. Our contribution lies in how these roles are defined and connected, so that each part supports a different need in handling degraded inputs. This design links degradation-guided fusion with sample-specific latent refinement.
-
RFfusion adapts a VAE for fusion and prioritizes efficient one-step RF sampling. Its frequency and saliency losses encourage information preservation, but it does not explicitly learn degraded-to-clean latent transport. GranFlow trains a dual-branch VAE to restore and fuse degraded inputs, then learns conditional RF refinement between degraded and clean fused latents, directly targeting residual corruption.
-
DiTFuse uses VAE, DiT, and masked learning for instruction-controlled fusion and takes explicit text instructions as input. GranFlow requires neither text prompts nor degradation labels: image-derived wavelet cues and local latent interactions guide spatial-channel mixing, while current displacement supplies state information.
-
OmniFuse first restores each source latent with diffusion, then learns weights to combine the restored features. Its contrast, texture, and color losses also use the reconstructed source images as references. When source restoration leaves noise or distorts details, these errors can enter both the features being fused and the references used to train fusion. The method does not learn a further degraded-to-clean transformation of the fused latent to directly correct such remaining errors. GranFlow addresses this limitation by training a dual-branch VAE to restore and fuse the inputs, followed by conditional RF trained between degraded and clean fused latents.
The modules of GranFlow work together to select useful features and then correct the remaining errors. The wavelet branch describes the degradation characteristics of each modality. The dual-branch VAE uses these conditions to restore modality features, while the spatial-channel gate combines the conditions with local latent features to decide how much each modality contributes at each location and channel. This produces the fused latent that conditional RF further restores, with DiT predicting the updates. Within this refinement stage, scale-gated normalization controls residual feature scale, while the displacement-scale embedding supplies the magnitude of each sample's current latent change:
s_t=\operatorname{std}(\delta_t),\qquad r_t=\operatorname{MLP}\bigl(\log(1+s_t)\bigr).$$ Here, $z_L^f$ is the degraded starting latent and $z_t$ is the current latent. Their difference describes the change during restoration, and $s_t$ measures its size. Small residuals retain their original scale, while larger ones receive more normalization. The embedding $r_t$ separately supplies the size of the change, so the rectified flow network can use both its pattern and magnitude. A fixed summary of the degraded input supplies scene context throughout refinement, including the starting step when $s_t=0$. Together, these conditions let the flow network use the input context and the current sample's state when predicting the next update. Table 4 tests our design at the same or a similar parameter count: | Conditioning arrangement | SCD ↑ | MSE ↓ | PSNR ↑ | MS-SSIM ↑ | | ------------------------ | ----------:| ----------:| -----------:| ----------:| | Global gate | 1.2711 | 0.0482 | 14.8469 | 0.6754 | | Unified conditioning | 1.6737 | 0.0481 | 15.6486 | 0.6761 | | Timestep-scheduled scale | 1.8592 | 0.0348 | 15.9845 | 0.7281 | | GranFlow | **1.8639** | **0.0311** | **16.0177** | **0.7344** |Response to Reviewer rs2e (Part.3)
Table 4 shows that the tested components serve their intended roles in restoration and fusion. The spatial-channel gate selects information locally. We designed the gate to adjust the contribution of each modality at different image locations. When the global-gate control forces these weights to be the same across locations within each channel, PSNR falls from 16.02 to 14.85 and MS-SSIM from 0.73 to 0.67. The two stages benefit from conditions matched to their tasks. The first stage uses modality-specific frequency information to guide restoration and fusion. The second stage uses the current latent change to guide further refinement. The unified-conditioning control merges the modality descriptors into one global condition and changes how both stages use the conditions. Compared to our approach, it has resulted in a certain decrease in performance. The displacement scale provides useful information about the current sample. We use this scale to tell the refinement network how much each sample’s latent has changed. The timestep-scheduled control replaces it with an average scale from training data, so every sample receives the same scale value at a given timestep.
Taken together, Table 4 shows that the local gate, task-specific conditions, and sample-specific displacement scale each serve a distinct role. Their combination helps GranFlow select useful modality features and refine the fused latent, leading to better fusion quality.
W4/Q4 — Direct Analysis of Gating and Displacement Conditioning
Figures 13–15 explain why we use both frequency information and latent displacement, and show how the gate adjusts the contribution of each modality across the image.
Figure 13: Different degradations cause different amounts of latent change. We compare clean and degraded versions of the same scene using the same frozen autoencoder. The curves show how much the fused latent changes as each degradation becomes stronger. Noise and blur show large early changes that diminish at higher strengths. Haze continues to produce larger changes over the tested range. Infrared stripes produce much smaller latent changes. These trends show that latent change depends on both the type and strength of degradation. A single degradation-type scale would miss these differences, thus causing the model to lose information.
Figure 14: Frequency changes and latent displacement provide different information. Each point relates the frequency content of the input degradation to the resulting latent change. The input degradation is measured by the difference between degraded and clean inputs, split into low- and high-frequency bands. For haze, larger low-frequency changes accompany larger latent displacement; for blur, high-frequency detail changes relate clearly to latent displacement. Noise behaves differently: its high-frequency changes cover a wide range, while latent displacement grows much less. Infrared stripes also show changes in input frequency content while the latent displacement remains small. These patterns explain why we use both kinds of information. Frequency features describe how the inputs are affected, while displacement scale measures how much the latent representation changes. The two signals therefore serve different roles: frequency information guides restoration and fusion of the source features, and the current displacement scale guides further refinement of the fused latent.
Figure 15: The gate adjusts modality weights by region. In our fusion rule, weights visible features and weights infrared features. The last column compares the gate under degradation with the gate for the clean image pair. Red means that the visible weight increases; blue means that it decreases. Under visible noise, the gate lowers visible weights over much of the central background, while increasing them in some other regions. Under haze, visible weights increase around the pedestrians and decrease in several background regions. Under blur, the changes are mixed and follow different parts of the scene. Thus, the gate adjusts the visible–infrared balance locally according to both image content and degradation. It can retain useful visible features in one region while giving infrared features more weight in another. In the infrared-stripe example, the visible input is unchanged and the gate difference is close to neutral. The displayed modality weights therefore remain close to those for the clean pair. Across the examples, the gate shows different responses to different degradations and different regions. This illustrates its intended role in selecting and combining modality features locally.
Together, these analyses show how the parts of GranFlow work together to handle degraded inputs. The visual analyses and controlled experiments support the effectiveness of our method: its components serve different roles and work together to select useful information and correct the errors that remain after fusion.
Response to Reviewer rs2e (Part.1)
Thank you for your comments. We have clarified the design rationale and methodological contribution, expanded the evaluation under unseen degradations, and added capacity-comparable conditioning varients comparisons and direct analyses of the conditioning mechanisms. The relevant evidence is presented in Sec. 4.5 (p. 9, Line 451), App. 11 (p. 24, Line 1262), and App. 12 (p. 25, Line 1346). Together, these results support the rationale for our design and demonstrate its effectiveness under the evaluated conditions. We address each point below.
W1/Q1 — Design Motivation and Limitations of Existing Fusion Methods
Different degradations differ in their statistical characteristics, including frequency content and latent displacement. These differences reflect the intrinsic relationships in image restoration under different degradations and can be used as the conditioning signals to achieve adequate restoration of complexly degraded images while preserving their useful information. We therefore use them to motivate our design.
Existing methods for degraded infrared-visible image fusion still have design limitations that leave some restoration needs insufficiently addressed. Methods such as AMG-Fuse use modality-contribution masks to guide degradation suppression and cross-modal interaction. However, masks based on intensity-mixing relationships struggle to separate true colors, differences between modalities, and degradation effects. This leads to substantial color bias in the fused results, as shown in the haze example of Fig. 4. All-in-one methods such as TG-ECNet use experts or prompts within a shared architecture to handle different degradation types and strengths. This limits how well the model can meet the restoration needs of an individual sample. Methods such as ControlFusion use frequency conditions and text-based modeling to describe degradation globally. However, a global condition alone cannot provide local information aggregation within the image. These limitations make it important to fully use frequency information for restoration at the degradation-type level and latent displacement scale for refinement at the sample level. GranFlow follows this design: modality-specific frequency features guide restoration and fusion, while the current latent displacement scale guides further refinement for each sample. The other two reviewers also recognize this motivation: Reviewer vAJJ describes our design as "well motivated," and Reviewer K4b3 finds the motivation "reasonable." The analysis above explains how this motivation leads to our two-stage design.
We have also clarified the different pipeline types in Fig. 1: Fig. 1(c) shows guidance from input-derived prompts, while Fig. 1(d) specifies our spatial-channel gating and displacement-conditioned refinement. This distinction makes the motivation behind our design clearer.
W2/Q2 — Unseen Degradations and Transfer to Exposure Changes
Table 19 directly evaluates fusion under low light and overexposure. The fusion results measure image quality under these two exposure changes. We also test stronger noise and blur, a different infrared-noise setting, and local degradation. Tables 18–21 give the individual results, Fig. 11 shows example outputs, and Table 6 gives the averages.
GranFlow has the best MSE, PSNR, and MS-SSIM under both low light and overexposure among the compared methods. It also has the best SCD under low light and QNCIE under overexposure. Figure 11 shows clearer targets and background structure in the low-light example. In the overexposure example, the bright area spreads less and nearby content remains visible. These results directly support the method's ability to fuse images under the tested exposure changes. GranFlow uses image features rather than a fixed set of degradation labels. The wavelet statistics and learned features come from the input images. The gate also uses local latent features, while the rectified flow network uses the fused latent and its displacement. Neither stage needs a degradation label at inference. The model can therefore respond to changes in brightness, contrast, and structure without assigning the image to a named degradation class first. Haze, low light, and overexposure have different physical causes. However, handling them can involve some shared needs, such as adjusting contrast and intensity while keeping useful structure. These shared needs motivate transfer to exposure changes, and Table 19 directly demonstrates this transfer in the tested settings.