acceptodds
Withdrawn from ICLR 2027

RouteFuse: Reliability-Conditioned Routing for Multi-Modality Image Fusion under Adverse Weather

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multi-modality image fusion integrates complementary sensor information, yet under adverse weather the need for infrared evidence varies spatially and across frequency bands because visible degradation is non-uniform and affects coarse structure and fine detail differently. Many existing methods rely on feature saliency or modality contribution, which do not directly measure local visible fidelity. We propose RouteFuse, a one-step diffusion-based framework that casts adverse-weather infrared-visible fusion as reliability-conditioned evidence allocation. RouteFuse represents infrared evidence as residuals relative to a primary multimodal representation and routes them separately in low- and high-frequency bands across multiple feature scales. Routing gates are conditioned on a learned visible reliability map and the cross-modal feature relation. A fusion-oriented diffusion learning scheme trains the predictor to estimate an explicit multimodal fusion target directly from sampled diffusion states, enabling one-step diffusion at inference. Mechanistic analyses show that predicted reliability aligns with local visible fidelity and that lower reliability is associated with stronger realized transfer of infrared residuals. Extensive experiments under rain, haze, and snow show that RouteFuse achieves state-of-the-art performance on all six primary fusion metrics with the lowest FLOPs and fastest inference among state-of-the-art competing methods.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
multi-modality image fusion, infrared-visible image fusion, adverse weather, one-step diffusion, reliability-conditioned routing, residual routing

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_thnVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes RouteFuse for infrared–visible fusion under rain, haze, and snow. Infrared evidence is injected as gated residuals relative to a primary multimodal representation. The injection is routed separately in low- and high-frequency bands at five feature scales, with gates conditioned on a learned visible-reliability map, a cross-modal feature relation descriptor, and the diffusion timestep. The predictor is trained with v-prediction plus image-space losses toward a hand-crafted fusion target built from clean visible and infrared images, and is applied once at t=299 at inference. Experiments on AWMM-100K report the best overall scores on six fusion metrics, the best detection results, and the lowest FLOPs and runtime. A mechanistic analysis, including an energy-matched Anti-Reliability vs. Random reallocation, supports the claim that reliability-conditioned allocation matters.

Critical strengths
  1. The framing of adverse-weather fusion as reliability-conditioned evidence allocation is clear and well motivated.

  2. The energy-matched reallocation experiment (Fig. 3c) is a thoughtful causal test that goes beyond correlation and is rarely seen in fusion papers.

  3. Consistent overall gains, particularly on SCD and VIF, together with strong efficiency in FLOPs and runtime.

  4. The paper is well written, and the appendix gives detailed loss definitions and weights.

Critical weaknesses
  1. The “one-step diffusion” framing. (1) At t=299 with T=300 and a cosine schedule, α_t ≈ 0, so x̂₀ ≈ −v̂ and the input x_t is essentially pure noise. The model therefore behaves as a noise-conditioned regressor; (2) A8 shows a gain over a deterministic encoder–decoder, but this could stem from multi-timestep training acting as augmentation or regularization rather than from diffusion modeling per se.
  2. Weak reliability estimates and limited contribution of adaptive gating. Despite direct supervision toward R*, predicted reliability correlates only weakly with local fidelity (ρ = 0.203 and 0.344), and Fig. 3(a) shows many images with negative correlation. In Table 4, trained static gates (A5) are only marginally worse than the full model (e.g., QMI 0.4460 vs. 0.4529), which suggests that most of the gain comes from the residual pathway (A4) rather than from adaptive reliability-conditioned gating.
  3. Synthetic-only evaluation. The reliability target requires V_gt, and all test results are on synthetic degradations. Results on real adverse-weather captures (e.g., the real-world portion of AWMM-100K reported by AMG-Fuse) would substantially strengthen the claims.
  4. Baseline fairness and missing baselines. (1) Six of nine baselines are run from author checkpoints after WeatherRemover preprocessing, without retraining on AWMM-100K, while RouteFuse is trained in-domain; (2) Methods discussed in the related work and designed for this setting are not compared: AWFusion (which introduced AWMM-100K), AWM-Fuse, ControlFusion (which covers rain and haze), TG-ECNet, and MdaIF; and (3) The omission of AWFusion and AWM-Fuse is particularly notable given that they target this exact benchmark.
  5. Selective reporting. (1) Under haze, RouteFuse trails CAWM-Mamba on QMI, QG, and QAB/F, but Table 1(b) shows only QMI, SSIM, and SCD; (2) On clean M3FD, RouteFuse’s VIF (0.4087) ranks near the bottom, and this is not discussed; and (3) The model has the largest parameter count (213M), which is not mentioned alongside the efficiency claims.
  6. Evaluation references are unspecified. All six metrics require source images as references, but the paper does not state whether the visible reference is the degraded V_d or the clean V_gt. With V_d, higher scores may reward preserving weather artifacts. With V_gt, the evaluation closely matches the training target x₀, which is constructed from V_gt, an advantage the WR-based pipelines do not share. This must be clarified for Table 1 to be interpretable.
  7. Minor. (1) Please clarify on which images the YOLO11x detector was trained; (2) DiffFuseNet (Wang et al., 2026, Pattern Recognition 180:114259) could not be located; please provide a link; and (3) The WeatherRemover volume and page metadata appear inconsistent with the available version.
Questions for authors
  1. Which visible image (V_d or V_gt) is used as the reference for each metric?
  2. Can you retrain or fine-tune the main baselines on AWMM-100K, and add AWFusion and AWM-Fuse?
  3. How does RouteFuse compare with the same network trained deterministically under the same multi-noise-level augmentation?
  4. Can you report results on real adverse-weather images?
  5. Given the modest gap between A5 and the full model, what evidence shows that input-adaptive reliability conditioning, rather than the residual pathway itself, drives the improvement?
Reviewer_P3VEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RouteFuse, a framework for infrared–visible image fusion under adverse weather. The method estimates a visible reliability map to guide the injection of infrared feature residuals across frequency bands and feature scales. The diffusion predictor is trained with an explicitly constructed fusion target and generates the fused image in a single step at inference. Comparisons with other methods and ablation studies demonstrate its effectiveness.

Critical strengths
  1. The motivation is clear. Weather degradation varies across regions, so adjusting the use of infrared information based on local visible reliability is reasonable.
  2. The experiments are fairly comprehensive, covering fusion metrics, downstream detection, and runtime. The residual reallocation experiment also provides evidence for the role of spatial routing.
Critical weaknesses
  1. The need for one-step diffusion is unclear. The model learns a fusion target generated by fixed rules. Although A8 compares against a deterministic network, the paper does not clearly state whether the training settings and losses are matched. The gain from diffusion training itself therefore remains unclear.
  2. The contributions of the handcrafted fusion target and the routing framework are not clearly separated. Equation (10) already specifies how low-frequency content and details are fused, but the paper does not report the quality of this target itself. The current experiments do not clearly show whether the gains mainly come from the routing design or from the fusion target and additional supervision.
  3. Generalization under adverse weather is not sufficiently evaluated. The relevant experiments are mainly conducted on AWMM-100K. The M3FD experiment uses only clean inputs, so it does not show how the method performs under adverse weather on other datasets.
Questions for authors
  1. How good is the fusion target constructed by Eq. (10)? Could the authors report its metrics and analyze the gap between the final output and this target?
  2. Since the primary condition already contains infrared information, why is a second infrared-conditioned encoding pass needed? What advantage does residual routing offer over directly fusing the two feature streams?
  3. How much would performance improve if the predicted reliability map were replaced with the ground-truth reliability target? How much do reliability estimation errors affect the results?
  4. The current evaluation provides limited evidence of generalization. Can the method maintain its advantage on other degradation datasets without fine-tuning?
Reviewer_nsr3OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses infrared–visible image fusion under adverse weather through a one-step diffusion framework (RouteFuse). RouteFuse routes low- and high-frequency infrared residuals across multiple scales, conditioned on predicted visible reliability and cross-modal feature relations. The model is trained with an explicit fusion target constructed from clean visible and infrared images and produces the fused image in a single prediction. Experiments demonstrate effective and efficient fusion under adverse weather, and ablations further analyses the routing design.

Critical strengths

The mechanistic analysis is convincing. The paper reports correlations between predicted reliability, local visible fidelity, and realized residual transfer. The Anti-Reliability and Random reallocation experiments further support the importance of spatial allocation to fusion quality.

Critical weaknesses

1. Visible fidelity does not directly indicate the complementary value of infrared information. Reliability in Section 3.2 measures visible-image distortion relative to a clean counterpart, whereas fusion also requires determining whether infrared contains useful corresponding information. Infrared could lack the required textures where visible observations are severely degraded, and could still provide important thermal cues where visible observations are reliable. The authors need to clarify how this reliability measure supports effective information allocation.

2. Cross-dataset generalization under adverse weather lacks validation. The main adverse-weather experiments use held-out scenes from AWMM-100K, while the M3FD evaluation in Appendix C uses clean inputs. Thus, the results do not establish whether reliability estimation and routing generalize across data sources and degradation distributions. Additional experiments are needed to demonstrate this generalization.

Questions for authors

See Weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.