acceptodds
Withdrawn from ICLR 2027

Hybrid Fusion: Minute-Scale Training for Zero-Shot Cross-Domain Image Fusion

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Image fusion integrates complementary information from multiple sources, yet most recent systems still learn to reconstruct every output pixel with patch training, long schedules, or large pretrained backbones. We propose Hybrid Fusion, a source-allocation formulation in which a U-Net predicts only a dense guidance map and a fixed Laplacian-pyramid kernel performs synthesis. The output therefore remains an explicit multi-scale mixture of the two sources, while the learnable task is reduced from image generation to deciding where each source should contribute. This design supports full-resolution optimization, reaches strong infrared-visible fusion quality after one to two epochs, improves downstream detection, and transfers without retraining to medical and video fusion. A unified single-GPU benchmark further shows that the practical advantage comes from rapid adaptation rather than a selectively cheap iteration. For accessibility, our Colab demo reproduces the one-epoch result in about five minutes on a free Google T4 GPU.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
image fusion, infrared-visible fusion, cross-domain generalization, efficient training, Laplacian pyramid, low-level vision, source allocation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_2dDnOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper combines learnable source allocation with fixed Laplacian pyramid synthesis to achieve competitive fusion results after a small number of training epochs. This characteristic has practical value, and the author's exploration of fast training and cross domain applications is worthy of recognition. However, there are still shortcomings in the innovative positioning, method description, and experimental comparison.

Critical strengths

The proposed method achieves competitive fusion performance after only a few training epochs, reducing training time and making it easier to reproduce and use with limited computational resources.

Critical weaknesses

1.The method mainly combines a conventional U-Net with a traditional Laplacian pyramid, using a predicted weight map to guide fusion. Its novelty is limited, which weakens the methodological contribution. 2.Most comparison methods are relatively old, and comparisons with recent relevant approaches are insufficient. This limits the evidence that the proposed method is competitive with the current state of the art.

Questions for authors

1.Could the authors clarify the substantive novelty of the proposed method compared with existing approaches that learn fusion weights or combine deep networks with pyramid-based fusion? The current description primarily presents a combination of a conventional U-Net and a fixed Laplacian pyramid. A clearer explanation of the differences would help readers understand the contribution. 2.Could the authors explain why more recent relevant methods were not included in the comparison? Including these methods would help provide a more complete assessment of the proposed method’s competitiveness. 3.Could the authors provide the parameter counts and inference times of the proposed method and the compared method under the same hardware and input resolution? 4.In the supplementary material, directly using the visible or infrared source image achieves higher scores than the fused results on some metrics. Could the authors explain this observation?

Reviewer_LiASOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Hybrid Fusion separates learned source allocation from fixed multiscale synthesis: a U-Net predicts spatial weights, and a Laplacian pyramid mixes source coefficients and reconstructs the fused image. The authors report useful first-epoch infrared-visible results, detection gains, repeated-run statistics, and medical and video transfer from MSRS. The design has practical value, and the main results support useful short training schedules. However, detection protocols, medical checkpoint selection, and efficiency controls require clarification, while several supporting analyses and configurations are not reported in the paper.

Critical strengths
  1. Separating allocation learning from synthesis is a useful design choice. The network predicts an inspectable source-preference map, while a fixed operator handles coefficient mixing and reconstruction. This gives the method a clear mechanism and keeps the dependency stack small.
  2. The method reaches a useful operating point quickly. By restricting learning to source allocation and keeping synthesis fixed, it gives the model a constrained output space and a direct path to improve fusion quality. This is a practical advantage when retraining time matters.
  3. The evaluation covers several practical views of fusion, including standard quality metrics, downstream detection, medical transfer, frame-wise video application, and hardware cost. This gives the reader a broader view of the method's practical scope.
Critical weaknesses

Major

  1. The medical cross-domain pipeline is not fully defined. The method is specified for an RGB visible image and a single-channel infrared image, but PET, CT, SPECT, and MRI inputs are generally grayscale medical modalities. The paper does not explain how these inputs are mapped to the two branches, normalized, or registered. Without this preprocessing definition, the zero-shot medical results are difficult to reproduce and interpret.
  2. The downstream detection comparison and checkpoint choices are not fully specified. The detector setup and fusion checkpoint are unclear, while the two- and ten-epoch medical results are combined with literature values without a clear rule fixed before target-domain evaluation. These omissions limit the comparability of the detection and transfer claims.
  3. The efficiency and reproducibility evidence does not fully isolate the proposed formulation. Batch sizes, schedules, and reconstruction controls are not consistently matched, and the paper does not fully report loss settings, search criteria, ablations, hardware details, or the DTPF teacher/student variant. These gaps make the source of the reported gain difficult to assess. Minor Comments
  4. For clarity, distinguish coefficient provenance under Equation (1) from an empirical guarantee of artifact-free reconstruction. This is mainly a wording issue and does not indicate an error in the fusion operator.
  5. The video experiment is frame-wise and uses EN, MI, and SD, so it does not measure temporal consistency. The video claim should remain limited to frame-wise application unless a temporal consistency analysis is added.
  6. The scope of the method should be stated more narrowly. It assumes spatially aligned and complementary inputs, and its cross-domain results depend on preprocessing and registration choices. The limitations section should make these conditions explicit.

Suggestions for Rebuttal The rebuttal should address the three main concerns in order of importance. First, define the medical cross-domain pipeline, including how grayscale PET, CT, SPECT, and MRI inputs are mapped, normalized, and registered for the two branches, and state how the medical checkpoints were chosen. Second, specify the detector architecture, training and test protocol, fusion checkpoint, and comparison rule. Third, isolate the efficiency gain with matched batch size, schedule, initialization, loss, reconstruction, and tuning budgets, and report the key loss, search, ablation, hardware, and DTPF details. The paper should also state the spatial-alignment assumption and limit its video claim to frame-wise evaluation.

Questions for authors
  1. What detector architecture and version, initialization, data split, and fusion checkpoint underlie the downstream detection comparison? Were all methods trained and evaluated under the same detector protocol, and was the fusion checkpoint fixed before test evaluation?
  2. How are the grayscale PET, CT, SPECT, and MRI inputs mapped and normalized for the two branches, and what registration procedure is used? How were the two-epoch and ten-epoch checkpoints selected, and was any target-domain test result used in that choice?
  3. Under matched losses, initialization, batch size, update budget, and tuning effort, how do the proposed allocation model, direct reconstruction, and residual reconstruction compare in quality versus wall-clock time?
  4. What are the component loss definitions and default weights, and what validation data and criterion were used for the 240-point search? Please also identify the DTPF teacher/student variant and report the scale-map and hardware results relevant to the efficiency claim.
Reviewer_BwA1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents Hybrid Fusion, a method that treats image fusion as a source-allocation problem instead of directly reconstructing the fused image. It uses a U-Net to predict a guidance map and a fixed Laplacian pyramid to combine information from the source images at different scales. The goal is to make training faster while allowing optimization at full resolution. The experiments show good results on infrared-visible image fusion and downstream object detection, and the authors also test the method on medical images and videos without retraining. Overall, the approach is interesting, but the efficiency comparisons and cross-domain results need more evidence to fully support the claims.

Critical strengths
  1. The idea of separating source allocation from image synthesis is simple and easy to understand. It also makes the overall fusion process more interpretable.
  2. The paper evaluates the method from several perspectives, including fusion quality, object detection, cross-domain transfer, and training stability. This gives a broader view of its performance than fusion metrics alone.
  3. One practical advantage is that the method can achieve good results with only a few training epochs and relatively limited hardware resources. This makes it worth considering for applications where training time and computing resources are limited.
Critical weaknesses
  1. The idea of separating source allocation from image synthesis is simple and easy to understand. It also makes the overall fusion process more interpretable.
  2. The paper evaluates the method from several perspectives, including fusion quality, object detection, cross-domain transfer, and training stability. This gives a broader view of its performance than fusion metrics alone.
  3. One practical advantage is that the method can achieve good results with only a few training epochs and relatively limited hardware resources. This makes it worth considering for applications where training time and computing resources are limited.
Questions for authors

1.Some baseline results are taken from published papers, while others are reproduced using different pretrained weights and training configurations. How much do these differences affect the reported performance gap? Could the authors provide additional comparisons under more consistent settings?

2.The paper compares Hybrid Fusion with direct U-Net reconstruction, but the training setups and computational costs are not fully matched across all methods. Could the authors provide a more controlled comparison to better isolate the contribution of the source-allocation formulation to faster convergence?

3.The cross-domain results vary across datasets and evaluation metrics, and the video model processes frames independently. How robust is the method to temporal inconsistencies and imperfect image alignment? Further evaluation under these conditions would help clarify its practical limitations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.