acceptodds
Under review as a conference paper at ICLR 2027

GeoMFF: Reconstructing a Latent Sharp 3D Scene for Misaligned Multi-Focus Image Fusion

Abstract

Multi-focus image fusion (MFIF) aims to integrate focused information from different focal planes into an all-in-focus image. However, existing methods typically perform focus estimation and fusion in the 2D image space, assuming pixel-wise correspondence among source images. They therefore struggle to handle spatial misalignment caused by camera motion and depth-dependent parallax during practical acquisition. To address this issue, we are the first to formulate misaligned MFIF as the recovery of a latent sharp 3D scene from multiple defocused observations, and propose GeoMFF, a geometry-aware fusion framework. GeoMFF initializes a 3D Gaussian scene using camera parameters and point clouds predicted by VGGT, and jointly optimizes the scene representation, camera poses, and view-dependent spatially varying defocus model to aggregate focused information from different focal planes in 3D space. Within this framework, the Gaussian-Initialized Mixture Blur Network (GMBN) compactly models spatially varying defocus by adaptively combining a shared set of learnable blur kernels. We further construct a real-world misaligned multi-focus image dataset comprising two-image pairs and seven-image sequences. Experiments on our dataset and multiple public benchmarks demonstrate that our method achieves state-of-the-art performance on misaligned image pairs, remains highly competitive on aligned benchmarks, and extends effectively to multi-focus image sequences.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multi-focus image fusion, Misaligned images, All-in-focus Image Reconstruction, Defocus Modeling, Spatially Varying Blur

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_KHQSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper formulates misaligned multi-focus image fusion (Misaligned MFIF) as a latent sharp 3D scene reconstruction problem. It proposes the GeoMFF framework, which combines VGGT initialization, camera pose residual optimization, and GMBN defocus blur modeling. In addition, the authors construct a real-world misaligned dataset containing two-image pairs and seven-image sequences.

Critical strengths
  1. Formulating misaligned MFIF as a 3D scene reconstruction problem is reasonable. Traditional 2D methods struggle with accurate registration under defocus and parallax. Using 3DGS to perform geometric alignment and sharp information aggregation in 3D space avoids the error propagation of explicit 2D registration.

  2. To address the failure of COLMAP initialization under small parallax and severe defocus in MFIF, the paper introduces VGGT to provide initial point clouds and camera parameters, combined with pose residual refinement. The implementation is complete and effective.

  3. GMBN represents spatially varying defocus blur using linear combinations of shared Gaussian kernel atoms and incorporates blur mask regularization, which constrains parameter degrees of freedom while preventing over-smoothing in sharp regions.

  4. The paper constructs a real-world misaligned multi-focus dataset containing two-image pairs and seven-image sequences (GeoMFF2 and GeoMFF7), which is practically useful for future research.

Critical weaknesses
  1. The baseline comparison in the main experiment (Table 1) is unreasonable. All 10 MFIF baselines in Table 1 are pure 2D models that assume aligned inputs. Directly comparing a 3D alignment model with pure 2D models on misaligned data is unfair.

  2. Relevant baselines are missing. Methods specifically designed for misaligned MFIF mentioned in Section 2 (e.g., DSAF-Net, Chen et al., 2024; Ji et al., 2023; Zhu et al., 2025) are not included in the experimental comparisons. For 3D defocus reconstruction, only CoCoGaussian is compared, omitting highly related methods such as BAGS (Peng et al., 2024, from which this paper directly adopts the progressive training strategy) and DOF-GS (Wang et al., 2025b).

  3. Camera pose estimation is the most challenging part of 3D scene optimization, yet the paper does not explain how the camera poses (especially the target rendering pose) are specifically selected. Without a clear explanation, there may be a major omission in this paper. Meanwhile, Section 4.1 mentions using RoMa to align each source-image pair for metric computation, but does not specify which reference view the outputs of different methods and the RoMa-aligned source images are in. Inconsistent reference views or resampling interpolation on source images will affect the accuracy of the objective metrics.

  4. Comparisons of computational cost and runtime are missing. The proposed method relies on per-scene 3DGS optimization requiring 15K to 22K iterations on an RTX 5090, whereas most compared deep learning MFIF baselines are feed-forward networks taking milliseconds or seconds. Per-scene optimization time, GPU memory usage, and parameter counts are not reported in the main text.

Questions for authors
  1. Please provide comparisons with methods specifically designed for misaligned MFIF (e.g., DSAF-Net) and related 3D defocus reconstruction methods (BAGS, DOF-GS).

  2. Please clarify how the camera poses (especially the target camera pose for all-in-focus rendering) are determined and selected. When using RoMa to align source images for metric calculation, which reference view are the fused outputs and the aligned source images placed in?

  3. Please report the per-scene optimization runtime, GPU memory usage, and number of parameters, and compare them with the baselines.

3 comments
AuthorsOpenReview ↗

Response to Reviewer KHQS

W1: The baseline comparison in the main experiment (Table 1) is unreasonable

The original ten 2D MFIF baselines show how conventional fusion methods perform on misaligned inputs. To broaden the comparison, we added methods designed for unaligned inputs, including DSIFT, DSAF-Net and MA-MFIF, as well as the 3D reconstruction methods Deblur-NeRF and BAGS. The new comparison results are shown in Table 1. On real misaligned data, GeoMFF outperforms DSAF-Net and MA-MFIF on most MFIF metrics. Comparisons with the 3D methods also support its effectiveness on this task.

Table 1. Quantitative comparison on the GeoMFF2 dataset.

MethodNMI ↑MAE ↓SD ↑SF ↑QAB/F ↑QY ↑SCD ↑SSIM ↑QCB ↑VIFF ↑
DSIFT0.578421.276042.835017.00210.43140.7061-0.18930.57410.48890.5795
GFF0.396721.180640.882514.53410.41740.5565-0.08420.60760.49390.5913
IFCNN0.385221.436140.125215.27760.37550.5587-0.09180.63570.43480.4237
DRPL0.406724.586843.404419.13700.35810.4292-0.16100.52670.45150.3712
DSAF-Net0.511916.226442.084611.36060.34610.82980.13650.71630.47420.5892
MA-MFIF0.344421.985241.058314.48690.30890.7286-0.19070.61400.43040.3627
U2Fusion0.339523.295739.449512.33200.32570.8211-0.10880.63290.43120.3728
MFF-GAN0.411619.107739.898015.10980.38510.6099-0.01930.66960.45760.4910
SwinFusion0.379620.721638.679412.93960.37490.5405-0.11820.65320.42980.4502
ZMFF0.401422.689442.293916.43760.41250.5142-0.15190.56510.47930.4891
DMANet0.420521.394641.265215.37210.41630.5358-0.12140.59590.47430.5249
SD-Fuse0.529419.526640.529612.87970.41380.6346-0.08140.65310.49070.5779
Text-MFF0.384976.619019.83107.40520.13280.3718-0.91210.56470.18740.3071
Deblur-NeRF0.523817.342943.565814.69080.30930.77050.11940.65590.47830.5487
CoCoGaussian0.492318.413342.561111.87730.36810.60910.15950.71210.49240.6461
BAGS0.713515.895944.109416.29730.46170.86900.41400.76200.55630.8491
BAGS+Pose0.717814.554443.270613.86300.46990.86790.36690.76010.56180.8523
Ours0.737114.510643.567614.81730.47580.88560.36710.75140.56570.8549

We further evaluated the methods on WHU-MFM using its all-in-focus ground truth and the full-reference metrics PSNR, SSIM, and LPIPS in Table 2. GeoMFF outperforms DSAF-Net and MA-MFIF on all three and ranks among the leading methods overall. These results support our conclusions through comparisons with misalignment-aware baselines and ground-truth evaluation.

AuthorsOpenReview ↗

Response to Reviewer KHQS

Table 2. Quantitative comparison on WHU-MFM using full-reference and standard fusion metrics.

MethodPSNR ↑SSIM ↑LPIPS ↓NMI ↑MAE ↓SD ↑QAB/F ↑QY ↑QCB ↑VIFF ↑
DSIFT27.29260.79900.28310.75156.546828.05150.42460.86560.42240.6878
GFF28.12810.80950.26770.66396.866227.72120.41980.87560.42290.6799
IFCNN27.94630.82110.29460.69015.989426.91620.33760.88900.36400.5365
DRPL26.02050.78350.30090.70728.035928.36450.36980.85050.42200.5521
DSAF-Net30.64610.85710.29660.73323.751528.05410.30150.93750.36040.5897
MA-MFIF26.55790.82630.29960.70515.804927.42710.37950.90170.40170.6414
U2Fusion25.83170.81340.28070.619410.717825.32840.30350.88480.38640.5094
MFF-GAN26.76220.80260.34260.58417.988426.44700.28320.87130.36840.4664
SwinFusion28.31770.82450.27390.68655.995426.73540.34320.89950.39490.6009
ZMFF27.65980.80080.27950.66146.682127.85150.40330.87290.41380.6231
DMANet27.79050.81070.26180.69166.660127.66580.40110.87590.40440.6564
SD-Fuse28.05550.81740.27190.68926.161027.07500.39010.88720.41620.6383
Text-MFF10.65460.65300.38940.650473.431513.37520.13150.73780.12590.3230
Deblur-NeRF28.48630.79780.29960.66655.336828.22840.25040.87310.38300.5181
CoCoGaussian28.00410.83270.31580.71244.519928.40120.28590.90210.41660.6864
BAGS30.65720.88790.24370.98473.391029.23110.40330.93110.47150.8778
BAGS+Pose31.20580.88290.23430.98153.205928.55780.39190.93180.47230.8791
Ours31.59700.88700.21961.01133.074229.39430.47130.93370.49821.0018

W2: Relevant baselines are missing

In the revised manuscript, we added DSAF-Net and MA-MFIF for unaligned inputs, Deblur-NeRF and BAGS as 3D baselines, and retained CoCoGaussian. We retrained DSAF-Net for two-image input using its public code. Ji et al. (2023) and Zhu et al. (2025) are also relevant, but their code has not been released, making reliable comparison under the same protocol difficult. The new results are in W1. For DOF-GS, we attempted to run its public implementation, but its data preparation requires COLMAP for geometric reconstruction and point cloud generation. On the multi-focus inputs studied here, which have limited viewpoint variation and substantial defocus, COLMAP could not complete this process reliably. Consequently, we were unable to obtain usable results covering all test scenes.

W3: The paper does not explain how the camera poses are specifically selected.

GeoMFF uses VGGT to initialize each source image’s camera intrinsics and pose. During joint optimization of the scene and defocus model, it fixes the intrinsics and refines each pose through a learnable residual rigid transformation. All results labeled “Ours,” for both qualitative and quantitative evaluation, are rendered from the shared 3D scene at source image A’s optimized pose.

We use A as the reference, align B to A, and define a common evaluation region from valid overlap and matching confidence. Fused outputs are not resampled with RoMa; all methods use the same reference view, aligned source image, and evaluation region. We have clarified the rendering view and RoMa’s role in Section 4.1 of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer KHQS

W4: Comparisons of computational cost and runtime are missing

We measured parameter counts and runtimes on WHU-MFM~\citep{chen2024defocus}, as shown in Table 3. GeoMFF uses approximately 30,000 Gaussian primitives per scene, and its GPU memory usage is not the highest among the compared methods. Its main computational cost comes from per-scene optimization. Although this optimization is slower than the inference of feed-forward MFIF methods, conventional MFIF methods are prone to ghosting and structural discontinuities when the inputs are misaligned. Our main contribution is an approach that addresses this problem by recovering a shared latent sharp 3D scene. In future work, we will explore feed-forward networks to reduce per-scene optimization costs and accelerate end-to-end processing.

Table 3. Parameter counts and average runtime of representative methods on WHU-MFM.

MethodsIFCNNDRPLDSAF-NetU2FusionMFF-GANSwinFusionDeblur-NeRFCoCoGaussianBAGSOurs
Total parameters (M)0.0841.072.6270.6590.040.9741.2151.831.6481.468
Training time (s)——————21060366.7385.4374.2
Inference time (ms)166010.291100318.7538.23512.6719900.74960.8551.27

— indicates that the method does not require per-scene optimization.

Q1: Please provide comparisons with methods specifically designed for misaligned MFIF

Please see our response to W1 for the added comparisons with misaligned MFIF and 3D defocus reconstruction methods. We also attempted to run the public implementation of DOF-GS. However, the COLMAP-based geometric reconstruction and point cloud generation required for its data preparation could not be completed reliably on the inputs studied here, which have limited viewpoint variation and substantial defocus. We were therefore unable to obtain usable results for all test scenes.

Q2: Clarify how the camera poses are determined and selected

GeoMFF uses VGGT to initialize each source image’s camera parameters, fixes the intrinsics, and refines the poses through residual rigid transformations during joint optimization. The all-in-focus result is rendered directly from the shared 3D scene at source image A’s optimized pose, without GMBN. For metrics requiring source-image correspondences, A is also the reference view, and RoMa aligns B to A. The fused output is not resampled. All methods use the same aligned source image and evaluation region. We have clarified these settings in Section 4.1 of the revised manuscript.

Q3: Report the per-scene optimization runtime, GPU memory usage, and number of parameters

Please see our response to W4.

We are grateful for the reviewer’s time and valuable feedback. We hope the additional experiments and revisions address your concerns, and we welcome further discussion.

Reviewer_JrgvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GeoMFF, a geometry-aware framework for multi-focus image fusion that addresses spatial misalignment caused by camera motion. The method formulates fusion as the recovery of a latent sharp 3D scene from multiple defocused observations. Initialized with camera parameters and point clouds predicted by VGGT, GeoMFF jointly optimizes a 3D Gaussian scene representation, camera poses, and a spatially varying defocus model implemented through the GaussianInitialized Mixture Blur Network (GMBN). The paper also introduces a real-world misaligned multi-focus dataset and reports experiments on both image pairs and multi-image sequences.

Critical strengths

Good presentation:

This paper clearly describes the overall architecture of GeoMFF and the roles of its main components, making the training pipeline and joint optimization process easy to follow.

Strong quantitative performance:

The experimental results show that GeoMFF consistently outperforms the compared baselines across the reported quantitative metrics on GeoMFF2 and GeoMFF7, demonstrating its effectiveness on both misaligned image pairs and multi-image sequences

Critical weaknesses

As stated in line 192, the proposed approach to mitigating misalignment in MFIF essentially consists of reconstructing an all-in-focus scene from defocused images. However, this reconstruction strategy has already been explored in NeRF- and Gaussian-splatting-based methods, including BAGS and CoCoGaussian. Moreover, residual rigid transformations for refining initial camera poses have been used in prior work, such as LighthouseGS and KeyGS. The design of GMBN also appears similar to that of the Blur Proposal Network in BAGS.

Since GMBN applies defocus blur to the rendered images, how can it be ensured that these images are sharp before being processed by GMBN and that the defocus blur is introduced entirely by GMBN?

GeoMFF requires per-scene optimization, which may introduce substantial computational overhead compared with feed-forward MFIF methods. Moreover, the number of parameters in its Gaussian scene representation depends on the number of Gaussian primitives. Please provide a comparison of training times and the number of model parameters.

This paper does not explicitly specify the camera poses used to render the results labeled “Ours” in the visual comparisons or those used for quantitative evaluation. This omission makes it difficult to reproduce the reported quantitative and qualitative results

Questions for authors

An in-depth comparison between GeoMFF and existing NeRF/GS-based reconstruction methods is missing. Please include experimental comparisons with Deblur-NeRF and BAGS and report GeoMFF’s performance on the defocus dataset used in Deblur-NeRF.

What is the size of the local window \Omega? Please include an ablation study on the effect of window size. Additionally, line 336 of the paper mentions the use of eight Gaussian basis kernels. Please conduct an ablation study to evaluate the effect of the number of these kernels

2 comments
AuthorsOpenReview ↗

Response to Reviewer Jrgv

W1: Novelty relative to existing 3D reconstruction methods and components

We would like to clarify the task addressed in this paper: GeoMFF targets misaligned multi-focus image fusion, rather than 3D reconstruction from defocused images as an end goal, as in BAGS and CoCoGaussian. We do not claim these components individually as novel. Rather, it applies the strategy to a task setting that has not been sufficiently investigated: generating an all-in-focus fused image from a specified viewpoint using multi-focus inputs with limited viewpoint variation, misalignment, and complementary sharp regions. In this setting, limited viewpoint variation and defocus make geometry and pose initialization more difficult, while input misalignment makes 2D pixel correspondences unreliable. To the best of our knowledge, GeoMFF is the first to formulate misaligned MFIF as the recovery of a shared latent sharp 3D scene and design a corresponding solution for this task. Although residual rigid transformations have precedents, our design focuses on jointly optimizing camera poses, scene geometry, and the defocus model for misaligned multi-focus inputs, so that pose refinement directly supports the fusion of sharp content across views. For blur modeling, the BPN in BAGS directly predicts a per-pixel convolution kernel. Our GMBN instead constructs local defocus kernels from a shared set of learnable, Gaussian-initialized kernel atoms using position-dependent weights, to model defocus variation in the MFIF task.

We have also added comparisons with 3D reconstruction methods, including BAGS[1] and Deblur-NeRF[2], in the revised manuscript. With adaptations for multi-focus inputs and the fusion objective, these methods could also serve as alternative implementations of the 3D solution proposed in this paper. Overall, GeoMFF is the first to propose a 3D solution for misaligned MFIF and designs corresponding joint optimization and defocus modeling mechanisms for this task. This task formulation also provides a direction for accelerating fusion through feed-forward methods in future work.

W2: Ensuring the rendered image is sharp before GMBN

We do not assume that the 3D scene can render an all-in-focus image at initialization. Sharp content in the rendered images is gradually recovered during joint optimization using complementary sharp regions from different source images. These regions constrain the shared scene through the blur mask’s direct-rendering branch, while GMBN applies view-dependent reblurring to match defocused regions in the source images. Blur-mask regularization suppresses unnecessary reblurring in sharp regions. Together, these observations and constraints encourage the shared scene to represent sharp content and GMBN to represent defocus degradation, without guaranteeing complete separation in advance. Ablations removing GMBN and blur-mask regularization support these designs.

W3: Per-scene optimization cost and Gaussian parameter count

We measured parameter counts and runtime on WHU-MFM~\citep{chen2024defocus}, as shown in Table 1. GeoMFF’s GPU memory usage is not the highest among the compared methods, and each scene uses approximately 30,000 Gaussian primitives. Its main computational cost comes from per-scene optimization. Our primary contribution is a new 3D approach to misaligned MFIF; future feed-forward networks could reduce the optimization cost and accelerate end-to-end processing.

Table 1. Parameter counts and average runtime of representative methods on WHU-MFM.

MethodsIFCNNDRPLDSAF-NetU2FusionMFF-GANSwinFusionDeblur-NeRFCoCoGaussianBAGSOurs
Total parameters (M)0.0841.072.6270.6590.040.9741.2151.831.6481.468
Training time (s)——————21060366.7385.4374.2
Inference time (ms)166010.291100318.7538.23512.6719900.74960.8551.27

— indicates that the method does not require per-scene optimization.

W4: does not explicitly specify the camera poses

Thank you for pointing out this ambiguity. The images labeled “Ours” in the qualitative comparisons and those used for quantitative evaluation are rendered using the same setting: we use source image A as the target view and directly render the all-in-focus result from the shared 3D scene using its camera intrinsics and optimized camera extrinsics, without passing the result through GMBN. We have clarified this setting in Section 4.1 of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer Jrgv

Q1: Comparisons with NeRF/GS methods and the Deblur-NeRF dataset

In the original manuscript, the GeoMFF2 comparison in Table 1 included CoCoGaussian as the only 3D reconstruction baseline. In the revised manuscript, we added the original BAGS and Deblur-NeRF using the same input preprocessing as GeoMFF. Their results are reported in Section 4.2 and Table 1 of the revised main paper and reproduced in Table 2 below. GeoMFF performs better overall in the misaligned MFIF setting.

Table 2. Quantitative comparison on the GeoMFF2 dataset.

MethodNMI ↑MAE ↓SD ↑SF ↑QAB/F ↑QY ↑SCD ↑SSIM ↑QCB ↑VIFF ↑
DSIFT0.578421.276042.835017.00210.43140.7061-0.18930.57410.48890.5795
GFF0.396721.180640.882514.53410.41740.5565-0.08420.60760.49390.5913
IFCNN0.385221.436140.125215.27760.37550.5587-0.09180.63570.43480.4237
DRPL0.406724.586843.404419.13700.35810.4292-0.16100.52670.45150.3712
DSAF-Net0.511916.226442.084611.36060.34610.82980.13650.71630.47420.5892
MA-MFIF0.344421.985241.058314.48690.30890.7286-0.19070.61400.43040.3627
U2Fusion0.339523.295739.449512.33200.32570.8211-0.10880.63290.43120.3728
MFF-GAN0.411619.107739.898015.10980.38510.6099-0.01930.66960.45760.4910
SwinFusion0.379620.721638.679412.93960.37490.5405-0.11820.65320.42980.4502
ZMFF0.401422.689442.293916.43760.41250.5142-0.15190.56510.47930.4891
DMANet0.420521.394641.265215.37210.41630.5358-0.12140.59590.47430.5249
SD-Fuse0.529419.526640.529612.87970.41380.6346-0.08140.65310.49070.5779
Text-MFF0.384976.619019.83107.40520.13280.3718-0.91210.56470.18740.3071
Deblur-NeRF0.523817.342943.565814.69080.30930.77050.11940.65590.47830.5487
CoCoGaussian0.492318.413342.561111.87730.36810.60910.15950.71210.49240.6461
BAGS0.713515.895944.109416.29730.46170.86900.41400.76200.55630.8491
BAGS+Pose0.717814.554443.270613.86300.46990.86790.36690.76010.56180.8523
Ours0.737114.510643.567614.81730.47580.88560.36710.75140.56570.8549

The Deblur-NeRF defocus dataset evaluates sharp-view reconstruction from defocused multi-view images. GeoMFF instead targets fusion of misaligned multi-focus inputs with limited viewpoint variation and complementary sharp regions. We therefore did not report results on that reconstruction benchmark. Instead, we evaluated Deblur-NeRF and BAGS on the same MFIF task and inputs as GeoMFF.

Q2: Ablations on blur window size and kernel atom count

The local window Ω\Omega is 17×1717\times17. We have added an ablation on window size to the supplementary material. The ablation on the number of Gaussian kernel atoms, including the eight-atom setting used in GeoMFF, is also provided in Table 3 and Table 4.

Table 3. Ablation on the number of Gaussian kernel atoms MM.

MMNMI ↑MAE ↓SD ↑SF ↑QAB/F ↑QY ↑SCD ↑SSIM ↑QCB ↑VIFF ↑
20.724114.565943.613414.56380.46580.89130.36800.75340.55750.8336
40.719514.665643.683314.82670.46610.88850.36540.75020.55730.8333
80.737114.510643.567614.81730.47580.88560.36710.75140.56570.8549
160.722114.546643.602514.55650.46820.89570.36300.75650.55850.8421

Table 4. Ablation on the local window size Ω\Omega.

Window sizeNMI ↑MAE ↓SD ↑SF ↑QAB/F ↑QY ↑SCD ↑SSIM ↑QCB ↑VIFF ↑
5×50.742614.190342.995714.37930.46290.87700.33150.81390.56130.8403
9×90.727414.450743.331314.55110.46500.87260.34800.80790.55830.8345
17×170.737114.510643.567614.81730.47580.88560.36710.75140.56570.8549
33×330.726814.473943.870014.06570.47010.88110.40570.81370.55480.8479

Thank you for the detailed and helpful comments. We have addressed them in the revised manuscript and in our responses, and remain happy to provide further clarification.

Reviewer_KRSbOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The author argues that for multifocal images with spatial mismatch, establishing cross-view geometric consistency through a shared latent sharp three-dimensional scene, and jointly optimizing camera poses with a view-dependent spatially varying defocus model, can more reliably integrate complementary sharp information from different focal images than fusion paradigms that rely on two-dimensional correspondences.

Critical strengths
  1. This article appears to propose a novel research framework that differs from the conventional Registration-then-Fusion paradigm, in which the authors advocate integrating geometric alignment with feature restoration.
  2. The experimental results appeared to be highly effective.
  3. Supports multiple image inputs.
Critical weaknesses
  1. The idea is intuitive and leans toward a practical, engineering-oriented approach.
  2. The geometric correspondence of the same 3D point across different camera viewpoints does not necessarily imply that explicit 3D scene reconstruction is required for fusion. Does the causal relationship between “We therefore formulate misaligned MFIF xxxx” and this premise hold?
  3. The author consistently emphasizes geometric consistency throughout the paper, yet the experimental section fails to address or validate this aspect, which is rather puzzling.
  4. “As conventional MFIF metrics assume spatially corresponding source images, we use RoMa to align each source-image pair for metric computation.” However, the authors note that two-dimensional registration is unreliable. Does this not present a contradiction? This is merely a point of curiosity.
Questions for authors
  1. The problem of misalignment has been largely addressed in the literature, for example through the use of optical flow or attention-based alignment mechanisms. A natural question, then, is why these approaches are not discussed in the present work. A straightforward and readily conceivable experiment would involve adding an alignment network before existing methods that are trained for aligned scenarios. The authors do not explain why such experiments were not considered.
  2. The author's description of geometric consistency does not appear to receive any corresponding feedback in the experimental section. It is recommended that this issue be discussed, as it seems to constitute a central argument of the entire paper.
  3. Do the evaluation metrics conflict with the starting point of this paper?
  4. Is coupling really better than separation?
4 comments
AuthorsOpenReview ↗

Response to Reviewer KRSb

W1: The idea is intuitive and leans toward a practical, engineering-oriented approach.

To the best of our knowledge, GeoMFF is the first to formulate misaligned MFIF as recovering a shared latent sharp 3D scene from images captured at different camera poses and focal settings. This formulation brings geometric correspondence and complementary focused content into a common scene representation. It applies to both image pairs and multi-image sequences, and our experiments show reduced ghosting and boundary discontinuities. The formulation is not tied to GMBN; defocus-aware reconstruction methods such as BAGS can also be adapted to this task.

To make this formulation work under misalignment MFIF, GeoMFF jointly optimizes the Gaussian scene, camera poses, and blur model. Pose changes shift scene content across views, while defocus weakens the cues needed to estimate geometry. GMBN models the resulting spatially varying blur with location-dependent mixtures of shared Gaussian-initialized kernel atoms. Together, the task formulation and these components provide a concrete method for misaligned MFIF.

W2: The geometric correspondence of the same 3D point across different camera viewpoints does not necessarily imply that explicit 3D scene reconstruction is required for fusion.

The cross-view correspondence equation shows that shared geometry and camera parameters provide a common reference; it does not establish that explicit 3D reconstruction is the only solution. We chose a 3D representation because depth-dependent parallax and occlusion complicate pairwise registration, while defocus weakens matching cues. As shown in Fig. 2, RoMa pre-registration reduces misalignment, yet fusion PSNR and SSIM still decline as camera displacement increases. A shared latent sharp 3D scene lets us keep each input in its original view and jointly optimize geometry, camera poses, and view-dependent defocus. We have clarified this rationale and the equation’s role in the revised Motivation section.

W3: Geometric consistency lacks experimental validation.

Our method targets misaligned multi-focus image fusion. By geometric consistency, we mean that the shared 3D scene provides consistent projections of the same scene point across views to support the fusion of focused information. High-precision 3D reconstruction is not a separate objective of this work. Following the reviewer’s suggestion, we added depth-map visualizations to the supplementary material and evaluated cross-view geometric correspondence using reprojection error. Specifically, we use the GT depth and camera parameters provided by WHU-MFM to identify reference points visible in both views. We then project these points using each method’s own rendered depth and camera parameters and measure the pixel distance between the predicted positions and the GT correspondences. All methods use the same reference point set. Table 1 reports the median reprojection error, P90, PCK@1/3/5px, and valid projection coverage.

Table 1. Cross-view reprojection accuracy on WHU-MFM.

MethodsMedian ↓ (px)P90 ↓ (px)PCK@1 ↑PCK@3 ↑PCK@5 ↑
CoCoGaussian2.640113.81370.20%60.68%70.61%
BAGS2.4855.97970.90%65.83%81.83%
Ours2.31634.24631.65%69.79%85.55%

GeoMFF achieves lower median and P90 errors than BAGS and CoCoGaussian, as well as the highest accuracy at all three PCK thresholds. Its P90 is 4.2463 px, compared with 5.9797 px for BAGS, indicating fewer large reprojection errors. All three methods achieve 100% coverage, so differences in valid projection rates do not affect this comparison. These results provide direct evidence of GeoMFF’s cross-view geometric correspondence and complement the experiments focused on fusion quality.

AuthorsOpenReview ↗

Response to Reviewer KRSb

W4: Using RoMa for metric computation appears inconsistent with the paper’s criticism of 2D registration.

Here, 2D registration is used at two different stages and does not constitute a contradiction. During fusion, Using registration as a fixed preprocessing step for fusion can pass errors caused by defocus, parallax, and occlusion into the fused result. GeoMFF instead takes the unregistered images as input and relates them through a shared 3D scene and camera poses.

For evaluation, RoMa is used only to establish a common coordinate system for metrics that depend on pixel-wise correspondence with the source images; it does not participate in fusion by GeoMFF or the other methods. We consistently use the viewpoint of source image A as the reference, align source image B to A, and define the evaluation region based on their valid overlap and matching confidence. All methods are evaluated in the same reference view and valid region. We also report PSNR, SSIM, and LPIPS against the all-in-focus ground truth on WHU-MFM; these full-reference results broadly corroborate the standard fusion metrics. We have clarified the distinct roles of registration in the revised evaluation protocol.

Q1: Why are optical-flow or attention-based alignment methods and alignment-network baselines not considered?

In the revised manuscript, we have expanded the Related Work discussion of 2D alignment methods, including optical flow and feature-based alignment (Section 2). We have also added comparisons with DSIFT, DSAF-Net [1], and MA-MFIF, as well as Deblur-NeRF, CoCoGaussian, BAGS, and BAGS+Pose, which adds our pose refinement to BAGS (Section 4.2 and Table 1). DSAF-Net was retrained for two-image input to match the other methods. also include BAGS+Pose, which augments BAGS with our camera pose refinement. The results on GeoMFF2 are shown in Table 2.

Table 2. Quantitative comparison on the GeoMFF2 dataset.

MethodNMI ↑MAE ↓SD ↑SF ↑QAB/F ↑QY ↑SCD ↑SSIM ↑QCB ↑VIFF ↑
DSIFT0.578421.276042.835017.00210.43140.7061-0.18930.57410.48890.5795
GFF0.396721.180640.882514.53410.41740.5565-0.08420.60760.49390.5913
IFCNN0.385221.436140.125215.27760.37550.5587-0.09180.63570.43480.4237
DRPL0.406724.586843.404419.13700.35810.4292-0.16100.52670.45150.3712
DSAF-Net0.511916.226442.084611.36060.34610.82980.13650.71630.47420.5892
MA-MFIF0.344421.985241.058314.48690.30890.7286-0.19070.61400.43040.3627
U2Fusion0.339523.295739.449512.33200.32570.8211-0.10880.63290.43120.3728
MFF-GAN0.411619.107739.898015.10980.38510.6099-0.01930.66960.45760.4910
SwinFusion0.379620.721638.679412.93960.37490.5405-0.11820.65320.42980.4502
ZMFF0.401422.689442.293916.43760.41250.5142-0.15190.56510.47930.4891
DMANet0.420521.394641.265215.37210.41630.5358-0.12140.59590.47430.5249
SD-Fuse0.529419.526640.529612.87970.41380.6346-0.08140.65310.49070.5779
Text-MFF0.384976.619019.83107.40520.13280.3718-0.91210.56470.18740.3071
Deblur-NeRF0.523817.342943.565814.69080.30930.77050.11940.65590.47830.5487
CoCoGaussian0.492318.413342.561111.87730.36810.60910.15950.71210.49240.6461
BAGS0.713515.895944.109416.29730.46170.86900.41400.76200.55630.8491
BAGS+Pose0.717814.554443.270613.86300.46990.86790.36690.76010.56180.8523
Ours0.737114.510643.567614.81730.47580.88560.36710.75140.56570.8549

We also tested the suggested alignment-then-fusion approach by applying RoMa before existing 2D MFIF methods. In the controlled synthetic experiment, pre-registration reduces misalignment, but fusion PSNR and SSIM still decline overall as camera displacement increases, as shown in Table 3. Real-scene results show residual ghosting and structural discontinuities. We have added these experiments and the corresponding discussion to Section 3.1 of the main paper and Section 3 of the supplementary material.

AuthorsOpenReview ↗

Response to Reviewer KRSb

Table 3. PSNR / SSIM under varying camera displacement.

PSNR (dB) / SSIM at different camera displacements.

Camera displacementIFCNNIFCNN+RoMaU2FusionU2Fusion+RoMaZMFFZMFF+RoMaDMANetDMANet+RoMaOurs
0.00624.510 / 0.77025.232 / 0.86023.422 / 0.72523.487 / 0.81323.916 / 0.77025.263 / 0.85425.020 / 0.79125.764 / 0.86928.376 / 0.896
0.01123.183 / 0.72825.096 / 0.85722.381 / 0.68623.393 / 0.81121.885 / 0.71225.162 / 0.85222.752 / 0.73625.522 / 0.86627.824 / 0.888
0.01722.720 / 0.71425.003 / 0.85521.882 / 0.67223.344 / 0.80921.409 / 0.69024.928 / 0.84821.828 / 0.71225.325 / 0.86427.230 / 0.876
0.02222.320 / 0.69824.947 / 0.85321.496 / 0.65823.296 / 0.80820.797 / 0.67125.030 / 0.84821.288 / 0.69425.245 / 0.86227.263 / 0.870
0.02821.924 / 0.68524.816 / 0.85121.140 / 0.64723.194 / 0.80620.416 / 0.65624.962 / 0.84520.881 / 0.68125.054 / 0.85927.038 / 0.869
0.03321.572 / 0.67424.696 / 0.84920.791 / 0.63723.095 / 0.80420.137 / 0.64524.873 / 0.84420.476 / 0.66724.901 / 0.85726.667 / 0.879
0.03921.274 / 0.66324.548 / 0.84620.464 / 0.62722.991 / 0.80219.240 / 0.62424.392 / 0.83920.087 / 0.65424.710 / 0.85326.052 / 0.871
0.04420.977 / 0.65424.500 / 0.84420.143 / 0.62022.952 / 0.80019.079 / 0.61924.380 / 0.83619.693 / 0.64424.711 / 0.85126.392 / 0.877
0.05020.706 / 0.64524.407 / 0.84419.849 / 0.61222.907 / 0.80018.583 / 0.60924.259 / 0.83819.362 / 0.63324.579 / 0.85026.650 / 0.873
0.05520.487 / 0.63624.301 / 0.84119.598 / 0.60422.832 / 0.79718.611 / 0.60024.157 / 0.83319.085 / 0.62424.438 / 0.84726.538 / 0.868

Q2: How is geometric consistency evaluated?

Please see our response to W3. Our primary objective is misaligned multi-focus image fusion; geometric consistency supports this objective through the shared 3D representation rather than serving as a separate reconstruction goal. We have added depth visualizations and GT-based cross-view reprojection results to evaluate it directly.

Q3: Are the evaluation metrics consistent with the paper’s motivation?

Thank you for the further question. We believe that our evaluation approach does not conflict with the motivation of this work. Our goal is to enable fusion methods to directly process unregistered source images. For real captured images without all-in-focus ground truth, we use source-image-based MFIF metrics to evaluate information preservation and fusion quality. Metrics that depend on pixel-wise correspondence require a common reference view during evaluation.

Specifically, we align source image B to A only during evaluation and define a common evaluation region based on their valid overlap and matching confidence. All methods, including GeoMFF, are evaluated in the same reference view and region. This alignment is used solely to establish the correspondences required by the metrics; it does not participate in generating the fused results. We also evaluate on WHU-MFM[2], using its all-in-focus ground truth. GeoMFF achieves the highest PSNR and lowest LPIPS, with the second-highest SSIM. These full-reference results, together with the qualitative comparisons, support the conclusions drawn from the MFIF metrics.

AuthorsOpenReview ↗

Response to Reviewer KRSb

Table 4. Quantitative comparison on WHU-MFM using full-reference and standard fusion metrics.

MethodPSNR ↑SSIM ↑LPIPS ↓NMI ↑MAE ↓SD ↑QAB/F ↑QY ↑QCB ↑VIFF ↑
DSIFT27.29260.79900.28310.75156.546828.05150.42460.86560.42240.6878
GFF28.12810.80950.26770.66396.866227.72120.41980.87560.42290.6799
IFCNN27.94630.82110.29460.69015.989426.91620.33760.88900.36400.5365
DRPL26.02050.78350.30090.70728.035928.36450.36980.85050.42200.5521
DSAF-Net30.64610.85710.29660.73323.751528.05410.30150.93750.36040.5897
MA-MFIF26.55790.82630.29960.70515.804927.42710.37950.90170.40170.6414
U2Fusion25.83170.81340.28070.619410.717825.32840.30350.88480.38640.5094
MFF-GAN26.76220.80260.34260.58417.988426.44700.28320.87130.36840.4664
SwinFusion28.31770.82450.27390.68655.995426.73540.34320.89950.39490.6009
ZMFF27.65980.80080.27950.66146.682127.85150.40330.87290.41380.6231
DMANet27.79050.81070.26180.69166.660127.66580.40110.87590.40440.6564
SD-Fuse28.05550.81740.27190.68926.161027.07500.39010.88720.41620.6383
Text-MFF10.65460.65300.38940.650473.431513.37520.13150.73780.12590.3230
Deblur-NeRF28.48630.79780.29960.66655.336828.22840.25040.87310.38300.5181
CoCoGaussian28.00410.83270.31580.71244.519928.40120.28590.90210.41660.6864
BAGS30.65720.88790.24370.98473.391029.23110.40330.93110.47150.8778
BAGS+Pose31.20580.88290.23430.98153.205928.55780.39190.93180.47230.8791
Ours31.59700.88700.21961.01133.074229.39430.47130.93370.49821.0018

Q4: Does joint modeling outperform a separated approach?

We evaluated joint modeling against a registration-then-fusion pipeline in both synthetic and real scenes. In the controlled experiment in Section 3.1, we applied RoMa before conventional MFIF methods. Pre-registration improves alignment, but the PSNR and SSIM of the fused results still decline overall as camera displacement increases. On real scenes, RoMa followed by 2D fusion leaves local ghosting and structural discontinuities, especially near depth boundaries and occlusions (Supplementary Figs. 3 and 4). We also added DSAF-Net and MA-MFIF, which are designed for misaligned inputs, alongside Deblur-NeRF, CoCoGaussian, and BAGS. GeoMFF outperforms DSAF-Net and MA-MFIF on most fusion metrics for the real dataset and on PSNR, SSIM, and LPIPS for the synthetic dataset with all-in-focus ground truth. The ablation without camera pose refinement further shows the value of updating cross-view geometry during optimization.

GeoMFF takes unregistered images as input and jointly optimizes the shared scene, camera poses, and view-dependent defocus. This allows cross-view relationships to be refined with the fusion objective rather than fixed by a separate registration step. Together, these results support joint modeling over registration followed by fusion in the settings we study.

[1] Chen P, Jiang J, Li L, et al. A defocus and similarity attention-based cascaded network for multi-focus and misaligned image fusion[J]. Information Fusion, 2024

We appreciate the reviewer’s careful reading and constructive suggestions. We hope our responses clarify the key points and would be glad to address any remaining concerns.

Reviewer_5BopOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes GeoMFF, a geometry-aware multi-focus image fusion framework that formulates the misalignment problem as the recovery of a latent sharp 3D scene using 3D Gaussian Splatting. The core idea of solving 2D misalignment from a 3D geometric perspective is innovative. However, the manuscript would benefit from broader comparisons with related 3D defocus-aware methods, a more comprehensive literature review.

Critical strengths
  1. The paper successfully addresses the highly challenging misalignment problem in Multi-Focus Image Fusion (MFIF). By elevating the task from 2D pixel-level alignment to a unified 3D scene recovery paradigm, it fundamentally avoids the artifacts introduced by traditional 2D registration-fusion pipelines.
  2. Experiments on the constructed real-world misaligned datasets (GeoMFF2 and GeoMFF7) demonstrate outstanding performance. The method shows great robustness to camera displacement.
  3. The overall structure of the methodology is logical, and the integration of the GMBN module with the 3DGS framework is technically sound.
Critical weaknesses
  1. Although framed as an image fusion task, the technical pipeline closely aligns with 3DGS-based defocus deblurring methods. However, the paper lacks comparisons with state-of-the-art defocus-aware 3D reconstruction methods, particularly BAGS (Blur Agnostic Gaussian Splatting), which shares a very similar philosophy of using masks and kernel mixtures.
  2. The Introduction fails to provide a systematic review of existing methods that attempt to jointly handle registration and fusion. The Related Work also lacks a thorough analysis of this specific sub-field.
  3. The proposed pipeline requires 22K optimization iterations per scene. This massive computational overhead makes real-time deployment extremely difficult. The authors should report the training/inference time and parameter counts.
Questions for authors
  1. The definition of symbol \pi_i should be introduced when they first appeares.
  2. In the GMBN module, the network predicts a spatial mask \tilde{M}_i and the mixture weights of M kernel atoms separately. Intuitively, if one of the kernel atoms was fixed as a Delta (identity) kernel, the network could theoretically learn to output sharp pixels, thereby obviating the need for rendering the separate blur mask.
  3. As shown in Fig. 2, the manuscript does not explain how the point clouds from different source images are processed or merged.
  4. The paper lacks a discussion on failure cases given its fundamental reliance on the presence of parallax.
2 comments
AuthorsOpenReview ↗

Response to Reviewer 5Bop

We appreciate the reviewer’s insightful questions and provide our responses below.

W1: The core idea of solving 2D misalignment from a 3D geometric perspective is innovative. However, the manuscript would benefit from broader comparisons with related 3D defocus-aware methods, a more comprehensive literature review.

We thank the reviewer for highlighting the connection to BAGS. A central contribution of this work is the use of a shared latent sharp 3D scene to solve misaligned MFIF. To our knowledge, this is the first formulation of the fusion task in 3D way. BAGS, Deblur-NeRF, and CoCoGaussian were developed for defocus-aware 3D reconstruction. Once adapted to misaligned MFIF, they can be viewed, at the task-formulation level, as alternative variants of this 3D fusion framework.

Table 1 compares these variants on the same fusion task. GeoMFF’s final design jointly refines camera poses and models spatially varying defocus with shared Gaussian-initialized kernel atoms. The free-form kernel variant in our original ablation, reported as BAGS+Pose, instead retains per-pixel free-form kernels while adding pose optimization. GeoMFF achieves the best results on six of the ten MFIF metrics among the compared 3D methods, supporting the value of its task-specific design.

Table 1. Quantitative comparison of GeoMFF and alternative 3D implementations of the misaligned MFIF formulation on GeoMFF2.

MethodNMI ↑MAE ↓SD ↑SF ↑QAB/F ↑QY ↑SCD ↑SSIM ↑QCB ↑VIFF ↑
Deblur-NeRF0.523817.342943.565814.69080.30930.77050.11940.65590.47830.5487
CoCoGaussian0.492318.413342.561111.87730.36810.60910.15950.71210.49240.6461
BAGS0.713515.895944.109416.29730.46170.86900.41400.76200.55630.8491
BAGS+Pose0.717814.554443.270613.86300.46990.86790.36690.76010.56180.8523
Ours0.737114.510643.567614.81730.47580.88560.36710.75140.56570.8549

Unlike per-pixel free-form kernels, GeoMFF constructs local defocus kernels from shared, Gaussian-initialized kernel atoms to model spatially varying defocus in MFIF. The full model outperforms both the original BAGS and the free-form kernel variant on multiple MFIF metrics, supporting the kernel representation and joint optimization design. We have added these comparisons and clarified the contributions and method differences in Section 4.2 and Table 1 of the revised manuscript.

W2: The Introduction fails to provide a systematic review of existing methods

Thank you for pointing out this gap. We have revised the Introduction and Related Work to provide a fuller account of research on misaligned MFIF. The revised manuscript further discusses these methods’ alignment mechanisms and applicable settings.

Thank you for pointing out this gap. We expanded the Introduction to summarize two main lines of work on misaligned MFIF:

Some studies improve tolerance to misalignment through multi-scale focus measures, local descriptors, robust sparse representation, or image matting. Others establish cross-image correspondences through feature matching, homography estimation, or optical flow before or during fusion.

We also added a dedicated “Misaligned multi-focus image fusion” subsection in Related Work. It discusses representative methods in both lines, including DSIFT, JointRF, Ji et al. (2023), DSAF-Net, MA-MFIF, and FM-MFIF, as well as the stage-wise approach of Zhu et al. (2025). The revised discussion explains the remaining difficulty:

Although these 2D methods reduce misalignment, a single global transform cannot model depth-dependent parallax, and local correspondences may be uncertain under strong defocus or occlusion.

Please see Sections 1 and 2 of the revised manuscript for details.

AuthorsOpenReview ↗

Response to Reviewer 5Bop

W3: report the training/inference time and parameter counts

We thank the reviewer for raising the issue of computational cost. Table 2 reports parameter counts, per-scene optimization times, and inference times on WHU-MFM [1]. Our work introduces a new 3D approach to misaligned MFIF by recovering a shared latent sharp scene. Its strong results on both standard fusion and full-reference metrics demonstrate the value of this approach and provide a basis for more efficient methods. In future work, we will explore feed-forward 3D reconstruction to reduce optimization cost and accelerate end-to-end processing.

Table 2. Comparison of total parameter counts, average per-scene optimization time, and average inference time across methods.

MethodsIFCNNDRPLDSAF-NetU2FusionMFF-GANSwinFusionDeblur-NeRFCoCoGaussianBAGSOurs
Total parameters (M)0.0841.072.6270.6590.040.9741.2151.831.6481.468
Training time (s)——————21060366.7385.4374.2
Inference time (ms)166010.291100318.7538.23512.6719900.74960.8551.27
— indicates that the method does not require per-scene optimization.

Q1: Definition of πi\pi_i at first use

Thanks! We have clarified in the teaser figure caption that πi\pi_i denotes the camera parameters of the ii-th input image.

Q2: Blur mask versus an identity kernel

We thank the reviewer for this suggestion. A fixed Delta kernel can theoretically represent an identity mapping. We implemented a Delta kernel variant without the separate blur mask and evaluated it on GeoMFF2, as shown in Table 3.

MethodsNMI↑MAE↓SD↑SF↑QAB/F↑QY↑SCD↑SSIM↑QCB↑VIFF↑
Ours0.737114.510643.567614.81730.47580.88560.36710.75140.56570.8549
Delta Kernel0.661915.183544.133015.35570.45300.84850.40220.78140.54100.7912

The Delta kernel variant scores higher on SD, SF, SCD, and SSIM, while our original design performs better on the other six metrics. Visual comparisons also show uneven transitions and oversharpening artifacts in some regions of the Delta kernel results. The explicit mask separates the decision of whether to apply blur from how to model it, reducing unnecessary processing in focused regions. Together, the quantitative and visual results support retaining the separate blur mask.

Q3: Processing and merging point clouds

After blur-adaptive confidence filtering, we concatenate the retained points from the VGGT point maps and keep the highest-confidence point in each occupied voxel. This removes redundant points without averaging foreground and background geometry near depth boundaries. We have clarified this procedure in Section 3.2 of the revised manuscript.

Q4: Failure cases under limited parallax

GeoMFF initializes scene geometry from VGGT predictions rather than relying solely on parallax-based triangulation. It can nevertheless fail when those predictions are inaccurate. As shown in Supplementary Fig. 9, insufficient texture can lead to erroneous point clouds and camera estimates, which affect joint optimization and produce ripple artifacts in the fused image.

[1] Chen P, Jiang J, Li L, et al. A defocus and similarity attention-based cascaded network for multi-focus and misaligned image fusion[J]. Information Fusion, 2024, 103: 102125.

We thank the reviewer for their time and thoughtful feedback. We hope our revisions and responses address the concerns raised, and we welcome any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.