- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper formulates misaligned multi-focus image fusion (Misaligned MFIF) as a latent sharp 3D scene reconstruction problem. It proposes the GeoMFF framework, which combines VGGT initialization, camera pose residual optimization, and GMBN defocus blur modeling. In addition, the authors construct a real-world misaligned dataset containing two-image pairs and seven-image sequences.
Critical strengths
-
Formulating misaligned MFIF as a 3D scene reconstruction problem is reasonable. Traditional 2D methods struggle with accurate registration under defocus and parallax. Using 3DGS to perform geometric alignment and sharp information aggregation in 3D space avoids the error propagation of explicit 2D registration.
-
To address the failure of COLMAP initialization under small parallax and severe defocus in MFIF, the paper introduces VGGT to provide initial point clouds and camera parameters, combined with pose residual refinement. The implementation is complete and effective.
-
GMBN represents spatially varying defocus blur using linear combinations of shared Gaussian kernel atoms and incorporates blur mask regularization, which constrains parameter degrees of freedom while preventing over-smoothing in sharp regions.
-
The paper constructs a real-world misaligned multi-focus dataset containing two-image pairs and seven-image sequences (GeoMFF2 and GeoMFF7), which is practically useful for future research.
Critical weaknesses
-
The baseline comparison in the main experiment (Table 1) is unreasonable. All 10 MFIF baselines in Table 1 are pure 2D models that assume aligned inputs. Directly comparing a 3D alignment model with pure 2D models on misaligned data is unfair.
-
Relevant baselines are missing. Methods specifically designed for misaligned MFIF mentioned in Section 2 (e.g., DSAF-Net, Chen et al., 2024; Ji et al., 2023; Zhu et al., 2025) are not included in the experimental comparisons. For 3D defocus reconstruction, only CoCoGaussian is compared, omitting highly related methods such as BAGS (Peng et al., 2024, from which this paper directly adopts the progressive training strategy) and DOF-GS (Wang et al., 2025b).
-
Camera pose estimation is the most challenging part of 3D scene optimization, yet the paper does not explain how the camera poses (especially the target rendering pose) are specifically selected. Without a clear explanation, there may be a major omission in this paper. Meanwhile, Section 4.1 mentions using RoMa to align each source-image pair for metric computation, but does not specify which reference view the outputs of different methods and the RoMa-aligned source images are in. Inconsistent reference views or resampling interpolation on source images will affect the accuracy of the objective metrics.
-
Comparisons of computational cost and runtime are missing. The proposed method relies on per-scene 3DGS optimization requiring 15K to 22K iterations on an RTX 5090, whereas most compared deep learning MFIF baselines are feed-forward networks taking milliseconds or seconds. Per-scene optimization time, GPU memory usage, and parameter counts are not reported in the main text.
Questions for authors
-
Please provide comparisons with methods specifically designed for misaligned MFIF (e.g., DSAF-Net) and related 3D defocus reconstruction methods (BAGS, DOF-GS).
-
Please clarify how the camera poses (especially the target camera pose for all-in-focus rendering) are determined and selected. When using RoMa to align source images for metric calculation, which reference view are the fused outputs and the aligned source images placed in?
-
Please report the per-scene optimization runtime, GPU memory usage, and number of parameters, and compare them with the baselines.
3 comments
Response to Reviewer KHQS
Table 2. Quantitative comparison on WHU-MFM using full-reference and standard fusion metrics.
| Method | PSNR ↑ | SSIM ↑ | LPIPS ↓ | NMI ↑ | MAE ↓ | SD ↑ | QAB/F ↑ | QY ↑ | QCB ↑ | VIFF ↑ |
|---|---|---|---|---|---|---|---|---|---|---|
| DSIFT | 27.2926 | 0.7990 | 0.2831 | 0.7515 | 6.5468 | 28.0515 | 0.4246 | 0.8656 | 0.4224 | 0.6878 |
| GFF | 28.1281 | 0.8095 | 0.2677 | 0.6639 | 6.8662 | 27.7212 | 0.4198 | 0.8756 | 0.4229 | 0.6799 |
| IFCNN | 27.9463 | 0.8211 | 0.2946 | 0.6901 | 5.9894 | 26.9162 | 0.3376 | 0.8890 | 0.3640 | 0.5365 |
| DRPL | 26.0205 | 0.7835 | 0.3009 | 0.7072 | 8.0359 | 28.3645 | 0.3698 | 0.8505 | 0.4220 | 0.5521 |
| DSAF-Net | 30.6461 | 0.8571 | 0.2966 | 0.7332 | 3.7515 | 28.0541 | 0.3015 | 0.9375 | 0.3604 | 0.5897 |
| MA-MFIF | 26.5579 | 0.8263 | 0.2996 | 0.7051 | 5.8049 | 27.4271 | 0.3795 | 0.9017 | 0.4017 | 0.6414 |
| U2Fusion | 25.8317 | 0.8134 | 0.2807 | 0.6194 | 10.7178 | 25.3284 | 0.3035 | 0.8848 | 0.3864 | 0.5094 |
| MFF-GAN | 26.7622 | 0.8026 | 0.3426 | 0.5841 | 7.9884 | 26.4470 | 0.2832 | 0.8713 | 0.3684 | 0.4664 |
| SwinFusion | 28.3177 | 0.8245 | 0.2739 | 0.6865 | 5.9954 | 26.7354 | 0.3432 | 0.8995 | 0.3949 | 0.6009 |
| ZMFF | 27.6598 | 0.8008 | 0.2795 | 0.6614 | 6.6821 | 27.8515 | 0.4033 | 0.8729 | 0.4138 | 0.6231 |
| DMANet | 27.7905 | 0.8107 | 0.2618 | 0.6916 | 6.6601 | 27.6658 | 0.4011 | 0.8759 | 0.4044 | 0.6564 |
| SD-Fuse | 28.0555 | 0.8174 | 0.2719 | 0.6892 | 6.1610 | 27.0750 | 0.3901 | 0.8872 | 0.4162 | 0.6383 |
| Text-MFF | 10.6546 | 0.6530 | 0.3894 | 0.6504 | 73.4315 | 13.3752 | 0.1315 | 0.7378 | 0.1259 | 0.3230 |
| Deblur-NeRF | 28.4863 | 0.7978 | 0.2996 | 0.6665 | 5.3368 | 28.2284 | 0.2504 | 0.8731 | 0.3830 | 0.5181 |
| CoCoGaussian | 28.0041 | 0.8327 | 0.3158 | 0.7124 | 4.5199 | 28.4012 | 0.2859 | 0.9021 | 0.4166 | 0.6864 |
| BAGS | 30.6572 | 0.8879 | 0.2437 | 0.9847 | 3.3910 | 29.2311 | 0.4033 | 0.9311 | 0.4715 | 0.8778 |
| BAGS+Pose | 31.2058 | 0.8829 | 0.2343 | 0.9815 | 3.2059 | 28.5578 | 0.3919 | 0.9318 | 0.4723 | 0.8791 |
| Ours | 31.5970 | 0.8870 | 0.2196 | 1.0113 | 3.0742 | 29.3943 | 0.4713 | 0.9337 | 0.4982 | 1.0018 |
W2: Relevant baselines are missing
In the revised manuscript, we added DSAF-Net and MA-MFIF for unaligned inputs, Deblur-NeRF and BAGS as 3D baselines, and retained CoCoGaussian. We retrained DSAF-Net for two-image input using its public code. Ji et al. (2023) and Zhu et al. (2025) are also relevant, but their code has not been released, making reliable comparison under the same protocol difficult. The new results are in W1. For DOF-GS, we attempted to run its public implementation, but its data preparation requires COLMAP for geometric reconstruction and point cloud generation. On the multi-focus inputs studied here, which have limited viewpoint variation and substantial defocus, COLMAP could not complete this process reliably. Consequently, we were unable to obtain usable results covering all test scenes.
W3: The paper does not explain how the camera poses are specifically selected.
GeoMFF uses VGGT to initialize each source image’s camera intrinsics and pose. During joint optimization of the scene and defocus model, it fixes the intrinsics and refines each pose through a learnable residual rigid transformation. All results labeled “Ours,” for both qualitative and quantitative evaluation, are rendered from the shared 3D scene at source image A’s optimized pose.
We use A as the reference, align B to A, and define a common evaluation region from valid overlap and matching confidence. Fused outputs are not resampled with RoMa; all methods use the same reference view, aligned source image, and evaluation region. We have clarified the rendering view and RoMa’s role in Section 4.1 of the revised manuscript.
Response to Reviewer KHQS
W4: Comparisons of computational cost and runtime are missing
We measured parameter counts and runtimes on WHU-MFM~\citep{chen2024defocus}, as shown in Table 3. GeoMFF uses approximately 30,000 Gaussian primitives per scene, and its GPU memory usage is not the highest among the compared methods. Its main computational cost comes from per-scene optimization. Although this optimization is slower than the inference of feed-forward MFIF methods, conventional MFIF methods are prone to ghosting and structural discontinuities when the inputs are misaligned. Our main contribution is an approach that addresses this problem by recovering a shared latent sharp 3D scene. In future work, we will explore feed-forward networks to reduce per-scene optimization costs and accelerate end-to-end processing.
Table 3. Parameter counts and average runtime of representative methods on WHU-MFM.
| Methods | IFCNN | DRPL | DSAF-Net | U2Fusion | MFF-GAN | SwinFusion | Deblur-NeRF | CoCoGaussian | BAGS | Ours |
|---|---|---|---|---|---|---|---|---|---|---|
| Total parameters (M) | 0.084 | 1.07 | 2.627 | 0.659 | 0.04 | 0.974 | 1.215 | 1.83 | 1.648 | 1.468 |
| Training time (s) | — | — | — | — | — | — | 21060 | 366.7 | 385.4 | 374.2 |
| Inference time (ms) | 1660 | 10.29 | 1100 | 318.75 | 38.23 | 512.67 | 1990 | 0.7496 | 0.855 | 1.27 |
— indicates that the method does not require per-scene optimization.
Q1: Please provide comparisons with methods specifically designed for misaligned MFIF
Please see our response to W1 for the added comparisons with misaligned MFIF and 3D defocus reconstruction methods. We also attempted to run the public implementation of DOF-GS. However, the COLMAP-based geometric reconstruction and point cloud generation required for its data preparation could not be completed reliably on the inputs studied here, which have limited viewpoint variation and substantial defocus. We were therefore unable to obtain usable results for all test scenes.
Q2: Clarify how the camera poses are determined and selected
GeoMFF uses VGGT to initialize each source image’s camera parameters, fixes the intrinsics, and refines the poses through residual rigid transformations during joint optimization. The all-in-focus result is rendered directly from the shared 3D scene at source image A’s optimized pose, without GMBN. For metrics requiring source-image correspondences, A is also the reference view, and RoMa aligns B to A. The fused output is not resampled. All methods use the same aligned source image and evaluation region. We have clarified these settings in Section 4.1 of the revised manuscript.
Q3: Report the per-scene optimization runtime, GPU memory usage, and number of parameters
Please see our response to W4.
We are grateful for the reviewer’s time and valuable feedback. We hope the additional experiments and revisions address your concerns, and we welcome further discussion.
Response to Reviewer KHQS
W1: The baseline comparison in the main experiment (Table 1) is unreasonable
The original ten 2D MFIF baselines show how conventional fusion methods perform on misaligned inputs. To broaden the comparison, we added methods designed for unaligned inputs, including DSIFT, DSAF-Net and MA-MFIF, as well as the 3D reconstruction methods Deblur-NeRF and BAGS. The new comparison results are shown in Table 1. On real misaligned data, GeoMFF outperforms DSAF-Net and MA-MFIF on most MFIF metrics. Comparisons with the 3D methods also support its effectiveness on this task.
Table 1. Quantitative comparison on the GeoMFF2 dataset.
We further evaluated the methods on WHU-MFM using its all-in-focus ground truth and the full-reference metrics PSNR, SSIM, and LPIPS in Table 2. GeoMFF outperforms DSAF-Net and MA-MFIF on all three and ranks among the leading methods overall. These results support our conclusions through comparisons with misalignment-aware baselines and ground-truth evaluation.