acceptodds
Withdrawn from ICLR 2027

S2WMamba: A Wavelet-Assisted Mamba-Based Dual-Branch Network For Pansharpening

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Pansharpening fuses a high-resolution panchromatic (PAN) image with a low-resolution multispectral (MS) image to produce a high-resolution multispectral (HRMS) image. A key difficulty is that jointly processing PAN and MS features often entangles spatial detail enhancement with spectral fidelity. To address this feature entanglement, we propose S2WMamba, a framework that disentangles modality-specific frequency information to guide cross-modal fusion. Concretely, unlike global frequency transforms, a localized 2D Haar DWT is applied to PAN features to isolate spatial edges and textures. Concurrently, a novel channel-wise 1D Haar DWT treats each pixel's spectrum as a 1D signal, re-parameterizing it as a shared spectral base plus band-specific variations; operating in this basis limits spectral distortion. The resulting Spatial branch injects wavelet-extracted spatial details into MS features, while the Spectral branch refines PAN features using spectra from the DWT1D process. To fuse these decoupled sub-bands, the two branches exchange information via Mamba-based cross-modulation, which models long-range dependencies across the sub-bands with linear complexity. On WV3, GF2, and QB datasets, S2WMamba matches or surpasses strong baselines (FusionMamba, CANNet, U2Net, PanNet), improving reduced-resolution PSNR by up to 0.23 dB and reaching a Hybrid Quality with No Reference (HQNR) of 0.956 on full-resolution WV3. Ablations show that both branches are necessary and that parallel dual-branch fusion outperforms sequential variants.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
pansharpening, remote sensing, image fusion, state space model, Mamba, wavelet transform, multi-modal fusion

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_zCW2OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes S2WMamba, a pansharpening framework that combines modality-specific wavelet decomposition with Mamba-based cross-modal fusion. Specifically, 2D Haar DWT extracts spatial details from PAN features, while channel-wise 1D Haar DWT decomposes MS spectral information. A dual-branch architecture integrates these representations through FMamba modules to generate high-resolution multispectral images. Experiments on WV3, GF2, and QB demonstrate competitive performance against existing pansharpening methods.

Critical strengths
  1. The paper addresses the important problem of balancing spatial detail enhancement and spectral fidelity in pansharpening.
  2. The proposed dual-branch architecture provides a reasonable way to separately process spatial and spectral information.
  3. The experiments cover three established benchmarks and include comparisons with several strong baselines.
Critical weaknesses
  1. Insufficient validation of the core wavelet design. The channel-wise 1D Haar DWT is presented as a central contribution, but its independent effectiveness is not adequately demonstrated. The ablation studies mainly remove entire branches rather than directly comparing 1D DWT with alternative spectral representations, such as standard convolutions, DCT, or learnable transforms. Consequently, the necessity of the proposed wavelet design remains unclear.
  2. Marginal improvements over strong baselines. The performance improvements over the strongest baselines are relatively small. For instance, the PSNR gains are only 0.017 dB on WV3 and 0.045 dB on QB. Since the experiments use a single random seed without reporting variance or statistical significance, it is unclear whether these improvements are robust and meaningful.
  3. Insufficient computational efficiency analysis. Although the paper emphasizes the linear complexity of Mamba-based fusion, it mainly provides theoretical complexity comparisons. Practical measurements of inference time, FLOPs, and GPU memory consumption are missing. Therefore, the claimed computational advantages of the overall framework are not sufficiently supported.
  4. Weak justification for spectral fidelity preservation. The paper argues that channel-wise Haar decomposition helps preserve spectral consistency. However, an invertible wavelet transform alone does not guarantee spectral fidelity after nonlinear feature modulation and fusion. The method also relies solely on an L1 reconstruction loss without explicit spectral consistency constraints. Stronger theoretical justification and targeted experiments are needed to substantiate this claim.
Questions for authors

Please refer to the weaknesses outlined above.

Reviewer_rW66OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes S2WMamba for full-color sharpening. The method performs two-dimensional Haar wavelet decomposition on the PAN features to extract spatial sub-bands, and one-dimensional Haar wavelet decomposition on the MS channels to represent spectral changes; subsequently, it completes cross-modal interaction through two cascaded branches and modifies the FMamba module of FusionMamba; finally, it predicts the residuals and adds them to the upsampled MS. The experiments cover downsampling and full-resolution evaluation of WV3, GF2, and QB, and compare single branch, parallel-series fusion, and different fusion modules.

Critical strengths

The spatial details guided by PAN are separated from the spectral correction guided by MS for processing, which is reasonable in the full-color sharpening task. Table 2 shows that deleting any branch on WV3 will cause the PSNR to drop by approximately 0.6 dB, indicating that the dual-branch design has a certain complementary effect.

The experiments cover three sensors and report both downscaled and full-resolution metrics. Compared to FusionMamba, the PSNR of GF2 increases from 49.678 dB to 49.909 dB; the full-resolution HQNR of QB reaches 0.942 (Table 1). These results indicate that this method is competitive on the existing benchmarks.

Critical weaknesses

The paper has not fully demonstrated that wavelet decomposition can better preserve spectral information. Table 2 only removes the entire branch without separately replacing 1D or 2D DWT, so it cannot rule out that the performance improvement comes from additional fusion modules or model capacity. The full-resolution Dλ on WV3 is still worse than FusionMamba (0.024 vs. 0.019, Table 1).

The current results are not sufficient to show that the method is more stable than the strong baseline. The PSNR of WV3 compared to FusionMamba only increases by 0.017 dB, and the QB compared to CANNet only increases by 0.045 dB. Each sensor has only 20 test samples, and the training only reports one random seed; the comparison method uses public weights and does not show re-training results under unified conditions. For such a small gap, the lack of per-image results and repeated experiments makes it difficult to determine whether the gain is stable.

Questions for authors
  1. Does 1D DWT act on the original MS band or the learned features? Equation (5) is inconsistent with the input of Algorithm 1. If it is a feature channel, how can we ensure that adjacent channels still have spectral correlation?
  2. Can we keep the dual-branch structure unchanged and replace the 1D/2D DWT with non-wavelet operations of comparable parameter quantity? Please also report SAM, full-resolution Dλ and PSNR.
  3. Is the 0.017 dB improvement of WV3 and the 0.045 dB improvement of QB stable under different random seeds? Please provide the results for each graph and compare them with the strongest baseline under the same training conditions.
  4. Compared with FusionMamba, what are the actual processing time and peak memory usage for full-resolution inference? The appendix only provides the theoretical complexity, without the measured efficiency results.
Reviewer_GtRYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents S2WMamba, a dual-branch pansharpening network that combines modality-specific wavelet decomposition with Mamba-based cross-modal fusion. It uses a 2D Haar transform to extract spatial structure from PAN features and a channel-wise 1D Haar transform to represent MS spectral information. The motivation is clear, and the ablations support the complementary roles of the two branches. However, the independent contribution of the wavelet design, its practical benefits, and its distinction from earlier spatial-spectral wavelet methods require stronger evidence.

Critical strengths

1 Clear task motivation. The architecture is motivated by a central challenge in pansharpening: transferring spatial detail from the PAN image while preserving the spectral information of the MS image. Applying different wavelet decompositions to the two modalities, followed by spatial and spectral processing branches, provides an understandable rationale for the design. The correspondence between the roles of the input modalities and the proposed processing paths is clearly articulated in Section 2.

2 Ablations support the dual-branch architecture. The experiments in Table 2 provide evidence that both branches contribute to reconstruction quality. On WV3, removing the spectral branch or the spatial branch reduces PSNR by 0.622 dB and 0.657 dB, respectively. The complete model also outperforms the two sequential fusion variants by 0.198 dB and 0.286 dB. These results support the complementary roles of spatial and spectral processing and the choice of parallel fusion over the tested sequential alternatives.

Critical weaknesses

1 Missing direct ablations of the core wavelet design. The ablations in Table 2 remove entire branches, change fusion order, or replace FMamba, but do not isolate the contribution of the core wavelet design. Branch removal changes both information flow and model capacity, so its performance impact cannot be attributed specifically to wavelet decomposition. Controlled comparisons without the 1D spectral transform, the 2D spatial transform, or both, as well as against a learned channel transform, would better establish the necessity of the proposed design.

2 Practical benefits remain insufficiently demonstrated. Some gains over the second-best method are small: PSNR improves by only 0.017 dB on WV3 and 0.045 dB on QB, and HQNR by 0.001 on WV3. Figure 5 also does not show an obvious visual advantage over strong baselines, while single-seed evaluation leaves the robustness of these margins unclear. Moreover, no downstream classification, detection, or segmentation results are provided. Repeated runs, clearer local visual comparisons, and a downstream evaluation would help demonstrate practical benefits.

3 Insufficient distinction from prior spatial-spectral wavelet methods. Joint 2D spatial and 1D spectral wavelet processing has precedents in hyperspectral denoising [1] and compression [2], where the latter evaluates a hybrid 1D-2D DWT baseline. Although these methods do not establish that the specific PAN/MS fusion architecture already exists, combining the two transform dimensions is not itself a new idea. The paper should clarify its distinction from these precedents and substantiate the additional contributions of modality-specific decomposition and Mamba-based cross-modal fusion.

References [1] J. Huang, Y. Ma, X. Mei, and F. Fan. A hybrid spatial-spectral denoising method for infrared hyperspectral images using 2DPCA. Infrared Physics & Technology, 79: 68-73, 2016. DOI: 10.1016/j.infrared.2016.09.009. [2] G. Tzagkarakis, W. Charle, and P. Tsakalides. Data Compression for Snapshot Mosaic Hyperspectral Image Sensors. 24th European Signal Processing Conference (EUSIPCO), pp. 1558-1562, 2016. See Section IV for the hybrid 1D spectral and 2D spatial DWT baseline.

Questions for authors

1.Can you isolate the contribution of the 1D spectral and 2D spatial wavelet transforms through controlled ablations that retain comparable model capacity and the dual-branch architecture? How do they compare with learned transformations?

2.Are the small performance gains on WV3 and QB consistent across multiple random seeds? Can you provide clearer visual comparisons or downstream-task evidence to demonstrate their practical significance?

3.How does your wavelet design differ from prior methods combining 2D spatial and 1D spectral wavelet processing, such as Huang et al. (2016) and the hybrid DWT baseline evaluated by Tzagkarakis et al. (2016)? What specific methodological advance does S2WMamba introduce beyond adapting this principle to PAN/MS fusion?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.