acceptodds
Under review as a conference paper at ICLR 2027

SEPARATE AND VARY TOGETHER: CROSS-SENSOR PANSHARPENING WITH REVERSIBLE SPECTRAL TRANSPORT AND PAIRED SENSOR TRANSFORMATIONS

Abstract

Pansharpening fuses a low-resolution multispectral (MS) image with a panchromatic (PAN) image. Existing cross-sensor methods rely on per-sensor training, multi-sensor data, or target-time adaptation, which are costly and require target labels or weight updates. We study a restrictive setting: train once on a source sensor and deploy the same weights on unseen sensors. The challenge is that different sensors observe different band counts, centers, and spectral responses, and the PAN–MS relation varies. A fixed input–output mapping learned on one sensor is therefore tied to that sensor’s observation process and difficult to transfer. We instead learn how the output should change when the sensing process changes, formulated as approximate equivariance under sensor transformations: fusion should commute with a sampled transformation applied jointly to the source MS observations and their HRMS targets. Since the transformation is sampled from source data and paired targets are exact, this behavior is supervised without target data, enabling source-only training. We propose EquiPan, which promotes this approximate equivariance through two complementary mechanisms. A Spectral–Contrast Transformation (SCT) first samples virtual observation models from source data and applies each transformation consistently to the MS observations and their HRMS target, providing exact supervision of how the fusion output should change with the sensor. A Reversible Spectral Transport (RST) then projects the resulting heterogeneous observations into a shared latent space and maps predicted residuals back to their native bands through an analytic left inverse on the observed spectral degrees of freedom. A parameter-free PAN mean alignment (PMA) operation removes branch-dependent global offsets while preserving the measured PAN contrast. Across seven backbones and three unseen sensors, EquiPan improves zero-shot transfer over traditional and interpolation-only baselines.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
pansharpening, Cross-Sensor, Generalization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_8mngOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

EquiPan addresses cross-sensor pansharpening through single-source training and fixed-weight deployment on unseen sensors. It combines paired spectral transformations, reversible spectral transport, and PAN mean alignment. Experiments across seven backbones and three unseen sensors show generally improved transfer over interpolation-only baselines.

Critical strengths
  • The method addresses a practical problem: applying a trained model to another sensor without collecting new training labels or updating its weights.
  • The framework works with several different fusion networks and generally improves their cross-sensor performance, suggesting that its benefits are not tied to a particular architecture.
Critical weaknesses
  • I would have liked to see direct comparisons with closely related cross-sensor methods. Without these comparisons, I find it difficult to assess the advantage over existing approaches.
  • The paper explains the method through approximate equivariance, but does not clearly establish what this contributes beyond paired data augmentation and statistical alignment.
Questions for authors
  1. How does EquiPan compare with closely related methods such as UniPAN under comparable conditions?
  2. What evidence shows that equivariance explains the improvement, beyond the use of paired spectral augmentation?
  3. What does the reversible mapping contribute beyond simpler spectral mapping and normalization?
  4. Since RST uses statistics from the target image, how does its deployment procedure differ from existing distribution-alignment methods?
3 comments
AuthorsOpenReview ↗

We sincerely thank Reviewer 8mng for the constructive comments. We have revised the manuscript (highlighted in blue) to incorporate your suggestions. Below we address all concerns and questions point by point.

1. Comparison with Closely Related Methods Under Comparable Conditions (W1 & Q1)

  • Manuscript Revision: In the revised Introduction (Section 1, Table 1) and Related Work (Section 2), we explicitly compare the operational assumptions of existing cross-sensor paradigms: foundation models (FoundPS) require multi-sensor pretraining on custom backbones; test-time adaptation (ERFT, SWIFT) requires target-side fine-tuning and target MTF priors; and distribution alignment (UniPAN) is backbone-agnostic and zero-shot, making it directly comparable to EquiPan.
  • New Experiments (Added as Group III in Revised Tables 3 & 4 + Multi-Source Transfers in Appendix D): Using the exact same ABFNet backbone, we first compare EquiPan against both zero-shot distribution alignment (UniPAN) and target-fine-tuned adaptation (ERFT, trained on WV3 and fine-tuned on each unseen target dataset) under unified 8-band WV3 training (top block). Furthermore, we evaluate bidirectional zero-shot transfers between UniPAN and EquiPan when trained from scratch on 4-band GF2 and 4-band QB (bottom block). Since UniPAN and ERFT operate on fixed channel counts, we pair them with wavelength interpolation/truncation for 8-to-4 and 4-to-8 band transfers:
Source -> TargetMethod (ABFNet Backbone)PSNRSAMERGASHQNRDλD_\lambdaDsD_s
WV3 (8b) -> WV3 (Source)UniPAN / ERFT / EquiPan36.79 / 36.21 / 37.323.35 / 3.38 / 3.262.35 / 2.52 / 2.210.9449 / 0.9574 / 0.95280.0249 / 0.0213 / 0.02000.0311 / 0.0218 / 0.0280
WV3 (8b) -> WV2 (8b)UniPAN / ERFT (FT) / EquiPan29.24 / 28.95 / 31.855.58 / 5.26 / 4.604.20 / 4.35 / 3.100.9293 / 0.9360 / 0.93410.0383 / 0.0301 / 0.02600.0338 / 0.0350 / 0.0410
WV3 (8b) -> QB (4b)UniPAN / ERFT (FT) / EquiPan32.73 / 34.25 / 35.107.15 / 6.11 / 5.996.90 / 5.79 / 5.020.9165 / 0.9098 / 0.91210.0530 / 0.0487 / 0.04500.0330 / 0.0454 / 0.0460
WV3 (8b) -> GF2 (4b)UniPAN / ERFT (FT) / EquiPan38.30 / 32.50 / 44.002.62 / 2.70 / 1.612.68 / 2.58 / 1.500.8447 / 0.8356 / 0.92120.0460 / 0.0605 / 0.03500.1143 / 0.1103 / 0.0450
GF2 (4b) -> GF2 (Source)UniPAN / EquiPan41.61 / 44.101.07 / 0.880.97 / 0.740.9226 / 0.96000.0370 / 0.02800.0420 / 0.0120
GF2 (4b) -> WV3 (8b)UniPAN / EquiPan26.30 / 32.257.90 / 5.378.14 / 3.940.7243 / 0.92950.1810 / 0.03000.1170 / 0.0420
GF2 (4b) -> WV2 (8b)UniPAN / EquiPan25.82 / 28.367.02 / 5.726.36 / 4.550.8868 / 0.93520.0660 / 0.03200.0500 / 0.0340
GF2 (4b) -> QB (4b)UniPAN / EquiPan30.13 / 34.207.88 / 6.849.56 / 5.800.8555 / 0.90830.0680 / 0.04400.0810 / 0.0510
QB (4b) -> QB (Source)UniPAN / EquiPan38.13 / 38.214.24 / 4.163.68 / 3.650.9077 / 0.90230.0430 / 0.03600.0520 / 0.0650
QB (4b) -> WV3 (8b)UniPAN / EquiPan30.70 / 31.115.68 / 5.874.88 / 4.460.9386 / 0.93110.0270 / 0.03300.0360 / 0.0380
QB (4b) -> WV2 (8b)UniPAN / EquiPan28.69 / 28.715.72 / 5.764.50 / 4.360.9326 / 0.91670.0400 / 0.04100.0280 / 0.0440
QB (4b) -> GF2 (4b)UniPAN / EquiPan31.20 / 31.903.50 / 2.223.29 / 2.920.8516 / 0.87120.0380 / 0.04300.1149 / 0.0900
  • Advantage Over Zero-Shot UniPAN Across Three Source Sensors: EquiPan consistently outperforms UniPAN on all source sensors (+0.53 dB on WV3, +2.49 dB on GF2, +0.08 dB on QB) with lower DλD_\lambda, avoiding the native radiometric distortion of per-band histogram warping. In cross-band transfer, EquiPan surpasses UniPAN by +2.61 dB on WV2, +2.37 dB on QB, and +5.70 dB on GF2 (from WV3), and by +5.95 dB on WV3 (+0.2052 HQNR) and +4.07 dB on QB (from GF2). Even on the challenging QB -> GF2 transfer, EquiPan leads across PSNR (31.90 vs. 31.20 dB), SAM (2.22 vs. 3.50), ERGAS (2.92 vs. 3.29), and HQNR (0.8712 vs. 0.8516).
  • Advantage Over Target-Fine-Tuned ERFT: Even though ERFT is fine-tuned on each unseen target dataset (WV2, QB, and GF2) using target MTF priors, fixed-weight EquiPan outperforms target-fine-tuned ERFT across all unseen sensors in RR fidelity (+2.90 dB on WV2, +0.85 dB on QB, and +11.50 dB on GF2) as well as FR HQNR on QB (0.9121 vs. 0.9098) and GF2 (0.9212 vs. 0.8356), while achieving the lowest spectral distortion DλD_\lambda across all four sensors without any target-domain adaptation.

2. Evidence That Equivariance Explains Improvements Beyond Augmentation (W2 & Q2)

  • Manuscript Revision: As clarified in revised Sections 2, 3.1, and 3.2, conventional unilateral spectral augmentation breaks radiometric correspondence. In contrast, SCT applies Tg\mathcal{T}_g bilaterally (Lg=Tg(L)L_g = \mathcal{T}_g(L) and Yg=Tg(Y)Y_g = \mathcal{T}_g(Y)) around the fixed PAN spatial anchor. By the triangle inequality, minimizing the joint loss Lθ=12∥Y^−Y∥1+12∥Y^g−Yg∥1\mathcal{L}_\theta = \frac{1}{2}\lVert \widehat{Y} - Y \rVert_1 + \frac{1}{2}\lVert \widehat{Y}_g - Y_g \rVert_1 (Eq. 21) strictly upper-bounds the equivariance error ∥Φθ,λg(Mg,Lg,P)−Tg(Φθ,λ(M,L,P))∥1\lVert \Phi_{\theta,\lambda_g}(M_g,L_g,P) - \mathcal{T}_g(\Phi_{\theta,\lambda}(M,L,P)) \rVert_1.

  • New Ablations: We ablate: (i) no_virtual (retaining RST+PMA alignment at test time but removing the virtual equivariance loss ∥Y^g−Yg∥1\lVert \widehat{Y}_g - Y_g \rVert_1), (ii) SCT Fixed 8 (fixing virtual band count K=8K=8 without 4-band sampling), (iii) SCT Selection (discrete band selection BgB_g only, without continuous Gaussian response mixing RgR_g), and two non-equivariant physical augmentations: (iv) Gaussian_Blur (random blur on MS and GT) and (v) Pan_noise (random noise on PAN):

Variant (ABFNet, WV3 Source)WV3 PSNR / HQNRWV2 PSNR / HQNRQB PSNR / HQNRGF2 PSNR / HQNR
EquiPan (Full)37.32 / 0.952831.85 / 0.934135.10 / 0.912144.00 / 0.9212
no_virtual (w/o ∥Y^g−Yg∥1\lVert \widehat{Y}_g - Y_g \rVert_1)37.47 / 0.952329.78 / 0.932733.50 / 0.832240.77 / 0.9298
SCT Fixed 8 (K=8K=8 only)37.40 / 0.952730.26 / 0.933434.43 / 0.761936.24 / 0.9337
SCT Selection (BgB_g only, τg=0\tau_g=0)37.33 / 0.954730.70 / 0.938835.24 / 0.791136.42 / 0.9253
Gaussian_Blur (MS & GT Blur)37.23 / 0.955230.34 / 0.941635.28 / 0.765042.10 / 0.8372
Pan_noise (PAN Noise)37.26 / 0.953530.19 / 0.936235.24 / 0.773543.67 / 0.9420

Removing the virtual equivariance loss (no_virtual) while keeping RST/PMA drops zero-shot PSNR by -2.07 dB on WV2, -1.60 dB on QB (HQNR drops to 0.8322), and -3.23 dB on GF2. Restricting SCT to fixed 8-band views (SCT Fixed 8) or discrete selection (SCT Selection) collapses GF2 PSNR by -7.76 dB and -7.58 dB and drops QB HQNR to 0.7619–0.7911. Adding non-equivariant spatial blur (Gaussian_Blur) or PAN noise (Pan_noise) violates task equivalence, lowering WV2/GF2 PSNR by 0.33–1.90 dB and collapsing QB HQNR to 0.7650 / 0.7735 (DsD_s: 0.172 / 0.196 vs. 0.046).

3. Contribution of Reversible Mapping Beyond Simpler Mapping & Normalization (Q3)

As detailed in revised Section 3.3 and Appendix A, bridging C≤DC \le D bands precludes a global bijection on RD\mathbb{R}^D by rank-nullity. RST uses Helmert bases (Hobs,HlatH_{\text{obs}}, H_{\text{lat}}) to isolate common intensity Lˉ\bar{L} from zero-mean chroma, combining orthogonalized embedding (Eλ⊤Eλ=IC−1E_\lambda^\top E_\lambda = I_{C-1}) with full Cholesky whitening SLS_L to guarantee an exact analytical left inverse (FoutFin=ICF_{\text{out}}F_{\text{in}} = I_C) and preserve task equivalence (Y^=L\widehat{Y}=L when Δ=0\Delta=0). We ablate RST_no_diagonal (per-channel variance scaling without off-diagonal Cholesky covariance SLS_L) and RST_no_stats (geometry alignment EλE_\lambda only, without statistical whitening/recoloring):

Variant (ABFNet, WV3 Source)WV3 PSNR / HQNRWV2 PSNR / HQNRQB PSNR / HQNRGF2 PSNR / HQNR
EquiPan (Full RST)37.32 / 0.952831.85 / 0.934135.10 / 0.912144.00 / 0.9212
RST_no_diagonal (Per-channel std)37.37 / 0.953330.11 / 0.937135.14 / 0.803836.55 / 0.9297
RST_no_stats (Geometry EλE_\lambda only)37.41 / 0.951430.22 / 0.928734.76 / 0.789632.00 / 0.8894

Simplifying full Cholesky whitening to per-channel scaling (RST_no_diagonal) drops GF2 PSNR by -7.45 dB (to 36.55 dB), WV2 PSNR by -1.74 dB, and QB HQNR to 0.8038. Removing statistical alignment (RST_no_stats) collapses GF2 PSNR by -12.00 dB (to 32.00 dB).

4. Deployment Difference Between RST and Distribution-Alignment Methods (Q4)

As clarified in revised Section 2 and Section 3.3, distribution-alignment methods (e.g., UniPAN) apply 1D marginal CDF warping independently per input band while predicting native-space residuals, requiring fixed band counts and distorting inter-band ratios. In contrast, RST is a closed-loop forward-and-inverse subspace transport: target statistics (bL,SL)(b_L, S_L) estimated on stationary zero-mean chroma residuals r~\tilde{r} not only whiten input chroma before the backbone, but also analytically recolor and project the predicted latent residual Δ\Delta back to the native CtC_t-band geometry via HobsEλ⊤SLHlat⊤rD(Δ)H_{\text{obs}} E_\lambda^\top S_L H_{\text{lat}}^\top r_D(\Delta) (Eq. 20), enabling lossless zero-shot transfer across arbitrary channel counts (8-to-4 and 4-to-8).

Reviewer_c7wJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes EquiPan, a single-source cross-sensor pansharpening framework based on approximate sensor equivariance. It integrates Spectral–Contrast Transformation (SCT) for generating virtual sensor observations, Reversible Spectral Transport (RST) for aligning heterogeneous spectral representations, and PAN Mean Alignment (PMA) for reducing intensity discrepancies. Experiments on seven backbones demonstrate improved zero-shot generalization from WorldView-3 to three unseen sensors, particularly GF2 and WV2, compared with interpolation-only baselines.

Critical strengths
  1. The paper investigates a practically meaningful challenge of achieving cross-sensor pansharpening generalization without relying on target-domain data or model adaptation.
  2. The framework is compatible with multiple existing pansharpening architectures, and the evaluation across seven different backbones provides useful evidence of its architectural flexibility.
Critical weaknesses
  1. Insufficient justification of sensor equivariance. The proposed equivariance is not explicitly enforced in the loss function, making it unclear whether the improvements stem from equivariant learning or conventional spectral data augmentation.
  2. Limited methodological novelty. SCT, RST, and PMA primarily combine established techniques, including spectral augmentation, orthogonal projection, and statistical normalization, without sufficiently demonstrating their novelty over simpler alternatives.
  3. Incomplete comparisons with existing methods. The experiments lack direct comparisons with dedicated cross-sensor pansharpening approaches, such as UniPAN, ERFT, and SWIFT, limiting the assessment of the method's competitiveness.
  4. Limited modeling of real sensor variations. The simulated transformations mainly consider spectral mixing and contrast changes, while neglecting important discrepancies such as PAN spectral responses, spatial degradation, and sensor noise.
  5. Insufficient generalization validation. All experiments use WV3 as the source sensor, without evaluating alternative source sensors or reverse transfer scenarios. This limits the evidence supporting the framework's general applicability.
Questions for authors

Please refer to the Critical Weaknesses section above for the main concerns and questions.

5 comments

We sincerely thank Reviewer c7wJ for the constructive comments. We have substantially revised the manuscript (highlighted in blue) and conducted extensive new experiments to address all five Critical Weaknesses (W1--W5) individually below.

Response to W1: Justification of Sensor Equivariance vs. Conventional Spectral Augmentation

  • Manuscript Revisions (Sections 1, 2, 3.1 & 3.2): In the revised manuscript, we explicitly clarify why EquiPan enforces genuine approximate sensor equivariance rather than conventional spectral augmentation:

    1. Implicit Bound via Bilateral Paired Supervision: Unlike unsupervised/self-supervised settings that require an explicit consistency loss because transformed targets are unknown, SCT applies the sampled sensor transformation Tg\mathcal{T}_g bilaterally and identically to both the MS inputs (M,L)(M,L) and the ground-truth HRMS target YY, yielding the exact transformed target Yg=Tg(Y)Y_g = \mathcal{T}_g(Y) (Eq. 10). Training both branches with shared parameters θ\theta via Eq. (21), Lθ=12∥Y^−Y∥1+12∥Y^g−Yg∥1\mathcal{L}_\theta = \frac{1}{2}\lVert \widehat{Y}-Y \rVert_1 + \frac{1}{2}\lVert \widehat{Y}_g-Y_g \rVert_1, simultaneously enforces Φθ,λ(M,L,P)≈Y\Phi_{\theta,\lambda}(M,L,P) \approx Y and Φθ,λg(Mg,Lg,P)≈Tg(Y)\Phi_{\theta,\lambda_g}(M_g,L_g,P) \approx \mathcal{T}_g(Y). By the triangle inequality, Eq. (21) strictly upper-bounds the empirical equivariance error in Eq. (6):

      ∥Φθ,λg(Mg,Lg,P)−Tg(Φθ,λ(M,L,P))∥1≤∥Y^g−Yg∥1+∥Tg(Y^)−Tg(Y)∥1\lVert \Phi_{\theta,\lambda_g}(M_g,L_g,P) - \mathcal{T}_g(\Phi_{\theta,\lambda}(M,L,P)) \rVert_1 \le \lVert \widehat{Y}_g - Y_g \rVert_1 + \lVert \mathcal{T}_g(\widehat{Y}) - \mathcal{T}_g(Y) \rVert_1

    2. Departure from Unilateral Spectral Augmentation: As highlighted in revised Section 2 and Section 3.2, conventional spectral augmentation perturbs inputs unilaterally while keeping labels fixed to teach invariance, which breaks physical radiometric correspondence in pansharpening. In contrast, EquiPan teaches equivariance: how the fused output must predictably commute with changes in band count, center wavelengths, and spectral contrast around a fixed spatial PAN anchor while preserving source task equivalence (Tg−1(Φ(Tg(X)))≈Φ(X)\mathcal{T}_g^{-1}(\Phi(\mathcal{T}_g(X))) \approx \Phi(X)).

  • Empirical Verification (no_virtual Ablation): To isolate the contribution of the equivariant supervision in Eq. (21), we evaluated a no_virtual variant on ABFNet (trained on WV3), which retains RST and PMA alignment during training and inference but removes the virtual equivariance loss term 12∥Y^g−Yg∥1\frac{1}{2}\lVert \widehat{Y}_g-Y_g \rVert_1:

Variant (ABFNet, WV3 Source)WV3 (PSNR / SAM / HQNR)WV2-Zero (PSNR / SAM / HQNR)QB (PSNR / SAM / HQNR)GF2 (PSNR / SAM / HQNR)
no_virtual (w/o ∥Y^g−Yg∥1\lVert \widehat{Y}_g-Y_g \rVert_1)37.47 / 3.22 / 0.952329.78 / 4.95 / 0.932733.50 / 7.46 / 0.832240.77 / 2.46 / 0.9298
EquiPan (Full, Eq. 21)37.32 / 3.26 / 0.952831.85 / 4.60 / 0.934135.10 / 5.99 / 0.912144.00 / 1.61 / 0.9212

Without the virtual equivariant supervision branch (no_virtual), zero-shot transfer degrades severely across all unseen sensors: PSNR drops by -2.07 dB on WV2, -1.60 dB on QB (with SAM worsening from 5.99 to 7.46 and HQNR dropping from 0.9121 to 0.8322, as DλD_\lambda more than doubles from 0.045 to 0.107), and -3.23 dB on GF2 (SAM worsening from 1.61 to 2.46). This confirms that equivariant supervision via Eq. (21) directly drives cross-sensor generalization.

Response to W2: Methodological Novelty of SCT and RST Over Established Techniques and Simpler Alternatives

While we agree that PMA is a lightweight, parameter-free zero-order mean alignment inspired by classical component substitution, we respectfully clarify (in revised Sections 1, 2, 3.2, and 3.3) that SCT and RST are not existing techniques in the literature, nor can they be replaced by simpler alternatives:

  1. Novelty of SCT: In existing pansharpening literature, augmentation is limited to geometric flips/rotations because unilateral spectral perturbations break input–target radiometric fidelity. SCT is the first bilateral virtual sensor observation operator in pansharpening that continuously blends discrete subset selection (BgB_g) and Gaussian spectral response mixing (RgR_g) across variable band counts (K∈{4,8}K\in\{4,8\}) with bijective, mean-preserving chroma scaling Gγg\mathcal{G}_{\gamma_g}, applied symmetrically to (M,L,Y)(M,L,Y) alongside dynamically updated metadata λg=Agλ\lambda_g = A_g\lambda.

  2. Novelty of RST: As clarified in revised Section 3.3 and Appendix A, bridging heterogeneous band counts (C≤DC \le D) precludes a global bijection on RD\mathbb{R}^D by rank-nullity, while standard normalizations (or UniPAN's 1D marginal warping) are lossy and restricted to fixed channel counts. RST uses fixed orthonormal Helmert bases (Hobs,HlatH_{\mathrm{obs}}, H_{\mathrm{lat}}) to decouple common luminance Lˉ\bar{L} from zero-mean chroma, orthogonalizes only the chromatic wavelength map (Eλ⊤Eλ=IC−1E_\lambda^\top E_\lambda = I_{C-1}), and couples it with regularized Cholesky covariance transport SLS_L. This establishes an exact analytical left inverse (FoutFin=ICF_{\mathrm{out}}F_{\mathrm{in}} = I_C) on the observed subspace, restoring native spectral scales with zero algebraic error (Y^=L\widehat{Y}=L when Δ=0\Delta=0).

  3. New Ablations Against Simpler SCT and RST Alternatives: We evaluated four simpler alternatives on ABFNet (WV3 source): SCT Fixed 8 (K=8K=8 only, no 4-band sampling), SCT Selection (discrete selection BgB_g only, removing Gaussian response mixing RgR_g), RST_no_diagonal (per-channel variance scaling instead of full Cholesky covariance SLS_L), and RST_no_stats (geometry projection EλE_\lambda only, without statistical whitening/recoloring):

Variant (ABFNet, WV3 Source)WV3 (PSNR / ERGAS / HQNR)WV2-Zero (PSNR / ERGAS / HQNR)QB (PSNR / ERGAS / HQNR)GF2 (PSNR / ERGAS / HQNR)
SCT Fixed 8 (K=8K=8 only)37.40 / 2.19 / 0.952730.26 / 3.68 / 0.933434.43 / 5.63 / 0.761936.24 / 1.83 / 0.9337
SCT Selection (BgB_g only)37.33 / 2.21 / 0.954730.70 / 3.50 / 0.938835.24 / 5.12 / 0.791136.42 / 1.79 / 0.9253
RST_no_diagonal (Per-band std)37.37 / 2.20 / 0.953330.11 / 3.75 / 0.937135.14 / 5.17 / 0.803836.55 / 1.80 / 0.9297
RST_no_stats (EλE_\lambda only)37.41 / 2.19 / 0.951430.22 / 3.70 / 0.928734.76 / 5.45 / 0.789632.00 / 2.80 / 0.8894
EquiPan (Full)37.32 / 2.21 / 0.952831.85 / 3.10 / 0.934135.10 / 5.02 / 0.912144.00 / 1.50 / 0.9212

Simplifying SCT to SCT Fixed 8 or SCT Selection collapses GF2 PSNR by -7.76 dB and -7.58 dB, while dropping QB HQNR from 0.9121 to 0.7619 / 0.7911 (DsD_s surges from 0.046 to 0.206 / 0.1698). Simplifying RST's Cholesky covariance to diagonal scaling (RST_no_diagonal) or removing statistical transport (RST_no_stats) drops GF2 PSNR by -7.45 dB (to 36.55 dB) and -12.00 dB (to 32.00 dB), and drops QB HQNR to 0.8038 / 0.7896.

Response to W3: Comparisons with Existing Cross-Sensor Approaches (UniPAN, ERFT, SWIFT, FoundPS)

  • Manuscript Revisions (Section 1, Table 1 & Section 2): We added Table 1 (tab:paradigm_comparison) in the Introduction and expanded Section 2 to systematically categorize existing cross-sensor paradigms and their operational assumptions: FoundPS requires multi-source pretraining on custom backbones; ERFT and SWIFT require target-side fine-tuning on target samples and target MTF priors; and UniPAN operates under the same single-source, fixed-weight, backbone-agnostic zero-shot setting as EquiPan.

  • Direct Quantitative Comparison with UniPAN and ERFT (Added as Group III in Revised Tables 3 & 4, Section 4.2): All three methods are trained uniformly on the source WV3 dataset using the exact same ABFNet backbone. While WV3 evaluates in-domain source performance, the three unseen datasets (WV2, QB, and GF2) evaluate cross-sensor transfer—where UniPAN and EquiPan are evaluated zero-shot with fixed WV3 weights, whereas ERFT is fine-tuned on each target dataset (pairing UniPAN and ERFT with wavelength interpolation/truncation for 8-to-4 band evaluation on QB and GF2):

DatasetMethod (WV3-Trained, ABFNet)PSNR ↑\uparrowSAM ↓\downarrowERGAS ↓\downarrowHQNR ↑\uparrowDλD_\lambda ↓\downarrowDsD_s ↓\downarrow
WV3 (8-band, Source)UniPAN36.793.352.350.94490.02490.0311
ERFT36.213.382.520.95740.02130.0218
EquiPan (Ours)37.323.262.210.95280.02000.0280
WV2 (8-band, Target)UniPAN (Zero-shot)29.245.584.200.92930.03830.0338
ERFT (Target fine-tuned)28.955.264.350.93600.03010.0350
EquiPan (Ours, Zero-shot)31.854.603.100.93410.02600.0410
QB (4-band, Target)UniPAN (Zero-shot)32.737.156.900.91650.05300.0330
ERFT (Target fine-tuned)34.256.115.790.90980.04870.0454
EquiPan (Ours, Zero-shot)35.105.995.020.91210.04500.0460
GF2 (4-band, Target)UniPAN (Zero-shot)38.302.622.680.84470.04600.1143
ERFT (Target fine-tuned)32.502.702.580.83560.06050.1103
EquiPan (Ours, Zero-shot)44.001.611.500.92120.03500.0450
  • In-Domain Source Retention (WV3): Under unified WV3 training, EquiPan achieves the highest source RR fidelity (37.32 dB PSNR vs. 36.79 dB for UniPAN and 36.21 dB for ERFT) and the lowest spectral distortion (Dλ=0.0200D_\lambda = 0.0200).

  • Transfer to Unseen Sensors vs. Zero-Shot UniPAN: Across the three unseen target datasets, EquiPan outperforms UniPAN by +2.61 dB on WV2, +2.37 dB on QB, and +5.70 dB on GF2 (+0.0765 HQNR on GF2), with consistently lower spectral distortion DλD_\lambda.

  • Transfer to Unseen Sensors vs. Target-Fine-Tuned ERFT: Even though ERFT is fine-tuned on each of the three unseen target datasets (WV2, QB, and GF2) using target MTF priors, fixed-weight EquiPan outperforms target-fine-tuned ERFT across all three unseen sensors in RR fidelity (+2.90 dB on WV2, +0.85 dB on QB, and +11.50 dB on GF2) as well as FR HQNR on QB (0.9121 vs. 0.9098) and GF2 (0.9212 vs. 0.8356), without requiring any target-domain fine-tuning. (Additional comparisons trained on GF2 and QB sources are provided in Response to W5 / Section 6.)

Reviewer_9qTKOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

This paper proposes EquiPan, a framework for cross-sensor pansharpening under a single-source training and fixed-weight deployment setting. The method formulates cross-sensor generalization as approximate equivariance under sensor transformations. It introduces Spectral–Contrast Transformation (SCT) to generate paired virtual sensor observations from source data, Reversible Spectral Transport (RST) to map heterogeneous spectral observations into a shared latent representation, and PAN Mean Alignment (PMA) to reduce global offsets between PAN and MS observations. Experiments on seven pansharpening backbones demonstrate improved zero-shot transfer from WorldView-3 to three unseen sensors, including WorldView-2, QuickBird, and Gaofen-2.

Critical strengths

The paper addresses a meaningful problem in cross-sensor pansharpening, particularly the challenging setting of transferring a source-trained model to unseen sensors without target-domain training or weight updates.

The proposed framework is evaluated across seven different backbones and three unseen sensors. The reported results show substantial improvements over interpolation-only baselines, suggesting that the approach can improve cross-sensor transferability.

Critical weaknesses

The methodological novelty is not sufficiently established. The proposed framework primarily combines spectral augmentation, wavelength-based interpolation, orthogonal spectral alignment, Cholesky normalization, and parameter-free PAN mean adjustment. Although these components are organized under the concept of sensor equivariance, their individual mechanisms are largely based on established mathematical operations and data augmentation principles. The paper should more clearly explain what fundamentally new learning principle is introduced compared with existing spectral normalization, domain generalization, and cross-sensor pansharpening approaches.

The central sensor-equivariance assumption lacks sufficient theoretical and physical justification. Equation (4) assumes that the observations of different sensors can be approximately related through a spectral transformation. However, real satellite sensors differ not only in band centers and spectral responses but also in spatial modulation transfer functions, point spread functions, noise characteristics, radiometric calibration, and PAN spectral sensitivity. The proposed SCT mainly simulates band mixing and spectral contrast variations while keeping the PAN observation unchanged. Consequently, the simulated transformations may not adequately represent the actual cross-sensor observation differences. The conditions under which the proposed equivariance principle remains valid are not sufficiently investigated.

The experimental comparisons do not fully establish superiority over existing cross-sensor generalization approaches. Although the paper discusses FoundPS, UniPAN, ERFT, and SWIFT, these methods are not included in the quantitative comparisons in Tables 2 and 3. The main controlled baseline is wavelength interpolation with the same backbone. While this comparison demonstrates improvements over a simple spectral adaptation strategy, it does not establish how EquiPan compares with stronger cross-sensor methods. The differences in training assumptions should be explicitly considered when designing fair comparisons.

The proposed reversible spectral transport requires stronger theoretical justification. RST employs Helmert bases, orthogonalized wavelength interpolation, and Cholesky normalization to construct a shared latent representation. However, the claimed reversibility is restricted to the observed spectral subspace rather than the full latent representation. More importantly, the paper does not sufficiently demonstrate that this mathematical reversibility guarantees preservation of physically meaningful spectral information or improved cross-sensor consistency. The sensitivity of the method to spectral interpolation, covariance estimation, and sensor response differences remains unclear.

The effectiveness of the individual components is not sufficiently investigated. Table 4 provides ablation results for RST, SCT, and PMA using ABFNet, but the analysis is limited to removing entire components. More fine-grained experiments are needed to distinguish the contributions of wavelength interpolation, orthogonal spectral alignment, Cholesky normalization, virtual band sampling, and spectral contrast scaling. In addition, the ablation results show that the complete framework does not consistently achieve the best reduced-resolution PSNR, raising questions about interactions among the components and potential trade-offs.

The generalization evaluation remains relatively limited. The experiments use WV3 as the only source sensor and WV2, QuickBird, and GF2 as unseen target sensors. Although the results are encouraging, they do not establish whether the proposed framework generalizes consistently across different source sensors, spectral response characteristics, or spatial degradation conditions. Reverse-transfer experiments and additional sensor combinations would provide stronger evidence for the claimed generalization capability.

The computational and deployment costs require further analysis. RST involves spectral transformations, orthogonalization, and covariance-based normalization, while the training framework introduces additional virtual observations and paired supervision. The paper does not provide a sufficiently comprehensive evaluation of the resulting computational overhead, memory requirements, training cost, and inference efficiency across the seven backbones.

Questions for authors

What are the fundamental methodological differences between EquiPan and existing cross-sensor pansharpening or domain generalization methods? How does the proposed equivariance formulation go beyond conventional spectral augmentation and feature normalization?

Equation (4) assumes that cross-sensor observations can be approximately related through a spectral transformation. How valid is this assumption when sensors have substantially different spectral response functions, spatial degradation operators, and PAN sensitivities?

Equation (6) formulates an approximate equivariance requirement, whereas Equation (21) optimizes reconstruction losses on the source and transformed branches. Why is no explicit equivariance-consistency loss included? Can the authors quantitatively demonstrate that the proposed training objective actually improves equivariance?

The proposed SCT keeps the PAN image unchanged while modifying the MS observations and their targets. How can this strategy adequately simulate cross-sensor variations when the PAN spectral response differs significantly between sensors?

The proposed RST is reversible only on the observed spectral subspace. What guarantees that the spectral information is sufficiently preserved during the forward and inverse transformations? How sensitive is the method to the selection of latent spectral dimensions and interpolation strategies?

Why are representative cross-sensor methods such as FoundPS, UniPAN, ERFT, and SWIFT not included in the quantitative comparisons? Can the authors provide additional comparisons while clearly distinguishing different training and deployment assumptions?

Table 4 shows that removing RST or PMA can slightly improve PSNR on some target sensors. How do the authors explain these results? More detailed ablation studies are needed to clarify the necessity of each mathematical operation.

How does the method perform when QuickBird, GF2, or WV2 is used as the source sensor? Would the same framework remain effective when the source sensor contains fewer spectral bands than the target sensor?

RST estimates normalization statistics from the complete target scene during deployment. How sensitive is the model to scene composition, image size, and distribution changes? What happens when only small image tiles are available?

What are the additional training and inference costs introduced by SCT and RST? More detailed comparisons of runtime, GPU memory consumption, and computational complexity would be valuable.

7 comments

We sincerely thank Reviewer 1 for the thorough review and insightful questions. We have substantially revised the manuscript (all changes highlighted in blue) and conducted extensive new experiments to address all Critical Weaknesses (W1--W7) and Questions (Q1--Q10) point by point below.

1. Response to W1 & Q1: Fundamental Learning Principle, Methodological Novelty, and Distinctions from Existing Paradigms

  • Manuscript Revisions (Section 1, Table 1, Section 2 & Section 3): We revised the Introduction, Related Work, and Method sections to clarify why EquiPan introduces a fundamentally new source-only equivariant learning principle rather than a heuristic combination of existing augmentation and normalization operations:

    1. The Dual-Constraint Principle (Revised Section 1 & Section 3.1): Single-source cross-sensor pansharpening faces two competing constraints: generalizing across sensors requires synthetic observation diversity (cross-domain manifold reachability), yet unconstrained spectral perturbations corrupt input–target radiometric correspondence and cause source collapse (in-domain task equivalence). EquiPan resolves this dilemma by requiring the fusion predictor to commute with subspace-invertible, physically admissible sensor transformations (Φ∘Tg≈Tg∘Φ\Phi\circ\mathcal{T}_g \approx \mathcal{T}_g\circ\Phi).

    2. Why SCT is Beyond Conventional Spectral Augmentation (Revised Section 2 & Section 3.2): In low-level pansharpening, augmentation is traditionally restricted to geometric flips/rotations because unilateral spectral perturbations break input–target radiometric correspondence and teach disruptive invariance. SCT is the first bilateral virtual sensor observation operator in pansharpening: it combines row-stochastic band mixing AgA_g (bridging discrete band selection BgB_g and continuous Gaussian response mixing RgR_g across K∈{4,8}K\in\{4,8\}) with bijective, mean-preserving chroma scaling Gγg\mathcal{G}_{\gamma_g}, applied symmetrically to both MS inputs (M,L)(M,L) and HRMS targets YY (Yg=Tg(Y)Y_g=\mathcal{T}_g(Y)) around the fixed PAN spatial anchor.

    3. Why RST is Beyond Feature Normalization & Marginal Alignment (Revised Section 2, Section 3.3 & Appendix A): Standard feature normalizations are lossy, while marginal distribution alignment (UniPAN) applies independent 1D CDF warping per band on fixed channel counts, distorting inter-band ratios. RST uses fixed orthonormal Helmert bases (Hobs,HlatH_{\mathrm{obs}}, H_{\mathrm{lat}}) to isolate per-pixel common luminance Lˉ\bar{L} from zero-mean chroma, orthogonalizes only the chromatic map (Eλ⊤Eλ=IC−1E_\lambda^\top E_\lambda = I_{C-1}), and couples it with regularized Cholesky covariance transport SLS_L. This guarantees an exact analytical left inverse (FoutFin=ICF_{\mathrm{out}}F_{\mathrm{in}}=I_C) on the observed subspace, ensuring zero algebraic loss and strict task equivalence (Y^=L\widehat{Y}=L when Δ=0\Delta=0).

2. Response to W2, Q2 & Q4: Physical Validity of Eq. (4), Unmodeled Sensor Variations (MTF/PSF, Noise), and Fixed PAN Anchor

  • Manuscript Revisions (Sections 1, 3.1 & 3.2): We revised Section 3.1 and Section 3.2 to clarify the physical rationale and scope of Eq. (4) and our fixed-PAN design:

    1. Relative PAN–MS Residual Formulation (Why Fixed PAN Simulates Cross-Sensor PAN–MS Variations, Q4): As shown in Eq. (20), our fusion network predicts high-frequency detail corrections added to the upsampled MS base (Y^=L+δ−μ(δ)\widehat{Y} = L + \delta - \mu(\delta)) driven strictly by the relative discrepancy between the PAN spatial anchor P∗P^{\ast} and the latent MS representation UU. Thus, cross-sensor transfer depends on the relative spectral overlap and contrast ratio between PAN and MS rather than their isolated absolute responses. Perturbing MS spectral responses and contrast via SCT while holding PAN fixed already trains the predictor across continuously varying relative PAN–MS overlap shifts, while PMA removes global offsets μ(L)−μ(P)\mu(L)-\mu(P) without altering spatial contrast P−μ(P)P-\mu(P).

    2. Why Injecting Spatial Blur (MTF/PSF) or Sensor Noise Violates Task Equivalence (W2 & Q2): Operational pansharpening benchmarks use radiometrically calibrated products where electronic gain/bias offsets are pre-rectified. Crucially, the measured PAN image serves as the sole high-resolution spatial ground truth at test time. Corrupting PAN with synthetic noise or blurring MS/GT with non-invertible spatial MTF kernels destroys high-frequency supervision, violates task equivalence, and induces over-smoothed reconstructions.

  • New Experimental Verification (Gaussian_Blur and Pan_noise): We evaluated two physical perturbation variants on ABFNet (WV3 source): (i) Gaussian_Blur (applying random Gaussian blur to MS and GT during training to simulate cross-sensor MTF shifts) and (ii) Pan_noise (injecting random Gaussian noise into PAN to simulate sensor noise differences):

Variant (ABFNet, WV3 Source)WV3 (PSNR / HQNR / DλD_\lambda / DsD_s)WV2-Zero (PSNR / HQNR / DλD_\lambda / DsD_s)QB (PSNR / HQNR / DλD_\lambda / DsD_s)GF2 (PSNR / HQNR / DλD_\lambda / DsD_s)
Gaussian_Blur (MS & GT MTF Blur)37.23 / 0.9552 / 0.022 / 0.02330.34 / 0.9416 / 0.031 / 0.02835.28 / 0.7650 / 0.050 / 0.17242.10 / 0.8372 / 0.081 / 0.096
Pan_noise (PAN Sensor Noise)37.26 / 0.9535 / 0.021 / 0.02630.19 / 0.9362 / 0.025 / 0.04035.24 / 0.7735 / 0.037 / 0.19643.67 / 0.9420 / 0.027 / 0.032
EquiPan (Ours)37.32 / 0.9528 / 0.020 / 0.02831.85 / 0.9341 / 0.026 / 0.04135.10 / 0.9121 / 0.045 / 0.04644.00 / 0.9212 / 0.035 / 0.045
  • Gaussian_Blur reduces RR PSNR by -1.51 dB on WV2 and -1.90 dB on GF2, and collapses full-resolution HQNR on 4-band targets: QB HQNR drops from 0.9121 to 0.7650 (DsD_s surges from 0.046 to 0.172) and GF2 HQNR drops from 0.9212 to 0.8372 (DλD_\lambda: 0.081 vs. 0.035; DsD_s: 0.096 vs. 0.045).

  • Pan_noise lowers WV2 PSNR by -1.66 dB and collapses QB HQNR from 0.9121 to 0.7735 (DsD_s jumps over 4x from 0.046 to 0.196). This confirms that keeping PAN as an uncorrupted spatial anchor is critical.

3. Response to Q3 (and W1/W2): Why Paired Supervision Controls Equivariance Error Without a Separate Consistency Loss

  1. Mathematical Bound Connecting Eq. (6) and Eq. (21):

    When transformed ground truth is unavailable, a consistency loss can directly encourage the predictor to commute with a transformation. In EquiPan, SCT instead generates the exact transformed target Yg=Tg(Y)Y_g = \mathcal{T}_g(Y) (Eq. 10), allowing direct supervision of both branches.

    Let Y^=Φθ,λ(M,L,P)\widehat{Y}=\Phi_{\theta,\lambda}(M,L,P) and Y^g=Φθ,λg(Mg,Lg,P)\widehat{Y}_g=\Phi_{\theta,\lambda_g}(M_g,L_g,P). For each sampled bounded linear transformation Tg\mathcal{T}_g, define its induced operator norm as cg=∥Tg∥1→1c_g=\lVert\mathcal{T}_g\rVert_{1\to1}. By the triangle inequality:

    ∥Y^g−Tg(Y^)∥1≤∥Y^g−Yg∥1+∥Tg(Y)−Tg(Y^)∥1\lVert \widehat{Y}_g-\mathcal{T}_g(\widehat{Y}) \rVert_1 \le \lVert \widehat{Y}_g-Y_g \rVert_1 + \lVert \mathcal{T}_g(Y)-\mathcal{T}_g(\widehat{Y}) \rVert_1.

    By the definition of the induced operator norm:

    ∥Tg(Y)−Tg(Y^)∥1≤cg∥Y^−Y∥1\lVert \mathcal{T}_g(Y)-\mathcal{T}_g(\widehat{Y}) \rVert_1 \le c_g\lVert \widehat{Y}-Y \rVert_1.

    Combining these inequalities gives:

    ∥Y^g−Tg(Y^)∥1≤2max⁡(1,cg)Lθ\lVert \widehat{Y}_g-\mathcal{T}_g(\widehat{Y}) \rVert_1 \le 2\max(1,c_g)\mathcal{L}_\theta,

    where Lθ=12∥Y^−Y∥1+12∥Y^g−Yg∥1\mathcal{L}_\theta=\frac{1}{2}\lVert\widehat{Y}-Y\rVert_1+\frac{1}{2}\lVert\widehat{Y}_g-Y_g\rVert_1 is the joint supervised loss in Eq. (21), using the same norm convention throughout.

    Thus, paired reconstruction supervision controls an upper bound on the equivariance error, with a factor determined by the sampled transformation. For each fixed bounded transformation, the equivariance error vanishes as both reconstruction errors approach zero. If the transformation family has a uniform bound cg≤cmax⁡c_g\le c_{\max}, the same conclusion extends to the expected loss over sampled transformations with factor 2max⁡(1,cmax⁡)2\max(1,c_{\max}).

    This provides a mathematical rationale for using Eq. (21) without a separate consistency penalty: at zero reconstruction error, consistency follows automatically from exact paired supervision. At finite reconstruction error, an additional consistency term may still affect optimization or generalization; the bound does not establish that such a term would be ineffective. Our claim is that Eq. (21) already supplies a supervised mechanism for controlling equivariance error.

  2. Empirical Evidence from no_virtual and SCT Sub-Component Ablations:

    To evaluate whether the joint objective in Eq. (21) and the sampled transformation family contribute to cross-sensor generalization beyond RST/PMA alignment, we evaluated no_virtual (retaining RST and PMA during training and inference, but removing the virtual reconstruction loss 12∥Y^g−Yg∥1\frac{1}{2}\lVert \widehat{Y}_g-Y_g \rVert_1), SCT Fixed 8 (fixing K=8K=8 without 4-band sampling), and SCT Selection (discrete selection BgB_g only, without continuous Gaussian mixing RgR_g):

Variant (ABFNet, WV3 Source)WV3 (PSNR / SAM / ERGAS / HQNR)WV2-Zero (PSNR / SAM / ERGAS / HQNR)QB (PSNR / SAM / ERGAS / HQNR)GF2 (PSNR / SAM / ERGAS / HQNR)
no_virtual (w/o ∥Y^g−Yg∥1\lVert \widehat{Y}_g-Y_g \rVert_1)37.47 / 3.22 / 2.18 / 0.952329.78 / 4.95 / 3.87 / 0.932733.50 / 7.46 / 6.29 / 0.832240.77 / 2.46 / 2.16 / 0.9298
SCT Fixed 8 (K=8K=8 only)37.40 / 3.24 / 2.19 / 0.952730.26 / 4.78 / 3.68 / 0.933434.43 / 6.25 / 5.63 / 0.761936.24 / 1.99 / 1.83 / 0.9337
SCT Selection (BgB_g only)37.33 / 3.26 / 2.21 / 0.954730.70 / 4.77 / 3.50 / 0.938835.24 / 6.03 / 5.12 / 0.791136.42 / 1.90 / 1.79 / 0.9253
EquiPan (Full Eq. 21)37.32 / 3.26 / 2.21 / 0.952831.85 / 4.60 / 3.10 / 0.934135.10 / 5.99 / 5.02 / 0.912144.00 / 1.61 / 1.50 / 0.9212

Removing the virtual reconstruction loss (no_virtual) reduces zero-shot PSNR by 2.07 dB on WV2, 1.60 dB on QB, and 3.23 dB on GF2. On QB, HQNR drops from 0.9121 to 0.8322, while DλD_\lambda more than doubles from 0.045 to 0.107. Restricting the transformation family to SCT Fixed 8 or SCT Selection reduces GF2 PSNR by 7.76 dB and 7.58 dB, respectively, and lowers QB HQNR to 0.7619 and 0.7911.

These results support the contribution of virtual paired supervision and the full SCT transformation family to cross-sensor reconstruction performance. They complement the mathematical bound above, but do not directly measure the equivariance error in Eq. (6) or establish that adding an explicit consistency penalty could not provide further benefits.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.