acceptodds
Under review as a conference paper at ICLR 2027

Entropy-Guided Heteroscedasticity Modeling for Fast Zero-Shot Remote Sensing Image Fusion

Abstract

Hyperspectral pansharpening (HyPan) and hyperspectral-multispectral image fusion (HMIF) reconstruct a high-resolution hyperspectral image (HR-HSI) from a low-resolution hyperspectral image (LR-HSI) and a high-spatial-resolution panchromatic or multispectral observation (PAN/MSI), a fundamental and highly ill-posed inverse problem in remote sensing. We address the zero-shot setting, in which the result must be obtained from the input observation pair alone without any external training data, and focus on optimization-based zero-shot methods, which optimize an untrained network directly on the input observations and stay faithful to the physical observation model. Uniformly weighting their spatial and spectral fidelity terms is, however, problematic: the spatial term is mainly informative at edges and fine structures, while the flat background that dominates remote sensing scenes dilutes these sparse constraints, and the spectral term is dominated by low-frequency content, so the two objectives drive the network in conflicting directions and the optimization settles at a sub-optimal compromise. We propose a fast zero-shot optimization framework with entropy-guided heteroscedasticity modeling: an information gate computed from the local Shannon entropy down-weights uninformative regions and strengthens genuine high-frequency structure, aligning the two objectives, and a two-stage schedule first anchors the spectral manifold and then refines spatial details. On HyPan and HMIF benchmarks, our method improves spectral fidelity and spatial sharpness over state-of-the-art methods, reconstructing an image in about 17 seconds without any external training data.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Hyperspectral pansharpening, Hyperspectral-multispectral image fusion, Remote sensing

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_vZKqOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a fast zero-shot framework for hyperspectral pansharpening and hyperspectral–multispectral image fusion. The method is based on a relatively straightforward idea: the spatial and spectral objectives are decoupled to reduce optimization conflict, while local Shannon entropy is used to reweight the spatial loss so that structurally informative regions receive greater emphasis. A two-stage optimization strategy is further introduced, with spectral warm-up followed by spatial refinement, together with a lightweight CNN and subspace projection to improve efficiency. The proposed method achieves strong quantitative results on several standard HyPan and HMIF benchmarks and is considerably faster than many existing zero-shot approaches.

While the method is effective, I find the overall contribution somewhat incremental. The main ingredients—loss decoupling, staged optimization, entropy-based weighting, and lightweight zero-shot reconstruction—are conceptually intuitive and closely related to ideas that have already been explored in related literature. As a result, the paper reads more like an effective adaptation and combination of mature techniques for a specific remote sensing fusion problem than a substantially new methodological contribution.

My main concern is therefore not whether the method works, but whether the paper provides sufficiently deep and generalizable insight beyond the reported performance improvements. The current analysis is still relatively limited, particularly regarding why entropy is the appropriate proxy for spatial reliability, how robust the proposed strategy is under different observation conditions, and whether the same optimization principle consistently benefits other recent fusion frameworks. In addition, the evaluation is largely based on classical benchmark datasets, which makes it harder to assess the practical relevance and generality of the conclusions.

Overall, the paper presents a reasonable and efficient solution, but I believe substantially stronger analysis and broader validation would be necessary to make the contribution sufficiently compelling for ICLR.

Critical strengths
  • Clear and intuitive motivation. The paper identifies a meaningful optimization issue in zero-shot hyperspectral fusion: the spatial and spectral fidelity objectives may provide conflicting gradients, particularly because spatial information is concentrated around edges and fine structures while spectral supervision is dominated by lower-frequency content. The proposed entropy-guided weighting provides a simple way to alleviate this imbalance.

  • Strong efficiency–performance trade-off. The proposed method achieves competitive or state-of-the-art results on the evaluated HyPan and HMIF benchmarks while remaining substantially faster than many recent zero-shot and diffusion-based approaches. On the HyPan benchmarks, the runtime is around 16–17 seconds per image.

  • Simple and practical zero-shot formulation. The framework does not require external training data or pretrained generative models. The combination of subspace projection, a lightweight CNN, and two-stage optimization makes the method relatively easy to implement and computationally manageable.

  • Reasonable empirical analysis of the optimization behavior. The paper goes beyond reporting final benchmark scores by examining gradient cosine similarity, convergence behavior, spectral-band reconstruction quality, and spatial-frequency characteristics, which provides some evidence for the intended optimization mechanism.

  • Good practical effectiveness despite the simplicity of the design. Although the individual components are conceptually straightforward, their combination produces consistent gains across multiple standard datasets, suggesting that the proposed optimization strategy is practically useful for this task.

Critical weaknesses
  1. The methodological novelty is limited, while the current analysis is not sufficiently deep.
    As the authors themselves note, gradient conflict is a well-known phenomenon in areas such as multi-task learning, and systematically modulating conflicting objectives can substantially benefit model optimization. Applying this idea to hyperspectral image fusion in a coherent manner is certainly a strength of the paper. However, I find the accompanying analysis rather limited. The paper mainly provides quantitative comparisons, efficiency measurements, reconstruction visualizations, gradient cosine similarity, and frequency-domain analyses.

    I believe the authors could employ richer analytical tools to better characterize both the original optimization issue and the behavior after applying the proposed strategy. For example, visualizing the optimization or loss landscape, or more systematically tracking the dynamics of the individual gradients throughout training, could provide a clearer understanding of how and why the proposed decoupling strategy works. For a paper centered around a relatively focused methodological contribution rather than a broader systematic study, I consider the depth of analysis particularly important. Overall, the current level of analysis is not yet sufficient to meet the acceptance standard I apply to ICLR 2026 submissions.

  2. The evaluation settings are relatively limited.
    I would encourage the authors to evaluate the proposed method under more realistic and controlled degradations, such as mild sensor noise, stripe noise, structured artifacts, or asymmetric corruptions. The current evaluation mainly follows standard synthetic degradation settings, including Gaussian blur and 4×4\times downsampling. It would be important to understand whether the proposed entropy-guided optimization strategy remains effective when the spatial observation itself contains non-negligible corruption, particularly because local entropy may also respond strongly to noise or structured artifacts.

  3. The efficiency claim deserves a more systematic analysis.
    Achieving approximately 17 seconds per image is a clear practical advantage of the proposed method. 23645_Entropy_Guided_Heterosce However, the paper does not sufficiently explain why the competing zero-shot methods are substantially slower, nor does it identify the dominant computational bottleneck of the proposed framework itself. If “fast zero-shot fusion” is intended to be one of the main contributions, I believe a more detailed breakdown of runtime, memory consumption, optimization stages, and computational bottlenecks would significantly strengthen this claim.

  4. The selected datasets are classical but relatively dated.
    Although the current benchmarks are standard and widely used, they are also relatively old and may not sufficiently reflect the diversity of more recent hyperspectral data. I recommend adding at least one newer and more heterogeneous dataset to strengthen the evidence for generalization. For example, a dataset containing a larger number of scenes with diverse spatial content would provide a more convincing evaluation of whether the proposed optimization strategy remains reliable across different scene characteristics.

The authors may also consider evaluating the proposed method on HyperAD, introduced in PromptHSI: Universal Hyperspectral Image Restoration With Vision–Language Modulated Frequency Adaptation (IEEE TGRS, 2026). Compared with the current benchmarks, HyperAD contains a larger number of hyperspectral images and covers more heterogeneous scenes with diverse content. I believe evaluation on such a dataset could provide stronger evidence regarding the robustness and generalizability of the proposed optimization strategy across different scene characteristics.

C.-H. Cheng, C.-M. Lee, W.-T. Liao, Y.-F. Lin, Y.-C. Cheng, F.-E. Yang, Y.-C. F. Wang, and C.-C. Hsu, “PromptHSI: Universal Hyperspectral Image Restoration With Vision–Language Modulated Frequency Adaptation,” IEEE Transactions on Geoscience and Remote Sensing, vol. 64, pp. 1–16, 2026.

Questions for authors

Please provide a strong and detailed rebuttal addressing the weaknesses raised above.

4 comments

We thank the reviewer for the constructive report and for suggesting HyperAD, which we found to be a useful benchmark.

W1. Methodological novelty is limited; analysis is not sufficiently deep (only quantitative comparisons, efficiency, visualizations, gradient cosine, frequency analysis). Suggest richer analytical tools (optimization/loss landscape, systematic gradient dynamics tracking).

We do not assemble existing components unchanged. The contribution is this: having observed a gradient conflict between the two fidelity terms of zero-shot fusion, we relieve it with an entropy weight read off the observation, and we design the model and the schedule that this relief makes possible. The two analyses below are the evidence for it.

1. The gradient conflict, and the cause we identify for it. In the later stage of the optimization the gradients of the spatial and spectral terms turn against each other and the result settles at a compromise. Our gradient analysis locates the cause in the spatial constraint itself: it is informative only in a sparse set of regions. It still acts on smooth regions, but there it adds nothing to the restoration; and early in the fit, which is mostly low-frequency, it has little to do either.

2. An entropy gate that pays the spatial budget where it is informative, and a schedule that follows from the same analysis. The gate is read off the observation, so it suppresses the constraint where it is redundant and keeps it where it drives detail; and because the term carries little early, the first stage lowers its weight and, in our design, removes it entirely, so a spectral-only loss fitted with SGD anchors the subspace quickly.

3. A model as light as the objective allows. Once the objective is accurate, less has to be supplied by the network's own capacity, so a four-layer network suffices.

4. Different from existing zero-shot methods, we do not strengthen the prior to fit the loss; we make the loss accurate and keep the prior minimal. They spend on a stronger prior (CS2DIPs couples two U-Nets with a tensor factorization for 5000 iterations, CLoRF evaluates two SIREN networks at every pixel for 30000 epochs, DM-ZS runs thousands of diffusion steps) and then fit a uniformly weighted spatial term with it. Removing that expense gives 0.16 M parameters, the best PSNR on every benchmark, about 17 seconds per image, and 5 to 130 times faster.

Loss landscape (Fig. 7a). Two runs leave the same warmed-up state and differ only in the weight map. On the unweighted criterion they separate within a few hundred iterations and end in different minima, L=0.0025\mathcal{L}=0.0025 with the gate against 0.00290.0029 with a constant weight: the gate changes which solution is reached, not only how fast, so the conflict leaves a lasting effect rather than a transient one.

Gradient dynamics (Fig. 7b,c). At three iterations of stage II the two gradients occupy different bands. Their median radial frequency is 3 for the spectral gradient and 39 for the spatial one, and below 32, the Nyquist band of the 4× downsampled observation, 97% of the spectral gradient energy lies against 45% of the spatial one. About half of the spatial budget is therefore spent where the spectral term already works, which is what the gate reallocates. This is also why the conflict appears late: once the low-frequency band has been fixed by the warm-up, the spectral gradient keeps pulling it back while the spatial gradient keeps pushing everywhere.

Both analyses are Figure 7 and Appendix F of the revision.

W2. Evaluation settings are relatively limited; suggest realistic/controlled degradations (mild sensor noise, stripe noise, structured artifacts, asymmetric corruptions), since local entropy may respond strongly to noise/artifacts.

Both halves of the question are answered. We added the four settings on three CAVE scenes, each one shared corrupted copy that every method reads: Gaussian noise on both observations (σ=0.01\sigma=0.01 and 0.020.02), an MSI stripe, and an asymmetric corruption that removes 5 rows and 2% of the 8×88\times8 blocks from the MSI only. To isolate the gate, the same network, initialization, schedule and budget are run with the weight map replaced by a constant:

cleannoise 0.01noise 0.02stripemissing
Const42.62441.05634.46235.07821.983
Proposed44.14942.00136.32936.89929.833
gain of the gate+1.53+0.95+1.87+1.82+7.85

The gate stays above the constant in all five settings, so it remains effective when the observation is corrupted. Additive noise is the harder case: it raises the local entropy everywhere and compresses the gate toward a uniform map, which is where our margin is smallest (0.95 dB at σ=0.01\sigma=0.01). In the other direction, corruption that destroys structure (the stripe and the removed rows and blocks) is locally flat and is therefore down-weighted rather than fitted, which is where the margin is largest (7.85 dB). Comparing with the five baselines on the same corrupted inputs, we are first in three of the five settings, level with the best baseline on the clean input, and second on the block-removal case.

W3. Efficiency claim deserves more systematic analysis: why competing zero-shot methods are slower, and what is the dominant computational bottleneck of the proposed framework. Suggest runtime breakdown, memory consumption, optimization stages, bottlenecks.

Why they are slower. The cost is the prior. Ours is a 0.16 M network, 76% of it in stage II, 195 MiB peak. DM-ZS runs thousands of reverse steps, each a full-resolution pass of a multi-band denoiser plus a data-consistency gradient; CLoRF evaluates two implicit networks at every pixel of the full-resolution grid, and its per-iteration cost therefore scales with the number of pixels, over 10410^4 iterations; CS2DIPs couples two DIPs with a tensor factorization for 5×1025\times10^2--10310^3 iterations; ZSL trains on patches of downsampled images and needs many steps to converge. In all four the cost is the prior.

Where our cost goes. We instrumented the HMIF scene:

preprocessphase Iphase IIper iterationpeak memory (phase II)parameters
0.17 s5.4 s17.6 s5.9 ms195 MiB0.16 M

Phase II dominates at 76% of the total, and each of its iterations is one forward--backward pass of a 0.16 M network on 12 bands, so the cost is linear in the iteration count and does not grow with ChsiC_{\text{hsi}}; the subspace projection and the loss evaluation add little. The entropy map costs under 0.2 s, but built with a one-hot histogram it sets the transient memory peak (1.9 GB); chunking that computation removes the peak.

We also tested whether the cost is launch overhead rather than computation: running two and three scenes concurrently takes 2× and 3× the wall time with no throughput gain, and a single run already keeps the GPU at about 90%, so the cost is the phase-II computation itself, not scheduling or memory.

W4. Selected datasets are classical but relatively dated; suggest adding at least one newer, more heterogeneous dataset with more scenes and diverse spatial content. Suggest evaluating on HyperAD (from PromptHSI, IEEE TGRS 2026), which contains more hyperspectral images and more heterogeneous scenes.

Thank you for pointing us to HyperAD. It is now in Section 4.1 and in the bottom part of Table 1: 20 of its 100 test images, chosen uniformly in name order, at 256×256×172256\times256\times172, generated with the same downsampling protocol as the other HyPan datasets, and compared with one representative of each family.

MethodPSNRSSIMSAMERGASRMSETime (s)
ZSL41.370 ±\pm 4.9990.941 ±\pm 0.0464.041 ±\pm 2.9823.063 ±\pm 1.7450.0148 ±\pm 0.008680.32
PLRDiff30.480 ±\pm 4.8470.698 ±\pm 0.16914.262 ±\pm 8.83516.303 ±\pm 12.4290.0701 ±\pm 0.05087.05
DM-ZS40.466 ±\pm 3.7460.924 ±\pm 0.0494.180 ±\pm 2.2693.061 ±\pm 1.3200.0147 ±\pm 0.006314.45
CLoRF31.534 ±\pm 3.1050.676 ±\pm 0.12710.322 ±\pm 5.17812.126 ±\pm 7.2990.0344 ±\pm 0.010245.51
Proposed42.715 ±\pm 3.9630.953 ±\pm 0.0353.155 ±\pm 1.6282.353 ±\pm 1.0930.0117 ±\pm 0.00517.26

On this dataset the method is the best on all five metrics, faster than the most accurate competitor. We did not run all nine methods of Table 1 on HyperAD: the four reported are one representative each of the pseudo-paired, diffusion-prior, diffusion-guidance and implicit-representation families. Because the diffusion and implicit-representation baselines take a long time per image, this response evaluates 20 of the 100 test images; the final version will evaluate all 100.

(Question in Paper Summary) Whether the same optimization principle consistently benefits other recent fusion frameworks?

Transfer to other frameworks. We tried it. The gate is only a change to the loss, so we inserted it into PLRDiff and into CLoRF and left everything else untouched; on both, the PSNR moves marginally. The reason is not that the gate is useless there, but that those two methods already carry a prior strong enough to fit a plain loss, so a better weighted loss has little left to improve and its advantage is masked instead of absent. It pays off only together with the rest of our design, a minimal network and the identity warm-up, which is why we present the three as one choice.

We hope these clarifications address the reviewer's concerns, and we are happy to provide any further detail or run additional experiments during the discussion.

Thank for your response, I am satisfied about my concerns. I will increase my score.

Good lucks!

Reviewer_T8NtOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper presents a fast zero-shot framework for hyperspectral pansharpening and hyperspectral-multispectral fusion. The method offers a useful solution to zero-shot fusion, with strong reported performance and a convincing reduction in runtime relative to several baselines. A four-layer CNN predicts coefficients in a spectral subspace, while a spatial fidelity term is weighted by squared, normalized local entropy from the PAN/MSI observation. An identity warm-up is followed by joint spatial and spectral optimization. The method requires no external training data or pretrained model. Experiments on three HyPan scenes and the CAVE and Harvard datasets show competitive reconstruction quality of the proposed method.

Critical strengths

The main strength is the quality-efficiency trade-off without external training. The method obtains the best reported PSNR, SSIM, and SAM on all three HyPan scenes in about 16-17 seconds per image, and the highest PSNR on CAVE and Harvard. These results make the approach practically useful when paired high-resolution hyperspectral training data are unavailable. The speed advantage over the more expensive neural baselines is substantial, even though some classical methods remain faster. The design is simple enough to understand and implement. The observation-derived gate adds little computational overhead, and Table 3 includes a tuned constant-weight baseline rather than only a weak default. Entropy improves PSNR over this baseline on all three HyPan scenes. Together with the warm-up and architecture studies, this gives useful evidence for the individual design choices. I view this as a worthwhile contribution to efficient zero-shot fusion. In addition, the entropy-guided heteroscedasticity framework is relatively general and has the potential to be incorporated into other methods.

Critical weaknesses

The interpretation of entropy as observation uncertainty needs clearer justification. Local histogram entropy measures intensity variation, whereas homogeneous regions may still provide reliable radiometric constraints. The gradient analysis illustrates favorable alignment, but whether this behavior holds across scenes and initializations remains unclear. The gains over tuned constant weighting are useful but modest. Overall, the results support the practical value of the gate more clearly than the broader claim that it removes gradient conflict.

Questions for authors
  1. Please specify the entropy window, histogram bins, MSI channel aggregation, subspace dimension, and loss weights. Were these settings fixed across datasets, and how were they selected without access to high-resolution test targets?

  2. How representative is the gradient-cosine trajectory in Figure 1 across scenes and initializations? Is conflict removal intended as an empirical observation for the reported cases, or is there a stronger argument beyond the illustrated trajectory?

  3. On CAVE, the proposed method achieves the highest PSNR but reports a SAM of 4.594 ± 11.065. Which images account for this large variation?

  4. Please clarify whether the entropy gate models observation uncertainty or prioritizes structurally informative regions. These interpretations are not equivalent, since homogeneous regions may still provide reliable radiometric constraints.

2 comments

We thank the reviewer for the positive assessment and for the precise questions, which we answer in turn.

W1. Interpretation of entropy as observation uncertainty needs clearer justification; local histogram entropy measures intensity variation, while homogeneous regions may still provide reliable radiometric constraints.

This is the same point as Q4, where we answer it in full. In short, the gate measures how much the spatial residual contributes to detail in a region, not how reliable the observation is: it suppresses that residual where it adds nothing, and no supervision is removed, because the spectral term still constrains those regions.

W2. Gradient analysis illustrates favorable alignment, but whether this holds across scenes and initializations remains unclear.

This is the same question as Q2, where we answer it in full.

W3. Gains over tuned constant weighting are modest; results support practical value of the gate more clearly than the broader claim that it removes gradient conflict.

The gain over a tuned constant is small; the conflict it removes is a different quantity. On the three HyPan scenes the margin is 0.17 to 0.29 dB, and we also ran the same comparison on CAVE, where the tuned constant is 0.97 dB behind (Table 3 of the revision, which also contains the gradient-, variance- and edge-magnitude comparisons). The evidence for the conflict is not this margin, however: a constant weight still constrains every pixel, so it cannot show where the budget should be spent. The conflict shows in the optimization itself, and we quantify it in our reply to Q2.

variantChikusei ΔPSNRChikusei ΔSSIMPavia ΔPSNRPavia ΔSSIMHouston ΔPSNRHouston ΔSSIMCAVE ΔPSNRCAVE ΔSSIM
Entropy00000000
Const−0.165−0.0028−0.189−0.0031−0.290−0.0031−0.971−0.0003

What the gate provides is a way to spend the spatial budget where it is informative, at no extra cost.

Q1. Specify entropy window, histogram bins, MSI channel aggregation, subspace dimension, and loss weights. Were these settings fixed across datasets, and how were they selected without access to high-resolution test targets?

Settings. The entropy window is 5×55\times5 pixels and the histogram uses 32 bins over the intensity range of the observation; the entropy is computed once per image before optimization and held fixed, so the gate adds no measurable memory or time consumption. For MSI the entropy map is computed per channel, and each channel's own map weights that channel's residual: the spatial loss sums, over pixels and over MSI channels, the absolute residual of each channel weighted by the square of that channel's normalized entropy, and it reduces to the single-channel form for PAN. The subspace dimension is K=8K=8 for HyPan and K=12K=12 for HMIF, with the KK bands taken at even intervals across the spectral axis. The loss weights are α=1.0\alpha=1.0 and β=10.0\beta=10.0.

Are they fixed across datasets? Yes, except KK and the per-channel handling, which depend on the sensor rather than on the scene.

The Adam budget is per task: 1600 iterations for HyPan and 3000 for HMIF. The paper gave a single figure for both; the results are those of the per-task schedule, so only the description changes.

How were they selected without high-resolution targets? KK follows the rank the paired observation can support, which is why it is 12 for HMIF and 8 for pansharpening. The window, the bin count and α,β\alpha,\beta were fixed by a small grid on one scene per task, minimising the two consistency residuals that the observations themselves provide, ∥(B⊗X^)↓−Y∥1\|(\mathbf{B}\otimes\hat{\mathcal{X}})\downarrow-\mathcal{Y}\|_1 and ∥X^R−P∥1\|\hat{\mathcal{X}}\mathbf{R}-\mathcal{P}\|_1, and then frozen on all tasks.

Q2. How representative is the gradient-cosine trajectory in Figure 1 across scenes and initializations? Is conflict removal intended as an empirical observation for the reported cases, or is there a stronger argument beyond the illustrated trajectory?

Representative across scenes and initializations: the sign of the late-phase cosine similarity is stable on five further CAVE scenes and on the HyPan scenes. We repeated the protocol there; the gate enters only the second stage, so the warm-up itself is unweighted and is not part of this claim.

mean cosineentropy gateconstant weight
iterations 2500 to 3000+0.416+0.083
iterations 2700 to 3000+0.327−0.039

An empirical observation with a mechanism behind it, not only an illustration. The constant weight drifts towards zero and then negative as the optimization proceeds, while the gate stays positive. The mechanism is the frequency separation of the two gradients: the spectral gradient acts on the low-frequency band that is fitted first (median radius r=3r=3, with 97% below r=32r=32), the spatial gradient is spread over all frequencies (median r=39r=39, with 45% below r=32r=32), so once that band is settled the spatial term keeps pushing the solution back out of it. The loss landscape of Fig. 7a shows the consequence, since the two weightings reach different minima of the same unweighted criterion, and its effect on the optimization is not merely slower convergence.

Q3. On CAVE, the proposed method achieves highest PSNR but reports SAM of 4.594 ± 11.065. Which images account for this large variation?

One patch accounts for it. On the 44 CAVE patches all but one have a SAM below 5.7, and patch 24 alone has 76.8; that patch is a background tile of the 2×22\times2 split, with 82% of its MSI pixels below 0.01 and a mean radiance of 0.006 against 0.019 for the next darkest, while its PSNR is 42.9 dB, inside the range spanned by the other patches. SAM is ill-conditioned when the spectrum carries almost no energy.

patch setPSNRSSIMSAMERGAS
all 4449.22 ±\pm 4.520.987 ±\pm 0.0414.59 ±\pm 11.072.13 ±\pm 4.32
43, excluding patch 2449.37 ±\pm 4.460.994 ±\pm 0.0042.92 ±\pm 1.171.49 ±\pm 0.89

Q4. Clarify whether the entropy gate models observation uncertainty or prioritizes structurally informative regions. These interpretations are not equivalent, since homogeneous regions may still provide reliable radiometric constraints.

It prioritizes structurally informative regions; it is not an uncertainty model.

What the statistic measures is the dispersion of intensities inside a window. A window containing mm distinct levels has entropy at most log⁡m\log m regardless of how far apart those levels are. It therefore indicates the presence of structure that the low-resolution observation cannot reproduce, not the reliability of the measurement.

Homogeneous regions are indeed reliable, and that is exactly why they need no extra weight: a low-resolution pixel summarizes such a neighbourhood, so the spectral term already fixes their radiometric content, and a small spatial weight there removes a constraint that duplicates the spectral one rather than the supervision itself.

The argument is therefore about allocation instead of reliability.

We hope these clarifications address the reviewer's concerns, and we are happy to provide any further detail or run additional experiments during the discussion.

Reviewer_8nXGOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes an entropy-guided zero-shot framework for hyperspectral pansharpening and hyperspectral–multispectral image fusion. The method introduces a local Shannon entropy-based weighting strategy to balance spatial and spectral fidelity losses and mitigate gradient conflicts during optimization. A lightweight CNN with spectral subspace projection and a two-stage optimization strategy is employed to improve reconstruction quality and efficiency. Experiments on several benchmark datasets demonstrate competitive reconstruction performance, with an average runtime of approximately 17 seconds for hyperspectral pansharpening, without requiring external training data.

Critical strengths
  1. The paper addresses a meaningful problem in zero-shot hyperspectral image fusion. The proposed framework is lightweight and does not require external training data.

  2. The method achieves competitive reconstruction performance on several benchmarks, with relatively low computational cost.

Critical weaknesses
  1. The methodological novelty is limited. Entropy-based weighting, spectral subspace projection, and two-stage optimization are established techniques. The authors should better distinguish their contributions from existing zero-shot fusion methods.

  2. The theoretical justification for heteroscedasticity modeling is insufficient. Local Shannon entropy does not necessarily reflect observation uncertainty or noise variance. The assumption that high-entropy regions provide more reliable spatial guidance needs stronger evidence.

  3. The ablation studies do not convincingly establish the benefits of entropy weighting. The PSNR improvements over constant weighting are relatively small (0.17–0.29 dB). Comparisons with simpler gradient-, variance-, or edge-based weighting strategies are missing.

  4. The experimental validation requires further clarification. The reported 44 CAVE image pairs should be explained, as the original dataset contains 32 scenes. Furthermore, robustness under spatially varying noise and realistic degradation conditions has not been adequately evaluated.

Questions for authors

see above

3 comments

We thank the reviewer for the detailed report. W1 to W3 are answered in full and W4 in two parts with new experiments.

W1. Methodological novelty is limited (entropy weighting, subspace projection, two-stage optimization are established).

We do not assemble existing components unchanged. The contribution is this: having observed a gradient conflict between the two fidelity terms of zero-shot fusion, we relieve it with an entropy weight read off the observation, and we design the model and the schedule that this relief makes possible.

1. The gradient conflict, and the cause we identify for it. In the later stage of the optimization the gradients of the spatial and spectral terms turn against each other: convergence slows and the result settles at a compromise. Our gradient analysis locates the cause in the spatial constraint itself: it is informative only in a sparse set of regions. It still acts on smooth regions, but there it adds nothing to the restoration; and early in the fit, which is mostly low-frequency, it has little to do either.

2. An entropy gate that pays the spatial budget where it is informative, and a schedule that follows from the same analysis. The gate is read off the observation, so it suppresses the constraint where it is redundant and keeps it where it drives detail; and because the term carries little early, the first stage lowers its weight and, in our design, removes it entirely, which shortens the run. Figure 1 shows the result: the conflict is relieved and the final accuracy improves.

3. A model as light as the objective allows. Once the objective is accurate, less has to be supplied by the network's own capacity, so a four-layer CNN suffices.

4. Different from existing zero-shot methods, we do not strengthen the prior to fit the loss; we make the loss accurate and keep the prior minimal. They spend on the prior (CS2DIPs couples two U-Nets with a tensor factorization for 5000 iterations, CLoRF evaluates two SIREN networks at every pixel for 30000 epochs, DM-ZS runs thousands of diffusion steps) and then fit a uniformly weighted spatial term with it. Removing that expense gives 0.16 M parameters, the best PSNR on every benchmark, about 17 seconds per image, and 5 to 130 times faster.

W2. Theoretical justification for heteroscedasticity modeling is insufficient; local Shannon entropy does not necessarily reflect observation uncertainty/noise variance; assumption that high-entropy regions provide more reliable spatial guidance needs stronger evidence.

Point by point, in the order of the three concerns.

(a) The justification, stated more fully. With A(X)=(B⊗X)↓\mathbf{A}(\mathcal{X})=(\mathbf{B}\otimes\mathcal{X})\downarrow onto and Π=A⊤(AA⊤)−1A\boldsymbol{\Pi}=\mathbf{A}^{\top}(\mathbf{A}\mathbf{A}^{\top})^{-1}\mathbf{A}, the spectral term depends on X\mathcal{X} only through ΠX\boldsymbol{\Pi}\mathcal{X}, so ∂Lspe/∂(I−Π)X=0\partial\mathcal{L}_{\text{spe}}/\partial(\mathbf{I}-\boldsymbol{\Pi})\mathcal{X}=\mathbf{0} identically: the remaining content is constrained by the spatial term alone, and a per-pixel weight on it is well defined. A window with mm distinct levels has entropy at most log⁡m\log m, however far apart those levels are, so the gate reports whether the residual is non-negligible, not how large it is.

(b) Not an uncertainty estimate, and it does not need to be one. We thank the reviewer for pointing this out, and we agree that local entropy is not an estimate of the observation uncertainty or of the noise variance. It is not meant to be one: what it reflects is how informative the spatial residual is in a region, and informativeness is the property a per-pixel weight needs. Spending the weight by it is what relieves the gradient conflict between the two fidelity terms.

(c) We ran an experiment to see whether the entropy marks the locations that are hard to restore, and it does. The test is this: take the reconstruction obtained with a uniform weight, measure its per-pixel error against the ground truth, and ask with what AUC each candidate weight map ranks those failing locations before any training (Chikusei; the ground truth is used only for this measurement and never enters the gate):

mapentropyvarianceedgegradientdropoutuniform (const)
AUC0.7720.7630.7470.7280.4680.500

The entropy map ranks them highest, against 0.73 to 0.76 for variance, edge and gradient, and 0.468 for MC dropout. A uniform map is 0.500 by construction, since it emphasises nothing, and dropout is no better than it, which is why dropout buys nothing. The ranking shows that the entropy concentrates the spatial budget where the error is, which is what relieves the conflict; the other structure statistics capture most of the same ranking, which is also why edge magnitude is a close second.

We claim redundancy: every region stays fully supervised by the spectral term. These points are stated in the revision in Section 3.3.

W3. Ablation studies do not convincingly establish benefits of entropy weighting; PSNR gains over constant weighting are small (0.17–0.29 dB); comparisons with gradient-, variance-, or edge-based weighting are missing.

We also added the same comparison on CAVE. Five alternatives replace the entropy gate, with the network, initialization and β\beta held fixed; entries are differences from the gate, so a positive value means the variant is better. On the three HyPan scenes the alternatives differ from the gate by at most about 1 dB; on CAVE every one of them is behind.

variantChikusei ΔPSNRChikusei ΔSSIMPavia ΔPSNRPavia ΔSSIMHouston ΔPSNRHouston ΔSSIMCAVE ΔPSNRCAVE ΔSSIM
Entropy00000000
Const−0.165−0.0028−0.189−0.0031−0.290−0.0031−0.971−0.0003
Variance−0.851−0.0154−0.996−0.0163−0.464−0.0102−1.844−0.0008
Gradient−0.213−0.0028−0.313−0.0046−0.157−0.0033−1.052−0.0004
Edge+0.047+0.0016−0.074−0.0008+0.005−0.0001−1.174−0.0001
Dropout−0.047−0.0009+0.054+0.0006−0.023+0.0001−4.381−0.0028

On the three HyPan scenes the margin over a tuned constant is indeed small and edge magnitude ties. On CAVE the same comparison is several times wider. We attribute the wider gap to the MSI residual having a much larger spatial spread than the PAN residual, which a single global weight cannot follow; we report this as an observation, not as a proven mechanism. Variance is the worst alternative almost everywhere because min–max normalization lets a few high-contrast outliers set its scale. The full comparison is in Table 3 of the revision.

W4. Experimental validation requires clarification: reported 44 CAVE image pairs.

The 44 pairs are obtained from 11 representative CAVE images of 512×512×31512\times512\times31, each cut into four 256×256×31256\times256\times31 patches, following the split of PSRT (Deng et al., 2023); 256×256256\times256 is also the patch size at which the competing methods are evaluated in their own papers. The protocol is stated in Section 4.2 of the revision.

W4-2. Robustness under spatially varying noise and realistic degradation conditions not adequately evaluated.

We added four settings on three representative CAVE scenes, each one shared corrupted copy that every method reads: Gaussian noise on both observations (σ=0.01\sigma=0.01 and 0.020.02), an MSI stripe, and an asymmetric corruption that removes 5 rows and 2% of the 8×88\times8 blocks from the MSI only, the last two being the spatially varying cases in which the spectral observation stays clean. We compare with five representative methods; the strongest competitor in each column is:

cleannoise 0.01noise 0.02stripemissing
best competitor44.148 (ZSL)37.534 (ZSL)36.106 (CLoRF)36.531 (CTDF)30.432 (CLoRF)
Proposed44.14942.00136.32936.89929.833

We lead in three of the five settings, match the best competitor on the clean input (44.149 against 44.148) and are second on the asymmetric case by 0.6 dB: a smooth low-rank representation is insensitive to corrupted pixels almost by construction, whereas a per-pixel gate cannot fully discount a corrupted block that it reads as structure. Under noise CLoRF degrades less, since it starts 7 dB lower, but our absolute score is the highest at both noise levels (36.33 against 36.11 at σ=0.02\sigma=0.02). Replacing the gate by a constant weight makes every one of the five settings worse, by 0.95 to 7.85 dB. The full comparison is Table 7 of the revision (Appendix E).

We hope these clarifications address the reviewer's concerns, and we are happy to provide any further detail or run additional experiments during the discussion.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Author Response Summary

We thank Reviewers 8nXG, T8Nt and vZKq for their detailed and constructive evaluations. We uploaded a revised version of the paper, and we summarize the discussion below.

Status: vZKq confirmed that the response resolves the remaining concerns and raised the rating, which we greatly appreciate.

Revision: the contribution is stated against prior-side methods, with the conflict and its cause named (Section 1); Eq. 6 makes explicit what the spatial term constrains (Section 3.3); six weightings on four datasets (Table 3); four corrupted settings (Table 7, Appendix E); HyperAD (Table 1).

Discussion points:

Novelty - 8nXG and vZKq asked what is new. We thank 8nXG for framing the problem as meaningful and T8Nt for calling the framework worthwhile. What is new is the cause of the conflict and the reallocation of the fidelity term: existing methods strengthen the prior and fit a uniformly weighted fidelity term, we keep the prior minimal and make the fidelity term accurate (Section 1).

Entropy - 8nXG and T8Nt asked whether it stands for uncertainty. It does not: the gate prioritises structurally informative regions and the spectral term still supervises every pixel (Section 3.3).

Evidence - T8Nt asked whether the conflict holds beyond the trajectory, and vZKq suggested a landscape and gradient tracking; we thank both and added both (Figure 7, Appendix F), plus the cosine on five further CAVE scenes in the response.

Margin - T8Nt asked how far the gain over a constant weight reaches. On CAVE the same comparison is several times wider, because the HR-MSI residual has a much larger spatial spread than the PAN residual, which a single global weight cannot follow.

Weightings - 8nXG asked for variance, edge and gradient comparisons; Table 3 now reports all of them, and the gate leads every alternative on CAVE and has the best average across the four datasets.

Corruption - 8nXG and vZKq asked about degradations, and we thank vZKq for naming them: four settings in Table 7, where the gate is ahead of a constant weight in every setting, by up to 7.85 dB.

Efficiency - vZKq asked about the bottleneck while noting the 16 to 17 seconds as a clear advantage. The cost is stage II, 76% of the total at a 195 MiB peak, and a concurrency test shows it is computation and not scheduling; the breakdown is in the response.

Data - vZKq suggested a newer dataset and we thank them for pointing to PromptHSI: HyperAD is now in Table 1, where we are best on all five metrics. T8Nt asked for the entropy settings, given in the response.

We thank all reviewers for their time.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.