- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes a fast zero-shot framework for hyperspectral pansharpening and hyperspectral–multispectral image fusion. The method is based on a relatively straightforward idea: the spatial and spectral objectives are decoupled to reduce optimization conflict, while local Shannon entropy is used to reweight the spatial loss so that structurally informative regions receive greater emphasis. A two-stage optimization strategy is further introduced, with spectral warm-up followed by spatial refinement, together with a lightweight CNN and subspace projection to improve efficiency. The proposed method achieves strong quantitative results on several standard HyPan and HMIF benchmarks and is considerably faster than many existing zero-shot approaches.
While the method is effective, I find the overall contribution somewhat incremental. The main ingredients—loss decoupling, staged optimization, entropy-based weighting, and lightweight zero-shot reconstruction—are conceptually intuitive and closely related to ideas that have already been explored in related literature. As a result, the paper reads more like an effective adaptation and combination of mature techniques for a specific remote sensing fusion problem than a substantially new methodological contribution.
My main concern is therefore not whether the method works, but whether the paper provides sufficiently deep and generalizable insight beyond the reported performance improvements. The current analysis is still relatively limited, particularly regarding why entropy is the appropriate proxy for spatial reliability, how robust the proposed strategy is under different observation conditions, and whether the same optimization principle consistently benefits other recent fusion frameworks. In addition, the evaluation is largely based on classical benchmark datasets, which makes it harder to assess the practical relevance and generality of the conclusions.
Overall, the paper presents a reasonable and efficient solution, but I believe substantially stronger analysis and broader validation would be necessary to make the contribution sufficiently compelling for ICLR.
Critical strengths
-
Clear and intuitive motivation. The paper identifies a meaningful optimization issue in zero-shot hyperspectral fusion: the spatial and spectral fidelity objectives may provide conflicting gradients, particularly because spatial information is concentrated around edges and fine structures while spectral supervision is dominated by lower-frequency content. The proposed entropy-guided weighting provides a simple way to alleviate this imbalance.
-
Strong efficiency–performance trade-off. The proposed method achieves competitive or state-of-the-art results on the evaluated HyPan and HMIF benchmarks while remaining substantially faster than many recent zero-shot and diffusion-based approaches. On the HyPan benchmarks, the runtime is around 16–17 seconds per image.
-
Simple and practical zero-shot formulation. The framework does not require external training data or pretrained generative models. The combination of subspace projection, a lightweight CNN, and two-stage optimization makes the method relatively easy to implement and computationally manageable.
-
Reasonable empirical analysis of the optimization behavior. The paper goes beyond reporting final benchmark scores by examining gradient cosine similarity, convergence behavior, spectral-band reconstruction quality, and spatial-frequency characteristics, which provides some evidence for the intended optimization mechanism.
-
Good practical effectiveness despite the simplicity of the design. Although the individual components are conceptually straightforward, their combination produces consistent gains across multiple standard datasets, suggesting that the proposed optimization strategy is practically useful for this task.
Critical weaknesses
-
The methodological novelty is limited, while the current analysis is not sufficiently deep.
As the authors themselves note, gradient conflict is a well-known phenomenon in areas such as multi-task learning, and systematically modulating conflicting objectives can substantially benefit model optimization. Applying this idea to hyperspectral image fusion in a coherent manner is certainly a strength of the paper. However, I find the accompanying analysis rather limited. The paper mainly provides quantitative comparisons, efficiency measurements, reconstruction visualizations, gradient cosine similarity, and frequency-domain analyses.I believe the authors could employ richer analytical tools to better characterize both the original optimization issue and the behavior after applying the proposed strategy. For example, visualizing the optimization or loss landscape, or more systematically tracking the dynamics of the individual gradients throughout training, could provide a clearer understanding of how and why the proposed decoupling strategy works. For a paper centered around a relatively focused methodological contribution rather than a broader systematic study, I consider the depth of analysis particularly important. Overall, the current level of analysis is not yet sufficient to meet the acceptance standard I apply to ICLR 2026 submissions.
-
The evaluation settings are relatively limited.
I would encourage the authors to evaluate the proposed method under more realistic and controlled degradations, such as mild sensor noise, stripe noise, structured artifacts, or asymmetric corruptions. The current evaluation mainly follows standard synthetic degradation settings, including Gaussian blur and downsampling. It would be important to understand whether the proposed entropy-guided optimization strategy remains effective when the spatial observation itself contains non-negligible corruption, particularly because local entropy may also respond strongly to noise or structured artifacts. -
The efficiency claim deserves a more systematic analysis.
Achieving approximately 17 seconds per image is a clear practical advantage of the proposed method. 23645_Entropy_Guided_Heterosce However, the paper does not sufficiently explain why the competing zero-shot methods are substantially slower, nor does it identify the dominant computational bottleneck of the proposed framework itself. If “fast zero-shot fusion” is intended to be one of the main contributions, I believe a more detailed breakdown of runtime, memory consumption, optimization stages, and computational bottlenecks would significantly strengthen this claim. -
The selected datasets are classical but relatively dated.
Although the current benchmarks are standard and widely used, they are also relatively old and may not sufficiently reflect the diversity of more recent hyperspectral data. I recommend adding at least one newer and more heterogeneous dataset to strengthen the evidence for generalization. For example, a dataset containing a larger number of scenes with diverse spatial content would provide a more convincing evaluation of whether the proposed optimization strategy remains reliable across different scene characteristics.
The authors may also consider evaluating the proposed method on HyperAD, introduced in PromptHSI: Universal Hyperspectral Image Restoration With Vision–Language Modulated Frequency Adaptation (IEEE TGRS, 2026). Compared with the current benchmarks, HyperAD contains a larger number of hyperspectral images and covers more heterogeneous scenes with diverse content. I believe evaluation on such a dataset could provide stronger evidence regarding the robustness and generalizability of the proposed optimization strategy across different scene characteristics.
C.-H. Cheng, C.-M. Lee, W.-T. Liao, Y.-F. Lin, Y.-C. Cheng, F.-E. Yang, Y.-C. F. Wang, and C.-C. Hsu, “PromptHSI: Universal Hyperspectral Image Restoration With Vision–Language Modulated Frequency Adaptation,” IEEE Transactions on Geoscience and Remote Sensing, vol. 64, pp. 1–16, 2026.
Questions for authors
Please provide a strong and detailed rebuttal addressing the weaknesses raised above.
4 comments
W2. Evaluation settings are relatively limited; suggest realistic/controlled degradations (mild sensor noise, stripe noise, structured artifacts, asymmetric corruptions), since local entropy may respond strongly to noise/artifacts.
Both halves of the question are answered. We added the four settings on three CAVE scenes, each one shared corrupted copy that every method reads: Gaussian noise on both observations ( and ), an MSI stripe, and an asymmetric corruption that removes 5 rows and 2% of the blocks from the MSI only. To isolate the gate, the same network, initialization, schedule and budget are run with the weight map replaced by a constant:
| clean | noise 0.01 | noise 0.02 | stripe | missing | |
|---|---|---|---|---|---|
| Const | 42.624 | 41.056 | 34.462 | 35.078 | 21.983 |
| Proposed | 44.149 | 42.001 | 36.329 | 36.899 | 29.833 |
| gain of the gate | +1.53 | +0.95 | +1.87 | +1.82 | +7.85 |
The gate stays above the constant in all five settings, so it remains effective when the observation is corrupted. Additive noise is the harder case: it raises the local entropy everywhere and compresses the gate toward a uniform map, which is where our margin is smallest (0.95 dB at ). In the other direction, corruption that destroys structure (the stripe and the removed rows and blocks) is locally flat and is therefore down-weighted rather than fitted, which is where the margin is largest (7.85 dB). Comparing with the five baselines on the same corrupted inputs, we are first in three of the five settings, level with the best baseline on the clean input, and second on the block-removal case.
W3. Efficiency claim deserves more systematic analysis: why competing zero-shot methods are slower, and what is the dominant computational bottleneck of the proposed framework. Suggest runtime breakdown, memory consumption, optimization stages, bottlenecks.
Why they are slower. The cost is the prior. Ours is a 0.16 M network, 76% of it in stage II, 195 MiB peak. DM-ZS runs thousands of reverse steps, each a full-resolution pass of a multi-band denoiser plus a data-consistency gradient; CLoRF evaluates two implicit networks at every pixel of the full-resolution grid, and its per-iteration cost therefore scales with the number of pixels, over iterations; CS2DIPs couples two DIPs with a tensor factorization for -- iterations; ZSL trains on patches of downsampled images and needs many steps to converge. In all four the cost is the prior.
Where our cost goes. We instrumented the HMIF scene:
| preprocess | phase I | phase II | per iteration | peak memory (phase II) | parameters |
|---|---|---|---|---|---|
| 0.17 s | 5.4 s | 17.6 s | 5.9 ms | 195 MiB | 0.16 M |
Phase II dominates at 76% of the total, and each of its iterations is one forward--backward pass of a 0.16 M network on 12 bands, so the cost is linear in the iteration count and does not grow with ; the subspace projection and the loss evaluation add little. The entropy map costs under 0.2 s, but built with a one-hot histogram it sets the transient memory peak (1.9 GB); chunking that computation removes the peak.
We also tested whether the cost is launch overhead rather than computation: running two and three scenes concurrently takes 2× and 3× the wall time with no throughput gain, and a single run already keeps the GPU at about 90%, so the cost is the phase-II computation itself, not scheduling or memory.
W4. Selected datasets are classical but relatively dated; suggest adding at least one newer, more heterogeneous dataset with more scenes and diverse spatial content. Suggest evaluating on HyperAD (from PromptHSI, IEEE TGRS 2026), which contains more hyperspectral images and more heterogeneous scenes.
Thank you for pointing us to HyperAD. It is now in Section 4.1 and in the bottom part of Table 1: 20 of its 100 test images, chosen uniformly in name order, at , generated with the same downsampling protocol as the other HyPan datasets, and compared with one representative of each family.
| Method | PSNR | SSIM | SAM | ERGAS | RMSE | Time (s) |
|---|---|---|---|---|---|---|
| ZSL | 41.370 4.999 | 0.941 0.046 | 4.041 2.982 | 3.063 1.745 | 0.0148 0.008 | 680.32 |
| PLRDiff | 30.480 4.847 | 0.698 0.169 | 14.262 8.835 | 16.303 12.429 | 0.0701 0.050 | 87.05 |
| DM-ZS | 40.466 3.746 | 0.924 0.049 | 4.180 2.269 | 3.061 1.320 | 0.0147 0.006 | 314.45 |
| CLoRF | 31.534 3.105 | 0.676 0.127 | 10.322 5.178 | 12.126 7.299 | 0.0344 0.010 | 245.51 |
| Proposed | 42.715 3.963 | 0.953 0.035 | 3.155 1.628 | 2.353 1.093 | 0.0117 0.005 | 17.26 |
On this dataset the method is the best on all five metrics, faster than the most accurate competitor. We did not run all nine methods of Table 1 on HyperAD: the four reported are one representative each of the pseudo-paired, diffusion-prior, diffusion-guidance and implicit-representation families. Because the diffusion and implicit-representation baselines take a long time per image, this response evaluates 20 of the 100 test images; the final version will evaluate all 100.
(Question in Paper Summary) Whether the same optimization principle consistently benefits other recent fusion frameworks?
Transfer to other frameworks. We tried it. The gate is only a change to the loss, so we inserted it into PLRDiff and into CLoRF and left everything else untouched; on both, the PSNR moves marginally. The reason is not that the gate is useless there, but that those two methods already carry a prior strong enough to fit a plain loss, so a better weighted loss has little left to improve and its advantage is masked instead of absent. It pays off only together with the rest of our design, a minimal network and the identity warm-up, which is why we present the three as one choice.
We hope these clarifications address the reviewer's concerns, and we are happy to provide any further detail or run additional experiments during the discussion.
Thank for your response, I am satisfied about my concerns. I will increase my score.
Good lucks!
We thank the reviewer for the constructive report and for suggesting HyperAD, which we found to be a useful benchmark.
We do not assemble existing components unchanged. The contribution is this: having observed a gradient conflict between the two fidelity terms of zero-shot fusion, we relieve it with an entropy weight read off the observation, and we design the model and the schedule that this relief makes possible. The two analyses below are the evidence for it.
1. The gradient conflict, and the cause we identify for it. In the later stage of the optimization the gradients of the spatial and spectral terms turn against each other and the result settles at a compromise. Our gradient analysis locates the cause in the spatial constraint itself: it is informative only in a sparse set of regions. It still acts on smooth regions, but there it adds nothing to the restoration; and early in the fit, which is mostly low-frequency, it has little to do either.
2. An entropy gate that pays the spatial budget where it is informative, and a schedule that follows from the same analysis. The gate is read off the observation, so it suppresses the constraint where it is redundant and keeps it where it drives detail; and because the term carries little early, the first stage lowers its weight and, in our design, removes it entirely, so a spectral-only loss fitted with SGD anchors the subspace quickly.
3. A model as light as the objective allows. Once the objective is accurate, less has to be supplied by the network's own capacity, so a four-layer network suffices.
4. Different from existing zero-shot methods, we do not strengthen the prior to fit the loss; we make the loss accurate and keep the prior minimal. They spend on a stronger prior (CS2DIPs couples two U-Nets with a tensor factorization for 5000 iterations, CLoRF evaluates two SIREN networks at every pixel for 30000 epochs, DM-ZS runs thousands of diffusion steps) and then fit a uniformly weighted spatial term with it. Removing that expense gives 0.16 M parameters, the best PSNR on every benchmark, about 17 seconds per image, and 5 to 130 times faster.
Loss landscape (Fig. 7a). Two runs leave the same warmed-up state and differ only in the weight map. On the unweighted criterion they separate within a few hundred iterations and end in different minima, L=0.0025 with the gate against 0.0029 with a constant weight: the gate changes which solution is reached, not only how fast, so the conflict leaves a lasting effect rather than a transient one.
Gradient dynamics (Fig. 7b,c). At three iterations of stage II the two gradients occupy different bands. Their median radial frequency is 3 for the spectral gradient and 39 for the spatial one, and below 32, the Nyquist band of the 4× downsampled observation, 97% of the spectral gradient energy lies against 45% of the spatial one. About half of the spatial budget is therefore spent where the spectral term already works, which is what the gate reallocates. This is also why the conflict appears late: once the low-frequency band has been fixed by the warm-up, the spectral gradient keeps pulling it back while the spatial gradient keeps pushing everywhere.
Both analyses are Figure 7 and Appendix F of the revision.