acceptodds
Under review as a conference paper at ICLR 2027

Response-Pair Spherical Attention for Hyperspectral and Multispectral Image Fusion

Abstract

Hyperspectral and multispectral image fusion (HMF) combines spatially coarse hyperspectral observations with fine-scale multispectral guidance to reconstruct a high-resolution hyperspectral image. In attention-based fusion, reconstruction quality depends on how neighboring responses are weighted relative to each target. Yet conventional dot-product attention couples candidate and target response amplitudes solely through their product. Consequently, response pairs with the same amplitude product and directional agreement receive identical affinity scores even when their relative amplitude allocations differ. We introduce response-pair spherical attention (RPSA), which decomposes each learned candidate–target response pair into overall effective amplitude, relative amplitude allocation, and orientation difference. Its affinity preserves sensitivity to joint response strength while using amplitude balance and directional agreement to regulate candidate contributions. We embed RPSA in RPSA-Net, an alternating cross-resolution architecture that alternates high-resolution feature refinement under multispectral guidance with feedback that updates a low-resolution hyperspectral state. Experiments on three simulated benchmarks and a real-world dataset demonstrate that RPSA-Net achieves state-of-the-art spectral fidelity and spatial reconstruction while maintaining fast inference.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Hyperspectral and multispectral image fusion, attention mechanisms, pairwise affinity modeling

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_UvnJOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper tackles hyperspectral and multispectral image fusion by addressing magnitude bias in conventional dot-product attention, where excessively large candidate amplitudes cause spectral distortion. The authors introduce Response-Pair Spherical Attention to decouple pairwise interactions into effective amplitude, directional alignment, and an amplitude balance factor based on harmonic means, allowing target responses to constrain candidate weights. Built into an alternating cross-resolution network, the method achieves improved reconstruction fidelity and low inference latency on standard benchmark datasets.

Critical strengths

The paper provides a compelling and insightful perspective by identifying the magnitude bias in conventional dot-product attention during hyperspectral fusion. Formulating this issue through response-pair spherical geometry and introducing a harmonic-mean balance factor is conceptually fresh, well-motivated, and mathematically sound, offering an intuitive mechanism to prevent spectral distortion.

The experimental evaluation demonstrates consistently superior performance across benchmark datasets under multiple scaling factors, accompanied by low inference latency. Furthermore, the component-wise and parameter ablation studies are well-conducted and reasonably justify the design choices of the proposed modules.

Critical weaknesses

In the ablation study on Chikusei ×4\times4, the baseline model without any attention mechanism (Local fusion) already achieves 51.80 dB in PSNR, which is remarkably competitive and closely trails most established SOTA methods in Table 1. Given the mathematical and architectural complexity introduced by RPSA, the actual performance gain yielded by the proposed attention mechanism is marginal (only around 0.2–0.3 dB). The authors need to thoroughly clarify whether the performance advantage primarily originates from the macro-architecture rather than RPSA itself, as the baseline alone is already near-SOTA.

The Harvard dataset is widely recognized as an indispensable benchmark in the hyperspectral fusion community, yet it is noticeably missing from the experimental evaluation. Furthermore, the core ablation studies and parameter analyses are conducted solely on the Chikusei dataset, which is insufficient to demonstrate whether the claimed mechanism and hyperparameter choices generalize robustly across diverse scene types (e.g., indoor vs. urban remote sensing).

Despite centering the paper around spherical geometry and amplitude balance, the authors provide no direct visualization of the learned spherical attention maps or response-pair feature distributions to intuitively demonstrate how RPSA mitigates magnitude bias compared to standard dot-product attention.

Questions for authors

see the Critical Weaknesses

Reviewer_ApqUOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies fusion of hyperspectral and multispectral images. It uses a low-resolution hyperspectral image and a high-resolution multispectral image to reconstruct a high-resolution hyperspectral image. RPSA-Net uses response strength, amplitude balance, and direction to calculate attention weights. The response pairs are described by a radius and spherical coordinates. The authors report better reconstruction metrics on simulated datasets and show visual results on real data.

Critical strengths
  1. The authors analyze the attention weights and explain how they implement the formulas.
  2. The method is tested on several datasets. The paper also includes ablations.
Critical weaknesses
  1. The paper is poorly written and does not explain the main idea clearly. The abstract is one example. It introduces response pairs, amplitude allocation, and spherical geometry, but I can only tell that the paper proposes an attention method for image fusion. It is not clear what is spherical about the method, why it is needed, or what the main contribution is.
  2. The visual improvement is not obvious. In Figures 4 and 5, the results look similar to the baselines, and the real-data examples are small. The output is a hyperspectral image, but most results are shown as RGB images. Figure 6 shows band-wise errors. I would also like to see individual bands and spectral curves compared with the main baselines.
  3. The experiments do not show the advantage of this attention clearly.
Questions for authors
  1. What is the main contribution of this work? Why is the spherical representation useful? How is it different from existing attention methods that consider response amplitudes or norms?
  2. Could the authors show clearer image results and spectral comparisons?
Reviewer_5hmfOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Response-Pair Spherical Attention (RPSA) for hyperspectral and multispectral image fusion. It jointly models response magnitude, amplitude balance, and directional agreement to improve feature aggregation. RPSA-Net integrates this mechanism into an alternating cross-resolution fusion architecture. Experiments on three simulated datasets and a real-world dataset demonstrate competitive reconstruction performance and fast inference.

Critical strengths
  1. The paper identifies a meaningful limitation of dot-product attention and introduces relative amplitude allocation to regulate neighboring feature contributions.
  2. RPSA provides a mathematically interpretable attention formulation, supported by theoretical analyses of candidate competition and affinity saturation.
Critical weaknesses
  1. The novelty needs stronger justification. The proposed affinity can be expressed as a harmonic-mean-based amplitude term combined with angular similarity. The spherical formulation appears to be an alternative parameterization, and its advantages over simpler magnitude-aware attention mechanisms remain unclear.
  2. The core motivation lacks direct experimental validation. Figure 1 highlights incorrect candidate preferences under dot-product attention, but the experiments do not directly demonstrate whether RPSA corrects these preferences or improves attention consistency with ground-truth spectral correspondence.
Questions for authors
  1. How does RPSA compare with simpler magnitude-aware similarity kernels or existing norm-modulated attention mechanisms under matched settings?
  2. The theoretical message sensitivity analysis assumes fixed value vectors. How do these conclusions extend to practical networks where feature values also change with candidate responses?
  3. The dot-product baseline uses Softmax, whereas RPSA employs direct affinity normalization. Could the authors isolate the effects of the similarity function and normalization strategy?
  4. Why is a two-dimensional response projection adopted? How sensitive is the reconstruction performance to the projection dimension and effective-amplitude attenuation parameter?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.