- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes ROTSA for unregistered HSI super-resolution. It learns neighborhood weights and a detail state from reference self-reconstruction, warps them onto the HSI grid via structural correspondence, and applies them recurrently; it also reweights the reference correction per spectral mode using a basis estimated from the LR-HSI. Results on simulated and real datasets beat prior work at all three upsampling scales, and the ablations are fairly complete.
Critical strengths
-
The method improves over prior work on both simulated and real datasets at all three scales, with clear margins and a much smaller model. That kind of across-the-board improvement is hard to dismiss.
-
The authors replace operators, vary alignment conditions, and ablate components one at a time under a fixed protocol. This makes the reported gains credible and easy to inspect. I appreciate.
Critical weaknesses
W1: I'm worried that the "operator transport" story does not match what the method actually does. The weights and state are both predicted from the reference image itself, so what is transported is really aligned reference content, not a reusable operator.
Furthermore, i struggle to see the real difference from CARAFE or DKN, and no evidence shows these operators transfer across contents or modalities.
W2: I also find it hard to tell which part does the work. The detail state Z is modulated by a gate that depends on the HSI features, so "operator transport works" and "the gate decides what to inject" are tangled together, with no w/o-gate ablation to separate them. Also, I do not see why local propagation learned on RGB should be a good prior for HSI features in the first place?
W3: I am not convinced by the motivation for the spectral adaptation. The oracle analysis only shows the HSI subspace varies per scene, which does not imply the reference correction should be damped per mode. The method also discards any correction outside the LR basis, which might be where the reference has the most to add, and this trade-off is never discussed.
Questions for authors
Is Figure 5 an AI drawing? And answer all questions carefully, scores are not fixed.
5 comments
W2. Gate attribution and RGB learning
Contribution of the detail gate. We add an ablation that disables the target-conditioned detail gate while retaining the complete operator and injection. This variant achieves 37.47 dB, exceeding the direct-fusion baseline (w/o RSOT) by 0.85 dB. Enabling the gate further improves PSNR by 0.28 dB to 37.75 dB. These results show that the transported reconstruction mechanism remains useful without input-dependent detail gating, and gating can provide an additional benefit.
| Setting | PSNR ↑ | SSIM ↑ | SAM ↓ |
|---|---|---|---|
| w/o RSOT (direct feature fusion) | 36.62 | 0.965 | 0.046 |
| w/o Detail Gate | 37.47 | 0.970 | 0.043 |
| ROTSA (Ours) | 37.75 | 0.973 | 0.042 |
RGB-to-HSI transfer. Local edges and geometric arrangements are often shared across RGB and HSI measurements even though their channel responses differ. For fixed and channel-shared neighborhood weights , local difference propagation is independent of channel semantics and commutes with a pointwise linear channel map :
This property (Appendix A.3) supports transferring the spatial propagation mechanism, while structural correspondence locates its application and the HSI-conditioned projections and gate adapt it to the target representation. The additive state can still encode reference detail, so the HSI branch must learn when to use it. Previous studies have also found RGB spatial SR learning useful for HSI reconstruction (Li et al., WACV 2022; Su et al., AAAI 2025). These considerations motivate the cross-modal transfer without assuming that all RGB textures are valid HSI details.
Li K, Dai D, Van Gool L. Hyperspectral image super-resolution with RGB image super-resolution as an auxiliary task[C]//2022 IEEE/CVF Winter Conference on Applications of Computer Vision. IEEE, 2022: 4039-4048. Su X, Shen X, Wan M, et al. EigenSR: Eigenimage-bridged pre-trained RGB learners for single hyperspectral image super-resolution[C]//Proceedings of the AAAI conference on artificial intelligence. 2025, 39(7): 7033-7041.
W3. Spectral modulation and complete-basis correction
Motivation for mode-wise correction of spectral adaptation. Fig. 2 illustrates scene-dependent spectral characteristics and LR–HR factor compatibility, motivating adaptive processing rather than directly determining correction weights. The usefulness of a reference correction depends not only on its spectral energy but also on its agreement with the missing target signal. As derived in Appendix C.5 (Eqs. 41–42), for the target residual and proposed correction , the oracle weight minimizing mode-wise squared error is
This analysis explains why uniformly accepting reference corrections may be suboptimal: a correction can have substantial energy without improving reconstruction. ASC addresses this issue by learning mode-dependent weights from observable LR-HSI scene statistics and predicted residual/correction energies.
Complete basis and continuous adaptation. ASC constructs a complete orthonormal basis from the LR-HSI and reconstructs the output as
Unlike fixed-rank truncation, this formulation represents corrections across all spectral directions while retaining the HSI-only estimate as the reconstruction base. The spectral basis is constructed from the combined global–local covariance of the LR-HSI, with its eigenvectors ordered by descending eigenvalues. The learned weights generally concentrate on leading energy-ordered modes and decrease toward higher-order modes, with scene-dependent variations particularly evident in intermediate modes. Rather than imposing a fixed strategy, ASC continuously adjusts the contribution of each mode according to scene and correction statistics.
Importantly, complete-basis representation does not imply unrestricted acceptance of every correction. Small weights can substantially attenuate reference information in weakly supported modes. This reflects a reliability–detail trade-off: suppressing potentially incompatible corrections helps preserve spectral fidelity but may also limit useful reference detail. Our ASC provides one practical approach to balancing these effects, while more flexible correction-allocation strategies remain an interesting direction for our future research.
Experiments. We compare three spectral correction strategies on REAL ×8 (Table 5). Fixed rank = 3 uses binary weights ( for , otherwise ), accepting reference corrections only in the first three modes. Full capacity sets for all modes, accepting the complete guided correction without modulation. In contrast, ROTSA learns continuous and input-dependent weights across the complete basis, which is better than other compared strategies. Appendix Table 10 in the revised manuscript further shows that ASC outperforms both global scalar (37.55 dB) and prior-only (37.60 dB), supporting the benefit of learned mode-specific correction allocation.
| Variant | PSNR ↑ | SSIM ↑ | SAM ↓ |
|---|---|---|---|
| Fixed rank = 3 | 37.56 | 0.972 | 0.043 |
| Full capacity | 37.45 | 0.971 | 0.044 |
| ROTSA (Ours) | 37.75 | 0.973 | 0.042 |
Q. Figure 5 provenance
Figure construction. Fig. 5 is a Matplotlib visualization constructed from our measured ablation results, not drawn directly by AI. AI assisted with drafting the plotting code, while we reviewed the implementation, verified the data, and manually refined the final visualization. The plotting script can be provided for reproducibility.
We thank the reviewer again for the insightful questions. We hope our explanations and additional evidence clarify the proposed approach. We will strengthen the methodological discussion into the revised manuscript and would be glad to provide further details if needed.
Reviewer WF1A
We appreciate the reviewer for recognizing our improvements across datasets and scales, as well as the careful assessment of our ablations. We address these points below.
W1. Operator Learning and Transfer
Definition of operator learning and transport. The learned fields (K,Z) are conditioned on each RGB reference, which are not fixed and content-independent parameters. In the reference self-reconstruction branch, they parameterize the spatial update
\mathcal K_K(U)(p)=\sum_{q\in\mathcal N_8(p)}\kappa_{p,q}[U(q)-U(p)].$$ For fixed $(K,Z)$, this is affine in $U$: $K$ specifies a channel-shared rule for propagating differences in the input signal, while $Z$ provides an additive detail state that can produce variation even in a locally flat input. The state $Z$ itself carries reference-dependent detail information. **What is transported?** Structural correspondence maps these parameter fields from the RGB grid to the HSI grid. The neighborhood weights are then applied to HSI features, not used to copy RGB features directly. The aligned detail state is projected into HSI feature space and modulated by an HSI-conditioned gate. Thus, the RGB self-reconstruction operation is affine for fixed fields, while its target-side realization is nonlinear and conditioned on HSI content. The claim is the transport of a reference-derived reconstruction rule and detail state, not the transfer of one fixed and general operator across all images. **Relation to CARAFE and DKN.** CARAFE reassembles nearby features using predicted kernels, and DKN further learns deformable sampling. Such propagation is effective where informative variation already exists. Our ROTSA explicitly couples local difference propagation with an additive state to address missing detail beyond local reweighting, and transfers the resulting fields across misregistered modalities. The methodological distinction lies in the complete reconstruction-and-transfer formulation, beyond the use of an adaptive local kernel alone. **Component and replacement experiments (REAL ×8).** Removing $Z$ or $K$ reduces PSNR by 1.28 dB or 0.14 dB, respectively. Under matched replacement controls, the complete ROTSA design outperforms CARAFE-only and DKN-only variants. | Operator configuration | PSNR ↑ | SSIM ↑ | SAM ↓ | | :-------------------------------------- | --------: | --------: | --------: | | Neighborhood propagation only (w/o $Z$) | 36.47 | 0.965 | 0.045 | | Additive detail state only (w/o $K$) | 37.61 | 0.972 | **0.042** | | CARAFE only | 33.48 | 0.939 | 0.049 | | DKN only | 36.50 | 0.965 | 0.046 | | **ROTSA** | **37.75** | **0.973** | **0.042** | **Reuse across content and modalities.** To assess transfer of the learned operator rule, we train the RGB predictor through self-reconstruction on REAL, freeze its parameters, and train the HSI branch on REAL or ICVL. Each new reference still generates its own $(K,Z)$ fields, where only the predictor is shared. The frozen variant remains 0.36/0.33 dB below joint training on REAL/ICVL but retains competitive HSI reconstruction, indicating partial transferability with a remaining benefit from joint adaptation. | Setting | REAL PSNR ↑ | REAL SSIM ↑ | REAL SAM ↓ | ICVL PSNR ↑ | ICVL SSIM ↑ | ICVL SAM ↓ | | :--------------------------- | ----------: | ----------: | ---------: | ----------: | ----------: | ---------: | | UAFL | 37.23 | 0.972 | 0.046 | 35.78 | 0.963 | 0.039 | | Frozen RGB predictor on REAL | 37.39 | 0.970 | 0.043 | 37.59 | 0.975 | **0.037** | | **ROTSA (joint training)** | **37.75** | **0.973** | **0.042** | **37.92** | **0.977** | **0.037** |