acceptodds
Under review as a conference paper at ICLR 2027

Recurrent Operator Transport with Spectral Adaptation for Unregistered Hyperspectral Image Super-Resolution

Abstract

Unregistered hyperspectral image (HSI) super-resolution transfers high-resolution spatial detail across misaligned sensors while preserving spectral fidelity. Existing methods typically transfer detail through cross-modal feature fusion, leaving the spatial reconstruction process implicit in the interaction between reference and HSI features. In this paper, we propose Recurrent Operator Transport with Spectral Adaptation (**ROTSA**), a framework that explicitly learns spatial reconstruction operators from the reference and transfers them to HSI restoration. Specifically, a self-reconstructing reference branch models cross-scale spatial transformations using neighborhood weights and an affine detail state, capturing local propagation relationships and missing high-frequency detail. Structural correspondence then transports these operators to the HSI grid, where they recurrently guide spatial restoration. Besides, to adapt the transferred correction to scene-dependent spectral characteristics, we further introduce adaptive spectral capacity, which represents the correction in a complete spectral basis estimated from the low-resolution HSI and continuously adjusts its contribution across the basis. This preserves the reconstruction supported by the observed HSI while allowing reference guidance to adapt to the spectral content of each scene. Experiments on simulated and real datasets demonstrate the effectiveness of the proposed method.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Unregistered hyperspectral image super-resolution, Spatial operator transport, Adaptive spectral capacity

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_WF1AOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ROTSA for unregistered HSI super-resolution. It learns neighborhood weights and a detail state from reference self-reconstruction, warps them onto the HSI grid via structural correspondence, and applies them recurrently; it also reweights the reference correction per spectral mode using a basis estimated from the LR-HSI. Results on simulated and real datasets beat prior work at all three upsampling scales, and the ablations are fairly complete.

Critical strengths
  1. The method improves over prior work on both simulated and real datasets at all three scales, with clear margins and a much smaller model. That kind of across-the-board improvement is hard to dismiss.

  2. The authors replace operators, vary alignment conditions, and ablate components one at a time under a fixed protocol. This makes the reported gains credible and easy to inspect. I appreciate.

Critical weaknesses

W1: I'm worried that the "operator transport" story does not match what the method actually does. The weights and state are both predicted from the reference image itself, so what is transported is really aligned reference content, not a reusable operator.

Furthermore, i struggle to see the real difference from CARAFE or DKN, and no evidence shows these operators transfer across contents or modalities.

W2: I also find it hard to tell which part does the work. The detail state Z is modulated by a gate that depends on the HSI features, so "operator transport works" and "the gate decides what to inject" are tangled together, with no w/o-gate ablation to separate them. Also, I do not see why local propagation learned on RGB should be a good prior for HSI features in the first place?

W3: I am not convinced by the motivation for the spectral adaptation. The oracle analysis only shows the HSI subspace varies per scene, which does not imply the reference correction should be damped per mode. The method also discards any correction outside the LR basis, which might be where the reference has the most to add, and this trade-off is never discussed.

Questions for authors

Is Figure 5 an AI drawing? And answer all questions carefully, scores are not fixed.

5 comments
AuthorsOpenReview ↗

Reviewer WF1A

We appreciate the reviewer for recognizing our improvements across datasets and scales, as well as the careful assessment of our ablations. We address these points below.

W1. Operator Learning and Transfer

Definition of operator learning and transport. The learned fields (K,Z)(K,Z) are conditioned on each RGB reference, which are not fixed and content-independent parameters. In the reference self-reconstruction branch, they parameterize the spatial update

\mathcal K_K(U)(p)=\sum_{q\in\mathcal N_8(p)}\kappa_{p,q}[U(q)-U(p)].$$ For fixed $(K,Z)$, this is affine in $U$: $K$ specifies a channel-shared rule for propagating differences in the input signal, while $Z$ provides an additive detail state that can produce variation even in a locally flat input. The state $Z$ itself carries reference-dependent detail information. **What is transported?** Structural correspondence maps these parameter fields from the RGB grid to the HSI grid. The neighborhood weights are then applied to HSI features, not used to copy RGB features directly. The aligned detail state is projected into HSI feature space and modulated by an HSI-conditioned gate. Thus, the RGB self-reconstruction operation is affine for fixed fields, while its target-side realization is nonlinear and conditioned on HSI content. The claim is the transport of a reference-derived reconstruction rule and detail state, not the transfer of one fixed and general operator across all images. **Relation to CARAFE and DKN.** CARAFE reassembles nearby features using predicted kernels, and DKN further learns deformable sampling. Such propagation is effective where informative variation already exists. Our ROTSA explicitly couples local difference propagation with an additive state to address missing detail beyond local reweighting, and transfers the resulting fields across misregistered modalities. The methodological distinction lies in the complete reconstruction-and-transfer formulation, beyond the use of an adaptive local kernel alone. **Component and replacement experiments (REAL ×8).** Removing $Z$ or $K$ reduces PSNR by 1.28 dB or 0.14 dB, respectively. Under matched replacement controls, the complete ROTSA design outperforms CARAFE-only and DKN-only variants. | Operator configuration | PSNR ↑ | SSIM ↑ | SAM ↓ | | :-------------------------------------- | --------: | --------: | --------: | | Neighborhood propagation only (w/o $Z$) | 36.47 | 0.965 | 0.045 | | Additive detail state only (w/o $K$) | 37.61 | 0.972 | **0.042** | | CARAFE only | 33.48 | 0.939 | 0.049 | | DKN only | 36.50 | 0.965 | 0.046 | | **ROTSA** | **37.75** | **0.973** | **0.042** | **Reuse across content and modalities.** To assess transfer of the learned operator rule, we train the RGB predictor through self-reconstruction on REAL, freeze its parameters, and train the HSI branch on REAL or ICVL. Each new reference still generates its own $(K,Z)$ fields, where only the predictor is shared. The frozen variant remains 0.36/0.33 dB below joint training on REAL/ICVL but retains competitive HSI reconstruction, indicating partial transferability with a remaining benefit from joint adaptation. | Setting | REAL PSNR ↑ | REAL SSIM ↑ | REAL SAM ↓ | ICVL PSNR ↑ | ICVL SSIM ↑ | ICVL SAM ↓ | | :--------------------------- | ----------: | ----------: | ---------: | ----------: | ----------: | ---------: | | UAFL | 37.23 | 0.972 | 0.046 | 35.78 | 0.963 | 0.039 | | Frozen RGB predictor on REAL | 37.39 | 0.970 | 0.043 | 37.59 | 0.975 | **0.037** | | **ROTSA (joint training)** | **37.75** | **0.973** | **0.042** | **37.92** | **0.977** | **0.037** |

W2. Gate attribution and RGB learning

Contribution of the detail gate. We add an ablation that disables the target-conditioned detail gate while retaining the complete (K,Z)(K,Z) operator and injection. This variant achieves 37.47 dB, exceeding the direct-fusion baseline (w/o RSOT) by 0.85 dB. Enabling the gate further improves PSNR by 0.28 dB to 37.75 dB. These results show that the transported reconstruction mechanism remains useful without input-dependent detail gating, and gating can provide an additional benefit.

SettingPSNR ↑SSIM ↑SAM ↓
w/o RSOT (direct feature fusion)36.620.9650.046
w/o Detail Gate37.470.9700.043
ROTSA (Ours)37.750.9730.042

RGB-to-HSI transfer. Local edges and geometric arrangements are often shared across RGB and HSI measurements even though their channel responses differ. For fixed and channel-shared neighborhood weights KK, local difference propagation is independent of channel semantics and commutes with a pointwise linear channel map PP: KK(PU)=PKK(U).\mathcal K_K(PU)=P\mathcal K_K(U).

This property (Appendix A.3) supports transferring the spatial propagation mechanism, while structural correspondence locates its application and the HSI-conditioned projections and gate adapt it to the target representation. The additive state can still encode reference detail, so the HSI branch must learn when to use it. Previous studies have also found RGB spatial SR learning useful for HSI reconstruction (Li et al., WACV 2022; Su et al., AAAI 2025). These considerations motivate the cross-modal transfer without assuming that all RGB textures are valid HSI details.

Li K, Dai D, Van Gool L. Hyperspectral image super-resolution with RGB image super-resolution as an auxiliary task[C]//2022 IEEE/CVF Winter Conference on Applications of Computer Vision. IEEE, 2022: 4039-4048. Su X, Shen X, Wan M, et al. EigenSR: Eigenimage-bridged pre-trained RGB learners for single hyperspectral image super-resolution[C]//Proceedings of the AAAI conference on artificial intelligence. 2025, 39(7): 7033-7041.

W3. Spectral modulation and complete-basis correction

Motivation for mode-wise correction of spectral adaptation. Fig. 2 illustrates scene-dependent spectral characteristics and LR–HR factor compatibility, motivating adaptive processing rather than directly determining correction weights. The usefulness of a reference correction depends not only on its spectral energy but also on its agreement with the missing target signal. As derived in Appendix C.5 (Eqs. 41–42), for the target residual tk=[E⊤(Y−B−RH)]kt_k=[E^\top(Y-B-R_H)]_k and proposed correction dk=[E⊤(RG−RH)]kd_k=[E^\top(R_G-R_H)]_k, the oracle weight minimizing mode-wise squared error is

ak⋆=clip⁡(⟨tk,dk⟩∣∣dk∣∣F2,0,1),dk≠0.a_k^\star=\operatorname{clip}\left(\frac{\langle t_k,d_k\rangle}{||d_k||_F^2},0,1\right),\qquad d_k\neq0.

This analysis explains why uniformly accepting reference corrections may be suboptimal: a correction can have substantial energy without improving reconstruction. ASC addresses this issue by learning mode-dependent weights from observable LR-HSI scene statistics and predicted residual/correction energies.

Complete basis and continuous adaptation. ASC constructs a complete orthonormal basis E∈RC×CE\in\mathbb{R}^{C\times C} from the LR-HSI and reconstructs the output as

Y^=B+RH+Ediag⁡(a)E⊤(RG−RH).\widehat{Y}=B+R_H+E\operatorname{diag}(a)E^\top(R_G-R_H).

Unlike fixed-rank truncation, this formulation represents corrections across all CC spectral directions while retaining the HSI-only estimate B+RHB+R_H as the reconstruction base. The spectral basis is constructed from the combined global–local covariance of the LR-HSI, with its eigenvectors ordered by descending eigenvalues. The learned weights generally concentrate on leading energy-ordered modes and decrease toward higher-order modes, with scene-dependent variations particularly evident in intermediate modes. Rather than imposing a fixed strategy, ASC continuously adjusts the contribution of each mode according to scene and correction statistics.

Importantly, complete-basis representation does not imply unrestricted acceptance of every correction. Small weights can substantially attenuate reference information in weakly supported modes. This reflects a reliability–detail trade-off: suppressing potentially incompatible corrections helps preserve spectral fidelity but may also limit useful reference detail. Our ASC provides one practical approach to balancing these effects, while more flexible correction-allocation strategies remain an interesting direction for our future research.

Experiments. We compare three spectral correction strategies on REAL ×8 (Table 5). Fixed rank = 3 uses binary weights (ak=1a_k=1 for k≤3k\leq3, otherwise 00), accepting reference corrections only in the first three modes. Full capacity sets ak=1a_k=1 for all modes, accepting the complete guided correction without modulation. In contrast, ROTSA learns continuous and input-dependent weights across the complete basis, which is better than other compared strategies. Appendix Table 10 in the revised manuscript further shows that ASC outperforms both global scalar (37.55 dB) and prior-only (37.60 dB), supporting the benefit of learned mode-specific correction allocation.

VariantPSNR ↑SSIM ↑SAM ↓
Fixed rank = 337.560.9720.043
Full capacity37.450.9710.044
ROTSA (Ours)37.750.9730.042

Q. Figure 5 provenance

Figure construction. Fig. 5 is a Matplotlib visualization constructed from our measured ablation results, not drawn directly by AI. AI assisted with drafting the plotting code, while we reviewed the implementation, verified the data, and manually refined the final visualization. The plotting script can be provided for reproducibility.

We thank the reviewer again for the insightful questions. We hope our explanations and additional evidence clarify the proposed approach. We will strengthen the methodological discussion into the revised manuscript and would be glad to provide further details if needed.

Reviewer_sjcYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ROTSA for unregistered HSI super-resolution using an HR RGB reference. RSOT learns spatial reconstruction operators through reference self-reconstruction, aligns them to the HSI grid, and applies them recurrently. ASC regulates the reference correction using continuous weights over a complete spectral basis. Experiments on ICVL and REAL, including misregistration tests and component ablations, demonstrate improved reconstruction accuracy with a compact model.

Critical strengths

1.The method addresses spatial transfer and spectral preservation through complementary components. Structural correspondence aligns the reference-derived operators, while ASC controls their spectral contribution. The separate HSI restoration path also provides a reference-independent reconstruction. 2.The experimental validation is substantial. Beyond comparisons with recent methods, the paper evaluates CARAFE and DKN replacements, alignment under different misregistration conditions, fixed-rank spectral alternatives, and reference pyramid construction. ROTSA improves reconstruction quality with only 1.39M parameters.

Critical weaknesses

1.The contribution of operator learning needs stronger evidence. Removing neighborhood weights or self-reconstruction reduces PSNR by only 0.14 or 0.09 dB, compared with 1.28 dB for the affine detail state. 2.Several ablation gains are small, but variability across training runs is not reported, making their significance difficult to assess. 3.Robustness to mismatched HSI blur, sensor noise, and spectral response changes remains unclear. 4.Practical efficiency is assessed mainly through parameter counts and FLOPs. End-to-end runtime and peak memory are needed to account for correspondence estimation and spectral eigendecomposition.

Questions for authors

1.Are the gains from neighborhood weights, self-reconstruction, and ASC consistent across random seeds and test scenes? Can you report paired scene-level improvements or variability across runs? 2.How does ROTSA perform when the test HSI blur or noise differs from training? Does ASC remain effective when the LR-HSI spectral basis is estimated from noisy observations? 3.What are the end-to-end inference time and peak memory relative to UAFL and SSDAN? Do the reported FLOPs include structural correspondence and spectral basis construction? 4.In Table 4, matched-condition ROTSA reaches 48.78 dB on clean ICVL ×8, while the slight-misregistration-trained model reaches 41.12 dB in the appendix. Were the competing methods also trained under each matched condition? Clarifying this would help distinguish adaptation to the training condition from robustness to unseen misregistration.

4 comments
AuthorsOpenReview ↗

Reviewer sjcY

We thank the reviewer for appreciating the complementary design, comprehensive comparisons, and compact architecture of ROTSA. We respond to each concern below.

W1. Operator learning

Complete operator representation. The proposed operators learn the complete (K,Z)(K,Z) representation: KK neighborhood weights and ZZ additive detail state. The neighborhood term propagates existing local differences, and the additive state supplies detail that cannot be recovered by reweighting a locally constant input. On REAL ×8, disabling these two applications reduces PSNR by 0.14 dB and 1.28 dB, respectively (Table 3). This supports their complementary contributions: propagation acts on existing spatial variation, while the additive state supplies detail that propagation alone cannot generate.

RGB self-reconstruction loss. The self-reconstruction ablation removes only the RGB self-reconstruction loss (λr=0)(\lambda_r=0), not the operator learner or the use of (K,Z)(K,Z). The final HSI reconstruction losses still train the reference-conditioned predictor through the transported updates. Its 0.09 dB drop therefore measures the incremental value of auxiliary RGB supervision. Tables 3 and 13 in the revised manuscript report the same loss-removal control.

Module-level experiments. At the module level, replacing RSOT with direct reference-feature fusion while retaining the HSI backbone, ASC, and training protocol decreases PSNR from 37.75 to 36.62 dB (Fig. 5). This more directly evaluates the contribution of the full spatial learning and operator transport mechanism.

W2/Q1. Training variability

Consistency across training seeds. We repeat the complete model and each ablation over three training seeds on REAL ×8. ROTSA achieves higher PSNR than each ablation in all three runs. The average gains are 0.100 dB over w/o neighborhood weights, 0.110 dB over w/o RGB self-reconstruction loss, and 0.247 dB over w/o ASC. Mean SSIM and SAM also improve, supporting consistent benefits across the tested training seeds.

VariantPSNR ↑ (mean ± SD)SSIM ↑ (mean ± SD)SAM ↓ (mean ± SD)
w/o Neighborhood Weights37.623 ± 0.04160.9710 ± 0.001000.0427 ± 0.00115
w/o RGB Self-reconstruction Loss37.613 ± 0.04510.9713 ± 0.000580.0427 ± 0.00058
w/o ASC37.477 ± 0.03060.9713 ± 0.000580.0437 ± 0.00058
ROTSA37.723 ± 0.03060.9727 ± 0.000580.0417 ± 0.00058

W3/Q2. Degradation robustness

Test protocol. We evaluate the pretrained models without additional fine-tuning under three separately applied test-time perturbations:

  • HSI blur: altered HSI Gaussian blur (8×8 kernel, sigma = 2).
  • HSI noise: additive Gaussian noise on LR-HSI (standard deviation 2/255).
  • RGB response: RGB channel-response mixing (strength 0.02) as a controlled proxy for response mismatch.

Results. ROTSA retains the highest PSNR and lowest SAM under each evaluated perturbation, with the best or tied-best SSIM at the reported precision. These experiments support the effectiveness of the proposed approach under the tested shifts.

Test conditionMethodPSNR ↑SSIM ↑SAM ↓
BlurSSDAN35.530.9610.050
BlurUAFL36.810.9700.047
BlurROTSA (ours)37.070.9710.042
NoiseSSDAN35.020.9570.076
NoiseUAFL35.840.9640.079
NoiseROTSA (ours)35.970.9640.075
Spectral responseSSDAN35.840.9610.049
Spectral responseUAFL37.130.9710.046
Spectral responseROTSA (ours)37.460.9720.043

W4/Q3. Runtime and memory

FLOPs coverage. The FLOPs reported in Table 1 account for the complete inference procedure, including structural correspondence, spectral basis construction, and other components.

End-to-end measurements. We additionally time the complete inference pass, including correspondence and spectral-basis computation, using CUDA synchronization. Memory denotes peak PyTorch-allocated GPU memory. With matched inference settings, the measured results are:

MethodRuntime, mean ± SD (ms/image) ↓Peak allocated GPU memory (MiB) ↓
SSDAN129.50 ± 7.13400.99
UAFL45.58 ± 1.13520.90
ROTSA (ours)35.22 ± 1.94291.50

The complete-pipeline measurements show that ROTSA has lower inference latency and lower peak memory in this comparison.

Q4. Evaluation protocols

Matched-condition protocol. In Table 4 and Appendix Table 9 in the revised manuscript, ROTSA and all comparison methods are trained separately for each matched training/test condition (Clean→Clean and Large→Large). The cross-condition comparisons instead reuse the same Slight-trained model without retraining, evaluated on Clean and Large test inputs.

For example, ROTSA scores 48.78 dB for Clean→Clean and 41.12 dB for Slight→Clean, due to different training protocols. Under the same Slight→Clean evaluation, ROTSA outperforms UAFL and SSC-HSR (41.12 vs. 38.03/35.93 dB). The advantage also holds for Slight→Large (34.74 vs. 33.89/33.34 dB), providing evidence of robustness to the evaluated changes in misregistration.

We appreciate the reviewer again for the valuable suggestions. We hope the additional experiments and clarifications adequately address your concerns. We will include these results in the revised manuscript and would be happy to address any remaining concerns.

Reviewer_sjcYOpenReview ↗

Follow-up on the Authors’ Response

Thank you for the clarifications and additional experiments. The operator-learning explanation, three-seed ablations, robustness tests, and end-to-end efficiency measurements largely address my concerns. The training and evaluation protocols are also clearer. I maintain my rating.

Dear Reviewer sjcY,

Thank you very much for your positive review and constructive feedback, and for taking the time to consider our responses. We sincerely appreciate your continued positive assessment of our work. We remain happy to provide any further clarifications or address additional concerns during the discussion.

Best regards,
Authors

Reviewer_4YWJOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses guided hyperspectral image super-resolution, reconstructing a high-resolution HSI from a low-resolution HSI (LR-HSI) using an unregistered high-resolution RGB reference image (HR-RGB).

Critical strengths

This paper makes two main contributions: 1.Rather than directly fusing reference features with HSI features, the method explicitly learns “spatial reconstruction operators” through RGB reference self-reconstruction. These operators are aligned with the HSI grid and transferred to the hyperspectral branch to guide reconstruction. 2.The method introduces “adaptive spectral capacity”, motivated by scene-dependent spectral variation. It estimates a complete spectral basis from global and local LR-HSI statistics and uses scene statistics and residual energies to predict continuous weights for the reference correction in each mode, rather than applying a fixed rank cutoff. The authors report better PSNR, SSIM, and SAM than all compared methods on ICVL and REAL across different upsampling scales, while using fewer parameters than most baselines.

The paper makes a clear contribution and appears technically sound. The reported evaluation follows an established benchmark protocol and shows substantial improvements over previous methods. The ablation studies examine the principal proposed components and provide evidence supporting their effectiveness.

Critical weaknesses

My main concern is that the connection between the paper’s two contributions is not sufficiently clear, either in terms of their motivation or their integration within the proposed architecture. The following five points require clarification.

Unclear connection between the two contributions. The motivation for combining spatial operator transport with spectral modulation is insufficiently explained. While the ablations support each component individually, they do not clearly establish how the components complement each other and form a coherent methodological contribution.

Insufficient justification for the two-branch architecture. One branch predicts a reconstruction residual using only LR-HSI features, while the other incorporates spatial operators transferred from the HR-RGB reference to predict a guided residual. Their difference is spectrally weighted and added to the HSI-only residual. Although this design preserves an HSI-only estimate, the rationale for using the difference between two residual predictions, rather than directly predicting a reference correction, remains unclear. A physical or signal-processing interpretation is also needed to justify applying spectral weighting specifically to this difference rather than to the entire guided residual.

Ambiguous interpretation of the spectral-capacity analysis. Figure 2 and Appendix D.3 use LR spectral bases paired with independently computed HR coefficient maps to motivate scene-dependent spectral capacity. The rationale for this analysis, rather than a conventional examination of each scene’s singular-value spectrum or cumulative explained energy, is unclear. In particular, the selected rank may reflect scene dimensionality, LR–HR basis compatibility, or both. This distinction and its connection to the proposed spectral adaptation require further discussion.

Insufficient validation of the covariance weighting. The spectral basis combines global covariance and local spectral-difference covariance with a fixed weight of λ_l = 0.5, but the selection of this value is not adequately justified. A sensitivity analysis, including the global-only case λ_l = 0, is needed to establish the contribution of local spectral differences and the robustness of this choice.

Limited evidence of scene-dependent spectral adaptation. Representative spectral weight profiles for individual scenes, together with their mean and variability across scenes, are needed to demonstrate how the weights adapt to scene content and how their behavior differs from fixed-rank selection.

Questions for authors

Please address the five concerns raised in the “Critical Weaknesses” section.

6 comments
AuthorsOpenReview ↗

Reviewer 4YWJ

We sincerely thank the reviewer for recognizing the methodological contributions, technical soundness, and experimental improvements. We address the five concerns below.

W1. Connection between the RSOT and ASC

Thank you for identifying the need to connect the two contributions more explicitly.

Complementary roles. RSOT (Recurrent Spatial Operator Transport) and ASC (Adaptive Spectral Capacity) address two coupled uncertainties in unregistered RGB-guided HSI super-resolution. RSOT learns reference-conditioned spatial reconstruction operator rules and aligns them with the HSI grid, thereby proposing a spatial correction. However, spatial correspondence does not guarantee that the proposed correction is spectrally appropriate: different sensor responses and reference-specific textures can make a well-located correction unreliable in some spectral directions. ASC therefore determines how much of the proposed spatial correction to accept in each HSI spectral mode, using the observed scene and predicted correction statistics.

Experimental analysis. For the interpolated input BB, reference-independent HSI residual RHR_H, and guided residual RGR_G, let ΔR=RG−RH\Delta R=R_G-R_H. We compare the following outputs from one forward pass of the jointly trained model:

YH=B+RH,Yraw=YH+ΔR,YASC=YH+Ediag⁡(a)E⊤ΔR.Y_H=B+R_H,\qquad Y_{\mathrm{raw}}=Y_H+\Delta R,\qquad Y_{\mathrm{ASC}}=Y_H+E\operatorname{diag}(a)E^\top\Delta R. Here EE is the complete spectral basis estimated from the LR-HSI and aa contains the predicted mode weights. Fig. 7 in Appendix B.4 of the revised manuscript compares their reconstructions and errors: the raw correction by RSOT adds useful spatial detail but can also produce excessive responses near displaced structures, whereas ASC moderates these responses. We additionally assess reconstruction improvement per spectral mode relative to YHY_H, showing how the spectral controller changes the effectiveness of the same RSOT-proposed correction. These within-model comparisons complement the independently trained component ablations: removing RSOT and ASC decreases PSNR by 1.13 dB and 0.30 dB, respectively (Fig. 5). These results demonstrate the effect of our RSOT and ASC for spatial learning and spectral modulation.

W2. Two-branch Architecture

Residual difference. Our reference-independent and guided branches share the same output decoder and differ through the transported reference update. Consequently, ΔR=RG−RH\Delta R=R_G-R_H measures the change induced by guidance under a common reconstruction mapping. It vanishes when the transported update vanishes. This decouples the incremental reference contribution instead of treating the full guided prediction as reference-dependent.

Equivalent variational interpretation. For fixed branch predictions and spectral weights ak∈(0,1)a_k\in(0,1), set ηk=(1−ak)/ak\eta_k=(1-a_k)/a_k. The proposed residual fusion is the solution of

\lVert S-R_G\rVert_F^2 +\sum_k\eta_k\lVert [E^\top(S-R_H)]_k\rVert_F^2 \right)$$ which gives $$\widehat S = R_H+W(R_G-R_H) = (I-W)R_H+WR_G, \qquad W=E\operatorname{diag}(a)E^\top.$$ Thus, this formulation regularizes the guided increment around an existing HSI estimate. The complementary gains sum to the identity, preserving components on which the two predictions agree, consistent with complementary filtering (Higgins, 1975). Weighting the entire guided residual instead shrinks it toward zero, potentially weakening reconstruction already supported by the HSI. **Controlled experiments (REAL ×8).** *Entire residual* applies spectral weighting to the guided residual itself. *Direct predictor* removes the separate HSI-only residual prediction and predicts a reference-guided output directly. The comparisons use the same benchmark setting. The proposed formulation gains 0.14 dB and 0.80 dB over these alternatives, supporting the default setting of our framework. | Setting | PSNR ↑ | SSIM ↑ | SAM ↓ | | ----------------- | --------: | --------: | --------: | | Entire residual | 37.61 | 0.972 | 0.043 | | Direct prediction | 36.95 | 0.965 | 0.044 | | Ours | **37.75** | **0.973** | **0.042** | Higgins W T. A comparison of complementary and Kalman filtering[J]. IEEE transactions on aerospace and electronic systems, 1975 (3): 321-325.

W3. Spectral capacity analysis

Additional spectral analysis. Following your suggestion, we add conventional LR/HR cumulative-energy curves and spectral projection-error analyses for 10 REAL scenes at ×8 (Fig. 9 in Appendix D.3 of the revised manuscript). Leading components contain most spectral energy, while HR self-projection and LR-basis projection errors approach numerical zero as the full basis is retained. These analyses characterize spectral energy concentration and distinguish conventional subspace approximation from the additional mismatch introduced by independent LR–HR factor pairing.

Motivation for factor pairing. Spectral capacity denotes the continuous strength of reference corrections across modes for unregistered HSI SR. The original Fig. 2 serves as an oracle analysis. Following the LR-basis/HR-coefficient factorization considered in UAFL, it combines the LR spectral basis ULU_L with coefficients independently computed in the HR basis UYU_Y:

= U_{L,k} U_{Y,k}^{\top} Y, \qquad \widehat{Y}^{\mathrm{proj}}_{k} = U_{L,k} U_{L,k}^{\top} Y.$$ Unlike conventional LR-subspace projection, the paired reconstruction can incur coefficient-coordinate mismatch even when more modes are retained. Its PSNR-optimal component count therefore reflects both retained scene information and LR–HR factor compatibility. It is not the intrinsic spectral rank of the scene without reference. We will clarify this scope in Fig. 2 and Appendix D.3. **Connection to ASC.** This perspective motivates considering both the observed scene and the correction introduced by guidance when allocating spectral contributions. ASC retains a complete basis from LR global–local statistics and learns continuous correction strengths using scene and prediction statistics. Projection and reconstruction use the same basis, making its squared-energy descriptors and weighted correction invariant to consistent basis sign flips. The scene-specific weight profiles (shown in W5) and trained spectral-controller ablations (Tables 5 and 10 in the revised manuscript) provide direct experimental analysis for adaptive correction allocation. ### **W4. Covariance weighting** **Effect of local covariance.** Global covariance $\Sigma_g$ captures overall spectral variation, whereas local spectral-difference covariance $\Sigma_\ell$ captures the joint variation of neighboring spectra. Their combination, $\Sigma_g+\lambda_\ell\Sigma_\ell$, lets the ASC basis reflect both scene-level spectral structure and local transitions relevant to spatial detail. We evaluate $\lambda_\ell\in\{0,0.25,0.5,1\}$, including the requested global-only case. The default $\lambda_\ell=0.5$ achieves the highest PSNR in both datasets, exceeding $\lambda_\ell=0$ by 0.18 dB on REAL and 0.21 dB on ICVL. Results vary by at most 0.10/0.15 dB among the tested nonzero weights on REAL/ICVL, respectively. | $\lambda_\ell$ | REAL PSNR ↑ | REAL SSIM ↑ | REAL SAM ↓ | ICVL PSNR ↑ | ICVL SSIM ↑ | ICVL SAM ↓ | | -------------- | ----------- | ----------- | ---------- | ----------- | ----------- | ---------- | | 0 | 37.57 | 0.972 | 0.043 | 37.71 | 0.976 | 0.040 | | 0.25 | 37.65 | **0.973** | 0.043 | 37.77 | 0.976 | 0.039 | | 0.5 (Ours) | **37.75** | **0.973** | **0.042** | **37.92** | **0.977** | **0.037** | | 1 | 37.69 | 0.972 | **0.042** | 37.86 | **0.977** | 0.038 | ### **W5. Scene adaptation** **Scene-specific weight profiles.** We analyze the predicted weights for 10 REAL ×8 test scenes, reporting three representative profiles and the across-scene mean and standard deviation for every ordered spectral mode. Mode indices refer to positions in each eigenvalue-ordered basis of the scene. The leading modes have comparatively similar weights, whereas intermediate modes show more substantial scene variation: the 5th ordered mode has weight 0.0048 in scene 1 and 0.1418 in scene 3. **Relation to fixed-rank selection.** Unlike a fixed-rank mask, ASC estimates continuous and scene-conditioned weights across the full basis. Table 5 shows 37.75 dB for ASC compared with 37.56 dB for fixed rank 3 and 37.45 dB for uniform full-capacity correction. These performance results support our scene-dependent spectral capacity rather than the fixed selection.
ModeScene 1Scene 2Scene 3Mean (10 scenes)SD (10 scenes)
10.29480.29220.28570.29400.0034
20.19740.19570.21590.19360.0147
30.15600.18190.18120.16270.0105
40.07490.15170.15140.12890.0276
50.00480.04990.14180.08460.0550
60.00270.01910.13360.05740.0494
70.00180.00950.07840.02400.0227
80.00130.00490.02550.01050.0082
99.769e-040.00350.01360.00580.0047
107.496e-040.00260.00810.00400.0035
116.343e-040.00200.00600.00300.0029
125.349e-040.00160.00450.00230.0024
134.629e-040.00120.00350.00190.0020
143.999e-040.00100.00290.00150.0017
153.354e-048.235e-040.00230.00130.0015
162.822e-046.784e-040.00190.00100.0012
172.385e-045.520e-040.00158.606e-040.0010
181.987e-044.420e-040.00127.124e-048.736e-04
191.654e-043.574e-049.127e-045.870e-047.232e-04
201.332e-042.845e-046.948e-044.762e-045.888e-04
211.073e-042.247e-045.225e-043.825e-044.726e-04
228.489e-051.739e-043.929e-043.028e-043.712e-04
236.460e-051.335e-042.859e-042.360e-042.853e-04
244.885e-059.986e-052.031e-041.819e-042.137e-04
254.627e-057.378e-051.430e-041.388e-041.532e-04
264.608e-055.243e-059.775e-051.039e-041.026e-04
274.596e-054.601e-056.230e-057.906e-056.283e-05
284.589e-054.594e-054.602e-056.146e-053.192e-05
294.580e-054.584e-054.582e-054.971e-059.242e-06
304.577e-054.580e-054.580e-054.573e-055.990e-08
314.570e-054.573e-054.576e-054.568e-054.970e-08

We thank the reviewer again for the constructive feedback. We hope our explanations and additional analyses address your concerns. We will incorporate these clarifications and results into the revised manuscript and would be happy to address any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.