acceptodds
Under review as a conference paper at ICLR 2027

Tensor Hourglass Decomposition for Hyperspectral and Multispectral Image Fusion

Abstract

Hyperspectral image super-resolution (HSISR) aims to reconstruct a high-resolution hyperspectral image from a low-resolution hyperspectral image and a high-resolution multispectral image. A central challenge is to recover fine spatial details from the multispectral observation while maintaining consistency with the spectral information provided by the hyperspectral observation. To address this challenge, we propose a tensor hourglass decomposition (THD) model that separately characterizes heterogeneous spatial and spectral structures while preserving their high-order interactions. THD organizes the spatial and spectral representations into two closed peripheral cycles and couples their spoke modes through a shared high-order waist core, thereby enabling flexible many-to-many interactions between spatial structures and spectral signatures. By exploiting the multilinearity of THD, the spatial degradation and spectral response operators are directly absorbed into the corresponding physical-mode-carrying factors, yielding a structured coupled observation model for HSISR. Extensive experiments on benchmark datasets demonstrate that the proposed method achieves competitive or superior spatial–spectral reconstruction performance.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Tensor Decomposition, Hyperspectral Image Fusion

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_8MCbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes THD for HSISR, reconstructing an HR-HSI from an LR-HSI and an HR-MSI. THD models spatial and spectral structures through two peripheral cycles coupled by a shared waist core. It absorbs degradation operators into the corresponding factors and solves the coupled model using matrix-free PAM. The paper provides rank bounds and descent analysis, with experiments on simulated and real data demonstrating competitive reconstruction performance.

Critical strengths

1.THD has a clear modeling motivation. Separate spatial and spectral representations capture their different structures, while the shared waist core models interactions between them. 2.The experiments provide substantial evidence of effectiveness. They cover multiple datasets, model-based and neural baselines, and different noise levels.

Critical weaknesses

1.The topology comparisons do not clearly separate the effects of structural design, parameter count, and rank configuration. Comparisons under matched parameter budgets would better establish the contribution of THD. 2.The selection of cyclic ranks, spoke ranks, and tensorization configurations for new scenes remains unclear, particularly when no HR-HSI reference is available. 3.The identifiability of THD is not fully characterized. Under what conditions is the decomposition unique up to inherent scaling and gauge ambiguities? 4.The convergence guarantees for the implemented optimizer need clarification. What assumptions are required, and do they ensure objective-value convergence or convergence to a stationary point? In particular, how do finite-step CG updates satisfy the sufficient-decrease condition?

Questions for authors

1.What parameter counts, rank configurations, and optimization budgets are used in the TT, TR, TW, and THD comparisons? Are results available under comparable parameter budgets to isolate the contribution of topology? 2.How are cyclic ranks, spoke ranks, and tensorization configurations selected? Are they fixed across datasets and noise levels? Does selection use the reference high-resolution image, and what procedure is recommended when no reference is available? 3.How do the finite-step conjugate gradient updates ensure sufficient decrease? What happens when the condition is not met? Does the analysis establish only convergence of objective values, or also convergence of the balanced iterates to a stationary point? 4.How do runtime and peak memory scale with image size, band count, and rank? Do the shared core or intermediate contractions become the main bottleneck? The paper presents a well-motivated model with strong experimental results. However, the contribution of topology remains difficult to separate from model capacity and rank selection, and the connection between the implemented updates and the descent guarantees needs clarification. Controlled comparisons and clearer explanations of parameter selection and optimization guarantees could lead me to raise my rating.

5 comments
AuthorsOpenReview ↗

Response to Reviewer 8MCb (1/2): Topology, Rank Selection, and Identifiability

We thank the reviewer for highlighting the need to distinguish topology, model capacity, rank selection, and optimization guarantees. We address Q1–Q2 and the identifiability concern with additional experiments and theory.


Q1. Controlled Topology Comparison

The original Fig. 7 was not parameter-matched: TT/TR/TW/THD used 36,398/29,645/25,578/40,170 parameters. We added 16 fits on PC (200×200×80200\times200\times80) and WDC (256×256×103256\times256\times103) at approximately matched 42k/84k budgets. The supplementary WDC cube retains 103 bands, unlike the 90-band main benchmark.

All models use identical ×4\times4, 15/15-dB observations, a common four-thread float64 CPU PAM-CG solver (30 sweeps, CG$\leq10$, ρ=ρC=0.3\rho=\rho_C=0.3), and one seed. Ranks follow a fixed dimension/count rule without consulting reference HR-HSI; parameter deviations range from −4.66% to +3.47% (Appendix Table 6).

Input/budgetTT ranks (params)TR ranks (params)TW ranks (params)THD ranks (params)
PC/42k[14,14] (43,120)[9,10,9] (42,480)[5,6,5] (42,027)[5,6,5,5,5,5] (42,954)
PC/84k[20,20] (85,600)[13,13,13] (81,120)[8,7,8] (82,587)[8,7,8,8,8,8] (83,865)
WDC/42k[12,12] (41,172)[8,9,8] (43,456)[4,6,4] (41,835)[4,6,4,4,4,4] (42,681)
WDC/84k[17,17] (80,087)[12,11,12] (82,416)[6,8,6] (84,879)[7,6,7,7,7,7] (80,823)

All active TW/THD spokes have dimension 3. The final results (PSNR in dB/SAM in degrees; Appendix Table 6) are:

Input/budgetTTTRTWTHD
PC/42k23.89/8.7426.47/10.2728.77/10.2130.05/8.71
PC/84k24.72/9.1527.48/9.8528.76/12.1229.99/8.74
WDC/42k21.97/10.9424.01/13.0323.26/10.6625.51/10.22
WDC/84k22.61/11.4724.62/12.8725.27/9.9025.39/9.63

THD attains the highest final PSNR and lowest SAM in all four settings, improving PSNR over TW by 1.28/1.23/2.25/0.11 dB. However, TW/THD runtimes are 5.75/11.78, 8.57/10.81, 6.57/17.51, 10.22/16.41 s, respectively. Fig. 9 shows that competitors may lead at earlier time cutoffs. Our claim is better endpoint accuracy at comparable model sizes, not equal-time superiority.


Q2. Ranks, Realizations, and Reference-Free Selection

Rank protocol. Main benchmark ranks and the (3,3) realization were chosen on a representative reference-based development setting, then reused across datasets, scales, and noise levels without scene-specific rank retuning. In contrast, the new 42k/84k allocation above does not use reference HR-HSI.

Peripheral cycles. We evaluated seven realizations, from (2,1) to (4,4), with paired initialization and ρ=ρC=0.7\rho=\rho_C=0.7. Total parameter counts differ by at most 0.93% per input. The (3,3) realization achieves the highest final PSNR on PC (30.6144 dB) and WDC (28.0426 dB) (Fig. 10; Appendix D.12.2); longer cycles do not guarantee gains.

Spoke sensitivity (Fig. 11 and Appendix Fig. 17). Fixing (3,3) and cyclic ranks:

Input/SNRssParamsPSNR/SAMTime (s)
PU/20 dB2616,20623.63/15.3024.44
3924,94233.55/4.88126.04
41,236,38033.53/5.45199.82
WDC/15 dB2616,38626.01/13.2016.28
3925,21227.44/8.8956.11
41,236,74024.75/15.64279.32

s=3s=3 attains the best PSNR and SAM in both tested conditions. For reference-free scenes, observation singular spectra and noise-level discrepancy may suggest candidate ranks but do not directly determine THD's internal bonds. An automatic reference-free rule has not been validated.


W3. Identifiability and Gauge Ambiguity

THD factors are generally non-identifiable. Proposition 6 (Appendix A.8) establishes diagonal spoke-gauge invariance: invertible spoke scalings can be absorbed by the corresponding waist-core modes without changing the reconstructed tensor. Cyclic bonds also permit compensating latent-basis changes between adjacent factors.

Spoke balancing (Appendix B.6) stabilizes scales but does not imply uniqueness; the rank bounds (Theorem 1; Proposition 2) concern capacity, not identifiability. We claim neither unique factors nor unconditional uniqueness of the recovered HR-HSI.

AuthorsOpenReview ↗

Response to Reviewer 8MCb (2/2): Inexact CG and Computational Complexity

We clarify what the optimization analysis guarantees, how it relates to truncated CG, and the measured accuracy–efficiency trade-off.


Q3. Finite-Step CG, Descent, and Convergence

Strongly convex subproblems. With all other THD blocks fixed, each proximal update is quadratic. Proposition 8 (Appendix B.1) shows that its normal operator is self-adjoint and positive definite for ρV>0\rho_V>0, so the block minimizer is unique despite the overall problem being nonconvex.

Sufficient-decrease requirement. Let Φ\Phi denote the unregularized fusion fidelity objective. A block update satisfying a nonincrease of the proximal subproblem objective relative to its previous value yields (Eq. B.55):

Φ(Θbefore)−Φ(Θafter)≥ρV2∥Vnew−Vold∥F2.\Phi(\Theta_{\mathrm{before}})-\Phi(\Theta_{\mathrm{after}})\geq \frac{\rho_V}{2}\|V^{\mathrm{new}}-V^{\mathrm{old}}\|_F^2.

This condition is sufficient for the descent result of Proposition 9; it is not by itself a stationary-point convergence guarantee.

Truncated CG. Each block CG solve is initialized at the previous block and terminates at relative residual 10−610^{-6} or 10 steps (Appendix B.5). In exact arithmetic, CG minimizes the positive-definite quadratic over expanding Krylov subspaces containing the initial point; its proximal objective therefore cannot increase. Finite-precision CG does not automatically ensure the same condition. If a computed step violates descent, Proposition 9 cannot be invoked for that step. Rejecting the update or continuing CG is a possible safeguard, but we do not claim such rejection was instrumented in our reported implementation.

What converges? For exact sweeps, or inexact sweeps satisfying the required per-block descent, Proposition 9 makes Φ\Phi nonincreasing. Since Φ≥0\Phi\geq0, its values converge and the squared pre-balancing block displacements are summable. Proposition 6 shows spoke balancing preserves the represented tensor and Φ\Phi. These results do not establish convergence of the balanced factor sequence to a stationary point, nor identifiability; the latter is discussed in Response 1/2.


Q4. Runtime, Working Memory, and Bottlenecks

Memory scaling. Define L=∏k=13LkL=\prod_{k=1}^3L_k, U=∏j=13UjU=\prod_{j=1}^3U_j, and Ω=hwS+HWs\Omega=hwS+HWs, where ss is the MSI band count. Appendix C.3 derives the asymptotic working-memory bound:

O(Ω+(HW+hw)L+(S+s)U+LU+PTHD).O\big(\Omega+(HW+hw)L+(S+s)U+LU+P_{\mathrm{THD}}\big).

Matrix-free CG evaluates normal-operator actions through tensor contractions and their adjoints, avoiding storage of explicit environment matrices and Kronecker-product normal coefficients. At fixed ranks, the spatial-group intermediates scale as (HW+hw)L(HW+hw)L (linear in spatial pixel count), while spectral-group intermediates scale as (S+s)U(S+s)U.

Rank-related costs. The waist core stores LULU scalars, and its CG updates cost O(NCLU(L+U))O(N_C LU(L+U)) (Appendix C.2). With six equal spoke dimensions sspoke=2,3,4s_{\mathrm{spoke}}=2,3,4, core sizes grow as 64, 729, and 4,096. Thus, both the waist core and group contractions can become significant bottlenecks as ranks increase; their relative costs depend on scene dimensions and rank allocation.

Measured runtime (Fig. 9).

Input/budgetTW time (s)THD time (s)
PC/42k5.7511.78
PC/84k8.5710.81
WDC/42k6.5717.51
WDC/84k10.2216.41

THD reaches higher final reconstruction quality but takes longer in all four common-solver tests. In the separate native MATLAB benchmark (Appendix D.8), THD averages 66.74 s, versus 63.17 s for GTNN and 109.41 s for LFCTNN. The two timing protocols should not be directly pooled.

Peak-memory scope. The derived storage bound is theoretical, not a measurement of peak resident memory; we therefore make no numerical peak-memory superiority claim.

We thank the reviewer for prompting a more precise account of theoretical guarantees and resource requirements.

Reviewer_8MCbOpenReview ↗

Remaining concerns after the response

Thank you for the detailed responses and additional experiments. The new comparisons under approximately matched parameter budgets largely address my concern about comparison fairness. The explanations regarding finite-step CG and the scope of the convergence results also clarify my questions on optimization.

I still have two issues. First, the additional comparisons primarily demonstrate reconstruction performance in the fusion task, while direct evidence of THD’s representation efficiency as a new decomposition remains limited. Second, Table 2 shows substantial sensitivity to the spoke dimension. For example, increasing it from 3 to 4 on WDC reduces PSNR by approximately 2.69 dB. The authors explain that the configuration selected during development was reused across datasets, but a reliable procedure for selecting configurations without reference ground truth remains unvalidated.

Reviewer_gnmXOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Tensor Hourglass Decomposition method for hyperspectral image super-resolution, which separates spatial and spectral representations into two peripheral loops, linked by a shared, higher-order waist core. The authors derive the algebraic properties of this method and construct a coupled fusion model. Experimental results on both simulated and real-world datasets demonstrate that the method achieves excellent reconstruction performance alongside highly competitive computational efficiency.

Critical strengths
  1. This paper proposes a partition-aware tensor network structure that models spatial and spectral dependencies separately while preserving the interactions between them. Its mathematical formulation is mathematically motivated and clearly presented, with a sound basis in the physics of degradation modeling.
  2. This method achieves excellent results across multiple datasets, varying noise levels, and a range of evaluation metrics. Its effectiveness is further validated through topological structure comparisons, ablation studies, and robustness analysis.
Critical weaknesses
  1. The advantages of this architecture, compared to existing tensor network models, could be articulated more clearly. Although THD introduces a topological structure distinct from tensor wheel decomposition, the specific benefits of partitioning the peripheral structure into two rings warrant further explanation.
  2. The experiments using real-world data, presented in Figure 6, rely primarily on visual comparisons via pseudo-color images. While THD reveals a clearer spatial structure, visual similarity alone is insufficient to adequately demonstrate its spectral fidelity.
Questions for authors

Q1. Could the authors further clarify the primary structural advantages of THD compared to tensor wheel decomposition? Q2. Figure 2 compares different parameterization methods for the waist-core. Could the authors elaborate on why a full waist core achieves superior reconstruction performance? Q3. Could the authors elaborate on how virtual singleton factors contribute to the representation capability of THD despite having unit physical dimensions? Q4. What are the key computational advantages of using a matrix-free conjugate gradient method as opposed to explicitly constructing environment matrices? Q5. In scenarios where ground-truth data is unavailable for qualitative comparison, how do the authors evaluate spectral consistency? Figure 6 presents qualitative comparisons on real data without reference HR-HSI. Could the authors further clarify how spectral consistency is evaluated and discuss the limitations of the visual assessment?

2 comments
AuthorsOpenReview ↗

Response to Reviewer gnmX (1/2): Structural Contributions and Core Parameterization

We sincerely thank the reviewer for recognizing THD's well-motivated architecture, coherent mathematical formulation, and strong experimental performance. We address Q1–Q3 through additional controlled topology and structural ablation studies.


Q1. Structural Advantages over Tensor Wheel (TW)

Partition-aware topology. Unlike TW, which places all physical modes in a single peripheral cycle, THD constructs two separate cycles for heterogeneous spatial and spectral groups, coupled through a shared high-order waist core. This separates within-group dependencies from cross-group coordination. The grouped form X⟨L∣R⟩=GCF⊤X_{\langle L|R\rangle}=GCF^\top (Theorem 1) characterizes this structure, where GG and FF are generated by their respective cyclic networks rather than unrestricted matrices.

Controlled topology validation. To complement the original Fig. 7, we added 16 fits of TT, TR, TW, and THD under approximately matched 42k/84k parameter budgets on PC and WDC (Fig. 9; Appendix D.12.1). All methods share identical ×4\times4, 15/15-dB observations and a unified float64 CPU PAM-CG protocol (30 sweeps, CG$\leq10$, ρ=ρC=0.3\rho=\rho_C=0.3). Parameter-count deviations remain below 5%.

Input / budgetTW PSNR/SAMTHD PSNR/SAMPSNR gain
PC / 42k28.77 / 10.2130.05 / 8.71+1.28
PC / 84k28.76 / 12.1229.99 / 8.74+1.23
WDC / 42k23.26 / 10.6625.51 / 10.22+2.25
WDC / 84k25.27 / 9.9025.39 / 9.63+0.11

PSNR in dB; SAM in degrees.

THD achieves the highest final PSNR and lowest SAM among all four topologies in each setting. This provides controlled empirical evidence for the effectiveness of group-specific peripheral organization beyond TW's shared-core coupling.

We also acknowledge the computational trade-off: THD requires longer optimization time to attain its final accuracy, and competing topologies can perform better at earlier time cutoffs (Fig. 9).


Q2. Why Use a Full Waist Core?

The waist core coordinates multiple spatial–spectral interaction channels. A rank-one core permits only one separable cross-group coupling channel, whereas higher-rank cores support multiple channels (Appendix A.6). The full parameterization avoids imposing additional rank or diagonal constraints on these interactions.

Beyond the approximation study in Fig. 2, we conducted a controlled fusion experiment on PC with fixed (3,3) realization, cyclic ranks, spoke dimensions, observations, and initial peripheral factors (Appendix D.11.3).

Core variantCore DoFTotal paramsPSNR ↑SAM ↓
Rank-one53722,31527.91459.3808
Diagonal27722,28928.58379.1441
Rank-four200722,46230.49988.8673
Full729722,99130.61448.8738

Rank-one and diagonal constraints reduce PSNR by more than 2 dB relative to the full core, demonstrating the importance of flexible cross-group coupling. The full core achieves the highest PSNR, while rank-four coupling approaches its accuracy with only 200 versus 729 core coefficients and slightly better SAM.

Thus, the results support flexible cross-group interactions, rather than suggesting that unrestricted coupling is necessary or optimal in every setting.


Q3. Role of Virtual Singleton Factors

In the (3,3) realization, G3G_3, F2F_2, and F3F_3 carry singleton physical dimensions, but their cyclic and spoke indices remain active (Eqs. 20–23). They therefore participate in latent contractions rather than introducing artificial spatial or spectral observations.

To investigate whether the virtual spectral cycle offers a practical benefit, we compared its contracted S×27S\times27 representation against a directly optimized free matrix of identical dimensions, retaining the spatial branch, waist core, and paired initialization (Appendix D.12.3).

Spectral modelParametersPSNR ↑SAM ↓Time (s)
Structured (3,3)722,99130.6144**8.8738 **95.84
Free spectral matrix722,93730.24088.792678.35

At nearly matched parameter counts, the structured spectral cycle improves PSNR by 0.3736 dB over the free-matrix parameterization, while the latter achieves slightly lower SAM (8.7926° vs. 8.8738°) and shorter runtime. These results suggest that structured spectral parameterization provides a beneficial reconstruction prior, rather than consistent superiority across all metrics.

These findings clarify that virtual factors contribute through their latent connectivity and structured optimization bias, rather than by increasing the number of physical tensor modes.

AuthorsOpenReview ↗

Response to Reviewer gnmX (2/2): Computational Efficiency and Real-Data Validation

We further address Q4–Q5 by clarifying the computational benefits of matrix-free CG and providing quantitative observation-consistency evidence on real HSI–MSI data.


Q4. Computational Advantages of Matrix-Free Conjugate Gradient

Avoiding explicit matrix construction. In THD's proximal alternating minimization, each factor or waist-core subproblem is quadratic with a positive-definite normal operator (Proposition 8):

HV=AY,V∗AY,V+λAZ,V∗AZ,V+ρVI,ρV>0.\mathcal{H}_V = \mathcal{A}_{Y,V}^{\ast}\mathcal{A}_{Y,V} + \lambda\mathcal{A}_{Z,V}^{\ast}\mathcal{A}_{Z,V} + \rho_V I, \qquad \rho_V > 0.

Instead of constructing the full environment matrices, Kronecker-product coefficients, or dense normal-equation matrices, our matrix-free implementation evaluates operator products through successive tensor contractions, degradation operators, and their adjoints (Appendix B.5).

Memory and computational complexity. Let L=∏k=13LkL=\prod_{k=1}^{3}L_k, U=∏j=13UjU=\prod_{j=1}^{3}U_j, and Ω=hwS+HWs\Omega=hwS+HWs, where ss denotes the number of MSI bands. Appendix C.3 derives the dominant working-memory bound:

O!(Ω+(HW+hw)L+(S+s)U+LU+PTHD).O!\left(\Omega+(HW+hw)L+(S+s)U+LU+P_{\mathrm{THD}}\right).

This avoids explicitly storing large coefficient matrices. For the adopted six spoke dimensions of 3, the waist core contains only 36=7293^6=729 coefficients. Its CG update is performed on the reshaped 27×2727\times27 core, with arithmetic cost scaling as O(NCLU(L+U))O(N_C LU(L+U)) (Appendix C.2).

Empirical runtime. Appendix D.8 reports average runtime across the three benchmark datasets and two noise levels, using the same MATLAB CPU environment for the model-driven methods:

MethodAverage runtime (s) ↓
CTDF27.89
GTNN63.17
THD66.74
LFCTNN109.41
GNLTN177.81
MHTNN226.22

THD is substantially faster than several tensor-based alternatives, although CTDF and GTNN are faster. Thus, the primary computational benefit of matrix-free CG is avoiding large explicit coefficient matrices while enabling structured block optimization, rather than guaranteeing the lowest runtime.

We also recognize that the shared waist core and intermediate contractions can become computational bottlenecks as latent ranks increase, consistent with the additional time–quality results in Fig. 9.


Q5. Spectral Consistency and Limitations of Real-Data Evaluation

Evaluation scope. Figure 6 presents pseudo-color reconstructions on a registered Hyperion–Sentinel-2 pair without reference HR-HSI. Although THD recovers clearer boundaries and richer spatial details, visual appearance alone cannot establish quantitative spectral fidelity.

Additional real-data observation consistency. To provide quantitative evidence, we evaluated a separate real-data crop (LR-HSI: 120×120×89120\times120\times89; HR-MSI: 360×360×4360\times360\times4) under common estimated degradation operators. We calculate the normalized residuals:

rY=∣Y−X^×1P1×2P2∣F∣Y∣F,r_Y=\frac{|\mathcal Y-\widehat{\mathcal X}\times_1P_1\times_2P_2|_F}{|\mathcal Y|_F},

rZ=∣Z−X^×3P3∣F∣Z∣F.r_Z=\frac{|\mathcal Z-\widehat{\mathcal X}\times_3P_3|_F}{|\mathcal Z|_F}.

MethodLR-HSI residual ↓HR-MSI residual ↓
Tensor Wheel (TW)0.0389640.048986
THD (Ours)0.0366190.045873

THD reduces the LR-HSI and HR-MSI residuals by approximately 6.0% and 6.4%, respectively, compared with TW. These results provide quantitative evidence of improved consistency with both observed sensor measurements under the estimated degradation model, complementing the qualitative results in Fig. 6.

Degradation-model sensitivity. We additionally evaluated PSF/SRF mismatch on simulated observations (Appendix D.12.6). PSF-width changes from σ=2.0\sigma=2.0 to 1.8 or 2.2 alter THD's PSNR by less than 0.1 dB, whereas 5% and 10% SRF perturbations reduce PSNR by 1.12 and 1.29 dB. These results highlight the practical importance of accurate spectral-response estimation.

We thank the reviewer for encouraging a clearer explanation of computational efficiency and more quantitative real-data validation.

Reviewer_8wF6OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes tensor hourglass decomposition (THD) for hyperspectral and multispectral image fusion. The spatial and spectral modes are assigned to separate peripheral cycles and connected through a shared core. The resulting coupled observation model is solved using proximal alternating minimization with matrix-free conjugate-gradient updates. Experimental results demonstrate the effectiveness of the proposed tensor decomposition.

Critical strengths
  1. THD uses separate peripheral cycles for spatial and spectral modes and couples them through a shared core, making the proposed grouping explicit in the model.
  2. The proposed method achieves substantial improvements over the compared methods.
  3. The optimization procedure is detailed and easy to follow.
Critical weaknesses
  1. Theorem 1 and Proposition 2 establish a grouped matrix representation and rank upper bounds. Neither result shows that THD achieves lower approximation error or requires fewer parameters than existing decompositions.
  2. The paper does not sufficiently explain the advantages of THD over TT, TR, and TW in terms of representation capacity, parameter efficiency, and computational cost. Figure 7 compares reconstruction errors, but the exact ranks and parameter counts are not reported.
  3. The experiments evaluate THD within the fusion framework. The absence of a basic approximation or low-rank completion experiment leaves its representation efficiency at matched parameter counts untested.
Questions for authors
  1. What specific advantage does THD offer over TT, TR, and TW beyond the different network topology? The current rank bounds do not establish lower approximation error at a fixed parameter budget or fewer parameters at a fixed error.
  2. What are the exact ranks and parameter counts used in Figure 7 and Table 4? Does the advantage of the (3,3) realization remain when parameter budgets are matched across configurations?
  3. Tensor completion is a standard task for evaluating new tensor decompositions. Why was THD not tested on this basic task to assess its effectiveness relative to existing decompositions?
  4. The authors should compare THD with coupled TT, TR, FCTN, and TW methods under the same experimental settings to establish its advantage over these decompositions in HSI fusion.
  5. Figure 8 varies the cyclic ranks while fixing all six spoke dimensions at 3. Does THD retain its advantage across different spoke dimensions, and what criterion is used to select these dimensions without access to the reference HR-HSI?
5 comments
AuthorsOpenReview ↗

Response to Reviewer 8wF6 (1/2): Representation and Controlled Topology Evaluation

We thank the reviewer for recognizing THD's explicit spatial–spectral grouping, reconstruction gains, and optimization details. We address Q1–Q2 with new controlled evidence (Figs. 9–10; Appendix D.11.1–D.11.2).


Q1. What Does THD Offer Beyond a Different Topology?

Structural contribution. The grouped identity X⟨L∣R⟩=GCF⊤X_{\langle L|R\rangle}=GCF^\top (Theorem 1) is not itself a new factorization. In THD, however, GG and FF are generated by separate spatial and spectral peripheral cycles (Eq. 12), whose spokes are coupled by a shared waist core. Unlike TW's single peripheral cycle, THD separates within-group structure from cross-group coordination. Virtual singleton factors preserve active latent bonds without adding physical modes (Eqs. 20–23).

Physical modeling and theoretical scope. Spatial and spectral degradation operators can be absorbed exactly into their corresponding physical factors under the assumed model (Eqs. 24–25). We agree that the grouped form and rank upper bounds do not prove smaller approximation error or fewer parameters than TT/TR/TW. We therefore evaluate the structural bias empirically, rather than claiming universal representational superiority.


Q2. Exact Ranks, Parameters, and Comparable-Budget Tests

The original Fig. 7 used unequal model sizes: 36,398 / 29,645 / 25,578 / 40,170 stored parameters for TT/TR/TW/THD, respectively. We added 16 fits on PC (200×200×80200\times200\times80) and WDC (256×256×103256\times256\times103), targeting 42k/84k parameters. All methods use identical ×4\times4, 15/15-dB observations and a common four-thread float64 CPU PAM-CG solver (30 sweeps, CG ≤10\leq10, ρ=ρC=0.3\rho=\rho_C=0.3). Ranks follow a fixed dimension/count rule without consulting reference HR-HSI; actual deviations are below 5% (Appendix Table 6).

Input/budgetTT ranks (params)TR ranks (params)TW ranks (params)THD ranks (params)
PC/42k[14,14] (43,120)[9,10,9] (42,480)[5,6,5] (42,027)[5,6,5,5,5,5] (42,954)
PC/84k[20,20] (85,600)[13,13,13] (81,120)[8,7,8] (82,587)[8,7,8,8,8,8] (83,865)
WDC/42k[12,12] (41,172)[8,9,8] (43,456)[4,6,4] (41,835)[4,6,4,4,4,4] (42,681)
WDC/84k[17,17] (80,087)[12,11,12] (82,416)[6,8,6] (84,879)[7,6,7,7,7,7] (80,823)

TW and THD spoke dimensions are 3. Final PSNR (dB)/SAM (degrees) are (Appendix Table 6):

Input/budgetTTTRTWTHD
PC/42k23.89/8.7426.47/10.2728.77/10.2130.05/8.71
PC/84k24.72/9.1527.48/9.8528.76/12.1229.99/8.74
WDC/42k21.97/10.9424.01/13.0323.26/10.6625.51/10.22
WDC/84k22.61/11.4724.62/12.8725.27/9.9025.39/9.63

THD attains the highest final PSNR and lowest SAM in all four settings, exceeding TW by 1.28, 1.23, 2.25, and 0.11 dB. The time–quality curves (Fig. 9) show the trade-off: TW takes 5.75–10.22 s versus 10.81–17.51 s for THD, and competitors can lead at early time cutoffs. Our evidence supports endpoint accuracy under comparable model sizes, not equal-time superiority.

Peripheral realizations. We also evaluated seven cycle configurations with paired initialization, 30 sweeps, and ρ=ρC=0.7\rho=\rho_C=0.7. Total parameter counts differ by at most 0.93% within each input (Fig. 10; Appendix Tables 9–10).

RealizationCore DoFPC totalPC PSNRWDC PSNR
(2,1)27722,18727.913625.4460
(2,2)81722,26829.873027.5318
(3,1)81722,28927.913627.7502
(3,3)729722,99130.614428.0426
(3,4)2,187724,47627.913627.8572
(4,3)2,187724,49730.074827.3142
(4,4)6,561728,89829.628626.0011

Spatial cyclic ranks are [4,150], [4,150,4], and [4,150,4,4] for spatial cycle lengths 2, 3, and 4; spectral cyclic bonds and spokes are 3. (3,3) achieves the highest final PSNR on both inputs. Notably, (3,4) and (4,3) have identical core DoF yet different PC PSNR (27.91 vs. 30.07 dB), suggesting that latent interaction organization matters beyond core size. The results do not establish a universally optimal topology.

AuthorsOpenReview ↗

Response to Reviewer 8wF6 (2/2): Independent Tasks, Coupled Baselines, and Spoke Selection

Q3. Why No Standalone Tensor Approximation or Completion Benchmark?

We agree that tensor approximation and completion are valuable benchmarks for general-purpose decompositions. Our study focuses on HSI–MSI fusion, where spatial and spectral modes have distinct physical roles and undergo different degradations.

Spatial modes encode geometry and textures, while spectral modes exhibit inter-band correlations. The LR-HSI undergoes spatial blur/downsampling (P1,P2P_1,P_2), whereas the HR-MSI undergoes spectral degradation (P3P_3). THD's group-specific cycles accommodate this asymmetry, with operators absorbed into their corresponding factors (Eqs. 24–26).

Unlike generic completion with missing tensor entries, fusion reconstructs a latent tensor from two differently degraded observations. A standalone completion benchmark would assess complementary representation properties, but would not directly evaluate the operator-aware modeling central to this work.

Although THD supports arbitrary mode partitions (Section 3; Appendix A), standalone tensor completion is not evaluated here. We therefore do not claim universally superior approximation or completion efficiency. Instead, the matched-budget TT/TR/TW comparisons (Fig. 9) and free-spectral-matrix control (Appendix Table 11) provide task-specific evidence.


Q4. Comparisons with Coupled TT, TR, FCTN, and TW

Controlled comparison. We implemented coupled TT, TR, TW, and THD under identical observations, fidelity objectives, approximately matched parameter budgets, and a unified PAM-CG solver (Fig. 9; Appendix D.11.1). THD achieves the highest final PSNR and lowest SAM in all four settings (Response 1/2).

FCTN–TR equivalence. For the third-order HR-HSI X∈RH×W×S\mathcal X\in\mathbb R^{H\times W\times S}, a standard three-factor FCTN forms a triangular graph with one bond between each pair of factors, topologically equivalent to a three-core TR up to latent-index reordering. Thus, our controlled TR experiment also evaluates this direct third-order FCTN topology. This equivalence concerns the contraction structure, not higher-order tensorized FCTN models or algorithms with different regularization and optimization.

Matched-Budget Comparison with Tensor Wheel (TW).

We further isolate the comparison between TW and THD under identical observations and a unified PAM-CG protocol, with approximately matched parameter budgets (42k/84k, within 5%; Appendix D.11.1, Table 6).

Input / BudgetTW PSNR / SAMTHD PSNR / SAMPSNR Gain
PC / 42k28.77 / 10.21°30.05 / 8.71°+1.28
PC / 84k28.76 / 12.12°29.99 / 8.74°+1.23
WDC / 42k23.26 / 10.66°25.51 / 10.22°+2.25
WDC / 84k25.27 / 9.90°25.39 / 9.63°+0.12

THD consistently achieves higher final PSNR and lower SAM across all four settings, supporting the benefit of group-specific peripheral coupling over TW's single peripheral cycle. However, THD requires longer optimization time, indicating an accuracy–efficiency trade-off.


Q5. Spoke-Dimension Sensitivity and Reference-Free Rank Selection

Spoke sensitivity. We varied all six spoke dimensions over s∈{2,3,4}s\in\{2,3,4\} with the (3,3) realization and cyclic ranks fixed (Fig. 11 and Appendix Fig. 17).

Input / SNRssParametersPSNR ↑SAM ↓Time (s)
PU / 20 dB2616,20623.6315.3024.44
3924,94233.554.88126.04
41,236,38033.535.45199.82
WDC / 15 dB2616,38626.0113.2016.28
3925,21227.448.8956.11
41,236,74024.7515.64279.32

PSNR in dB; SAM in degrees.

s=3s=3 achieves the highest PSNR and lowest SAM in both conditions. On PU, increasing ss from 2 to 3 improves PSNR by 9.92 dB, whereas s=4s=4 offers no further gain. On WDC, s=4s=4 reduces PSNR by 2.69 dB relative to s=3s=3, despite its higher cost. These results support moderate spoke dimensions under the tested conditions.

Rank selection. The main benchmark ranks were chosen on a representative reference-based development setting and then reused across datasets, scales, and noise conditions without scene-specific retuning.

Reference-free guidance. For new sensors, singular-value decay in observed HSI/MSI unfoldings may guide candidate dimensions, while observation residuals relative to noise levels may help compare configurations. These quantities do not directly determine THD's internal ranks; an automatic reference-free selection procedure remains unvalidated.

We thank the reviewer for highlighting these distinctions.

Reviewer_8wF6OpenReview ↗

Further Validation of THD as a Tensor Decomposition

Thank you for the additional experiments. The comparisons at similar parameter counts address my concern about the fairness of the original fusion comparison. My main concern is that the paper presents THD itself as a contribution, while the new comparisons are still limited to fusion. A direct comparison of approximation error at similar parameter counts would help assess how well THD represents the same data. Tensor completion and tensor robust PCA are also useful tests. For these tasks, I would like to see theory that explains when recovery is possible and experiments that compare THD with existing decompositions. If the authors prefer to evaluate THD through hyperspectral image processing, why limit the evaluation to fusion? Results on other hyperspectral imaging tasks would help show whether the same structure is useful across tasks.

Reviewer_wz5jOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Tensor Hourglass Decomposition (THD) for hyperspectral image super-resolution through HSI–MSI fusion. THD organizes spatial and spectral modes into two separate peripheral cycles coupled by a shared high-order waist core, allowing group-specific representations and cross-group interactions. The spatial degradation and spectral response operators are incorporated directly into the corresponding tensor factors, and the resulting coupled reconstruction problem is solved using matrix-free proximal alternating minimization. Experiments on three hyperspectral datasets and a real HSI–MSI pair demonstrate improvements in spatial and spectral reconstruction quality, with additional analyses of tensor topology, robustness, and computational efficiency.

Critical strengths
  1. Separating spatial and spectral modes into two peripheral structures is a reasonable extension of existing tensor-network decompositions. The shared waist core allows the two groups to interact without imposing the same network organization on spatial and spectral factors.

  2. The multilinear formulation makes it possible to incorporate spatial degradation and spectral response operators directly into the corresponding THD factors. Together with the matrix-free PAM solver, this provides a coherent optimization framework that explicitly respects the HSI–MSI observation model.

  3. The reconstruction results are consistently strong across different datasets, noise levels, and downsampling factors. In particular, the improvements in both PSNR and SAM under 15-dB noise are substantial, and the topology comparisons in Figure 7 provide additional evidence beyond comparisons with existing HSISR algorithms.

Critical weaknesses
  1. In the HSISR instantiation of Eqs. (20)–(23), the spectral cycle contains only one physical spectral factor, while the other two positions are virtual singletons. I would be interested in whether this construction offers a meaningful advantage over a simpler spectral matrix factor or Tucker-type representation under matched rank and parameter budgets.

  2. Tensor Wheel is the most relevant structural comparison, but it is absent from the main quantitative results in Table 1, while Figure 7 only evaluates it on one PU setting. I also could not find the detailed ranks, parameter counts, runtime, and peak memory promised in Section 5.1 for this topology comparison.

  3. The improvement from the balanced (3,3) realization in Appendix Table 4 is encouraging, but its runtime increases from 29.7 s to 162.5 s compared with the minimal (2,1) realization. A parameter-matched comparison, potentially including alternative spatial–spectral partitions, would better isolate the benefit of the proposed topology.

  4. Figure 2 shows that diagonal and low-rank waist-core parameterizations achieve reconstruction errors close to those of the full core. Further analysis of their performance across datasets and computational budgets would help clarify how much the unrestricted high-order coupling contributes.

  5. The quantitative experiments assume known spatial degradation and spectral response operators, while the real-data evaluation is mainly qualitative. Evaluating sensitivity to inaccurate PSF/SRF estimates and reporting observation-consistency errors on the real HSI–MSI pair would provide stronger evidence for practical cross-sensor reconstruction.

Questions for authors

pls see weakness

4 comments
AuthorsOpenReview ↗

Response to Reviewer wz5j (1/2): Structural and Topology Comparisons

We thank the reviewer for recognizing THD's structural motivation, physical formulation, and reconstruction performance. We address W1–W3 through new experiments in Figs. 9–10 and Appendix D.12.


W1. Virtual Spectral Factors versus a Free Matrix

In (3,3), F1F_1 carries the spectral mode, while virtual factors F2F_2 and F3F_3 retain active cyclic and spoke bonds (Eqs. 20–23). With spoke dimensions of 3, the spectral cycle contracts into a structured S×27S\times27 matrix.

We compare this against a directly optimized free S×27S\times27 matrix, retaining the spatial branch and waist core. Common factors share initialization, and the free matrix is initialized from the spectral-cycle contraction (Appendix Table 8; PC, ×4\times4, 15/15-dB observations, ρ=ρC=0.3\rho=\rho_C=0.3).

Spectral modelParametersPSNR (dB) ↑SAM (°) ↓Time (s) ↓
Structured (3,3)722,99130.61448.510895.84
Free spectral matrix722,93730.24088.594278.35

At nearly identical parameter counts (difference below 0.12%), the structured cycle improves PSNR by 0.3736 dB and reduces SAM by 0.0834°, although the free matrix is faster. Since the free matrix can represent the contracted cycle's output, this does not imply greater theoretical expressiveness. Instead, it demonstrates a beneficial structural and optimization bias under the tested finite-iteration setting.


W2. Fair TW and Topology Comparisons under Comparable Budgets

We agree that original Fig. 7 involved unequal model sizes. We therefore conducted 16 controlled fits of TT, TR, TW, and THD on PC (200×200×80200\times200\times80) and WDC (256×256×103256\times256\times103), at 42k and 84k target parameter budgets.

All models use identical ×4\times4, 15/15-dB observations and a common four-thread float64 CPU PAM-CG protocol (30 sweeps, CG ≤10\leq10, ρ=ρC=0.3\rho=\rho_C=0.3). Ranks follow a fixed dimension/parameter-count rule without reference-HSI tuning. Parameter-count deviations remain below 5% (Appendix Tables 6).

Input / budgetTT (PSNR/SAM)TR (PSNR/SAM)TW (PSNR/SAM)THD (Ours)
PC / 42k23.89 / 8.74°26.47 / 10.27°28.77 / 10.21°30.05 / 8.71°
PC / 84k24.72 / 9.15°27.48 / 9.85°28.76 / 12.12°29.99 / 8.74°
WDC / 42k21.97 / 10.94°24.01 / 13.03°23.26 / 10.66°25.51 / 10.22°
WDC / 84k22.61 / 11.47°24.62 / 12.87°25.27 / 9.90°25.39 / 9.63°

Entries denote final PSNR (dB)/SAM (degrees). THD achieves the highest final PSNR and lowest SAM in all four settings, exceeding TW by 1.28, 1.23, 2.25, and 0.11 dB in PSNR. These results provide controlled evidence for THD's partition-aware topology at comparable model sizes.

Accuracy–efficiency trade-off. Fig. 9 shows that THD requires longer optimization for its final accuracy. TW completes in 5.75–10.22 s versus 10.81–17.51 s for THD. Competing networks can lead at earlier time cutoffs. Thus, our claim concerns final reconstruction accuracy under comparable parameter budgets, not superiority at equal wall-clock time.


W3. Peripheral Configurations and Model Capacity

We evaluated seven realizations while preserving the spatial–spectral partition (Fig. 10; Appendix Tables 7). All use paired initialization, 30 sweeps, and ρ=ρC=0.7\rho=\rho_C=0.7, with total parameter counts differing by at most 0.93% within each input.

RealizationWaist-core DoFPC PSNR (dB)WDC PSNR (dB)
(2,1)2727.913625.4460
(2,2)8129.873027.5318
(3,1)8127.913627.7502
(3,3)72930.614428.0426
(3,4)2,18727.913627.8572
(4,3)2,18730.074827.3142
(4,4)6,56129.628626.0011

The balanced (3,3) realization achieves the highest final PSNR on both PC and WDC. Increasing the waist-core size from 729 to 2,187 or 6,561 coefficients yields no further gains. Moreover, configurations with similar core sizes show different performance, indicating that latent interaction organization matters beyond core parameter count.

These results support the adopted (3,3) realization while demonstrating that longer peripheral cycles or larger waist cores do not guarantee better reconstruction.

AuthorsOpenReview ↗

Response to Reviewer wz5j (2/2): Waist-Core Design and Practical Validation

W4. Full versus Restricted Waist-Core Coupling

We compared rank-one, diagonal, rank-four, and full waist cores on PC under identical observations and initial peripheral factors, fixing the (3,3)(3,3) realization, cyclic ranks [4,150,4,3,3,3][4, 150, 4, 3, 3, 3], and spoke dimensions of 3 (App. Table 9). Constrained variants employ projected CG updates with proximal descent checks:

Core VariantCore DoFTotal ParametersPSNR (dB) ↑SAM (°) ↓Time (s)
Rank-one53722,31527.91459.380828.81
Diagonal27722,28928.58379.144119.48
Rank-four200722,46230.49988.867348.88
Full Core729722,99130.61448.873835.47
  • Finding: Rank-one and diagonal coupling reduce PSNR by >2 dB>2\text{ dB} relative to the full core, highlighting the necessity of flexible cross-group interaction channels. The full core achieves the highest PSNR, exceeding rank-four by 0.1146 dB, while rank-four attains slightly lower SAM using only 200 core coefficients (vs. 729). These results indicate that flexible cross-group coupling provides the primary reconstruction benefit, whereas unrestricted coupling offers an additional modest gain.

W5. Sensitivity to PSF/SRF Estimation Mismatch

To evaluate robustness against operator estimation errors, we fixed the noisy PC observations (×4\times 4, 15-dB noise) and perturbed only the blur kernel σ\sigma or spectral response matrix P3P_3 supplied to the algorithms (App. Table 10). The reference PSF is a 7×77\times 7 Gaussian with σ=2.0\sigma=2.0; SRF perturbations have relative Frobenius errors of 5% and 10%, preserving nonnegativity and row sums:

Assumed OperatorTHD PSNR (dB)THD SAM (°)CTW PSNR (dB)CTW SAM (°)THD Δ\DeltaPSNR
Matched Operators30.61448.873825.025318.41800.00 dB
PSF σ=1.8\sigma = 1.830.56418.622425.154518.0940-0.05 dB
PSF σ=2.2\sigma = 2.230.70038.677824.864018.7597+0.09 dB
SRF Error 5%29.491110.533324.942618.5527-1.12 dB
SRF Error 10%29.326610.766724.861618.6820-1.29 dB

PSNR in dB; SAM in degrees. ΔPSNR\Delta\text{PSNR} denotes perturbed minus matched THD PSNR.

  • Finding: THD maintains superior absolute PSNR (by >4.4 dB>4.4\text{ dB}) and lower SAM than CTW across all five tested operator settings. PSF-width perturbations alter THD's PSNR by <0.1 dB<0.1\text{ dB}, whereas 5% and 10% SRF errors reduce PSNR by 1.12 and 1.29 dB. Although THD retains higher absolute accuracy, its relative SRF-induced decline is larger than CTW's, indicating that reconstruction is more sensitive to spectral-response calibration than to spatial blur errors.

Generalization to Tensor Completion and RPCA.

We further evaluate THD on hyperspectral images and video sequences through tensor completion and RPCA (Appendix D.13). For completion, we compare against TW-TC and its two improved variants, TWLRS and TW-FPAG, alongside TT/TR baselines. For RPCA, we include TW-RPCA, TTNN-3D, BRFCTN, and RTCUR. THD achieves the best PSNR in 10/12 completion settings and the best angular error and SSIM in all 12. For RPCA, it achieves the best PSNR and angular error in all 12 settings and the lowest sparse-component RSE in 9/12, demonstrating its applicability beyond fusion.


Real-Data Quantitative Observation Consistency

Beyond the qualitative visual comparisons in Fig. 6, we quantitatively evaluated observation-consistency errors on an independent crop of the real Hyperion–Sentinel-2 pair (LR-HSI 120×120×89120\times 120\times 89; HR-MSI 360×360×4360\times 360\times 4). Using common estimated degradation operators, we computed the normalized observation residuals:

r_Z = \frac{\|\mathcal{Z} - \widehat{\mathcal{X}} \times_3 P_3\|_F}{\|\mathcal{Z}\|_F}.$$ | Method | LR-HSI Residual $r_Y$ ↓ | HR-MSI Residual $r_Z$ ↓ | | :--- | :---: | :---: | | **Tensor Wheel (TW)** | 0.038964 | 0.048986 | | **THD (Ours)** | **0.036619** | **0.045873** | * **Finding:** THD achieves lower normalized residuals in both observation spaces, reducing LR-HSI and HR-MSI residuals by **6.0%** and **6.4%** relative to TW, respectively. This provides quantitative evidence of improved observation-model consistency, corroborating the sharper spatial boundaries in Fig. 6. Because true high-resolution references are unavailable, observation consistency does not directly verify full ground-truth spectral accuracy. Nevertheless, these results confirm THD's robust cross-sensor fidelity on real-world satellite data. We thank the reviewer again for prompting these constructive structural and practical evaluations.

Dear Authors,

Thank you for your detailed and thoughtful response. I have changed my score. Good Luck.

Thank you for your positive feedback and for reconsidering your score. We sincerely appreciate your time, thoughtful comments, and constructive suggestions, which have helped us improve our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.