acceptodds
Under review as a conference paper at ICLR 2027

Shared Transform Learning and Half Thresholding for Tensor Restoration

Abstract

Low-rank tensor models provide a global reference for removing sparse corruption, but a compact representation can omit genuine signal retained in the observations. We propose ST-HalfNet, an unfolded network with a learned shared tensor transform and Half thresholding for sparse corruption estimation. An orthogonal transform and nonuniform slice ranks are learned jointly under a total rank budget. Scaled factor updates estimate a reference for each input, and robust local regression predicts the signal omitted by that reference. Half thresholding then estimates corruption from the corrected residual. The restored image is obtained by subtracting this estimate from the observation, preserving signal outside the reference family. We establish optimal rank allocation for a fixed transform and derive a restoration error bound separating missed corruption, prediction error, and shrinkage. On full-resolution CAVE and ICVL images, ST-HalfNet achieves the highest mean PSNR across the tested salt-and-pepper densities and transfers to random-valued and sparse Gaussian corruption with unchanged weights, exceeding the strongest baseline by 2.94 to 13.14 dB in the transfer settings. Comparisons with the best clean approximation show that a learned reference can guide restoration beyond its own approximation accuracy.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
tensor robust principal component analysis, transform learning, deep unfolding, hyperspectral image denoising

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_vwbjOpenReview ↗
Rating
3(+2)
Confidence
3(−1)
Informativeness
2(+1)
Presentation
2(+1)
Soundness
2(+1)
Read review
Paper summary

The authors address the challenge that, although previous low-rank tensor models provide global references for noise removal, their compact representations may overlook some genuine signal components. To tackle this issue, they propose the ST-HALFNet model. This model employs learned shared tensor variations and a half-thresholding method to estimate sparse corruption, ultimately achieving effective image restoration. Comparative experiments were conducted on full-resolution CAVE and ICVL images to validate the effectiveness of the proposed model, along with several effective ablation analyses.

Critical strengths

(1) The authors appear to have proposed an effective low-rank tensor model that achieves superior performance on CAVE and ICVL images.
(2) The authors provide relatively sufficient comparison figures and experimental results, and the paper is generally well written and fluently expressed.

Critical weaknesses

(1) The authors report that the ST-HALFNet model achieves a 13.14 dB improvement over the current strongest baseline. Given that such a margin represents an extremely substantial gain within the existing baseline framework, yet the paper provides no mention of released code or any commitment to future open-source availability, I have serious doubts regarding the authenticity of the reported results, as the magnitude of this accuracy improvement appears to far exceed the pace typically observed in normal model iteration.
(2) The reference formatting in the paper is incorrect: normal citation linking between in-text citations and the REFERENCES section cannot be established, suggesting that the authors may have modified the template format themselves. It remains unclear whether this issue constitutes grounds for desk rejection. In addition, the formatting of figure and table citations in the paper is also incorrect.
(3) In Figure 1, the Reference image and the Restored image appear to exhibit no significant difference at the level of fine detail. Although the authors emphasize that the PSNR of the Reference image is only 43.26, whereas that of the Restored image reaches 51.81, my understanding of the PSNR metric suggests that once PSNR exceeds 40, the image is already very close to the original ground truth; further gains in accuracy may well be attributable to overfitting. It is also possible that information leakage is involved, and such improvements would carry no practical significance. As for Figure 2, the pipeline framework diagram appears to have been generated by AI. The authors should refine and revise the framework diagram on the basis of the AI-generated version rather than adopting it directly.
(4) The Introduction section is extremely poorly organized. On the one hand, the authors fail to clarify why previous methods encounter this problem; on the other hand, they do not articulate the core contributions of this paper.

Questions for authors

Please refer to the content of the Weakness section and provide detailed responses to each of the issues raised.

4 comments
AuthorsOpenReview ↗

Response to Reviewer vwbj

Thank you for the review. We address the four weaknesses below, as requested in Question 1.

Weakness 1. Code and the 13.14 dB gain

The original submission already included source code and the CAVE/ICVL ST-HalfNet checkpoints in its supplementary ZIP, and the manuscript's Reproducibility statement identified the accompanying implementation. These materials were available for review at submission. We provide the revised code again in the supplementary attachment and in the anonymous repository. The README gives data preparation, training, evaluation, and figure-generation commands. The repository link is now also in the introduction and Reproducibility statement.

The reported 13.14 dB gain measures transfer to a different corruption distribution. Table 2 evaluates how the same restoration methods perform when the corruption distribution changes: models selected for salt-and-pepper restoration are applied directly to random-valued and sparse Gaussian corruption, with no further training, and optimization methods retain their solver settings. On ICVL at 10% random-valued corruption, ST-HalfNet achieves 54.90 dB compared with 41.76 dB for DFT-TRPCA, the strongest listed comparator. This gives the reported 13.14 dB difference.

This comparison measures generalization to a different corruption distribution, where methods can differ substantially in how much accuracy they retain. It does not compare successive versions of one model. Table 2 and Appendix E.1 give the absolute scores and conditions; Figure 3 shows recovered structures across the corruption distributions. The supplied implementation allows these comparisons to be checked directly.

Weakness 2. Citations and formatting

Clickable citations are controlled by the hyperref package and do not require changing the conference template. That package was not loaded in the original source. We have checked the conference and bibliography style files against the official files and confirmed that they are unchanged. Citations and cross-references resolve correctly, and hyperlinks are now enabled.

Could the reviewer point to the specific figure or table reference that is incorrectly formatted? We have not found an unresolved or incorrectly numbered reference in our checks; a concrete example would help us identify and correct any remaining issue.

Weakness 3. Image accuracy and Figure 2

We would argue that further reconstruction accuracy remains meaningful above 40 dB. Figure 1 improves from 43.26 to 51.81 dB, while the enlarged regions and error maps in Figure 4 and Appendix E.2 show the recovered details and reduced errors. A small printed image may not reveal these differences clearly. Hyperspectral restoration also aims to preserve the measured signal across all bands, not only the appearance of a color display.

We rechecked the data splits and training and evaluation procedures and found no test-data leakage in these checks. Training and validation determine the learned parameters; ordinary restoration takes only the corrupted image as input. The supplementary code provides these procedures for inspection, as specified in Appendix C.1–C.4. A high test PSNR alone is not evidence of overfitting or leakage.

We used AI assistance and disclosed its use in the paper. Figure 2 has now been revised to show the main computation and actual intermediate images more clearly. The authors edited and checked the diagram.

Weakness 4. Motivation and contributions

We have rewritten the introduction to explain the problem and the two contributions explicitly. When a low-rank approximation is used as the restored image, genuine detail outside that representation is lost. Under sparse corruption, uncorrupted measurements can still contain details omitted by the low-rank reference. Treating the entire residual as corruption can therefore remove genuine signal. This explains why fitting a compact reference accurately is not sufficient for complete signal recovery.

The two contributions follow from this problem. Joint transform and rank learning improves the reference under a fixed budget, with optimal allocation for each fixed transform. Residual prediction and Half thresholding then estimate corruption while preserving the signal the reference omits. Section 1 now connects this motivation to the proposed computation in Figure 2. Table 3 isolates prediction at a fixed reference, Tables 4 and 9 separate the representation contributions, and Section 5 relates the components to restoration error.

Thank you for the authors’ careful and detailed response. I have also thoroughly examined the provided code. Although the corresponding pretrained weights were not released, the implementation is generally consistent with the descriptions in the paper, which alleviates some of my concerns regarding the reliability of the work.

Nevertheless, given the limited novelty of the proposed approach and the limitations of the experimental comparisons, I remain somewhat concerned that the paper may not yet meet the acceptance standard of ICLR. However, in light of the authors’ clarification and the consistency between the code and the paper, I will increase my score.

Thank you for examining the implementation and for reconsidering your assessment. We appreciate your finding that the code is consistent with the paper and that this addresses part of the reliability concern raised in the original review.

Released checkpoints. The two trained ST-HalfNet models used in the main restoration and transfer experiments were included in both the original supplementary ZIP and the revised source package:

The revised package additionally provides the learned representation states in pretrained/representation/{cave,icvl}.pt and the trained Table 4 controls in pretrained/controls/. It also includes the final configurations and the training and evaluation code for all learned comparison methods.

Novelty. We respectfully disagree with the assessment of limited novelty. ST-HalfNet changes the role of the low-rank model in tensor restoration: the estimated low-rank tensor LK\mathcal L_K is used as a compact reference for estimating sparse corruption, while the final image is recovered as X^=Y−S^\widehat{\mathcal X}=\mathcal Y-\widehat{\mathcal S}. The restored image can therefore retain reliable signal outside the reference family. This formulation is realized through three linked advances: (i) joint learning of a shared orthogonal spectral transform and nonuniform slice ranks under a fixed total rank budget, with the globally optimal rank allocation for each fixed transform (Section 3, Proposition 1); (ii) prediction of the signal omitted by the reference before Half thresholding (Section 4); and (iii) an error analysis that separates missed corruption, prediction error, and shrinkage and gives sufficient conditions under which the restored output has lower error than every tensor in the learned low-rank family (Section 5). Together, these elements define a distinct restoration formulation supported by theory and controlled experiments.

The controls isolate these contributions. Rank allocation provides the largest representation gain. After each basis receives its own optimal ranks determined from the training images, transform optimization still improves CAVE restoration by 0.14–0.38 dB over the initial spectral basis and yields the lowest approximation error on both datasets (Tables 4 and 9). With the learned representation fixed, a separately trained Soft variant lowers mean PSNR by 3.36–6.93 dB relative to Half (Table 4). Table 3 holds the estimated reference, thresholds, and candidate support fixed, yet residual prediction raises ICVL restoration from 51.57 to 55.81 dB at 10% corruption. The results demonstrate distinct empirical roles for rank allocation, transform learning, residual prediction, and Half thresholding.

Experimental comparisons. The revised evaluation provides broad and targeted support for the paper's claims. It compares four optimization methods and four learned baselines on two full-resolution hyperspectral datasets; evaluates salt-and-pepper corruption at 10%, 30%, and 50%; applies the same validation criterion to every learned method; tests transfer to random-valued and sparse Gaussian corruption without retuning; and includes dedicated controls for basis choice, rank allocation, prediction using the best clean approximation, and spatially structured corruption. After the common model-selection procedure, ST-HalfNet retains the highest mean PSNR in every main restoration and transfer condition in Tables 1–2. If the reviewer has a particular baseline or experimental setting in mind that is not covered by these evaluations, we would appreciate a specific suggestion.

Thank you for the authors’ further clarification. I acknowledge the substantial experimental evidence and the effectiveness of the proposed method. However, I still have reservations about whether the method and its practical significance fully meet the standards of ICLR.

Under this year’s four-point scoring system, I found it difficult to decide between a score of 2 and 3. Nevertheless, in light of the authors’ careful and detailed response, as well as the extensive experiments and demonstrated model effectiveness, I have decided to raise my score to 3.

I would like to clarify that this decision is primarily based on the authors’ thorough response, the breadth of the experimental evaluation, and the empirical effectiveness of the model, rather than on a strong conviction regarding the method’s novelty or practical impact. I hope the authors can understand my final score and evaluation in the context of this year’s scoring scheme.

If the paper is unfortunately not accepted, I sincerely wish the authors success with their future submissions.

Reviewer_sxbAOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ST-HalfNet for tensor restoration under sparse corruption. It learns a shared orthogonal transform and nonuniform slice ranks to construct a low-rank reference, and uses Half thresholding to estimate sparse corruption. Since the residual may also contain genuine image details missed by the low-rank model, the method applies direction-aware robust local regression to predict these residual signal components before the final corruption estimation. The paper further provides rank-allocation analysis and a restoration error bound. Experiments on CAVE and ICVL demonstrate strong performance under several sparse corruption types.

Critical strengths
  1. The paper highlights an important limitation of low-rank restoration: valid fine-scale structures may be lost by the low-rank approximation. Using the low-rank component as a reference for corruption estimation, together with residual prediction, helps preserve such details.

  2. The experimental results are strong, with ST-HalfNet achieving the best reported PSNR across multiple corruption settings on CAVE and ICVL. The ablation studies also clearly examine the contributions of the main components. They further provides theoretical analysis of optimal rank allocation and a restoration error bound, which helps clarify the roles of different error sources.

Critical weaknesses

The main limitation is that the method still largely follows the conventional RPCA paradigm. It assumes that the observation can be decomposed into a low-rank reference and a sparse corruption term, and most of the proposed components can be viewed as refinements of this framework: the transform and rank allocation improve the low-rank representation, Half thresholding improves sparse estimation, and local residual prediction attempts to recover signal components missed by the low-rank model. However, it is not fully clear whether the latter issue reflects an intrinsic limitation of the low-rank-plus-sparse model or simply an insufficiently accurate low-rank reference. Moreover, once locally predictable residuals are treated as genuine signal, structured or spatially correlated noise may also be mistakenly preserved. Therefore, the paper appears more as an engineered extension of RPCA than a fundamentally new restoration paradigm.

Questions for authors
  1. The proposed residual predictor is motivated by the fact that the low-rank reference may fail to represent fine image details. However, could this omitted signal mainly result from an insufficiently accurate low-rank representation, e.g., a suboptimal transform, rank allocation, or factor estimation? If the low-rank reference were sufficiently well fitted, would the residual-prediction module still provide a significant benefit? It would be helpful to separate the gain due to improving the low-rank reference from the gain due to predicting signal outside the low-rank family.

  2. The residual prediction module assumes that locally predictable structures correspond to genuine image details, while locally inconsistent components are more likely to be sparse corruption. However, some structured or spatially correlated noise may also exhibit local predictability. Could the authors analyze whether such noise patterns may be mistakenly preserved as image details?

3 comments
AuthorsOpenReview ↗

Response to Reviewer sxbA

Thank you for the questions about the distinction between reference accuracy and signal recovery, and the effect of predictable corruption.

Weakness 1 / Question 1. Reference accuracy and residual prediction

Our starting point is that a real image need not be exactly low rank. Even an accurately fitted low-rank component can miss textures and boundaries that remain visible in the corrupted observation. We therefore distinguish the low-rank reference from the clean image:

=\mathcal{L}+\mathcal{D}^\star+\mathcal{S}^\star.$$ The term $\mathcal{D}^\star$ is genuine signal omitted by the reference. Treating the whole residual $\mathcal{Y}-\mathcal{L}$ as corruption can remove that signal. ST-HalfNet predicts it before estimating sparse corruption, then restores $\widehat{\mathcal{X}}=\mathcal{Y}-\widehat{\mathcal{S}}$. The output can therefore preserve detail outside the low-rank family. Section 5 gives a sufficient condition for restoration to be more accurate than every member of the chosen reference family. Yes, the experiments show a benefit even when the computed reference is already close to the best approximation in its family. On ICVL at 10% corruption, these references achieve 43.92 and 44.04 dB, respectively. Keeping the computed reference, thresholds, and candidate support unchanged, prediction improves restoration from 51.57 to 55.81 dB (Table 3). The reference itself has not been refitted. To distinguish the remaining sources of error, let $\mathcal{L}^\star$ be the best clean approximation in the fixed family. Then $$\mathcal{Y}-\mathcal{L}_K=\mathcal{S}^\star+ \underbrace{(\mathcal{X}^\star-\mathcal{L}^\star)}_{\text{representation error}}+ \underbrace{(\mathcal{L}^\star-\mathcal{L}_K)}_{\text{factor-estimation error}}.$$ Even exact factor estimation can leave genuine signal in $\mathcal{X}^\star-\mathcal{L}^\star$. A stronger control replaces the reference with $\mathcal{L}^\star$ on all eight CAVE test images, removing factor-estimation error altogether. Thresholds, candidate support, the directional guide, and learned parameters remain fixed. Prediction still improves mean PSNR by 1.25 and 0.58 dB at 10% and 30% corruption; at 50%, it lowers PSNR by 1.41 dB with the same fixed predictor (Appendix D.3, Table 8). The positive gains with factor-estimation error removed establish a contribution from predicting signal outside the chosen family, separate from improving its representation or fitting its factors. Joint transform and rank learning improves the reference, with optimal rank allocation for each fixed transform (Proposition 1; Tables 4 and 9). Half reduces shrinkage of nonzero corruption estimates; its separately trained Soft control loses 3.36–6.93 dB (Table 4). Figure 2 shows how these components estimate corruption while preserving signal outside the reference. **Weakness 1 / Question 2. Structured corruption** Spatial correlation alone does not eliminate the benefit of prediction. On all eight CAVE test images at 30% additive sparse Gaussian corruption, prediction improves mean PSNR by 0.83–1.02 dB for dispersed corruption and radius-one disks, under both independent and correlated amplitudes. For dense lines and larger disks, however, coherent contamination can enter the local fit and prediction reduces accuracy. The comparison covers dispersed locations, one-pixel horizontal and vertical lines, and disks of radii one and three, each with independent or spatially correlated amplitudes. Density and amplitude variance are held fixed; all methods receive the same observations and retain their trained weights and solver settings (Appendix D.6, Table 10 and Figure 10). The error measurements explain the distinction. Prediction reduces error at uncorrupted entries in all ten conditions, but increases error at corrupted entries for lines and larger disks. It still protects genuine signal, while contaminated neighboring responses bias the local prediction and make the final corruption estimate too small. Appendix A.5 identifies this contribution from corrupted neighbors. The relevant condition is therefore the reliability of the fitting neighborhood, not simply whether the corruption amplitudes are spatially correlated. Full ST-HalfNet exceeds all three external baselines in nine of the ten conditions; correction without prediction exceeds them in all ten. The exception for the full model is larger correlated disks: 33.04 dB, compared with 36.92 dB for DFT-TRPCA and 41.05 dB without prediction. Thus the experiment separates the strength of the global reference and sparse correction from the local conditions under which prediction adds further accuracy. The experiment is included in the supplementary attachment and the [anonymous repository](https://anonymous.4open.science/r/ST-HalfNet-7591/).

Thank you for the detailed response and additional experiments. Your explanations have alleviated some of my concerns. However, the failure cases, particularly the negative gain at 50% corruption, suggest that the predicted residual does not necessarily correspond to genuine signal outside the low-rank representation.

Additionally, could the authors clarify how L⋆\mathcal{L}^\star is obtained? Overall, these remaining ambiguities make it difficult for me to change my current rating.

Thank you for the follow-up. We clarify how L⋆\mathcal L^\star is obtained and directly test whether adding the predicted residual to this best reference brings it closer to the clean image.

Construction of L⋆\mathcal L^\star. We fix the transform QQ and slice ranks r\mathbf r learned from the training images. For each clean test cube X⋆\mathcal X^\star, we apply TQ\mathcal T_Q, retain the leading rsr_s singular components of each transformed slice, and apply the inverse transform. Because QQ is orthogonal, this computes

L⋆∈argmin⁡L∈M(Q,r)∥X⋆−L∥F2.\mathcal L^\star\in\operatorname*{argmin}_{\mathcal L\in\mathcal M(Q,\mathbf r)}\lVert\mathcal X^\star-\mathcal L\rVert_F^2.

Table 8 uses this reference in place of LK\mathcal L_K to remove factor estimation error within the fixed family. The comparison retains the model parameters, thresholds, candidate support, and directional guide from the ordinary run.

Residual prediction and the 50% result. We compare the clean image with the reference alone, L⋆\mathcal L^\star, and with the reference plus the predicted residual, L⋆+D^\mathcal L^\star+\widehat{\mathcal D}. This directly tests prediction against the known omitted signal D⋆=X⋆−L⋆\mathcal D^\star=\mathcal X^\star-\mathcal L^\star, since

∥L⋆+D^−X⋆∥F2=∥D^−D⋆∥F2.\lVert\mathcal L^\star+\widehat{\mathcal D}-\mathcal X^\star\rVert_F^2=\lVert\widehat{\mathcal D}-\mathcal D^\star\rVert_F^2.

Here D⋆\mathcal D^\star is the evaluation target; D^\widehat{\mathcal D} is computed from the contaminated residual Y−L⋆\mathcal Y-\mathcal L^\star.

Table 8(b) reports MSE against the clean image, computed over each full cube and averaged equally across all eight CAVE test images. Lower is better. The sum L⋆+D^\mathcal L^\star+\widehat{\mathcal D} is used here to assess signal prediction; ST-HalfNet restores the image as Y−S^\mathcal Y-\widehat{\mathcal S}.

Salt-and-pepper corruptionReference alone L⋆\mathcal L^\starReference + predicted residual L⋆+D^\mathcal L^\star+\widehat{\mathcal D}
10%0.000047910.00003388
30%0.000047910.00003806
50%0.000047910.00006342

For each image, the reference is identical across the three corruption levels. At 10% and 30%, adding the predicted residual reduces mean MSE by 29.3% and 20.6%, respectively. Because L⋆\mathcal L^\star is already the best approximation within the fixed family, this reduction directly supports recovery of part of the signal that the representation omits. At 50%, adding the prediction increases mean MSE by 32.4%, consistent with the negative restoration gain in Table 8(a).

An optimal low-rank reference removes factor estimation error but does not remove contamination from the neighboring residuals used for prediction. Section 5 includes prediction error in the restoration bound, and Appendix A.5 separates gain bias, local approximation error, and neighbor contamination.

Ordinary restoration estimates LK\mathcal L_K from the corrupted observation. With this reference, prediction improves mean PSNR in all six settings in Table 3, including 50%: from 36.958 to 37.072 dB on CAVE and from 42.403 to 44.298 dB on ICVL.

Separately, the learned reference and sparse correction provide effective restoration without local prediction: this output exceeds DFT-TRPCA, Tucker unfolding, and HSDT-M in all ten structured-corruption conditions in Table 10.

Reviewer_nXSaOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ST-HalfNet for tensor restoration under sparse corruption. It learns a shared orthogonal spectral transform and nonuniform slice ranks under a total rank budget, then uses unfolded scaled factor updates to estimate a low-rank reference. A robust local regression predicts signal omitted by that reference, and half-thresholding estimates corruption from the corrected residual. The output subtracts this estimate from the observation, allowing it to retain signal outside the reference family. The paper gives optimal rank allocation for a fixed transform and a conditional restoration error decomposition.

Critical strengths

The distinction between a low-rank reference and the final restored output is well motivated and supported. Table 3 compares the best clean approximation, the estimated reference, and the corrected observation, showing that restoration can exceed the reference family's approximation accuracy. The paired local-prediction comparison holds the trained network, reference, thresholds, and candidate support fixed, making the contribution of residual prediction particularly clear.

Critical weaknesses
  1. The real-image representation ablation does not isolate the incremental value of transform optimization. Table 4 and Figure 5 compare learned coordinates with nonuniform ranks against a fixed DCT with uniform ranks. They support the joint representation, but do not show how much of the gain requires optimizing the transform rather than allocating ranks adaptively. The synthetic evidence in Appendix D.1 is useful, but a real-image comparison with the initialized spectral eigenbasis and a fixed DCT, each using the same training-derived optimal rank allocation, would clarify this central technical contribution.

  2. The learned methods use different checkpoint-selection distributions. ST-HalfNet uses 10%, 30%, and 50% validation corruption; HSDT and Tucker use 20%, and ILRNet and MP-HSIR use 10% and 30% (Appendix C.3–C.4). This leaves uncertainty about the robustness of the high-density comparisons under a common selection objective.

Questions for authors
  1. How much of the real-image representation gain remains when the fixed DCT and the initialized spectral eigenbasis each use the same training-derived optimal rank allocation as the learned transform? Approximation error and representative restoration results under the same budget, recovery training, and selection rule would help identify the incremental value of optimizing the transform.

  2. Can the key learned comparisons be evaluated after choosing candidate checkpoints with a common validation density and aggregation rule, reporting the selected checkpoints and any changes in restoration/transfer performance? Please also clarify the actual ICVL training subset used for Tucker: the supplied default configuration specifies 20 training scenes, while the full training split contains 162. This would distinguish a configurable default from the runs underlying the reported table.

3 comments
AuthorsOpenReview ↗

Response to Reviewer nXSa

Thank you for the suggestions on separating the representation contributions and making model selection comparable.

Weakness 1 / Question 1. Transform and rank allocation

We agree that the original comparison combines two effects. We have added fixed DCT and the initial spectral eigenbasis, with ranks optimized separately for each basis using the same training references and total budget R=512R=512. Their CAVE recovery parameters are also trained separately with identical training and selection settings. The results show that rank allocation explains most of the improvement over uniform DCT, but does not remove the benefit of optimizing the transform: the latter adds 0.14–0.38 dB over the initialized spectral basis after both use optimal ranks.

RepresentationCAVE approximation errorICVL approximation errorCAVE PSNR, 10%30%50%
DCT, uniform ranks0.009780330.0164577141.6337.7831.58
DCT, optimal ranks0.000906640.0010005549.9944.8736.87
Initial spectral basis, optimal ranks0.000841620.0004862750.0444.9636.69
Optimized transform, optimal ranks0.000822460.0004590250.1945.1337.07

Approximation errors are mean relative squared errors over all eight CAVE and twenty ICVL test images (Appendix D.4, Table 9). Restoration results cover all eight CAVE test images (Section 6.3, Table 4).

The large improvement from the first to the second row shows why learning the ranks is important: equal allocation spends capacity on slices that contribute little to the approximation. With DCT fixed, optimal allocation raises CAVE PSNR at 10% corruption from 41.63 to 49.99 dB. Changing the basis then changes which spatial singular directions are worth retaining. Optimizing the coordinates further improves what the same budget can retain, including beyond the data-derived spectral initialization. Our contribution is this joint optimization, with Proposition 1 providing the optimal allocation for each fixed transform.

Weakness 2 / Question 2. Model selection

We agree that the validation criterion should be the same when comparing performance at high corruption levels. We have repeated model selection for every learned method on both datasets using the same full-resolution validation observations and mean PSNR at 10%, 30%, and 50% salt-and-pepper corruption, with equal weights for scenes and densities. Three selected models change, and their complete restoration and transfer results have been updated. One selected model is used for every restoration and transfer condition.

This selects epoch 5 for CAVE ILRNet, update 1,000 at learning rate 3×10−43\times10^{-4} for ICVL HSDT-M, and update 4,000 for ICVL MP-HSIR. The other seven selections retain the submitted parameters. CAVE ILRNet shows the largest changes: PSNR rises from 29.77 to 31.87 dB at 50% salt-and-pepper corruption (Table 1), and from 34.67 to 38.04 dB at 30% sparse Gaussian corruption (Table 2).

ST-HalfNet still gives the highest mean PSNR in every restoration and transfer condition in Tables 1–2 and Appendix E.1. At 50% corruption, it remains ahead of the strongest comparator by 2.84 dB on CAVE and 9.52 dB on ICVL. Thus the high-density advantage persists after the comparisons use a common selection criterion. Appendix C.3–C.4 specifies the rule.

Question 2. Tucker training data

The reported Tucker calibration did use twenty ICVL training scenes; the default was not overridden in those runs. We repeated calibration using all 162 training scenes with the same architecture and self-supervised objective, five epochs at each of the two learning rates (10−310^{-3} and 5×10−35\times10^{-3}), and the common validation criterion. Validation selects the calibration starting checkpoint, so Tucker's main restoration and transfer results remain unchanged. The configuration and Appendix C.4 now specify the full training set. The experiments and reproduction commands are included in the supplementary attachment and the anonymous repository.

Thank the authors for providing a detailed reply and additional experiment results. After reading the reply, my concern about the practical value of transform optimization still remains. Specifically, I found the revised ablation separates the effect from rank allocation. However, once ranks are optimized for each basis, the learned transform improves restoration over the initial spectral basis by only small number of db (0.14–0.38). On ICVL, Table 9 reports approximation error without a matched restoration comparison, so it, in my point of view, remains unclear whether this modest gain carries over to another dataset. I found it difficult to change my current rating if these ambiguities remain.

Thank you for clarifying the remaining concern. We have now completed the matched restoration comparison on ICVL and present it alongside CAVE below. With ranks optimized separately for each basis, optimizing the transform beyond spectral initialization improves final restoration on both datasets.

For each dataset, all three bases use a total rank budget of R=512R=512, with ranks optimized separately from the same training references. Restoration parameters are trained separately using the same data, training budget, and validation criterion. All variants are evaluated on identical corrupted observations at the original image resolution.

DatasetBasis, with optimal ranks10%30%50%
CAVEFixed DCT49.99044.86736.866
CAVEInitial spectral basis50.04244.95936.695
CAVEOptimized transform50.18645.13137.072
ICVLFixed DCT53.75848.65642.246
ICVLInitial spectral basis55.67950.52844.154
ICVLOptimized transform55.80750.67244.298

Values are mean test PSNR in dB under salt-and-pepper corruption. Further optimization beyond the initial spectral basis gains 0.143, 0.172, and 0.377 dB on CAVE, and 0.128, 0.145, and 0.144 dB on ICVL. Relative to the initial spectral basis, both PSNR and SSIM improve on every test image at each corruption level: all eight CAVE images and all twenty ICVL images.

The initial spectral basis is itself estimated from the training images and forms the starting point of our transform learning procedure. Relative to fixed DCT, the final learned transform gains 0.20–0.26 dB on CAVE and 2.02–2.05 dB on ICVL. This comparison measures the overall benefit of learning the coordinates, including spectral initialization; the comparison with the initial spectral basis measures the additional benefit of optimizing the approximation objective.

Rank allocation determines how much capacity each transformed slice receives within a given basis. Transform optimization adjusts the basis to reduce approximation error under the same total rank budget (Section 3, Eq. (4)).

Table 4 reports the restoration results on both datasets; Appendix D.4 and Table 9 give the corresponding approximation errors and gains between bases. The two ICVL control checkpoints are included at pretrained/controls/dct_optimal/icvl/model.pt and pretrained/controls/spectral_init_optimal/icvl/model.pt in the updated supplementary source package.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.