acceptodds
Withdrawn from ICLR 2027

HSRest: Benchmarking Hyperspectral Satellite Image Restoration Across Tasks, Data Regimes, and Regions

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Hyperspectral satellite images provide valuable observations of the Earth, but their restoration is difficult to evaluate consistently because existing studies differ in data selection, degradation protocols, training regimes, and evaluation metrics. We introduce HSRest, a benchmark for systematic evaluation of hyperspectral image (HSI) restoration, comprising 23,228 non-overlapping 128 × 128 patches extracted from 946 EO-1 Hyperion scenes across 26 countries and six continents. HSRest covers three representative restoration tasks, namely Gaussian denoising, spatial super-resolution, and structured inpainting, with multiple degradation levels, and organizes the same corpus under three complementary evaluation settings. Setting I provides country- and scene-disjoint train/validation/test partitions for standard evaluation. Setting II defines fixed 10%, 20%, and 30% training subsets for low- resource evaluation. Setting III partitions all scenes into three geographically distinct groups for evaluation under geographic distribution shifts. We benchmark recent restoration methods using complementary spatial, spectral, and efficiency measures. Across tasks, training budgets, metrics, and held-out regions, method rankings change substantially, showing that conclusions drawn from a single split, degradation level, or reconstruction metric may not reliably characterize restoration performance. HSRest releases fixed partitions, degradation protocols, metadata, evaluation code, and baseline configurations, providing a reproducible testbed for data-efficient and region-robust HSI restoration.

Primary area:
datasets and benchmarks
Keywords:
Hyperion, denoising, inpainting, super resolution, hyperspectral, image restoration

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_BkppOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The main contribution of this paper is the proposal of the HSRest hyperspectral satellite image inpainting benchmark, which integrates standardized data partitioning, limited-data training, and cross-geographical testing into a unified evaluation protocol. This benchmark provides a reusable basis for comparing different methods under various inpainting tasks and evaluation conditions.

Critical strengths
  1. Addressing the differences in data selection, degradation methods, training and testing partitioning, and evaluation metrics in existing research, this paper provides a unified data partitioning and evaluation process, contributing to improved consistency and reproducibility in method comparisons.

  2. Cross-geographical experiments require the model to be trained on one geographic group and tested on other groups with non-overlapping countries and scenes. This design allows for a more direct examination of the model's performance under varying geographic distributions.

  3. The paper covers tasks such as denoising, super-resolution, and image inpainting, and compares the performance of methods under different training data sizes, degradation levels, geographic partitioning, and evaluation metrics, helping to reveal performance differences that are difficult to reflect with a single experimental setup.

Critical weaknesses
  1. In limited-data experiments, the training sample size and geographic composition change simultaneously.

Appendix Table 10 shows that 10% of the training subset comes from Mongolia and Canada, while the 20% and 30% subsets use different combinations of countries. Therefore, performance differences under different data proportions may be influenced by the sample size, land cover composition, texture features, and acquisition conditions simultaneously, and cannot be solely attributed to the amount of training data. The authors have acknowledged this in Section 4.3 and the Limitations section, defining this setting as a robustness test under limited data conditions. It is recommended to further add control experiments that maintain similar country composition and proportions, only changing the sample size, to more clearly analyze the impact of data scale. Furthermore, Table 10 summarizes the 20% subset into 3 countries and 3,243 tiles, but the column explicitly labelling the 20% of countries and tile counts is inconsistent with the summarized values; the authors are advised to verify and clarify this.

  1. Details regarding the input construction and task adaptation of some baseline models still need clarification.

For example, Fusformer originally used low-resolution hyperspectral images and high-resolution multispectral images as input, while the super-resolution degradation protocol in this paper primarily describes the downsampling process of hyperspectral images. While the paper states that methods within the same task use the same input, it fails to clearly explain how the second input for the Fusformer was obtained, whether the original model was modified, or whether different methods obtained equal amounts of input information. These details affect the fairness of baseline comparisons and the reproducibility of the experiments. It is recommended that the input, adaptation method, and training configuration of each model be clearly provided in the main text or appendix.

  1. The analysis of ranking changes can be further quantified.

The paper demonstrates the changes in leading methods under different conditions through specific experimental results, thus providing numerical evidence for the observation that "rankings change." However, the existing analysis does not systematically report the magnitude and frequency of ranking changes, and lacks repeated experiments or uncertainty estimates, making it difficult to determine whether some ranking changes with small differences are stable. Furthermore, cross-regional results are mainly summarized by geographical group, and it is not yet possible to determine which countries or land cover types primarily drive the changes. It is recommended to supplement the analysis with cross-conditional ranking consistency indicators, multi-random seed results, and analyses categorized by country or land cover type, and to indicate which methods remain relatively stable under multiple conditions.

Questions for authors

See weakness

Reviewer_ySW8OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces HSRest, a standardized benchmark for hyperspectral satellite image restoration, constructed from 946 EO-1 Hyperion Level-1Gst scenes. HSRest is built from 946 EO-1 Hyperion Level-1Gst scenes collected across 26 countries on six continents. There are 23,228 patches in total, each 128 × 128 pixels with 175 spectral bands. The paper reports experiments on denoising, spatial super-resolution (×2/×4/×8), and structured inpainting. Experiments are conducted using the full training set, subsets containing 10%, 20%, and 30% of the training data, and cross-geographic transfer. A benchmark covering these restoration tasks would be useful to the hyperspectral imaging community. However, I have serious concerns about the degradation models and the way the reference targets are constructed, which undermine the reliability of the evaluation. In addition, the selection and adaptation of baseline methods are insufficiently justified, the low-resource data splits introduce confounding effects due to geographic distribution without employing stratified sampling, and numerous numerical anomalies appear in the experimental result tables. Consequently, the methodological rigor and experimental reliability of the current manuscript are insufficient to support its publication.

Critical strengths
  1. For the remote sensing hyperspectral image restoration task, this paper proposes a standardized evaluation benchmark, HSRest, based on EO-1 Hyperion satellite data, making a valuable attempt to address the lack of unified evaluation protocols across denoising, spatial super-resolution, and image inpainting tasks.
  2. Using a dataset comprising 23,228 image patches, the authors designed three experimental settings: full-data training, low-resource training, and cross-geographic transfer. Combined with various spatial and spectral metrics, the paper conducts relatively comprehensive generalization experiments.
  3. The data processing pipeline and experimental settings are clearly described. The charts and metadata statistics provided in the appendix are intuitive, facilitating easy comprehension and data traceability.
Critical weaknesses
  1. There is a significant flaw in the degradation modeling. The authors define the MNF-denoised result as the Ground Truth, yet they apply synthetic Gaussian noise to the raw observations rather than to this reference image. This means the input images contain both the native sensor noise and the additionally applied noise, creating a misalignment with the Ground Truth target.
  2. The Ground Truth definitions are contradictory across different tasks. For instance, regarding the "Target" in Figure 2, the authors use the MNF-processed image as the Ground Truth for the Denoising task, whereas they use the raw, un-denoised original image as the Ground Truth for the Super Resolution and Inpainting tasks. This violates the fundamental definitions of image restoration. Such a fundamental contradiction in ground truth definitions across tasks raises serious concerns about the rigor of the paper.
  3. While the authors mention in the Related Work that hyperspectral restoration encompasses diverse paradigms such as CNNs, Transformers, state-space models (Mamba), and diffusion models, the baselines selected in Table 11 lack this diversity. All 5 baseline models for the super-resolution task are Transformers, and 4 out of 5 models for the denoising task are Transformers. Highly representative recent approaches, such as Mamba-based methods and diffusion models for denoising/SR, are completely absent. Furthermore, the baselines for the Inpainting task lack the latest advancements from 2025-2026.
  4. For the Super Resolution task, two of the five selected baseline models—Fusformer (GRSL 2022) and PSRT (TGRS 2023)—were originally designed as dual-branch networks specifically for hyperspectral and multispectral image fusion. However, in Section 4.1, the task is defined as single-image spatial super-resolution without any auxiliary imagery. The authors fail to explain how they forcibly modified the dual-stream fusion architectures of Fusformer and PSRT to accommodate a single-input task. Stripping away or tampering with the secondary branch of a fusion model fundamentally changes its network architecture, rendering this practice highly questionable.
  5. A fundamental principle of evaluating data efficiency is to maintain a consistent data distribution while proportionally reducing the sample size. However, in the low-resource evaluation experiments (e.g., Appendix Table 10), the "10% training set" exclusively consists of data from Mongolia and Canada, while the "20% training set" exclusively consists of data from South Africa, Ethiopia, and Spain. This could have been completely avoided by employing stratified sampling.
  6. The paper suffers from careless writing and cross-referencing errors. For example, in Section 4.1, immediately after introducing spatial super-resolution, the text abruptly states: "Complete inpainting results across the six corruption cases are reported in Table 16." Similarly, Section 4.3 claims that "The complete super-resolution results are reported in Tables 14 and 16," but Table 16 actually presents the Inpainting results, while Table 15 contains the super-resolution metrics.
  7. There are multiple numerical anomalies and potential copy-paste errors in the experimental results. For instance, in Tables 4 and 16, the Inpainting task for missing 50% pixels reports a zero-error result with a PSNR of inf and RMSE of 0.000. In Table 13, the ERGAS value reaches an abnormal 14817, and the RND model reports a SAM of 90.000 and CC of 0.000, clearly indicating training collapse, yet the authors did not investigate this. Furthermore, in Table 12, all four metrics for VolFormer under the 10% data, σ=90\sigma=90 setting are identical to those under the 30% data, σ=70\sigma=70 setting. Similarly, under the 30% data setting, all metrics for the SST model at σ=10\sigma=10 and σ=90\sigma=90 are exactly identical. These glaring discrepancies raise serious doubts about the authenticity of the experimental results.
  8. There are typographical errors in the main text. In lines 262 and 269, the word "Table" is missing when cross-referencing appendix materials, resulting in phrases like "summarized in 9" and "detailed in 10".
Questions for authors
  1. Why does the author add synthetic Gaussian noise to the original noisy image (X), rather than to the denoised reference image (X_{\mathrm{ref}}), in the denoising task? Furthermore, how does the author justify treating the original image (X) as too noisy to serve as ground truth in the denoising task, while directly using the same un-denoised, noisy image (X) as the ground-truth target for supervision in the super-resolution and image inpainting tasks?
  2. Fusformer and PSRT are dual-branch networks specifically designed for hyperspectral–multispectral image fusion. How exactly does the author modify their dual-branch architectures to enable training and evaluation for single-image super-resolution in the absence of any auxiliary multispectral image input?
  3. Could the author explain why several sets of results in Tables 12 and 13 contain exactly identical values? For example, all four metrics reported for VolFormer under the 10% data setting at (\sigma=90) are identical to those under the 30% data setting at (\sigma=70). Similarly, all metrics reported for SST under the 30% data setting at (\sigma=10) are identical to those at (\sigma=90). How did these results arise? In addition, why do PSNR (=\mathrm{inf}) and RMSE (=0.000) appear under the Missing50 condition in Tables 4 and 16?
Reviewer_jVMgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces HSRest, a benchmark for hyperspectral satellite image restoration constructed from EO-1 Hyperion imagery. The dataset contains 23,228 non-overlapping patches from 946 scenes across 26 countries and supports three restoration tasks: Gaussian denoising, spatial super-resolution, and structured inpainting. The authors establish standardized evaluation protocols covering full-data training, low-resource settings, and cross-geographic generalization, and benchmark existing restoration methods using spatial, spectral, and efficiency metrics. The results highlight changes in method rankings across degradation levels, training regimes, and geographic regions.

Critical strengths

The paper addresses a relevant problem in hyperspectral image restoration. Existing studies often adopt different datasets, degradation settings, and evaluation protocols, making direct comparison difficult. HSRest attempts to improve standardization through fixed scene- and country-disjoint partitions, consistent restoration tasks, and complementary spatial and spectral evaluation criteria. The geographic coverage and explicit low-resource settings are potentially valuable for studying restoration robustness beyond conventional small-scale benchmarks.

The benchmark provides relatively broad experimental coverage. The authors evaluate multiple restoration methods across denoising, super-resolution, and inpainting, and consider different degradation severities, training-data regimes, and geographic groups. The inclusion of spectral metrics alongside conventional image-quality measures is particularly appropriate for hyperspectral restoration. The paper also acknowledges several important limitations, including the use of a single sensor, synthetic degradations, and imperfect denoising references.

Overall, I appreciate the motivation to establish a more systematic and reproducible evaluation framework for hyperspectral restoration. Nevertheless, I am not yet convinced that the current benchmark construction and experimental analysis are sufficiently reliable and distinctive to support acceptance.

Critical weaknesses

The reliability of the reference construction and the consistency of the evaluation protocols are not sufficiently established. For denoising, the benchmark uses MNF–HySIME reconstructions as pseudo-ground-truth references rather than independently acquired clean observations. Although this limitation is acknowledged, the paper does not adequately justify why this particular reference construction is preferred over alternative denoising approaches, or investigate how its potential spectral and spatial biases affect model rankings. This is especially important because models may be rewarded for reproducing the characteristics of the reference estimator rather than recovering physically faithful signals. Moreover, the three restoration tasks employ different reference constructions, while some super-resolution baselines, such as Fusformer and PSRT, were originally developed for HSI–MSI fusion and may require additional high-resolution observations. The paper does not sufficiently clarify how these methods are adapted or whether all competing approaches operate under equivalent input information. The inpainting results also contain infinite or near-perfect PSNR values, raising further questions about metric aggregation and the interpretation of benchmark performance.

The distinctive contribution and experimental justification of the benchmark remain somewhat limited. Although HSRest combines multiple restoration tasks, low-resource evaluation, and geographic-transfer protocols, several underlying ideas have already been explored in existing hyperspectral datasets and restoration studies, including scene-disjoint evaluation in HySpecNet-11K, large-scale geographic coverage in SpectralEarth, and multi-task restoration evaluation in recent works. The paper does not sufficiently demonstrate why these protocols require a new Hyperion-based benchmark rather than extending existing hyperspectral resources. Furthermore, the main observations, such as changing method rankings across tasks, metrics, and training conditions, are relatively expected and provide limited new insight into restoration generalization. The low-resource subsets also vary in geographic composition, making it difficult to separate data-efficiency effects from distribution shifts. Finally, the complete dataset is not yet publicly available according to the linked repository, limiting independent verification of the benchmark and its reported results. These issues weaken the claimed reproducibility and broader research impact.

Questions for authors

What motivates the selection of MNF–HySIME for constructing the denoising reference images? How do the authors ensure that the resulting pseudo-ground-truth preserves meaningful spectral information rather than introducing algorithm-specific smoothing or reconstruction biases? Could the authors provide a reference-sensitivity analysis using alternative denoising methods and examine whether the benchmark rankings remain consistent?

Could the authors clarify the information provided to each super-resolution baseline? In particular, Fusformer and PSRT were originally designed for hyperspectral–multispectral fusion. Are additional high-resolution multispectral observations supplied during evaluation, and if so, how is information equivalence maintained across methods? If these models are adapted to single-image super-resolution, what modifications are made?

How are the infinite or near-perfect PSNR values in the inpainting experiments obtained and aggregated? Do they represent exact reconstruction over the entire evaluation set or individual samples with zero reconstruction error? A more explicit evaluation convention and corresponding finite-error statistics would help clarify these results.

What is the distinctive insight provided by HSRest that cannot be obtained by applying standardized restoration protocols to existing large-scale hyperspectral datasets such as HySpecNet-11K or SpectralEarth? Could the authors provide a more direct comparison with existing dataset protocols or demonstrate a previously overlooked restoration failure mode revealed specifically by HSRest?

For the low-resource experiments, the 10%, 20%, and 30% training subsets differ not only in size but also in geographic composition. Could the authors additionally evaluate nested or geographically balanced subsets to distinguish sample-efficiency effects from distribution shifts? The country-level subset assignments in Table 10 also appear inconsistent with the reported 20% sample count and would benefit from clarification.

Could the authors clarify the availability and completeness of the benchmark release? The linked repository indicates that the full dataset will be uploaded upon acceptance. Since the primary contribution is a reproducible dataset and evaluation framework, access to the complete reference data, fixed partitions, degradation scripts, and baseline configurations is important for independent verification during review.

Why are recent works such as MP-HSIR (ICCV 2025) and DAMP (ICML 2026) not discussed in the related work? Both have investigated unified HSI restoration across multiple degradation types and evaluated generalization under different restoration conditions, which partially overlaps with the motivation and evaluation scope of HSRest. Could the authors clarify how the proposed benchmark differs from and extends these existing evaluation settings, and what distinctive insights it provides beyond those already explored in recent multi-task restoration studies?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.