acceptodds
Withdrawn from ICLR 2027

UWHFormer: A Spectrally Coherent Transformer for Underwater Hyperspectral Image Restoration

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Hyperspectral imaging captures dense spatial–spectral information, enabling fine-grained material identification and scene understanding. Underwater acquisition, however, is degraded by wavelength-dependent attenuation, non-uniform scattering, insufficient illumination, and sensor noise, which jointly corrupt spatial structures and spectral signatures. We formulate underwater hyperspectral image (HSI) restoration as a physics-guided spatial–spectral reconstruction problem and propose UWHFormer, a spectrally coherent Transformer for restoring real-world UnderWater HSIs. UWHFormer first employs a water-conditioned state modeling module to estimate physically interpretable, wavelength-dependent degradation states. A physics-guided restoration transformer then integrates these states with image features to condition residual reconstruction, enabling adaptive compensation for attenuation and scattering while retaining reliable information from the observations. We further introduce a spectral–spatial refinement module that jointly aggregates spectral and spatial neighborhoods to suppress local artifacts without distorting spectral profiles. For systematic evaluation, we construct a paired real-world underwater HSI dataset spanning diverse objects, imaging distances, and degradation levels. Extensive experiments on this dataset show that UWHFormer achieves state-of-the-art performance, with a PSNR of 39.70, SSIM of 0.860, SAM of 1.339 and ERGAS of 4.34, which indicates improvements in both spatial reconstruction and spectral fidelity across a range of underwater conditions.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Underwater hyperspectral imaging, Image restoration, Physics-guided learning, Spectral-spatial modeling, Transformers

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_tHAzOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies underwater hyperspectral image (HSI) restoration, aiming to recover both spatial details and spectral information from degraded underwater HSIs. The proposed UWHFormer estimates water-related states from the degraded input based on an underwater image formation model, and uses them to guide a Transformer-based restoration network, followed by a refinement step. The authors also collect a real paired dataset, with short-exposure observations as inputs and long-exposure captures as references, and compare the method with existing HSI denoising methods, together with ablations.

Critical strengths

Real paired data are scarce in this area. The authors collect such a dataset and plan to release it, which could benefit future research. 2. An intuitive design with consistent results

Guiding restoration with a physical model is intuitive, and the figures are clear. The method achieves the best PSNR, SAM, and ERGAS among the compared methods, with SSIM on par with HSDT (Table 1), and each module brings improvements in the ablation (Table 2).

Critical weaknesses

The role of the physics guidance could be better supported

The data may not directly verify compensation for water-induced degradation. The abstract lists wavelength-dependent attenuation, scattering, insufficient illumination, and sensor noise as underwater degradations (L014–016), and states that the method compensates for attenuation and scattering (L022–023). However, both inputs and references are captured underwater and differ only in exposure time (L245–246, L357–360), so the data mainly reflect insufficient illumination and noise (L358–359). Since attenuation and scattering are also present in the references, whether the method compensates for them cannot be directly assessed. The estimated physical states are not quantitatively validated. They are described as "physically interpretable" (L020, L092), while the paper also notes that they serve as latent guidance rather than calibrated physical measurements (L245–246). The ablation adds each module as a whole (Table 2), so it is difficult to tell whether the gains come from the physics-based design (e.g., the physical bias in PSA) or from the module itself (e.g., an additional spectral attention layer, which also increases computation; App. F). Since physics guidance is the central contribution of the paper, these points are important for assessing it. 2. The experimental evidence is somewhat limited

The comparison includes four general HSI denoising methods from 2020–2023 (L417–422), while more recent methods mentioned in the related work (L151–153) are not compared. The evaluation scale is limited. The test set contains 24 samples (L360–361). The claim of robustness to imaging-distance shifts (L097–099) is evaluated only on an additional 6.3 m subset with four scenes, and only against the degraded input rather than other methods (App. C). As a result, the current experiments do not yet fully support the claims on performance and generalization.

Questions for authors

Scope of the task. Since the references are also captured underwater, only with longer exposure, which degradations does the method mainly address? How should the claimed compensation for attenuation and scattering be understood? Role of the physics guidance. Could you add controlled ablations, for example PSA without the physical bias, or WCS replaced by a generic conditioning encoder of similar size? In addition, do the estimated states (e.g., transmission) change with imaging distance as expected, for instance being lower on the 6.3 m subset? Comparisons and robustness. Could you also report the baselines' results on the 6.3 m subset? If feasible, could you include more recent methods as well? Data and other clarifications. Could you describe the acquisition environment, e.g., a water tank or open water, the water type, and the illumination? How does the task setting differ from that of Li et al. (2024) (L185–186)? Why does the SSIM of the full model differ between Table 1 (0.860) and Table 2 (0.865)?

Reviewer_PwNMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents UWHFormer, a Transformer-based method for underwater hyperspectral image restoration. It combines a water-conditioned state modeling module, physics-biased spectral attention, and a spectral-spatial refinement block. The authors also collect a real-world dataset with paired short- and long-exposure observations. The reported results are encouraging, but I have some concerns about the physical interpretation of the restoration task and whether the experiments sufficiently establish the contribution of the proposed design.

Critical strengths
  1. The real-world dataset is a useful contribution.

Collecting paired underwater hyperspectral observations is valuable, particularly given the difficulty of obtaining real underwater data with spatially aligned references. The object-disjoint split is also a reasonable choice for testing generalization to unseen objects, rather than relying on random patch-level splits.

  1. The method achieves consistent improvements across several metrics.

UWHFormer outperforms the evaluated baselines in PSNR, SAM, and ERGAS. The ablation study also shows progressive improvements as WCS, PSA, and SSR are added. While the gains are not large, the results suggest that the proposed components are useful for the restoration task.

Critical weaknesses
  1. The restoration target does not fully support the paper's physical claims.

The dataset uses 10 ms underwater observations as inputs and 50 ms underwater observations as references. A longer exposure can improve the signal-to-noise ratio, but it does not remove wavelength-dependent absorption or scattering. As a result, the current evaluation appears to measure restoration from short- to long-exposure observations more directly than recovery of the underlying scene spectrum before underwater degradation.

The paper also states that the estimated water states are latent guidance variables rather than calibrated physical measurements. This makes the claim of physically interpretable degradation modeling less convincing. The authors should clarify how exposure differences are handled during preprocessing and explain what aspects of underwater degradation the reference data actually allow the model to learn.

  1. The specific contribution of the proposed physics-guided design remains unclear.

The baseline Transformer already achieves 39.31 dB PSNR, close to the full model's 39.70 dB. Adding WCS alone improves PSNR by only 0.10 dB. Although the full ablation shows that the components improve performance, it does not establish that the proposed physical guidance is more effective than a simpler learned conditioning mechanism or conventional spectral attention.

There is also a small reporting inconsistency: Table 1 reports an SSIM of 0.860 for the full model, whereas Table 2 reports 0.865. Please clarify this difference.

Questions for authors
  1. How are the 10 ms inputs and 50 ms references normalized to account for their different exposure times? Are the references intended to represent higher-quality underwater measurements, or are they used as approximations to the underlying scene spectra?

  2. Can the authors provide a controlled comparison against a parameter-matched model using conventional spectral attention or generic learned conditioning? This would help determine whether the gains come specifically from the proposed physical guidance rather than from additional feature modulation.

Reviewer_LSpeOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This manuscript formulates underwater hyperspectral image (HSI) restoration as physics-guided spatial–spectral reconstruction. Specifically, the proposed method includes three modules: Water-Conditioned State modeling (WCS) estimates wavelength-dependent attenuation, transmission, depth, and reliability under the underwater image formation model; Physics-biased Spectral Attention (PSA) treats spectral bands as tokens and biases attention by attenuation difference and observation reliability; Spectral-Spatial Refinement (SSR) applies a residual 3D-conv correction. Moreover, a paired real-world dataset with 10 ms degraded inputs and 50 ms references is built for performance evalution. The experimental results indicate that the proposed method outperforms the SOTA method reported in the manuscript.

Critical strengths
  • Real dataset contribution. Paired underwater HSI data is genuinely scarce. The short-exposure-input / long-exposure-reference protocol with object-disjoint splits is well designed, and the supplementary transmission estimates (≈93% at 450 nm vs. ≈36% at 600 nm over 4.2 m) quantify the band-wise difficulty. If released, this is the most citable contribution of the paper.
  • The concept of water-conditioned state modeling of implementing the physica prior is interesting, simple and interpretable.
  • Acceptable runtime efficiency. 2.944M params, 28.8G FLOPs, 38.9 ms at 96×224×16, well below SERT's 87.4G FLOPs.
Critical weaknesses
  • Task motivation is disconnected from the evaluation. The introduction section motivates the work by marine perception, material recognition, and ecological monitoring, yet no downstream evaluation such as detection/classification exists. The performance evaluation of underwater HSI enhancement tasks only considers signal fidelity metrics such as PSNR/SSIM/SAM/RGAS, without considering semantic fidelity metrics for downstream tasks such as object detection.

  • The relevant work review is incomplete and outdated. The literature in the relevant work is from before 2024, and without covering the 2024-2026 wave of diffusion- and Mamba-based HSI restoration. The lack of a comprehensive review on HSI image enhancement and detection tasks makes it difficult to determine the cutting-edge nature of this work.

  • The setting, size and diversity of the dataset limit the contribution.

    • For example, the input/GT pair in the dataset only considers exposure parameters, with low exposure set to 10ms and high exposure set to 50ms. Why not add multiple exposure settings, such as 20ms and 30ms? Moreover, the ground truth is itself degraded. Both input and reference are captured underwater, so wavelength-dependent attenuation affects both. The supplementary admits the 600 nm reference retains only ~36% signal. The learned mapping is therefore essentially 10 ms-->50 ms exposure correction, not "removal of underwater wavelength-dependent attenuation": information already lost in the reference cannot be recovered, undermining the abstract's claim of "compensating attenuation."
    • Foreground categories (coral-like objects, sponges, fabrics, stones, bottles, cans) are limited and per-category counts are unreported, so "diverse targets" is unsubstantiated.
  • Baselines are old and the comparison is unfair in both directions in the experimental results part. All four baselines are 2020–2023 denoisers. It cannot answer how much gain physics modules add on top of a current SOTA.

Questions for authors
  • Since the reference itself suffers underwater attenuation (~36% remaining at 600 nm), how do you argue the model recovers scene-true spectra rather than "50 ms underwater spectra"? Is there any external ground truth for any band?
  • What do Restormer / NAFNet achieve when directly applied to the 16-band cube under the same split? This determines the net contribution of the physics modules.
  • Why only 10 ms / 50 ms? How does the model perform at 20 ms / 30 ms?
  • How many samples per category? Can you report downstream detection/classification accuracy on degraded vs. restored vs. GT to substantiate task significance?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.