acceptodds
Withdrawn from ICLR 2027

Native-Resolution Readout and Support-Centered Geographic Conditioning for Diffusion Cloud Removal

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Auxiliary observations provide complementary information for cloud removal, but their utility depends on how spatial structure is represented at reconstruction. Spatial coarsening can suppress fine variation in continuous observation features; categorical maps require class composition and missing coverage to be represented distinctly. We view these as two forms of conditioning-representation mismatch and introduce native-resolution readout and support-centered geographic conditioning for diffusion-based cloud removal. Native-resolution readout exposes joint cloudy-optical and synthetic aperture radar (SAR) features directly to residual reconstruction, retaining spatial variation beyond an existing multiscale conditioning pathway. Geographic conditioning encodes land use and land cover (LULC) as target-grid class-area fractions with explicit coverage. A support-centered transformation removes the patch-average class composition while retaining within-patch spatial deviations; these features combine with SAR and elevation to update an RGB reconstruction embedding. Matched controls isolate representation effects in a 13-band pixel-space system on SEN12MS-CR and a latent-space RGB system on CloudRGB. With aligned auxiliary SAR channels, preserving the native field improves peak signal-to-noise ratio (PSNR) by 1.323 dB and reduces learned perceptual image patch similarity (LPIPS) by 0.0785 relative to spatially pooled conditioning. With geographic inputs fixed, support-centering improves full-image PSNR by 1.09 dB and reduces LPIPS by 0.0227 over uncentered area fractions in controlled CloudRGB gap reconstruction. These results support conditioning representations that preserve native spatial variation and explicitly encode valid support and local categorical deviations.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Cloud removal, diffusion models, multimodal remote sensing, native-resolution readout, geographic conditioning, support-aware representation

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_NevgOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper identifies the loss of spatial detail and the inadequate representation of class composition and valid coverage in categorical geographic information as problems in the use of auxiliary information for cloud removal. To address these problems, the authors propose native-resolution readout and support-centered geographic conditioning to improve the use of auxiliary information in diffusion-based reconstruction. The proposed methods achieve competitive reconstruction results on SEN12MS-CR and CloudRGB.

Critical strengths

The paper focuses on how auxiliary information is represented and used in cloud removal reconstruction, identifying two problems: the loss of spatial detail and the conflation of class composition with missing coverage. These problems are directly related to differences in the spatial structure and coverage of multisource remote sensing data, and the research motivation merits recognition.

Critical weaknesses
  1. The proposed native-resolution readout and support-centered geographic conditioning are two independent methods developed for different settings. The two methods lack methodological coherence, and the authors’ attempt to unify them under “conditioning-representation mismatch” is somewhat unconvincing. Specifically, the authors investigate the loss of fine spatial detail caused by spatial coarsening of continuous optical–SAR conditioning features on SEN12MS-CR, while studying the representation of class composition and valid coverage in categorical geographic conditions on CloudRGB. Developing a separate method for each specific dataset provides insufficient evidence of applicability to broader real-world settings.

  2. The paper lacks technical details related to diffusion. Are these updates performed within each sampling step or after sampling is completed? Section 3.1 mentions reconstructing RGB through a codec and a dense readout, but Figures 1 and 3 do not separately show the codec or explain its specific connections to the latent DiT, the readout embedding, and the final RGB output. The main text does not specify the configuration of the base restoration model. Although Figures 1 and 3 indicate SAR inputs, they do not specifically explain how SAR is encoded and fused in the base latent-space restorer. These omissions limit the reproducibility of the work.

Questions for authors

Please address the concerns raised in the Critical Weaknesses section.

Reviewer_MrvVOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes two different methods for deep learning-based cloud removal from satellite optical images. In the first case, the authors consider cloud removal happening in pixel space and propose having a second residual stream that uses SAR images at the full resolution to produce a correction image that is then added to the output of the original cloud removal network. In the second case, they consider the problem of geography-conditioning in latent-space cloud removal methods. They use class proportions in each patch of the Digital Elevation Model (DEM) or Land-Use/Land-Cover (LULC) maps to perform support-centering and then input them to a geography adapter, the output of which is added to the latent embedding of the cloud removal network, before the final reconstruction step. Experiments on SEN12MSCR and CloudRGB datasets show that, with these ideas, possibly SOTA cloud removal performance can be obtained.

Critical strengths

The paper shows improved results using (a) residual stream that takes in SAR images to provide a correction to the cloud removal model and (b) support-centering to improve the representation of the geography conditioning. Both these ideas lead to good improvements for commonly used cloud removal benchmarks.

The ideas themselves seem simple to implement and test, especially the support-centering, which is a strength of the method.

Critical weaknesses

The paper has several weaknesses.

  1. There are gaps in the experiments which make the conclusions in the paper not fully substantiated. Table 1 shows that the performance of the proposed method is indeed better than ECRFormer in terms of PSNR and SSIM, and worse in terms of LPIPS. However this does not show that ECRFormer features are important. Indeed, in Table 3, it is shown that the "Current SAR" row actually performs better than the "Full Native" method. It is my understanding that the "Current SAR" refers to a simple convolutional network instead of the proposed ECRFormer + Multiscale Encoder-Decoder in the residual stream. This actually shows that the simpler convolutional network works just fine and is lighter. I think the main contribution of the paper is that it is showing that a residual stream to correct the outputs from the main model is important, rather than having this ECRFormer Native Context + Multiscale Encoder-Decoder

  2. Also related to the above, in Table 3, the authors compare "Full native" to a "Pooled Native" version where the SAR image is pooled and then upsampled again before feeding it to the network. "Pooled native" is shown to be worse than "Full native". I cannot see the point of this experiment. Of course, destroying the information by pooling it is worse than the full resolution inputs. Why would we expect anything different? I am failing to understand why this is important. Again, as I said above, the main contribution of the paper is that it is showing that a residual stream to correct the outputs from the main model is important, not that "native resolution" is important. The authors keep mentioning native context throughout the paper, but I don't see any clarity on this or evidence for this.

  3. About DEM/LULC as inputs, the experiments show that the performance improves significantly with these inputs. But I think this is missing the point a little bit. If the DEM/LULC is already known well, then cloud removal itself would be less useful. The reason we want to do cloud removal is partly because we might not know the exact land use patterns before looking at the clean optical image. So, this is a little confusing, as my understanding is that the LULC would be derived partly from a clean optical image of the same area.

  4. Table 5 shows that the proposed support-centering outperforms the uncentered inputs by about 1 dB in terms of PSNR. However, I am not able to understand why support-centering improves the performance. The authors should try to provide an explanation for this.

  5. The paper mentions diffusion models throughout the paper, including the title. However, I don't see the methods tied in any way to diffusion-based cloud removal. The methods would work the same way even if the cloud removal model was not diffusion-based. This might be a good thing. Yet, the authors focus on diffusion-based cloud removal. I cannot see why.

  6. The paper reads as two different papers as the residual stream idea is not connected in any way to the improved geography conditioning idea. As far as I can see, even the two sets of experiments don't use these two ideas simultaneously.

  7. Finally, while the ideas may be novel, the amount of novelty is rather limited, in my opinion. I am not sure how interesting such specific ideas like having a residual stream or support-centering for geography conditioning is, in a conference like ICLR.

Questions for authors
  1. Please address the weaknesses that I have mentioned above.

  2. Could you please provide an intuitive explanation as to why support-centering helps?

  3. Could you provide some clarification why using LULC as a conditioning is applicable in a real-world case if the point of cloud removal is to see what the optical image is and then determine LULC?

  4. Could you please address the issues with the experiments I have mentioned above?

Reviewer_j2rYOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents methods to improve satellite image cloud removal by using native-resolution joint optical–SAR features and support-centered geographic conditioning to preserve spatial detail better and represent categorical data. The data include optical bands, SAR, DEM, and LULC data from NASA/HLS and Landsat, mosaicked and filtered for cloud and land coverage. Images are clipped to specific ranges, normalized, and sampled at 30-meter resolution, with acquisition-quality metrics and spatial metadata recorded. The models are evaluated on multiple datasets and show competitive performance, with lower MAE, higher PSNR, and better perceptual scores. Qualitative comparisons feature scene reconstructions, error maps, and display transformations, illustrating differences in scene fidelity. Most content and qualitative results appear in the supplementary material. A demo implementation is also provided.

Critical strengths
  1. Native-resolution readout: retain a 96-channel, 256 × 256 ECRformer intermediate field at the reconstruction readout instead of AvgPool16 pooling it to 16 × 16 and upsampling it.

  2. Support-centered LULC representation: encode target-cell class-area fractions, retain coverage separately, and subtract the patch-level class mixture so that the adapter receives local deviations from that mixture.

  3. The empirical evidence is internally coherent: full native features outperform the authors’ pooled-feature control by 1.323 dB PSNR, and support-centered LULC outperforms uncentered area fractions by 1.09 dB.

  4. The paper is unusually careful in holding several factors fixed in its two ablations. In particular, the native study fixes the feature source, readout capacity, initialization, objective, data order, and auxiliary SAR alignment; the LULC study fixes parent, adapter capacity, coverage, and update budget. Those controls show that the authors’ chosen representations can matter.

Critical weaknesses

Limited conceptual leap. The native readout is a direct high-resolution skip/conditioning path from a frozen optical–SAR extractor to a residual head. Preserving spatial detail rather than heavily pooling it is intuitive, and the paper does not identify a new principle, failure mode, or general scaling law beyond this implementation-specific finding.

The geographic idea is a modest feature transformation. Support-centering is a reasonable formulation of local categorical composition, but it is essentially patch-wise mean removal with an explicit validity channel. The evidence does not establish why this transform should be broadly preferable to standard multi-scale encodings, learned categorical embeddings, attention-based fusion, or normalization schemes that already account for missingness.

Diffusion and DiT are not the demonstrated source of the gain. Both systems use 12-block DiTs and two-step DDIM inference, but there is no matched UNet or deterministic-restoration counterpart using the same native/context or geographic adapter. Accordingly, the paper cannot support a claim that diffusion or DiT is needed for, or especially effective at, exploiting these conditions.

The main CloudRGB gain is largely an auxiliary-information result. Adding DEM + LULC to SAR raises PSNR from 41.00 to 44.11 dB. The representation-specific gain is the 1.09 dB comparison against uncentered fractions. Since the baselines in the headline table receive RGB + SAR only, that table demonstrates a stronger input configuration rather than clear superiority over methods equipped with comparable geographic priors.

The practical problem remains incompletely characterized. Annual LULC can disagree with the contemporaneous target in changing environments. The paper evaluates almost complete LULC coverage (0.9999817) and synthetic fixed gaps from the same target acquisition, but not temporal mismatch, abrupt change, real-cloud artifacts, or circumstances in which LULC should be ignored. Therefore, it does not show that its explicit support treatment resolves the more consequential reliability problem of stale geographic priors.

Evaluation limits the practical claim. There is no cloud/gap-severity breakdown, no masked-region-only CloudRGB metrics, little uncertainty analysis outside the SEN native control, and no fair same-device method-level efficiency comparison. The reported efficiency table measures the CloudRGB method on CPU and baselines on GPU.

On CloudRGB, the headline improvement is confounded by the additional DEM and LULC inputs, while published comparison methods use RGB + SAR only. The representation-specific result is more limited: support-centering improves over uncentered area fractions within the same adapter. Further, the evaluation does not test reliability when annual LULC is stale or conflicts with the current scene, despite this being central to the usefulness of static geographic priors under construction, deforestation, flooding, or other rapid change. Overall, the paper provides promising engineering evidence, but it currently lacks compelling conceptual insight, broad validation, and practical trade-off analysis to establish a significant contribution to the community.

Questions for authors

I understand that you use RGB difference to compare qualitative results. However, for example, in Figure 5, ECRformer's results seem much better.

Have you considered, like in ECRformer a quantitative comparison of different methods under different cloud coverage conditions (PSNR ↑/SSIM ↑).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.