acceptodds
Under review as a conference paper at ICLR 2027

GeoCR: Learning a Generalist Cloud Removal Prior from Heterogeneous Observations

Abstract

Cloud removal (CR) methods are typically specialized to individual datasets and input configurations, limiting reuse across sensors, spectral bands, and observation settings. We introduce **GeoCR**, a generalist model that *firstly* unifies RGB-only-based CR and multispectral-based CR from single- or multi-temporal cloudy observations, with optional SAR guidance, within a single network. To accommodate different spectral and sensing domains, compact input and output stems extend a pretrained RGB autoencoder while keeping its encoder and decoder trunks frozen. This shared latent interface enables a single flow transformer to jointly model clean RGB and non-RGB latents, conditioned on separate cloudy-observation streams and optional SAR tokens. Through joint pretraining on the training splits of ten datasets comprising 883,331 cloud-free target images, GeoCR learns a *shared cloud removal prior* across these heterogeneous configurations. The same pretrained checkpoint supports direct inference without dataset-specific fine-tuning and efficient adaptation through low-rank adaptation (LoRA). We evaluate GeoCR against general image restoration and cloud removal methods on held-out test splits of the contributing datasets under full-band and RGB-only settings. GeoCR achieves the best FID and DISTS on full-band SEN12MS-CR and Sen2_MTC_New and RGB-only CUHK-CR2, *significantly* outperforming existing models and demonstrating the effectiveness of a reusable generative model across diverse cloud removal settings.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Cloud Removal, Remote Sensing, Generalist Models, Flow Matching, Multispectral Image Restoration, Multimodal Learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_fY8WOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduces GeoCR, which unifies cloud removal for RGB, multispectral, and SAR data within a single model using FLUX.2 and flow matching. The model is jointly trained on ten datasets and provides perceptual metrics and visual comparisons across multiple benchmarks.

Critical strengths

The experiments cover various observation configurations, enabling thorough visual comparisons.

The training settings, data splits, and evaluation protocols are described in detail. These materials facilitate reproducibility and provide useful references for future research on remote sensing cloud removal, regardless of the paper's acceptance.

Critical weaknesses

The contribution lies primarily in engineering integration. Flow matching and LoRA are both existing techniques, and the paper does not propose a novel de-clouding mechanism. Moreover, LoRA does not consistently improve performance across multiple datasets.

The paper does not report inference speed, memory usage, or computational cost. For large-scale deployment, it is necessary to compare the inference overhead with that of dedicated de-clouding models. Although optimal efficiency is not required, understanding this aspect remains important.

In Figure 1, some cloud-covered regions appear nearly pure white in the displayed RGB image, yet the output still contains clear surface details. What information does the model rely on to recover these details? Is this due to information preserved in other spectral bands or SAR data, or could there be overlap between the training and test samples?

Questions for authors

Clarify the technical contributions of the paper beyond its engineering integration.

Please specify whether the test images are disjoint from the ten training datasets, and explain how the model recovers surface details in thick cloud regions.

Report the single-image inference latency, memory usage, and computational cost of the proposed method, and compare them with baseline methods on the same hardware.

Reviewer_iKs2OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

In this paper, a general generation model for cloud removal, GeoCR, is proposed, with the goal of unified processing in a single network. Methods Based on the pre-trained RGB self-encoder, and then using the pre-trained image flow transformer as conditional flow matching, clean RGB and non-RGB latents were jointly modeled, and each cloudy optical observation and optional SAR were regarded as independent conditional flows. The author jointly pre-trained on 10 CR data sets and 883,331 cloud-free target, supporting direct reasoning (w/o FT) and LoRA adaptation. Experiments are evaluated on benchmark such as Sen2_MTC_New, CUHK-CR2, WHUS2-CRv, T-CLOUD, etc. It is reported that the best FID/DISTS are obtained on full-band Sen2_MTC_New and RGB-only CUHK-CR2, and the best FID/Dists have passed the stem training and phase conditions.

Critical strengths
  1. The framework design of unified heterogeneous observation configuration has substantial value.
  2. The experimental support of large-scale joint pre-training and direct reasoning across data sets is strong.
Critical weaknesses
  1. The evidence of generalization claims is insufficient, and there is the risk of optimistic deviation caused by the overlapping of evaluation set and pre-training set.
  2. Index selection emphasizes the quality of perception/distribution, while pixel and structural fidelity are not fully ahead, so the reliability of remote sensing application is in doubt.
Questions for authors
  1. Can you provide a generalization evaluation of true held-out?
  2. Can you report the spectrum/radiation fidelity index under the full-band setting?
  3. The benefits of LoRA adaptation are inconsistent. When should we use w/o FT and Lora? LoRA in Table 2-4 is sometimes slightly better and sometimes worse, and the author also claims that it provides selective improvements. Please explain the clear application conditions of LoRA and whether it is worthy of being used as a general adaptation scheme.
  4. Is the advantage of FID/DISTS still valid in severe cloud coverage and quantitative downstream tasks?
Reviewer_HTC9OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents GeoCR, a method for learning a common cloud removal prior from heterogeneous observations, spanning RGB, non-RGB (visible and NIR), and SAR modalities. The method builds on top of the FLUX.2 models, considering a shared pre-trained autoencoder for all the data modalities, which is kept frozen, while trainable stems account for non-RGB and SAR imaging modalities. In the second stage, a diffusion transformer (DiT) is trained, considering noisy target, cloudy optical, and optional SAR streams under a flow matching loss. The proposed method is evaluated on five widely used cloud removal datasets featuring different modalities and conditioning horizons across standard reconstruction quality metrics (FID, DISTS, LPIPS, SSIM, and PSNR) along with KID and DINO feature similarity.

Critical strengths
  • The proposed architecture allows learning a common prior that can seamlessly operate under different combinations of imaging modalities.
  • GeoCR results in improved performance on distributional and perceptual metrics compared to numerous baselines, spanning generic image restoration methods, single-image as well as multi-temporal cloud removal methods.
  • Qualitative comparison on the presented instances shows satisfying structural and color consistency.
  • Failure cases are presented and discussed in the Appendix.
Critical weaknesses
  1. To better assess the generalization ability of the method, it would be important to evaluate the architecture using different reference autoencoders, beyond the one of FLUX.2.

  2. The claim of a unified multi-temporal model relies on a restricted set of benchmarks (Sen2_MTC_{Old,New} and AllClear, conditioning on at most 3 images). To rigorously evaluate multi-temporal fusion, the model should be evaluated on dense multi-temporal benchmarks with constrained revisit intervals (such as SEN12MS-CR-TS). Constraining phenological shifts and surface change between acquisitions is critical to distinguish whether GeoCR is successfully aggregating real surface information across frames or simply relying on the DiT generative prior to generate plausible textures.

  3. As described in Section 3.2 and Appendix B, the K cloudy observations are processed with shared weights and joint attention, without temporal positional encodings or acquisition time offsets relative to the reference. The model appears to treat multi-temporal frames as an unordered bag of observations rather than a time-dependent sequence. Have the authors considered integrating acquisition timestamps or temporal encodings?

  4. In Table 3 (Sen2_MTC_New, K=3), how were single-image baselines (e.g., DiffCR, HI-Diff, pix2pixHD) adapted to handle multiple input frames? Furthermore, relevant multi-temporal baselines that explicitly model temporal sequences, such as [R1] and [R2] should be included for a fair comparison on multi-frame settings.

  5. To convincingly justify the adaptation framework, LoRA should instead be demonstrated in an out-of-distribution (OOD) or few-shot transfer setting adapting the pretrained checkpoint to a held-out dataset or geographic region not present in Stage 2 pretraining. Otherwise, the paper would be stronger by omitting LoRA and focusing entirely on the zero-shot capabilities of the unified checkpoint.

  6. Regarding the stems, have more complex projection layers (beyond a single convolution layer) been considered?

Minor comments

  • eq.2: Clarify that stem training is performed on both clean/cloudy images, as Table 5 suggests.
  • L.204-206: It is not clear from this text alone how the distinction between top-of-atmosphere and bottom-of-atmosphere reflectance imagery is exploited in the proposed method. A sentence should be added to clarify this, as in Appendix B.
  • Figure 5: Please specify the length of the context (temporal) considered for obtaining these results.

[R1] Christopoulos, et al. "CloudTran++: improved cloud removal from multi-temporal satellite images using axial transformer networks." Remote Sensing 2024.

[R2]Zhang, et al. "TSSMamba: A temporal–spectral–spatial state space model for multi-temporal remote sensing cloud removal." JAG 2026.

Questions for authors

Please address the questions raised in the Weaknesses section above.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Revised Manuscript and Follow-up

Dear Reviewers and AC,

Thank you again for your constructive feedback. We have revised the GeoCR manuscript to incorporate the main results and clarifications from our responses.

  • Held-out transfer and adaptation. Section 4.4 adds zero-shot and few-shot LoRA evaluations on SEN12MS-CR-TS, which was excluded from pretraining. Table 5 compares GeoCR with U-TAE and UnCRtainTS, and Appendix A examines the use of up to six cloudy acquisitions and acquisition-time conditioning.
  • Reconstruction quality. Appendices A and E add analyses of optical and SAR conditioning, performance across cloud-cover levels, full-band spectral and radiometric fidelity, and vegetation and water indices.
  • Inference cost and data overlap. Appendix B reports inference cost and acceleration results using the same checkpoint without retraining. Appendix H documents the pretraining–test overlap audit and reports how excluding geographically overlapping patches affects the results.

If you have any remaining questions about the revision or our responses, we would be happy to clarify.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.