acceptodds
Under review as a conference paper at ICLR 2027

EaSe-CR: Evidence-Aware Multimodal Representation Learning for Cloud Reconstruction

Abstract

Clouds obscure critical information in optical satellite imagery, making reliable reconstruction fundamentally dependent on how complementary observations are interpreted. Existing CR (cloud-removal/reconstruction) methods primarily fuse optical and synthetic aperture radar (SAR) observations without explicitly reasoning about whether the available evidence is reliable, degraded or mutually in- consistent. We introduce EaSe-CR (Evidence-aware Sensing experts for Cloud Reconstruction), a reconstruction model that jointly learns modality reliability, cross-modal evidence interaction, disagreement and latent uncertainty, allowing the reconstruction process to adapt to the quality and consistency of available evidence. Our analysis show that the learnt representation responds systematically to changes in evidence quality and cross-modal consistency, while external evaluation demonstrates that EaSe-CR retains an advantage over fixed-fusion baselines under zero-shot, geographically diverse conditions spanning Kerala, Rajasthan and Ladakh

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Multimodal Learning, Representation Learning, Remote Sensing, Cloud Removal, SAR–Optical Fusion, Evidence-Aware Learning, Uncertainty Modeling

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_M5YMOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces a method that predicts what a multispectral optical image would have looked like without clouds. The model is given the, at least partially cloudy, multispectral optical image and synthetic aperture radar (SAR) image as input. The method first separately encodes each modality, then performs "cross-modal evidence construction" and "evidence-conditioned routing" for processing by 3 experts. Finally, the data is processed by a "gated expert mixture", latent refinement, adaptive optical skip, and a multi-scale decoder. The method outperforms prior work, specifically Pix2Pix and SAR2OPT.

Critical strengths

none

Critical weaknesses

Regardless of the experimental soundness and novelty, I do not believe this paper would be a valuable addition to ICLR or even the ML4RS community within ICLR. The paper is on cloud removal which is a specific application of ML4RS (a niche in a niche). Secondly, as far as I know, cloud removal is not as useful to ML4RS as it once was, since we now have models that can process several time steps.

The paper also does not compare against well-known baselines like GLF-CR. The paper would be improved by comparing to more baselines and even using a pre-trained foundation model like OlmoEarth, which can receive many time steps and potentially be fine-tuned for cloud removal.

Questions for authors

none

Reviewer_qZyBOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents EaSe-CR, an evidence-aware multimodal hybrid expert model for cloud removal and image reconstruction in SAR-assisted optical satellite images. The paper also conducts analyses such as modal removal and artificial perturbation, and evaluates it on the SEN12MS subset and a self-built dataset.

Critical strengths

The paper has conducted a relatively comprehensive analysis of the internal representation, and the experimental perspective is of certain value.

Critical weaknesses
  1. The AI-generated elements in the paper are too obvious, such as strange language expressions, disordered bold fonts, abstract Overview diagrams, unnumbered formulas and the strange formatting with extra frame lines around them. The quality of the entire thesis is extremely poor. It seems to be a piece of research paper not written by a human.
  2. The main table lacks a large number of baseline methods from recent studies.
  3. The paper did not include any component ablation experiments.
  4. Table 1 provides a direct explanation for the opposite results.
Questions for authors

Apart from the weaknesses, I have no further questions.

Reviewer_XuGMOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes EaSe-CR, a multimodal model that reconstructs cloud-free optical imagery using cloudy Sentinel-2 images and Sentinel-1 SAR observations. It combines modality reliability, cross-modal interactions, disagreement, and estimated uncertainty to guide spatial routing among optical, SAR, and joint reconstruction experts. Controlled experiments examine how modality degradation and mismatch affect reconstruction quality and latent representations. Evaluation uses SEN12MS-CR Spring and an independent dataset spanning Kerala, Rajasthan, and Ladakh. The model reports improved PSNR and SSIM over the evaluated fusion baselines, although gains are not consistent across all metrics.

Critical strengths

The paper addresses a relevant multimodal reconstruction problem by explicitly distinguishing modality availability, reliability, and cross-modal consistency. Its spatially adaptive expert routing and multispectral reconstruction objective are well motivated for handling heterogeneous cloud cover. The study includes useful diagnostic analyses of modality degradation, mismatch, reliability, and latent representations, alongside reported improvements in SSIM and PSNR over the evaluated fusion baselines. Evaluation on an independent, geographically diverse dataset without fine-tuning provides a useful assessment of transfer beyond the training distribution. The authors also acknowledge the trade-offs across reconstruction metrics and appropriately qualify the interpretation of latent variance as an empirical uncertainty proxy.

Critical weaknesses

• The paper lacks component ablations isolating reliability, disagreement, uncertainty, and expert routing. The reported gains could therefore arise from additional capacity, skip connections, or the composite reconstruction loss. Representation changes under corruption demonstrate sensitivity, but do not establish improved robustness or meaningful expert specialization. Comparisons with matched architectural controls and final-model evaluations under identical corruptions and modality mismatches are needed. • Empirical modality utility, reliability targets, and the computation of latent variance are insufficiently specified. The latent variance analyzed in Section 3.5 is not explicitly connected to the learned uncertainty field used for routing. Only reconstruction losses are described, leaving the interpretation of these learned quantities insufficiently validated. The model or checkpoint used for each diagnostic analysis is also unclear, limiting interpretability and reproducibility. • Table 1 indicates a larger degradation after SAR removal, contradicting the claim that optical removal is more harmful. The aggregate and regional external metrics in Tables 4–5 also do not reconcile under a common sample-weighted averaging protocol. Different masks, subsets, or aggregation rules could explain the discrepancy, but these details and regional sample counts are missing.These issues require clarification before the conclusions about modality dependence and external generalization can be assessed confidently. • The main PSNR improvement is only 0.20 dB, while MAE and spectral angle worsen relative to Early Fusion. Baseline implementations, capacity matching, and loss consistency are insufficiently documented, weakening the assessment of the method’s contribution. The final comparisons lack variability across training runs, and regional baseline results are absent despite substantial geographic performance differences. Missing optimization settings, preprocessing details, and precise corruption procedures further limit reproducibility and confidence in the reported gains.

Questions for authors

Which ablations demonstrate the individual contributions of reliability, disagreement, uncertainty, and expert routing? How are reliability and latent uncertainty defined, computed, and validated in the final model? Can you reconcile the contradictory interpretation of Table 1 and the aggregate versus regional results in Tables 4–5? Are the reported gains consistent across training runs and regions when compared with baselines using matched capacity, losses, and training budgets?

2 comments

We sincerely thank the reviewer for the careful assessment of our work and for identifying several important aspects requiring clarification and further investigation.

We agree that the current submission does not provide controlled component-wise ablations isolating the individual contributions of modality reliability, cross-modal disagreement, representation uncertainty and spatial expert routing. Although our controlled experiments investigate how learned representations respond to modality degradation and cross-modal inconsistency, they do not independently quantify the contribution of each mechanism to final reconstruction performance. We will extend the experimental evaluation with targeted component-wise ablations to distinguish their respective contributions.

Regarding the mathematical formulation, our initial reliability investigation operationalizes modality reliability through relative unimodal reconstruction utility, using independently obtained optical-only and SAR-only reconstruction errors to construct empirical supervision targets. This investigation yielded Pearson correlations for optical and SAR reliability. In the final EaSe-CR architecture, reliability is extended to spatially resolved fields, which become part of the shared evidence representation. Similarly, we distinguish the perturbation-induced latent variance investigated in Section 3.5 from the learned uncertainty pathway in EaSe-CR. The former measures representation variability across eight controlled perturbation-induced evaluations and exhibits a positive association with reconstruction error; the latter produces a non-negative learned spatial representation (Section 4.1), which conditions expert routing. Importantly, this is a representation of evidence variability, not a calibrated probabilistic uncertainty estimate. We acknowledge that the earlier empirical analyses do not constitute direct calibration of the final EaSe-CR reliability and uncertainty fields, and will clarify this distinction in the revised manuscript.

We also appreciate the reviewer identifying the inconsistency in the modality-removal interpretation of Table 1. We will correct this interpretation while retaining the conclusion that the two modalities provide complementary, but asymmetric, evidence under the evaluated conditions. Regarding Tables 4–5, the aggregate PSNR, MSE and MAE are consistent with the sample-weighted regional means across the 337 matched external triplets (98 Kerala, 119 Rajasthan and 120 Ladakh). The apparent discrepancies in SSIM and SAM arise from differences in their evaluation implementations, the aggregate evaluation uses down-sampled SSIM and a valid-target-pixel mask for SAM, whereas the regional evaluation uses full-resolution SSIM and does not apply the same SAM mask. We will make these evaluation definitions explicit rather than present the metrics as directly interchangeable. We will also clarify that the PSNR values reported alongside the examples in Figure 6 correspond to individual representative patches, not regional averages.

Regarding training-run consistency, we acknowledge that the reported experiments use a single fixed random seed and do not establish variability across independently trained instances. The reported sample-level statistical analyses should therefore be interpreted as characterizing variation across evaluated observations, rather than across independent training runs. We will make this distinction explicit. For comparative evaluation, we used official released checkpoints where available, including the author-released pretrained DiffCR checkpoint, and the corresponding official implementations of the other baseline methods.

Finally, our independent external evaluation examines zero-shot reconstruction across three geographically distinct regions without fine-tuning or parameter updates. EaSe-CR achieves mean PSNR values of 30.066 dB in Kerala, 26.605 dB in Rajasthan and 17.312 dB in Ladakh. These differences reflect substantial geographical variation in reconstruction difficulty and reinforce the importance of evaluating evidence-conditioned reconstruction beyond a single environmental setting. We acknowledge that the reported regional analysis establishes EaSe-CR's performance across these regions, rather than comparative superiority over every baseline within each individual region, and will appropriately delimit this claim. More fundamentally, EaSe-CR seeks to advance the formulation of multimodal reconstruction from feature fusion towards evidence-conditioned inference, wherein the reliability, consistency and variability of available observations inform how evidence is represented, integrated and reconstructed.

We hope these clarifications address the reviewer's concerns and strengthen the presentation of the work. We would be grateful to address any further questions or points requiring clarification.

Overall, while the rebuttal improves some clarity and transparency of the work, it does not provide substantial evidence to resolve the central concerns regarding methodological effectiveness, robustness and fairness of comparison. Therefore, I would maintain my original evaluation and recommendation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.