acceptodds
Under review as a conference paper at ICLR 2027

GeoSET: Generalist Foundation Model for SAR-to-EO Image Translation

Abstract

Paired synthetic aperture radar (SAR) and electro-optical (EO) imagery is increasingly available across sensors, resolutions, and geographic regions. Yet existing SAR-to-EO image translation (SET) methods are typically trained on a single, limited-scale dataset, producing models specialized to particular sensing conditions. We introduce **GeoSET**, *the first generalist model for SET task*, built around a single pretrained parent that is adapted to downstream datasets under a common protocol. We curate over 3 million high-quality SAR–EO pairs from a collection of more than 10 million SAR observations, spanning diverse sensors, spatial resolutions, and ground sampling distances. To bridge the modality gap between SAR observations and a pretrained image generator, we develop a speckle-robust SAR encoder and pretrain the conditional generator on this heterogeneous corpus. The resulting parent supports efficient adaptation across downstream datasets through low-rank adaptation (LoRA), updating only 0.60% of the generator parameters and requiring approximately one hour per dataset. Across six downstream benchmarks, GeoSET achieves state-of-the-art results in FID and DISTS with full fine-tuning or LoRA, demonstrating effective transfer across heterogeneous SAR–EO domains. Code and pretrained checkpoints will be released upon acceptance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
SAR-to-EO image translation, remote sensing, generalist models, flow matching, speckle-robust representation learning, parameter-efficient adaptation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_A2Z5OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies a pretrain-once, adapt-many approach to SAR-to-optical image translation. GeoSET first trains a SAR encoder with robustness to speckle perturbations, then performs conditional generative pretraining on approximately 3.2 million curated SAR–EO pairs, and finally adapts to downstream datasets through full fine-tuning or LoRA. The method combines a natural-image generative model, SAR-specific encoding, and large-scale multisource data to reduce the need to build separate translation models for each new dataset. The main text reports perceptual-quality improvements across benchmarks and reductions in trainable parameters, memory, and adaptation time with LoRA.

Critical strengths

1. The main value of this work is a complete approach to shared pretraining and downstream adaptation. SAR encoding, paired-corpus pretraining, and dataset adaptation are connected coherently, and the main results cover multiple public benchmarks. Full fine-tuning reaches an FID of 24.5 on SAR2EO versus 52.7 for the best non-GeoSET method in the table, while LoRA reaches 87.9 on SpaceNet6. These results demonstrate practical effectiveness. The authors also acknowledge that pixel metrics are not consistently better and interpret the outputs as plausible SAR-conditioned optical appearances, making the performance claims easier to assess accurately.

2. The lightweight adaptation results also fit the motivation. In Table 4(b), LoRA updates approximately 0.60 percent of the generator parameters and takes about 0.98 hours per dataset, suggesting a practical benefit from sharing the pretrained model across tasks. Table 5 includes comparisons using the same FLUX.2 backbone, so the improvements cannot all be dismissed as a consequence of switching to a larger foundation model. Although the cumulative ablation does not fully separate the individual factors, it is more informative than comparisons only against methods using different backbones.

Critical weaknesses

1. Independence between downstream test scenes and the pretraining corpus is a key condition for interpreting the transfer results. Table 1 gives sources and counts, and Section 4.1 gives downstream splits, but the main text does not clearly explain whether overlap was checked using geographic footprints or original scenes. The same area could enter through different sources, neighboring tiles, or observations at different times; a split based only on files or sample identifiers would not necessarily resolve this. There is no evidence here establishing actual overlap, but for a paper emphasizing transfer across datasets, this condition directly affects whether the results demonstrate transfer to new scenes or adaptation to another observation of an already encountered scene.

2. The independent contributions of the main design choices remain insufficiently separated. Table 5 shows that the complete recipe works, but adding pretraining introduces both domain data and training steps, so the value of more in-domain training is not the same as the distinct value of multisource design. The relationship between SAR encoding and speckle augmentation raises a similar question: Stage 1 in Section 3 already uses perturbation training, while Table 5 introduces the encoder and augmentation as successive additions. Readers need to know which stages use perturbations and which weights change in each row to interpret the incremental gains. The results support the engineering value of the complete system; establishing the distinct contribution of each component requires a clearer connection between the experimental settings and the conclusions.

Questions for authors

1. Please clarify the actual separation rules between pretraining data and downstream test sets. Were geographic footprints, original scenes, and repeated observations across sources checked? Do both Stage 1 and Stage 2 follow the same constraints? These details directly affect the interpretation of transfer across datasets.

2. Please specify how the encoder is trained in the “+ SAR encoder” row of Table 5 and at which stages “+ Speckle aug.” introduces perturbations. A concise description of the settings changed and held fixed between the rows would resolve the configuration ambiguity.

3. Do the existing analyses distinguish the effects of data volume, additional training, and source diversity? If not, would the authors limit the corresponding claim to the benefit of the complete pretraining recipe? The response can clarify the scope of the existing evidence and align the contribution statement with the available results.

1 comment
AuthorsOpenReview ↗

Response to Reviewer A2Z5

We thank the reviewer for requesting clearer data-separation rules and component analysis. C2 provides the shared overlap audit; below, we clarify Table 5 and report additional Stage-2 controls.

W1 / Q1. Pretraining–test separation

C2 reports EO/SAR screening, geographic checks, and conservative re-scoring. The flagged SAR-1M files are absent from every GeoSET training list. SpaceNet6 has geographic overlap with TerraMesh, despite different sensors and 10 m versus 0.5 m GSD; its results therefore do not establish transfer to unseen locations. Coordinates are unavailable for several benchmarks, and the audit does not exhaustively exclude shared original scenes or repeated observations across sources for both stages. We will state these limits explicitly.

W2 / Q2. Settings of the Table 5 rows

“+ SAR encoder” already uses Stage-1 speckle perturbations; “+ Speckle aug.” adds them in Stages 2 and 3.

Table 5 rowSAR encoderS1 speckleS2 dataS2 speckleS3 speckleS2 updates
FLUX.2 LoRA (backbone)Frozen FLUX.2 AE—None—No0
+ Pretraining corpusFrozen FLUX.2 AE—3.2M pairsNoNo500k
+ SAR encoderStage-1 trainedYes3.2M pairsNoNo500k
+ Speckle aug. (GeoSET)Stage-1 trainedYes3.2M pairsYesYes500k

Stage 1 trains the SAR encoder and channel-specific stems with the decoder trunk fixed. Stage 2 trains the generator with both encoders and the decoder frozen. In every row, Stage 3 trains rank-16 LoRA adapters for 20,000 updates × batch 16. The three pretrained rows share the corpus and Stage-2 update count. This cumulative comparison measures the perturbation-trained encoder as a unit; it does not isolate encoder training from Stage-1 perturbations. We will clarify this in Table 5.

Q3. Source composition and training exposure

The following controls share the backbone, Stage-1 SAR encoder, and speckle augmentation in all applicable stages. Short Stage-2 runs use 30,000 updates × batch 64; all rows use the same rank-16 LoRA adaptation on all target training pairs for 20,000 updates × batch 16.

Stage-2 dataS2 sample presentationsSAR2EO FID ↓SAR2EO DISTS ↓SAR2EO LPIPS ↓SpaceNet6 FID ↓SpaceNet6 LPIPS ↓SpaceNet6 Bldg. IoU ↑
None049.20.2520.477102.90.3570.308
GUSO only1.92M37.50.2240.47490.20.3430.316
Full corpus (short)1.92M36.90.2250.47386.20.3400.332
Downstream training union1.92M24.80.1860.40192.20.3520.278
Full corpus (paper)128M29.50.2020.44087.90.3380.361

Source composition. GUSO improves all reported metrics over omitting Stage 2. At matched training exposure, the full corpus further improves the reported SpaceNet6 metrics, including building IoU 0.316 → 0.332; SAR2EO changes are small and mixed. This supports the tested source mixture, without separately isolating diversity from source content or unique-pair counts.

Training exposure. The longer full-corpus run improves SAR2EO metrics and SpaceNet6 LPIPS/IoU (0.332 → 0.361), while SpaceNet6 FID slightly favors the short run (86.2 versus 87.9). Both update count and batch size increase, so this comparison measures greater exposure rather than update count alone.

Target-domain exposure. The downstream-union row gives the strongest SAR2EO results but lower SpaceNet6 building IoU (0.278 versus 0.332). Both domains are included, so these are held-in-domain results. This row uses the union for Stage 2, followed by separate target adapters; the joint model in our response to Reviewer Mngz, W3 instead uses the union for one shared Stage-3 adapter. We will distinguish these protocols and scope attribution to the tested source mixtures and training recipe.

Reviewer_MngzOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents GeoSET, a large-scale image translation model from SAR to EO. The training is divided into three stages, offering two versions: full FT and LoRA FT. The experimental data and training scale were quite large, and ultimately, satisfactory results were achieved.

Critical strengths
  1. The author has made significant progress in the field of SAR to optical image translation, integrating large-scale data and completing the training of high-parameter models. If open-source code, especially open-source code related to weights, becomes more prevalent in the future, it will make a significant contribution to the remote sensing community.
  2. The paper provides a clear and transparent report on the adopted solution. In particular, the three-stage training process, the number of parameters, training time, GPU memory usage, and LoRA storage costs are all clearly reported, which is convincing.
Critical weaknesses
  1. The "First generalist model" mentioned in the paper seems unreasonable and lacks a comparison algorithm or a baseline. For instance, there has already been a method called Any2Any for cross-modal image conversion, and it can handle multiple modalities. In contrast, GeoSET only deals with the conversion direction from SAR to EO, and each downstream dataset still requires separate adaptation. Therefore, the positioning of "first generalist SET model" and "foundation model" must at least be significantly narrowed. Moreover, even if these two models do not have the same test set, it is inappropriate for the entire article to not mention methods like Any2Any at all.
  2. The technical contribution of GeoSET is relatively small, and it mainly consists of engineering-based combinations. For example, the FLUX.2 algorithm is used to generate the backbone, freeze the image autoencoder, fine-tune the SAR encoder, apply standard flow matching, and perform downstream adaptation with LoRA. The combination of these modules is reasonable in engineering terms, but the paper does not propose any new generation targets or cross-modal learning theories, nor does it present a unified modal representation mechanism. The so-called "speckle-robust encoder" essentially uses noisy reconstruction for modal adaptation, and the technical depth is insufficient to support the core innovations of the paper. This work might be more suitable for journals in the field of remote sensing, but I don't think it is very appropriate for ICLR which places greater emphasis on technical theories.
  3. All the main results require 20000 rounds of supervised adaptation on the complete target-paired data. Meanwhile, GeoSET additionally uses 3.2M pairs of pre-trained data and 3.85B parameter generators. Most baselines do not have access to the same data or computing resources. Therefore, the existing experiments mainly demonstrate that large-scale paired pre-training can provide better initialization, but do not prove that the proposed components have stronger generalization ability.Moreover, the comparison methods are omitted, so the SOTA conclusion is questionable.
Questions for authors

I have no further questions. I hope the author can provide their responses to the "Critical Weaknesses".

4 comments
AuthorsOpenReview ↗

Response to Reviewer Mngz (1/2)

We thank the reviewer for the thoughtful assessment and for encouraging clearer positioning and comparisons. C1 and C2 clarify the generalist scope, target-data requirements, and pretraining–test overlap. Below, we explain the learning contribution and report additional shared-model comparisons.

W1. Generalist scope and comparison with prior models

We agree that Any2Any [1] and TerraMind [2] are important prior work and will discuss both explicitly. C1 compares their task and SAR training coverage: both support SAR-to-optical generation and cover more modalities, while GeoSET spans more diverse SAR sensors and observation conditions within SAR-to-EO translation. We will replace the broad “first generalist” wording and scope the abstract, and “foundation model” terminology around a shared SAR-to-EO prior for downstream transfer and adaptation. This framing reflects reuse of one pretrained parent and makes the limits of direct zero-shot transfer explicit.

W2. Contribution beyond established components

We agree that flow matching, pretrained autoencoders, and LoRA are established techniques, and appreciate the request to clarify what GeoSET contributes. Our contribution is a unified learning framework for a shared SAR-to-EO prior across heterogeneous observation domains. It handles different SAR channel layouts and representations through channel-specific stems and representation-aware encoder training with a fixed decoder trunk, then conditions one generator through joint attention between spatial SAR and EO tokens. The encoder reconstructs the original SAR observation from a perturbed input, requiring neither speckle-free targets nor paired EO during this stage; the generator subsequently learns translation from paired observations.

Table 5 supports the value of these choices with the same generator: adding the SAR encoder and then speckle augmentation reduces SAR2EO DISTS from 0.250 → 0.220 → 0.202, with gains on SpaceNet6. The response to Reviewer A2Z5, Q2–Q3 clarifies these cumulative configurations and adds controls for source composition and training exposure. Together with C1's limited-data adaptation results, this evidence supports the framework's practical contribution when aligned target pairs are scarce. We will make this contribution and the role of each component clearer in the revision.

AuthorsOpenReview ↗

Response to Reviewer Mngz (2/2)

W3. Generalist baselines and comparison scope

We agree that model scale and additional pretraining must be considered when interpreting the gains. To address the missing shared-model comparison, we added joint training of cBBDM, C-DiffSET, and GeoSET on the union of the six downstream training sets. Each uses one shared model across all six benchmarks; GeoSET trains a single LoRA adapter on its Stage-2 parent.

Joint training on six benchmarks. Bold indicates the best result in each row.

BenchmarkMetriccBBDMC-DiffSETGeoSET
QXS-SAROPTFID ↓88.522.119.7
DISTS ↓0.2800.2730.258
LPIPS ↓0.5830.5810.565
PSNR ↑14.3214.8514.52
SAR2OptFID ↓116.8105.182.1
DISTS ↓0.2620.2620.214
LPIPS ↓0.5800.5580.546
PSNR ↑14.9616.3415.03
SAR2EOFID ↓88.643.529.3
DISTS ↓0.2300.2420.195
LPIPS ↓0.4400.4640.429
PSNR ↑20.2816.4419.44
SpaceNet6FID ↓218.1124.189.0
DISTS ↓0.2650.2390.189
LPIPS ↓0.3940.4400.339
PSNR ↑18.7516.1517.83
Bldg. IoU ↑0.0950.1740.362
CAP-BSGFID ↓130.5105.064.1
DISTS ↓0.3470.3610.330
LPIPS ↓0.6150.6140.583
PSNR ↑16.0215.1316.55
KOMPSATFID ↓172.389.273.9
DISTS ↓0.3720.3230.304
LPIPS ↓0.6590.6480.635
PSNR ↑10.9410.6810.51

GeoSET achieves the lowest FID, DISTS, and LPIPS on all six benchmarks and the highest SpaceNet6 building IoU (0.362) among these joint models. These results support the practical value of one jointly adapted GeoSET model across the six trained domains, without requiring separate per-benchmark adapters. Figure 8 provides visual comparisons.

Multimodal references. At the time of these experiments, Any2Any's code and model weights were unavailable to us for direct evaluation. We additionally compare TerraMind-1.0-large’s released checkpoint [2] with GeoSET’s Stage-2 parent without fine-tuning. Both use the same test items, crops, and evaluator.

Zero-shot comparison with TerraMind. Bold indicates the better result between the two models for each benchmark.

BenchmarkModelFID ↓DISTS ↓LPIPS ↓PSNR ↑
QXS-SAROPTTerraMind341.00.4660.7588.84
GeoSET61.00.3630.65711.27
SAR2OptTerraMind392.00.4730.75710.61
GeoSET147.90.3610.68411.68
SAR2EOTerraMind421.90.4990.7219.70
GeoSET184.00.3890.6749.28
SpaceNet6TerraMind435.20.4650.73812.71
GeoSET214.50.4080.71910.01
CAP-BSGTerraMind237.30.4100.66514.26
GeoSET128.50.4040.70914.44
KOMPSATTerraMind411.20.5260.74311.15
GeoSET160.70.4090.68611.17

GeoSET has lower FID and DISTS on all six benchmarks, lower LPIPS on five, and higher PSNR on four. TerraMind has lower LPIPS on CAP-BSG and higher PSNR on SAR2EO and SpaceNet6. Figure 6 provides visual comparisons. These results support GeoSET’s SAR-to-EO prior relative to this released multimodal checkpoint under the stated protocol, while direct zero-shot transfer remains limited.

References

[1] Haoyang Chen et al. “Any2Any: Unified Arbitrary Modality Translation for Remote Sensing.” ICML, 2026.

[2] Johannes Jakubik et al. “TerraMind: Large-Scale Generative Multimodality for Earth Observation.” ICCV, 2025.

Reviewer_MngzOpenReview ↗

Official Comment by Mngz

Thanks to the authors for the thorough response. The authors have addressed some of my concerns with additional experiments and revised the paper accordingly. I will take these into further consideration.

We sincerely thank the reviewer for the constructive feedback. The suggestions helped us improve the clarity of the paper, use more precise wording, and better define the scope of our claims. We have incorporated these improvements and the additional experimental results into the revised manuscript.

Reviewer_8a4YOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces GeoSET, presented as a "generalist" SAR-to-EO translation (SET) model under a pretrain-once, adapt-many framework. The authors curate 3M SAR–EO pairs from five heterogeneous sources with source-aware filtering, then pretrain in two stages: a speckle-robust SAR encoder (initialized from a natural-image autoencoder, trained to reconstruct clean SAR from speckle-perturbed input with a frozen decoder), and a repurposed FLUX.2 text-to-image generator whose text stream is replaced by a spatial SAR stream and trained with conditional flow matching. The shared checkpoint is adapted per dataset via full fine-tuning or LoRA (0.6% of parameters, ~1 hour). Across six benchmarks and a broad set of retrained baselines, GeoSET achieves best FID everywhere and best DISTS on five benchmarks, with ablations confirming complementary gains from corpus pretraining, SAR-specific encoding, and speckle augmentation.

Critical strengths

1: Meaningful problem formulation. Moving from per-dataset specialist models to a shared pretrained prior with PEFT adaptation is a conceptually meaningful contribution; the LoRA results (0.6% parameters, competitive with full fine-tuning) offer concrete evidence that this direction is practical. The core question — "can SET learn a reusable generative prior across heterogeneous SAR–EO sources?" — is well posed and answered affirmatively.

2: Simple but effective technical ideas. (a) Training the SAR encoder by reconstructing the original SAR from a speckle-perturbed input, with the decoder frozen to prevent an unconstrained SAR-specific codec, is an elegant way to bridge the modality gap without paired EO supervision. (b) Replacing the text stream with a spatial SAR stream initialized from the image stream is a clean, transferable trick for repurposing text-to-image backbones.

3: Thorough empirical protocol. Evaluations span six heterogeneous benchmarks across sensors, GSDs, and platforms, with all baselines retrained on identical splits. The paper reports a broad metric suite and detailed ablations.

Critical weaknesses

1: The "generalist" framing slightly oversells. The model does not generalize zero-shot; each benchmark requires paired training data and adaptation. The contribution is thus best described as large-scale multi-source pretraining + parameter-efficient adaptation, which is valuable but less novel as a concept than the title/abstract suggest. The paper mostly acknowledges this in the limitations, but the abstract's "generalist" claim and the "first generalist model" framing should be tempered.

2: Zero-shot evidence is asserted as limited but never shown. The appendix acknowledges that GeoSET "does not consistently achieve strong zero-shot performance," yet no zero-shot numbers or examples appear in the main paper. Since the central claim is a reusable cross-source prior, zero-shot (or few-shot) performance is the most direct evidence of whether that prior generalizes or merely provides a good initialization for per-dataset adaptation. The authors should report zero-shot/few-shot quantitative and qualitative results — including failure cases — or explicitly scope the "generalist" claim to adaptation rather than zero-shot transfer.

3: Pretraining–benchmark overlap is not clearly described. Given shared sensors and geographic regions, we cannot rule out that test areas appear in pretraining; if so, the reported gains may reflect memorization rather than transfer of a reusable prior. A deduplication statement or a small overlap analysis would suffice.

4: Model size and inference cost are not reported. The paper does not report parameter counts or training compute, so it is unclear how much of the reported gains come from the proposed recipe versus model scale. While Table 5 shows corpus pretraining helps, and no compute-matched comparison is provided. We are also concerned about inference cost, since multi-step methods such as GeoSET are substantially heavier than one-step approaches. Reporting per-method parameter counts and inference latency/FLOPs would clarify.

5: The wins concentrate in distributional metrics while faithfulness is systematically worse: on SpaceNet6, PSNR is ~18 dB vs. 19.4–19.7 for C-DiffSET/HI-Diff, with similar gaps in LPIPS/SSIM/PSNR on SAR2Opt and QXS-SAROPT. The multimodality/misregistration defense is reasonable, but the systematic direction of the trade-off raises the concern that GeoSET sacrifices SAR faithfulness for realism — problematic given its downstream-use motivation. Direct faithfulness evidence (e.g., structure/edge preservation or a task-based evaluation) would strengthen the claim that FID/DISTS gains reflect plausible SAR-conditioned outputs rather than merely prettier images.

Questions for authors
  1. Can you report quantitative and qualitative zero-shot/few-shot results, including failure cases? If the "generalist" claim is scoped to adaptation rather than zero-shot transfer, could the title and abstract be revised accordingly?

  2. Given shared sensors and geographic regions, can you rule out pretraining–benchmark overlap? A deduplication statement or a small overlap analysis would suffice.

  3. Can you report parameter counts, training compute, and inference latency/FLOPs for GeoSET and the baselines, so that gains can be separated from model scale and inference cost?

  4. What direct evidence can you provide on SAR faithfulness—e.g., structure/edge preservation or a downstream detection/segmentation comparison—to show that the distributional gains do not come at the expense of SAR fidelity?

  5. In §3.2, are the SAR and EO streams initialized from identical weights, and can the terminology ("image stream" vs. "EO tokens") be unified to avoid the impression of an additional stream?

3 comments
AuthorsOpenReview ↗

Response to Reviewer 8a4Y (1/3)

We thank the reviewer for the constructive comments. Common Responses C1 and C2 provide the transfer results and overlap audit; the response to Reviewer A2Z5, Q2–Q3 details the pretraining controls.

W1–W2 / Q1. Generalist scope and zero-shot/few-shot evidence

We will replace the broad “first generalist” claim and scope the manuscript to a shared SAR-to-EO prior for transfer and adaptation, explicitly discussing Any2Any and TerraMind. C1 reports zero-shot and 1%/10%/100% LoRA results on six benchmarks, with visual comparisons in Figure 6. Zero-shot performance is limited, while 1% adaptation improves FID, DISTS, and LPIPS on every benchmark. We will make the failure cases explicit.

W3 / Q2. Pretraining–test separation

C2 reports image screening, geographic checks, and conservative re-scoring. The flagged SAR-1M files are unused by GeoSET; excluding them and the flagged QXS-SAROPT EO near-duplicates changes LPIPS, PSNR, and SSIM only slightly. SpaceNet6 has geographic overlap with Stage-2 TerraMesh despite different sensors and 10 m versus 0.5 m GSD.

AuthorsOpenReview ↗

Response to Reviewer 8a4Y (2/3)

W4 / Q3. Model size, training cost, and inference efficiency

GeoSET’s 3.97B inference parameters comprise a 3.852B generator, 34.4M SAR encoder, and 84.0M autoencoder. It requires more compute and memory than compact specialized models. All methods use the same timing routine on idle B200 GPUs, batch size 1. Paired entries denote 256 / 512 px; Evals counts denoiser evaluations, including both CFG branches.

MethodParamsEvalsTFLOPs/imageLatency (ms)Peak memory (GiB)
E3Diff192M10.38 / 1.510 / 311.0 / 1.8
HI-Diff28.5M80.29 / 1.127 / 780.7 / 2.3
ResShift174M4 / 161.4 / 5.7155 / 5551.0 / 1.1
DDPM (SR3)192M5019.1 / 76.4494 / 1,4161.1 / 2.1
SD2.1 FT1.29B509.7 / 43.5824 / 1,2013.9 / 4.8
C-DiffSET1.29B509.7 / 43.5890 / 1,1933.9 / 4.8
ControlNet1.65B10025.1 / 1111,258 / 1,5863.3 / 3.7
BBDM292M20050.6 / 2041,467 / 1,7791.5 / 3.9
cBBDM292M20050.6 / 2041,567 / 1,7861.5 / 3.9
CondDiff164M / 166M250166 / 2795,754 / 8,7191.0 / 1.5
GeoSET (paper: 50 steps, CFG 2)3.97B100323 / 1,3891,873 / 3,42614.9 / 15.5

Methods use their released precision: fp32 except ResShift (fp16), ControlNet (bf16), and GeoSET (bf16 autocast). Latency is the median of 5–20 runs after warm-up.

Training cost. Stage 1 uses 50k updates × batch 432 and Stage 2 uses 500k × 256, taking 14.8 and 129.3 hours, respectively, on four B200s (Sec. 4.2). Each benchmark then uses 20k × 16 adaptation.

Attribution controls. Table 5 uses the same generator and downstream LoRA budget. Additional Stage-2 controls match 30,000 updates × batch 64, holding the backbone, Stage-1 encoder, and augmentation fixed. The full source mix improves SpaceNet6 building IoU 0.316 → 0.332 over GUSO alone; SAR2EO changes are small and mixed. The full configuration and control tables are provided in the response to Reviewer A2Z5, Q2–Q3. These experiments support the tested training recipe without establishing superiority over other methods at matched data and compute.

Sampling steps and guidance. We additionally evaluate 50, 25, and 10 sampling steps, with and without CFG, on three benchmarks. All settings use the paper’s LoRA adapters without retraining; only the sampler changes. Figure 9 provides visual comparisons.

QXS-SAROPT

StepsCFG wEvalsFID ↓DISTS ↓LPIPS ↓PSNR ↑
502100 (paper)19.30.2540.56414.67
2525019.00.2540.56414.72
1022020.50.2590.56014.99
50—5012.90.2570.57414.65
25—2513.30.2600.57514.72
10—1016.60.2690.57215.03

SAR2EO

StepsCFG wEvalsFID ↓DISTS ↓LPIPS ↓PSNR ↑
502100 (paper)29.50.2020.44019.19
2525030.40.2040.44119.22
1022033.80.2090.44219.46
50—5026.30.2140.46019.12
25—2526.80.2170.46219.17
10—1029.70.2270.46319.39

SpaceNet6

StepsCFG wEvalsFID ↓DISTS ↓LPIPS ↓PSNR ↑
502100 (paper)87.90.1890.33817.95
2525088.70.1890.33717.99
1022089.60.1890.33618.01
50—5082.40.1880.35017.92
25—2582.40.1880.35017.98
10—1082.60.1900.34918.16

Sampler trade-offs. With CFG retained, 25 steps halve denoiser evaluations with small measured quality changes. Ten steps reduce evaluations by 80% and eager latency from 1,873 / 3,426 ms to 382 / 716 ms. SpaceNet6 building IoU remains comparable (0.363 versus 0.361), while SAR2EO FID increases from 29.5 to 33.8. Removing CFG improves FID but worsens LPIPS on all three benchmarks and lowers 10-step SpaceNet6 building IoU from 0.363 to 0.318.

Training-free acceleration. We also combine sampler changes with lower precision, compilation, and CFG batching using the same LoRA checkpoints without retraining. Latency pairs denote 256 / 512 px; quality changes are relative to the paper configuration on QXS-SAROPT / SpaceNet6 (3,999 / 495 images).

SettingEvalsLatency (ms)ΔFIDΔDISTS × 1,000
bf16 + compile100600 / 1,4290.0 / 0.0−0.1 / 0.0
bf16 + compile, 25 steps50305 / 730−0.3 / +0.7+0.3 / −0.1
FP8 + compile + batched CFG, 10 steps2074 / 256+1.2 / +1.7+4.9 / +0.1

The 25-step setting gives 6.1× / 4.7× speedups; the fastest setting gives 25.3× / 13.4×, with a QXS-SAROPT DISTS increase of 0.0049.

AuthorsOpenReview ↗

Response to Reviewer 8a4Y (3/3)

W5 / Q4. Direct evidence of structural faithfulness

We agree that FID/DISTS alone do not establish SAR faithfulness. We therefore add building segmentation and edge-preservation evaluations.

(i) Building IoU. We train a U-Net/ResNet-50 only on real SpaceNet6 training EO, freeze it, and evaluate all methods against ground-truth building footprints. IoU is pooled over 495 test tiles; the evaluator’s held-out real-EO IoU is 0.764. Figure 7 provides visual comparisons.

MethodSpaceNet6 PSNR ↑Building IoU ↑
pix2pix17.070.143
CycleGAN16.880.186
pix2pixHD19.210.251
SPADE17.770.192
StegoGAN16.040.176
DDPM (SR3)12.450.027
SD2.1 FT19.290.188
BBDM18.030.048
ControlNet14.450.092
HI-Diff20.610.055
CondDiff14.680.170
E3Diff18.320.244
cBBDM19.720.067
C-DiffSET19.430.178
GeoSET LoRA17.950.361
GeoSET full FT17.990.339

GeoSET LoRA achieves building IoU 0.361 versus 0.251 for the strongest evaluated baseline (+11.0 points), despite lower PSNR. Full fine-tuning ranks second at 0.339, supporting preservation of task-relevant building structure beyond pixelwise agreement.

(ii) Edge F1. Canny edges (σ = 2) are matched to paired EO edges within two pixels. F1 balances precision and recall, penalizing both spurious and missing boundaries.

Edge F1 ↑ (two-pixel tolerance)

MethodQXS-SAROPTSAR2OptSAR2EOSpaceNet6CAP-BSGKOMPSAT
pix2pix0.3300.3550.2670.5270.1810.449
CycleGAN0.3090.3820.1320.5500.3220.452
pix2pixHD0.3090.4210.4180.5240.3500.449
SPADE0.3290.4140.2920.5390.2040.357
StegoGAN0.2820.3980.0890.5520.3190.071
DDPM (SR3)0.2770.3780.3520.5270.3250.489
SD2.1 FT0.3380.4160.3450.5230.2820.457
BBDM0.3520.3870.3870.2610.3120.454
ControlNet0.3540.4320.3170.5600.2480.409
HI-Diff0.0020.1250.0210.4990.2470.074
CondDiff0.3360.2680.3460.4750.2390.523
E3Diff0.3790.4460.4480.5590.2970.462
cBBDM0.3830.2420.4570.2230.2810.228
C-DiffSET0.3800.4240.3760.5150.3210.443
GeoSET LoRA0.3770.4510.3320.5690.3260.493
GeoSET full FT0.3850.4510.3780.5670.3560.496

Across its LoRA and full-FT variants, GeoSET attains the highest reported Edge F1 on four of six benchmarks; cBBDM leads on SAR2EO and CondDiff on KOMPSAT. Together with building IoU, this provides evidence of scene-structure preservation beyond distributional similarity.

Q5. Stream initialization and terminology

Yes. The SAR stream is initialized by copying the EO stream’s projection, attention, MLP, and modulation parameters (Sec. 3.2; App. C). The two streams then use separate parameters in double-stream blocks and shared parameters in single-stream blocks. “Image stream” refers to the EO stream processing noisy EO latent tokens; there is no third stream. In the revision, we will consistently use “EO (target) stream” and “SAR (condition) stream” throughout.

Comments on the paper

4 comments
AuthorsOpenReview ↗

Common Response to All Reviewers

We thank the reviewers for their constructive feedback and for recognizing the value of learning a shared SAR-to-EO prior across heterogeneous sensing conditions. Below, we address common concerns about the scope of the generalist claim, pretraining–test separation, and zero-shot/few-shot adaptation. Reviewer-specific concerns and additional experiments are covered in the individual responses.

We have added the supporting figures to the Appendix (Section E). We will incorporate the additional results, analyses, and clarifications into the manuscript and upload a consolidated revision 2–3 days before the discussion period ends.

AuthorsOpenReview ↗

Common Response to All Reviewers (1/2)

C1. Scope and zero-shot/few-shot adaptation

We thank the reviewers for highlighting the scope of “generalist.” Any2Any [1] and TerraMind [2] already support SAR-to-optical generation. GeoSET learns one shared SAR-to-EO prior from over three million heterogeneous pairs and adapts it across six benchmarks. We will replace the broad “first generalist” wording and scope the abstract and contributions to a shared SAR-to-EO prior for downstream transfer and adaptation.

Relationship to prior work. Any2Any translates among SAR, RGB, NIR, MS, and PAN; TerraMind supports optical/SAR generation, elevation, land cover, vegetation indices, captions, and coarse geolocation.

ModelSAR training coverage
Any2AnySEN1-2/SEN12MS: Sentinel-1 C-band, VV; approximately 10 m grids.
TerraMindTerraMesh: Sentinel-1 C-band, VV/VH (GRD/RTC); 10 m grid.
GeoSETFive sources spanning multiple SAR sensors and X/C/L bands; spatial scales of 0.16–12.5 m.

Both prior models cover more modalities and tasks, while GeoSET covers more diverse SAR sensing conditions within SAR-to-EO translation. Spatial scales denote reported resolutions or sampling intervals.

Transfer results. We evaluate the same Stage-2 parent without target adaptation and with rank-16 LoRA on 1%, 10%, or 100% of each target training set. Figure 6 in Appendix provides visual comparisons.

BenchmarkSettingTraining pairsFID ↓DISTS ↓LPIPS ↓PSNR ↑
QXS-SAROPTZero-shot061.00.3630.65711.27
QXS-SAROPTLoRA, 1%16138.10.2810.57315.11
QXS-SAROPTLoRA, 10%1,60117.80.2510.56314.75
QXS-SAROPTLoRA, 100%16,00119.30.2540.56414.67
SAR2OptZero-shot0147.90.3610.68411.68
SAR2OptLoRA, 1%15114.70.2690.60614.04
SAR2OptLoRA, 10%14577.40.2150.55615.10
SAR2OptLoRA, 100%1,45074.60.2000.53515.50
SAR2EOZero-shot0184.00.3890.6749.28
SAR2EOLoRA, 1%68229.50.2000.43619.41
SAR2EOLoRA, 10%6,81629.40.2000.43719.34
SAR2EOLoRA, 100%68,15129.50.2020.44019.19
SpaceNet6Zero-shot0214.50.4080.71910.01
SpaceNet6LoRA, 1%26122.40.2190.37816.91
SpaceNet6LoRA, 10%25689.30.1900.34517.92
SpaceNet6LoRA, 100%2,55887.90.1890.33817.95
CAP-BSGZero-shot0128.50.4040.70914.44
CAP-BSGLoRA, 1%74104.30.3240.57117.25
CAP-BSGLoRA, 10%73457.60.3050.55717.37
CAP-BSGLoRA, 100%7,33164.70.3180.56616.85
KOMPSATZero-shot0160.70.4090.68611.17
KOMPSATLoRA, 1%62123.40.3620.64110.95
KOMPSATLoRA, 10%61773.80.3050.62510.59
KOMPSATLoRA, 100%6,16772.00.2990.62510.64

Protocol: All LoRA settings use 20,000 updates and one run per setting. The 1% and 10% subsets are nested and fixed by one random permutation per benchmark; zero-shot uses no target training pairs.

Direct zero-shot performance is limited, but 1% adaptation improves FID, DISTS, and LPIPS over the unadapted parent on all six benchmarks. On five benchmarks, 256–1,601 target pairs yield FID close to or better than full-data adaptation. This data efficiency is useful because acquisition-time, viewing-geometry, and resolution differences make aligned SAR–EO pairs difficult to obtain. All LoRA settings retain the same update budget; we will distinguish data efficiency from compute efficiency and make zero-shot limitations and failure cases explicit.

References

[1] Haoyang Chen et al. “Any2Any: Unified Arbitrary Modality Translation for Remote Sensing.” ICML, PMLR 306, 2026.

[2] Johannes Jakubik et al. “TerraMind: Large-Scale Generative Multimodality for Earth Observation.” ICCV, pp. 7383–7394, 2025.

AuthorsOpenReview ↗

Common Response to All Reviewers (2/2)

C2. Pretraining–test separation and leakage audit

We agree that data separation is essential for interpreting transfer. We distinguish image-level matches, matches in unused source data, and geographic overlap. Before submission, we screened the EO side of GUSO, TerraMesh, SARLO-80, SAR-1M, and 3MOS; for the rebuttal, we additionally searched the SAR side.

The audit combines (i) filename matching, (ii) DINOv3 top-10 retrieval followed by SIFT/RANSAC and normalized cross-correlation verification, and (iii) test-location checks against tile footprints.

BenchmarkEO-side screeningSAR-side screeningGeographic overlap
QXS-SAROPT22 near-duplicates (0.55%)82 matches in SAR-1M’s unused SAR-only portionCoordinates unavailable
SAR2OptNo matches detected9 matches in SAR-1M’s unused SAR-only portionCoordinates unavailable
SAR2EONo matches detectedNo matches detectedCoordinates unavailable
SpaceNet6No matches detectedNo matches detectedAll 495 test-tile centres (100%) within TerraMesh footprints
CAP-BSGNo matches detectedNo matches detectedNo overlap detected in screened footprints
KOMPSATNo matches detectedNo matches detectedCoordinates unavailable

SEN1-2 was excluded as an evaluation benchmark before submission: all 52,967 test filenames occur in SAR-1M, including 49,225 (92.9%) in the training list.

The 82 QXS-SAROPT and nine SAR2Opt SAR matches occur in no GeoSET training list. They belong to SAR-1M’s unused SAR-only portion. We nevertheless re-score QXS-SAROPT after excluding these 82 pairs and the 22 flagged EO near-duplicates, retaining 3,895/3,999 pairs; excluding the nine SAR2Opt matches retains 618/627 pairs. Re-scoring uses the existing generated outputs and unchanged evaluator.

BenchmarkGeoSET settingLPIPS ↓PSNR ↑SSIM ↑
QXS-SAROPTLoRA0.564 → 0.56514.67 → 14.670.326 → 0.325
QXS-SAROPTFull FT0.553 → 0.55415.07 → 15.080.332 → 0.331
QXS-SAROPTZero-shot0.657 → 0.65811.27 → 11.280.224 → 0.223
SAR2OptLoRA0.535 → 0.53515.50 → 15.500.278 → 0.276
SAR2OptFull FT0.532 → 0.53215.66 → 15.660.279 → 0.278
SAR2OptZero-shot0.684 → 0.68211.68 → 11.690.152 → 0.149

Entries show original split → subset after exclusions. The small changes indicate limited sensitivity to these flagged pairs in LPIPS, PSNR, and SSIM; this subset analysis does not establish an entirely overlap-free pretraining corpus.

SpaceNet6 has geographic exposure: all 495 test-tile centres lie within Stage-2 TerraMesh footprints. TerraMesh uses 10 m Sentinel-1/2 observations, while SpaceNet6 uses 0.5 m imagery from different sensors. This 20-fold GSD difference limits exposure to comparable fine-scale detail, but these results do not establish transfer to geographically unseen locations.

AuthorsOpenReview ↗

Revised Manuscript and Follow-up

Dear Reviewers,

Thank you again for your feedback. We have revised the GeoSET manuscript to incorporate the main results and clarifications from our responses.

  • Scope and transfer. We revised the title and abstract to focus on a shared SAR-to-EO prior for transfer and adaptation, and added a discussion of Any2Any and TerraMind. Tables 7 and 13 report zero-shot comparisons with TerraMind and GeoSET’s LoRA results using 1%, 10%, and 100% of the target training pairs across six benchmarks.
  • Ablations and comparisons. Tables 5 and 6 clarify the stage-specific use of speckle augmentation and examine source composition and training exposure. Appendix A adds building and edge preservation evaluations and comparisons with jointly trained baselines.
  • Inference cost and data overlap. Appendix B reports inference costs and training-free acceleration results. Appendix C documents the pretraining–test overlap audit, including SpaceNet6’s geographic overlap and re-evaluation after excluding flagged pairs.

If you have any remaining questions about the revision or our responses, we would be happy to clarify.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.