- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper generates ground-level panoramas from low-resolution Sentinel-2 observations, AlphaEarth features, and geographic and temporal metadata. It introduces directional conditioning and a frozen Perspective-Inversion Critic (PIC), which maps generated panoramas back to high-resolution overhead imagery to provide structural supervision during training. Evaluation includes a geographically partitioned dataset, CVUSA, and component ablations. The internal results suggest that PIC improves perceptual and predicted-depth consistency. However, I have serious concerns about the numerical credibility and comparability of the reported FID advantage. Unresolved memorization concerns, incomplete panoramic calibration, and insufficient comparisons with simpler uses of high-resolution supervision further limit the evidence for the claimed contribution.
Critical strengths
-
The PIC ablation provides direct evidence that the added supervision improves the current generator. With the same Sentinel-2 and AlphaEarth inputs, predicted-depth AbsRel decreases from 0.1385 to 0.1057 and LPIPS from 0.5362 to 0.5038. These controlled results are more informative about PIC than the overall comparison with earlier systems and support the usefulness of its training signal.
-
Directional conditioning has measurable benefits in a component comparison. In Table 6, removing coordinate encoding from the Sentinel-2 plus high-resolution configuration increases AbsRel from 0.1007 to 0.1126 and LPIPS from 0.4912 to 0.5059. This provides empirical support for incorporating directional information when connecting overhead observations to panoramic outputs.
Critical weaknesses
-
The central performance and generalization claims are not supported by sufficiently verifiable evidence. CVUSA FID is reported as 6.73, compared with 21.30 for CS2S. Page 22 states that baseline FID, PSNR, and SSIM values were taken from other publications, but the paper does not establish equivalent evaluation implementations and preprocessing. The cited SatDreamer360 reports FID 17.00 for single-image generation on CVUSA and is absent from the comparison; Bifröst reports worse LPIPS, PSNR, and SSIM despite its much lower FID. Different metric rankings are possible, but this unusually large distributional advantage requires a reproducible, common evaluation. The reported panorama resolutions also differ, making the crop and resize pipeline consequential. I have serious concerns about accepting these FID values as evidence of the claimed improvement without that verification. The stated inability to release the dataset and code makes a reproducible evaluation package particularly important. Figures 14–15 additionally show striking similarities between generated images and their targets in capture-vehicle appearance, with similar cloud structure in Figure 14(g). Such details are not determined by the coarse overhead observations. These examples raise unresolved overfitting or memorization concerns that require sample provenance, training-neighbor searches, and capture-sequence deduplication before the generalization claim is convincing.
-
The specific methodological advance and geometric validity remain insufficiently established. The key question is whether PIC transfers high-resolution structural supervision more effectively than simpler alternatives. The paper lacks comparisons with a teacher-distillation or auxiliary overhead-reconstruction objective using the same backbone, supervision, and training budget. On the internal benchmark, the model without PIC already obtains FID 4.4003, while PIC improves it to 3.8707; most of the gap to earlier systems therefore predates the proposed critic. The geometric evidence also relies primarily on predicted depth and segmentation, leaving the accuracy of location-specific structure uncertain. Moreover, CVUSA panoramas are vertically cropped, while the method uses elevation-angle encoding. The vertical field of view, crop bounds, and pixel-to-ray calibration are needed to assess whether these directions are correct. Please report this calibration and validate perspective reprojection within the retained viewing range. A cropped panorama should not be treated as a complete spherical observation. These omissions affect both the interpretation of the geometric gains and the usability of the generated output.
Questions for authors
-
FID verification: What implementation, weights, feature layer, sample set, crop, resize, and sampling procedure produced the reported FID values? Under a common CVUSA evaluation pipeline, how do Bifröst, CS2S, and SatDreamer360 compare, with their input differences documented? Recomputable results would directly affect my assessment of the main performance claim.
-
Generalization verification: Which splits contain the examples in Figures 14–15, and what inputs and initialization were used to generate them? Do training-neighbor searches, capture-sequence deduplication, and evaluation with the capture vehicle masked support the interpretation that these results generalize to unseen locations?
8 comments
Response 1 - Continued
- Checking for Overfitting and Memorization (Appendix Section G in updated paper):
Thank you for highlighting the specific example of cloud structure similarities. We would like to draw the re- viewer’s attention to images on Fig 14 (a-e) where the model does not replicate the cloud structure. We believe that the model is most likely combining cloud structures it has learnt over the dataset, correlating it to the low resolution Sentinel-2 images and AlphaEarth embeddings where possible. Since AlphaEarth embeddings are trained on Sentinel-1 (which is good at penetrating through clouds) and Sentinel-2 (which has explicit bands for cloud cover identification), it is possible that the model learns to generate cloud structures.
To directly test for training-set memorization and capture-sequence leakage on CVUSA (35,532 training / 8,884 held-out validation pairs), we conducted an exhaustive nearest-neighbor audit across all three cyclic LR→SV conditioning configurations, EFM, Sentinel-2 multispectral, and EFM + multispectral, evaluating 16 uniformly stratified validation generations per model (48 queries to- tal) against all 35,532 training panoramas using L2-normalized 2048-D InceptionV3 embeddings. Visual inspection of Generated / Closest Training Neighbor / Validation GT triplets, including the global highest-similarity pairs across all 48 evaluations confirms zero near-duplicate copying: road curvature, lane markings, buildings, and tree/terrain silhouettes differ substantially from every retrieved training panorama. If predictions relied on memorizing adjacent frames from overlapping Street View drive sequences, nearest-neighbor retrieval over all 35,532 training panoramas would surface co-located sequence neighbors. Instead, retrieved neighbors are geographically dispersed across the continental US, and for identical validation queries evaluated across the three models, each model retrieves a different training panorama separated by 198–3,793 km across different states, confirming conditioning-driven scene synthesis rather than capture-sequence memorization. We have included this analysis in the updated draft with example visualizations (Figure 27-29).
Response 2
Weakness 2
We agree that most of the gap to earlier systems comes before the critic: our low-resolution model without PIC already reaches FID 4.40, and our HR-conditioned model reaches 3.93. The revised Section 4.2 states this. The contribution of PIC is measured by the matched comparison in Table 2 (S+A: FID 4.40 → 3.87, AbsRel 0.1385 → 0.1057).
To test simpler uses of the supervision, we ran two controls on CVUSA from the same Stage-1 checkpoint and budget (100 epochs, 13.8k steps, BS 256, LR 1e-5).
Table R2. Critic controls on CVUSA (paper: Table 9). Each cell lists S / A / S+A.
| Variant | FID ↓ | LPIPS ↓ | PSNR ↑ | SSIM ↑ |
|---|---|---|---|---|
| Matched-compute panorama LPIPS (no critic) | 13.14 / 12.61 / 12.10 | 0.61 / 0.59 / 0.59 | 11.84 / 12.26 / 12.22 | 0.34 / 0.35 / 0.35 |
| Blurred-target critic (σ = 8, ~3 m/px) | 7.78 / 7.32 / 7.48 | 0.61 / 0.60 / 0.60 | 11.91 / 12.29 / 12.33 | 0.32 / 0.34 / 0.34 |
| Sharp-target critic (Bifröst) | 7.15 / 6.63 / 6.73 | 0.62 / 0.59 / 0.59 | 11.69 / 12.14 / 12.17 | 0.32 / 0.33 / 0.33 |
Under these controls, the sharp-target critic gives the lowest FID for every input; pairwise metrics are similar, and PSNR and SSIM slightly favour the controls. We have not yet compared against teacher distillation or an auxiliary overhead-reconstruction head with the same backbone and budget, and we list this explicitly as future work (App. H.5).
Geometric evidence. Depth and segmentation are foundation-model proxies, so we present these results as improved structural consistency rather than verified location-specific accuracy (App. H.4).
Panorama calibration (Sec. 3.3, App. B.2)
Neither dataset is modelled as a full sphere. Both use a 90° vertical field of view (Shi et al., 2022; Dai et al., 2023), and perspective views are reprojected only within this retained band.
| Quantity | Value |
|---|---|
| Vertical crop | Central 50% of 512×1024 (rows [128, 384)) |
| Vertical field of view | 90°, elevation φ ∈ [−45°, +45°] |
| Azimuth of column c | θ(c) = 2π(c − W/2)/W ∈ [−π, π), north-centred |
| Elevation of row r | φ(r) = π/4 − (π/2)/(H − 1) · r |
| Perspective reprojection | v = ½(1 − 4φ/π)H − ½, with 360° azimuthal wrap-around |
References.
-
Ming Dai, Enhui Zheng, Zhenhua Feng, Lei Qi, Jiedong Zhuang, and Wankou Yang. Vision-based uav self-positioning in low-altitude urban environments. IEEE Transactions on Image Processing, 33:493–508, 2023
-
Y. Shi, D. Campbell, X. Yu, and H. Li. Geometry-guided street-view panorama synthesis from satellite imagery. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 44 (12):10009–10022, 2022
Question 1
FID Implementation Details and Baselines (Appendix section E.1 and E.2 in paper): Thank you for raising this. We have retrained our model and the baselines on CVUSA and reported the updated numbers in Table 1. Please refer to our detailed response to W1 above on the implementation and settings.
Question 2
Neighbourhood searches and concerns on overfitting (Appendix Section G): Thank you for raising this. We have examined the potential for overfitting and elaborated on the experiment settings and the evidence against it in the response to W1 above
We hope these changes address the reviewer's concerns, and we would be grateful to know whether any remain.
Response 1
Weakness 1
CVUSA Experiment Settings (Appendix E.1 Table 6 in updated paper): | | Sat2StrPanorama | Sat2Density | Sat2Density++ | Bifröst (S / A / S+A) | |---|---|---|---|---| | Test-time input | RGB aerial 256², zoom 19 | RGB aerial 256², zoom 19 | RGB aerial 256², zoom 19 | Sentinel-2 and/or AlphaEarth (no HR) | | Model | 64 height planes, concentric-cylinder projection | 65 density planes, 100 ray samples | 256²×96 tri-plane, 64×256 volume rendering, 4× StyleGAN2 SR | Pixel-space U-ViT; frozen critic in Stage 2 | | Losses | PatchGAN + VGG perceptual | Multi-scale panorama + perspective discriminators | Street + satellite discriminators, R1 | Sigmoid-weighted diffusion loss (+ PIC loss in Stage 2) | | Optimizer, LR | Adam, 2e-4 | Adam, 5e-5 | Adam, 2.5e-3 (generator) | AdamW, 1e-4 / 1e-5 | | Batch size | 16 | 16 | 32 | 512 / 256 | | Training | 35 epochs (77.7k steps) | 30 epochs (66.6k steps) | 19k [Still Running] | 41k + 14k steps |
FID & LPIPS Calculation Protocol (Appendix E.1 in updated paper).
We compute FID using the penultimate layer activations of a pre-trained Inception-v3 to quantify global visual manifold coverage. All methods use the same implementation over the full evaluation split, never per batch: RGB panoramas in [0, 1] are scaled to [0, 255], bilinearly resized from 256 × 1024 to 299 × 299 (TF- 1.x convention, align corners=False), normalized as (x − 128)/128, and passed through the standard inception-2015-12-05 weights to accumulate 2048-d pool3 statistics (μ, Σ), from which we compute the exact Frechet distance for the test sets. To assess local pairwise perceptual fidelity robust to spatial phase stochasticity, we compute LPIPS using a pre-trained VGG backbone (for our 556k benchmark) and AlexNet backbone for CVUSA.
Thank you for highlighting this. Since we are unable to release the proprietary data, we have instead included evals and ablations on the public CVUSA dataset, by retraining the baselines and our model on the publicly available data splits. In addition, we have included detailed experimental settings to reproduce the implementation as well, in Appendix sections D, E and F.