acceptodds
Under review as a conference paper at ICLR 2027

Loc2Drive: Beyond Visual History with Explicit 3D World State for Long-Horizon Driving Video Generation

Abstract

Long-horizon driving-video generation is vulnerable to autoregressive error accumulation because generated frames are repeatedly reused as future context. Yet some errors arise before propagation: future views may reveal static scene content that the current visual history has never observed. Turns and disocclusions make this gap especially clear, exposing road topology or appearance outside the previous field of view or behind occluders. We present Loc2Drive, which supplies missing static evidence from route-queryable satellite imagery. At each ego pose, Loc2Drive queries a heading-aligned satellite crop and reconstructs a shared 3D world state. The state combines semantic occupancy for geometry and visibility, a height-aware 3D feature volume that preserves appearance cues, and BEV road semantics learned with HD-map supervision. Projecting this state through six calibrated cameras yields pixel-aligned RGB, depth, semantic, and pseudo-HD-map conditions for video generation. Satellite evidence is re-queried along the route rather than inferred recurrently from generated frames; specified dynamic agents remain on a separate 3D-box path, while the video prior handles unconstrained appearance and dynamics. Without an external HD map at inference time, Loc2Drive reduces FVD by 14.7% at 16 frames and by 29.7% at 128 frames relative to UniMLVG with HD maps. With no visual references, it further reduces by 23.1%. Explicit world-aligned reconstruction also outperforms direct 2D satellite fusion.

Primary area:
generative models
Keywords:
driving video generation, long-horizon generation, world models, satellite imagery, 3D world representation, multi-view video generation, geographic grounding

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_aPJAOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Loc2Drive augments a pretrained six-camera driving-video generator with geographically indexed satellite evidence. At successive route positions, it predicts a local state comprising semantic occupancy, a height-aware feature volume, and HD-map-supervised BEV road semantics. Camera projection produces aligned RGB, depth, semantic, and pseudo-HD-map conditions. A modality-specific encoder and shared trunk produce satellite residuals, which Control–State Fusion integrates using the original control residual and the current DiT hidden state. Extensive experiments, including baseline comparisons, long-horizon autoregressive generation, evaluations with limited reference inputs, and component ablations on nuScenes, as well as qualitative cross-city analyses, demonstrate the effectiveness of the proposed method.

Critical strengths

Identify one or two critical strengths and explain why they support acceptance.

  1. A coherent way to supply persistent geographic evidence. The method combines occupancy-based visibility, lifted appearance features, and road semantics in a shared coordinate system before producing six-view conditions. Refreshing these conditions from satellite imagery provides a static information source that does not directly inherit errors in generated references. Separating this source from supplied dynamic-agent boxes is a useful design choice. The contribution is consequential as a practical integration of geographic reconstruction and controllable driving-video generation.
  2. Consistent gains in matched evaluations. With external HD-map control, standard-window FVD improves from 36.11 to 29.77, and FVD at 128 frames improves from 88.65 to 61.44 (Tables 1–2). The variant without an inference-time external HD map also improves standard FVD to 30.80. In the zero-reference setting, the full model reduces FVD from 106.01 to 73.96. These comparisons retain the same 3D-box controls and are matched within each setting. Together with the representation and integration ablations, they provide meaningful empirical support for the complete system.
Critical weaknesses

Identify one or two critical weaknesses and explain why they weigh against acceptance.

  1. The evidence does not isolate reduced autoregressive error propagation. Missing information about future static content and error propagation through generated history are related but distinct problems. A stronger condition can improve every prediction window without changing how existing errors propagate. Loc2Drive still reuses generated frames, and better rollout FVD alone does not separate these effects. Moreover, the eligible evaluation subset changes across horizons. Appendix C.2 acknowledges that the stress tests are not a causal decomposition. I therefore interpret the results as evidence for better long-horizon conditional generation, while the specific explanation in terms of error propagation remains insufficiently established. This limits the strength of the paper’s central motivation.
  2. The incremental contribution needs stronger attribution against close alternatives. SR-AD already uses future-route geographic retrieval, and SatDreamer360 already uses satellite-derived 3D tri-plane representations and ray-based querying. Loc2Drive’s distinction lies mainly in the explicit occupancy/appearance/road decomposition and its integration into a pretrained six-camera video model. The direct 2D satellite cross-attention baseline achieves an FID of 5.74, relatively close to the full method’s 5.31, although the full method provides a larger improvement in FVD (35.13 to 29.77). Since the comparison explicitly matches only Stage-II settings, clarifying the Stage-I supervision and total training resources would help assess how much of this improvement comes from the explicit 3D representation. In addition, rendered RGB alone with an expanded input reaches FVD 30.17, compared with 29.77 for the full method (Appendix E.1). This does not negate the 3D reconstruction, which supplies that RGB condition, but it leaves the added value of the complete multimodal fusion less compelling.
Questions for authors

Ask only questions whose answers could materially affect the assessment.

  1. Mechanism and scope. What evidence separates improved per-window prediction from reduced propagation of errors in generated history? A focused diagnostic on the same routes and targets, comparing generated-history conditioning with a real-history counterfactual, could help. If this separation is unavailable, can the central claim be restricted to improved generation under stronger static conditioning?
  2. Comparison and supervision. Does the direct 2D baseline receive supervision and training resources comparable to the complete Stage-I pipeline? Please clarify the exact differences. Do the advantages over the rendered-RGB-only variant persist in long rollouts, where the paper’s motivation is strongest?
  3. Robustness to alignment and reconstruction errors. Satellite-derived conditions face two sources of uncertainty: potential errors in geographic registration, and the limited information that overhead RGB imagery provides about building facades and vertical structure. Could these uncertainties introduce ambiguous or misleading structural cues, thereby compromising the geometric consistency of the generated videos across views or over time? Could the authors provide robustness analysis, failure cases, or quantitative evaluations of geometric consistency? For example, a pretrained model such as VGGT[1] could estimate camera poses from overlapping generated views or temporally adjacent frames. Comparing the recovered relative poses with the known conditioning poses could provide an additional measure of cross-view or temporal geometric consistency. [1] Jianyuan Wang et al. VGGT: Visual Geometry Grounded Transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.
2 comments
AuthorsOpenReview ↗

Response to Reviewer aPJA: per-window prediction vs. error propagation, attribution, long rollouts, robustness, and geometric consistency

W1/Q1. Per-window prediction and error propagation.

We thank the reviewer for the precise question. Loc2Drive improves per-window prediction and also reduces the propagation of errors in generated history. As the reviewer notes, its satellite state "does not directly inherit errors in generated references".

(1) Mechanism. A window's only history is the previous window's last three frames, which the GPS-indexed state never reads; re-queried along the route, the state reaches every window and improves every handoff.

(2) A better start alone does not slow propagation (Table R9). Querying the state only at the first frame separates the two effects. This variant improves the first window over the UniMLVG baseline (Chen et al., 2025c), yet its FVD still grows 2.44× by 128 frames, like UniMLVG (2.45×); re-querying at every frame limits the growth to 2.02×. A stronger start thus leaves propagation unchanged; re-querying slows it by re-anchoring every window's static scene to evidence free of the history's errors.

(3) One cohort for all horizons. Our description in the paper may have suggested that the subsets change with horizon; they do not. Every horizon is evaluated on the same 150 val clips, one per scene from its first frame; longer horizons roll the same clips further, so Table R9 measures growth on a fixed cohort.

Table R9. FVD growth on the same 150 val clips (three references).

MethodFVD₁₆ ↓FVD₁₂₈ ↓FVD₁₂₈ / FVD₁₆ ↓
UniMLVG (+H)36.1188.652.45
First-frame query (no H)34.3583.892.44
Every-frame query (no H)30.8062.332.02
Loc2Drive (+H)29.7761.442.06

W2/Q2. Attribution against close alternatives.

The gain comes from the explicit supervised state, which neither close alternative has.

(1) SR-AD (Jia et al., 2026) fuses satellite features and street views retrieved at the target location through 2D cross-attention. Its 5.60 FID / 29.97 FVD is self-reported under its own protocol; Loc2Drive reaches 5.31 / 29.77 without any target street views.

(2) SatDreamer360 (Ze et al., 2026) synthesizes ground views from a satellite tri-plane, without supervised occupancy and road semantics, box control, or route re-querying.

(3) The SR-AD-style 2D baseline is our controlled comparison. It uses the same satellite encoder, initialized from ImageNet, and is trained end to end in Stage II under the same schedule. It lacks only Stage I, whose occupancy, masked-RGB, depth, and perspective-semantic losses all require the lifted volume and its projection, so the comparison isolates this supervision. The explicit state lowers FVD₁₆ from 35.13 to 29.77 (Tab. 3b), and to 30.57 even without HD-map labels.

(4) Rendered RGB alone supports the explicit state. This RGB condition is rendered from the state (App. E.1 D) and already outperforms 2D fusion (30.17 against 35.13), so it tests how the state is injected, not whether it is needed.

Q2. Long rollouts (Table R10). Yes, the advantage persists and widens: the full design leads the rendered-RGB-only variant by 0.40, 2.21, and 4.33 FVD at 16, 64, and 128 frames.

Table R10. Three references, external HD map, clips as in Table 2a.

MethodFVD₁₆ ↓FVD₆₄ ↓FVD₁₂₈ ↓
UniMLVG (+H)36.1187.5688.65
Rendered RGB only (App. E.1 D)30.1768.9165.77
Full design29.7766.7061.44

Q3. Registration, vertical structure, and geometric consistency.

Neither uncertainty breaks geometric consistency, which we now measure on the generated videos.

(1) Robustness to misregistration. Training pairs already contain the imagery's registration error. At inference, shifting the query by 2 m or 4 m, or rotating it by 5°, gives FVD₁₆ of 31.19, 31.56, and 31.41, against 30.80 unperturbed and 36.11 for UniMLVG.

(2) Vertical structure. The state spans −1.0 m to 5.4 m, the volume that shapes the drivable scene; higher facades are left to the video prior, so the overhead imagery cannot inject a misleading vertical cue.

(3) Geometric consistency (Table R8). All six cameras sample one state, so a registration error shifts every view together instead of making the views disagree. As the reviewer suggests, we use VGGT (Wang et al., 2025) to recover the poses of the six generated views and compare the relative rotations of overlapping cameras with the calibration in the first and last windows of the rollout. Loc2Drive stays closer to real video in both windows, and its lead over UniMLVG grows from 1.45 to 3.08.

Table R8. Cross-view consistency on 150 val clips (three references): AUC@30 of the VGGT relative rotations of the six overlapping camera pairs against the calibration.

VideoFrames 1–16 ↑Frames 113–128 ↑
Real nuScenes video91.0391.65
UniMLVG (+H)87.2383.47
Loc2Drive (+H)88.6886.55

We hope these results strengthen the reviewer's confidence in the paper.

AuthorsOpenReview ↗

Follow-up on error propagation and the VGGT check

Dear aPJA, thank you for your positive assessment and for the precise questions, which helped us sharpen the paper.

Your main reservation was whether the long-horizon gain reflects reduced error propagation or only better per-window prediction. In our reply, querying the state only at the first frame separates the two: it improves the first window, yet FVD still grows 2.44× by 128 frames, like UniMLVG (2.45×), whereas re-querying at every frame slows the growth to 2.02×. We also ran the VGGT check you suggested: the generated views of Loc2Drive stay more consistent than UniMLVG's, with the lead growing from 1.45 to 3.08 AUC@30 over the rollout.

We would be grateful to know whether this addresses your concern about the central claim, and we are happy to clarify anything else before the discussion closes.

Reviewer_JY56OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents Loc2Drive, a satellite-grounded framework for long-horizon driving video generation. The authors argue that generation errors arise not only from autoregressive error accumulation but also from insufficient visual evidence of previously unseen static scene content, particularly during turning and disocclusion. To address this issue, the method retrieves route-aligned satellite imagery and reconstructs an explicit 3D world state consisting of semantic occupancy, height-aware appearance features, and road semantics. The reconstructed information is projected into six camera views and integrated into a pretrained video generator through Control–State Fusion. Experiments on nuScenes report improvements in video generation quality across different rollout horizons and visual-reference settings.

Critical strengths
  1. The paper offers an interesting perspective on the role of missing visual evidence in long-horizon driving video generation.
  2. The proposed framework combines geometric, semantic, and appearance information in a coherent representation.
  3. The experimental results show consistent improvements across different generation horizons and reference settings.
Critical weaknesses
  1. The proposed observability deficit is not directly examined. The paper argues that missing static scene evidence is an important source of generation errors. However, the evaluation primarily relies on global FID/FVD metrics, without distinguishing previously observed regions from newly revealed content. It is therefore difficult to determine whether the improvements specifically address the identified problem.

  2. The role of repeated geographic queries remains unclear. Re-querying satellite imagery along the route is presented as a key mechanism for mitigating long-horizon error propagation. Yet, the experiments do not compare this strategy with one-time or less frequent geographic conditioning, leaving its individual contribution uncertain.

  3. Geometric consistency deserves a more direct evaluation. While the explicit 3D representation outperforms direct 2D satellite fusion, the experiments lack quantitative measurements of cross-view geometric consistency or temporal scene stability. Such evidence would better support the motivation for reconstructing a shared 3D world state.

  4. Geographic grounding is demonstrated mainly through visual examples. The point-addressed experiments provide interesting qualitative results, but the absence of quantitative geographic correspondence metrics makes it difficult to assess how accurately the generated scenes reflect the queried locations.

Questions for authors
  1. How do the improvements vary between previously observed static regions and newly revealed areas during turning or disocclusion?
  2. An ablation comparing different satellite re-query frequencies, including one-time conditioning, would help clarify the importance of the proposed route-aligned updating strategy.
  3. It would be valuable to provide quantitative evidence of cross-view consistency and geographic correspondence, rather than relying primarily on overall generation metrics and qualitative examples.
6 comments
AuthorsOpenReview ↗

Response to Reviewer JY56: direct tests of newly revealed regions, re-query frequency, geometric consistency, and geographic correspondence

W1/Q1. Previously observed vs. newly revealed regions.

We thank the reviewer for finding our perspective interesting and examine the observability deficit directly. Most of what a rollout generates is unseen by its references; the state is equally accurate there, and the gain grows on turns.

(1) The references see little of what follows. Even at a single timestamp, the six cameras miss 34.78% of the occupied voxels (App. C.6). Over the next 16 keyframes (8 s), aligned with the reference keyframes on 3,120 val windows, this share rises to 87.4% with one reference keyframe and 83.0% with three.

(2) The state is equally accurate where the references are blind (Table R5). On these voxels, Stage I reaches 55.03 mIoU, on par with the observed voxels (54.92).

(3) The gain grows on turns, where new areas are revealed (Table R6). On turning clips, Loc2Drive reduces FVD₁₆ relative to the UniMLVG baseline (Chen et al., 2025c) by 15.45 (17.8%), against 7.37 (14.9%) on straight clips.

Table R5. Stage-I static mIoU (%) on the voxels of the next 16 keyframes.

Voxels1 ref. keyframe3 ref. keyframes
Unseen by the references53.5455.03
Seen by the references54.9154.92

Table R6. FVD₁₆ of 150 val clips by heading change over each 3.5 s clip (three references, +H). FVD is higher on smaller sets; compare within a row.

Heading change# clipsUniMLVG ↓Loc2Drive ↓Drop
Below 5°10449.6242.257.37
5° or above4686.5871.1315.45
All (Tab. 1)15036.1129.776.34

W2/Q2. Re-query frequency.

As requested, we compare one-time and half-rate conditioning (Table R7): re-querying along the route accounts for 82% of the 128-frame gain.

Between queries, the state stays fixed in world coordinates and is re-rendered at each new pose. Half-rate querying retains most of the gain. One-time conditioning still improves the first window, but by 128 frames it is close to UniMLVG, so re-querying supplies 21.56 of the 26.32 FVD gain.

Table R7. Re-query rate (no external HD map; clips as in Tab. 2a).

Re-query rateFVD₁₆ ↓FVD₁₂₈ ↓
Every frame (Tab. 2a)30.8062.33
Every 2 frames31.4365.12
First frame only34.3583.89
No satellite: UniMLVG +H (Tab. 2a)36.1188.65

W3/Q3. Cross-view geometric consistency and temporal stability.

We now measure consistency directly: the generated views of Loc2Drive stay closer to real video than UniMLVG's, and the lead grows from 1.45 to 3.08 over the rollout (Table R8). All six cameras render one state, so overlapping views agree by construction (App. F.1).

(1) The shared state is accurate. From satellite imagery alone, Stage I reaches 53.57 occupancy mIoU, above FlashOcc (Yu et al., 2023; 48.02) and within one point of SA-Occ (Chen et al., 2025a; 54.55, cameras plus satellite). Its rendered semantics reach 59.60–64.11 mIoU in each camera.

(2) The generated videos (Table R8). At every timestamp, we use VGGT (Wang et al., 2025) to recover the poses of the six generated views; in consistent views, the relative rotations of overlapping cameras match the calibration, which we test in the first and last windows of one rollout to also measure temporal stability.

Table R8. Cross-view consistency on 150 val clips (three references): AUC@30 of the VGGT relative rotations of the six overlapping camera pairs against the calibration.

VideoFrames 1–16 ↑Frames 113–128 ↑
Real nuScenes video91.0391.65
UniMLVG (+H)87.2383.47
Loc2Drive (+H)88.6886.55

W4/Q3. Geographic correspondence.

The generated videos follow the road layout of the queried location, on nuScenes and in six unseen cities.

(1) nuScenes. Without the external HD map, a fixed BEV road-segmentation probe on the generated videos (Tab. 1) gives Loc2Drive 71.27 road mIoU against the location's HD map, above UniMLVG given that map (70.81).

(2) Six unseen cities (Table R4). In the strict setting of Section 4.6 (no references, HD map, boxes, or text), the layout can only come from the satellite. Against OpenStreetMap, the generated videos reach 56.9% road IoU, within 4 points of the satellite state in every city, but only 17.4% against a mismatched location's roads.

Table R4. Road IoU (%) against drivable OpenStreetMap roads, strict setting. OSM roads are centerlines buffered to estimated widths, so the reference is coarse and the IoU conservative.

CityStage-I state ↑Generated video ↑
London60.158.0
New York57.956.2
Shanghai58.057.1
Dubai62.359.9
Paris59.455.8
Tokyo58.654.6
Mean59.456.9

Ethics flag.

As the flag states no specific concern, we address licensing, privacy, and misuse in a separate comment and welcome clarification.

We hope the direct tests above merit a reconsideration of the rating.

AuthorsOpenReview ↗

Discussion invitation: a brief summary of our reply

Dear JY56, thanks for taking the time and effort to help us improve this work.

We understand that long replies take time to read, so we put a brief summary below. You can skim it first. If anything is still a concern, you can find the details in our point-by-point reply above. We included these details because we hoped to address your concerns as fully as possible.

A brief summary

  • Observed vs. newly revealed regions. The gain over UniMLVG is larger where new areas are revealed: FVD₁₆ drops by 15.45 on turning clips against 7.37 on straight ones, and on voxels the references never saw, the state is as accurate as on observed ones (55.03 vs. 54.92 mIoU).
  • Re-query frequency. Re-querying supplies most of the long-horizon gain: one-time querying gives 83.89 FVD₁₂₈, close to UniMLVG (88.65), while every-frame querying gives 62.33.
  • Cross-view consistency and temporal stability. Measured with VGGT, the generated views of Loc2Drive are more consistent than UniMLVG's, and the lead grows from 1.45 AUC@30 in the first window to 3.08 in the last window of a 128-frame rollout.
  • Geographic correspondence. In six unseen cities, the generated videos reach 56.9% road IoU against the queried location's OpenStreetMap roads. Scored against the roads of another location in the same city, the same videos reach only 17.4%, so the road layout comes from the queried location.

We also addressed the ethics flag in a separate comment. We'd be happy to hear your thoughts or discuss any remaining concerns.

Thank you for the detailed responses and additional experiments. However, I still find the evaluation of consistency and geographic fidelity insufficient, as camera pose consistency and road IoU do not fully demonstrate the actual 3D consistency and overall scene fidelity of the generated videos. I therefore maintain my original score.

Reviewer_n6csOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents an interesting approach to improving long-horizon driving video generation by incorporating satellite imagery as additional geospatial information. However, the computational cost, geometric accuracy of the reconstructed 3D World State, temporal mismatch, and dependence on HD-map supervision remain insufficiently addressed. If the authors can address these concerns with additional experiments and analysis, I would be willing to reconsider my rating.

Critical strengths

1、The paper introduces the concept of Future-scene Observability Deficit, pointing out that autoregressive driving video generation models relying only on historical observations struggle to predict future static scenes. Using satellite imagery as additional geospatial information is an interesting idea that could help reduce accumulated scene structure errors in long-horizon generation. 2、The paper proposes a complete framework that reconstructs an explicit 3D World State from satellite imagery and integrates it into video generation through multi-view projection and adaptive condition fusion. The experimental results are promising, with an approximately 29.7% reduction in FVD over the baseline in the 128-frame setting. The ablation studies also provide evidence for the effectiveness of the proposed modules.

Critical weaknesses

1、The method uses a dense 200×200×16 voxel grid, with 89 depth samples for each camera ray. It also updates satellite imagery and the world state at 10 FPS, which could introduce considerable computational overhead, especially in multi-camera and long-horizon settings. However, the paper does not report inference latency or GPU memory consumption. 2、Satellite images do not directly provide height or facade information, yet the method uses them to construct an explicit 3D representation through learned height distributions and semantic features. The current evaluation mainly focuses on downstream video generation quality, which does not directly demonstrate the geometric accuracy or multi-view consistency of the reconstructed 3D World State. 3、The nuScenes driving videos were collected in 2019, while the satellite imagery mainly comes from 2025 and 2026. Changes in roads, buildings, and vegetation could introduce incorrect spatial information. It is unclear whether Background Purification and the year-based ablation studies are sufficient to address this issue. 4、Although the method does not require external HD maps during inference, the BEV Road Semantics branch in Stage I still relies on HD-map annotations for training.

Questions for authors

1、What is the computational overhead of the proposed 3D World State construction and projection pipeline? Can the authors report inference latency, GPU memory consumption, and throughput compared with the baseline? Is the method efficient enough for practical long-horizon or real-time generation? 2、How accurately does the reconstructed 3D World State reflect the actual scene geometry? Can the authors provide quantitative metrics, such as 3D occupancy IoU, height estimation error, depth accuracy, or multi-view geometric consistency, to validate the reconstruction? 3、How does the method perform when satellite imagery and driving videos have significant temporal gaps? Can the authors evaluate its robustness using outdated satellite images or images from different years? 4、How much does the method rely on HD-map annotations during training? Can the authors evaluate its performance without HD-map supervision? Can they also provide quantitative results on unseen cities to better demonstrate its generalization ability?

2 comments
AuthorsOpenReview ↗

Response to Reviewer n6cs: new experiments on cost, state accuracy, the temporal gap, HD-map reliance, and unseen cities

W1/Q1. Latency, memory, and throughput.

We thank the reviewer. The satellite state is not a bottleneck (Table R1): Loc2Drive is only 6.1% slower than the UniMLVG baseline (Chen et al., 2025c) end to end.

(1) The dense grid is cheap to render. The 89 samples per ray are marched at 64×112, a quarter of the 256×448 output per side, and two ×2 upsampling stages restore the resolution.

(2) The 10 Hz refresh is negligible. One Stage-I pass takes 0.0366 s per frame, so the 16 refreshes of a window add 0.59 s (0.13%).

(3) End-to-end throughput. Loc2Drive generates 1.96 new six-view frames per minute, against 2.08 for UniMLVG; both are bound by the shared 50-step sampler, not by the satellite state.

Table R1. One H800 GPU, batch size 1. A window has three references and 16 new frames (50 steps, CFG 3), including Loc2Drive's 16 state refreshes.

Latency (s)Peak memory (GB)
Projection, per frame0.03270.84
Stage I, per frame0.03661.52
UniMLVG, per window461.248.2
Loc2Drive, per window489.256.0

W2/Q2. Geometric accuracy of the reconstructed state.

From satellite imagery alone, the state outperforms a six-camera baseline with the same backbone and occupancy head and comes within one point of a camera-plus-satellite model.

(1) 3D occupancy IoU (Table R2). On 6,019 val frames, Stage I reaches 53.57 static mIoU, above FlashOcc (Yu et al., 2023; 48.02) and on par with SA-Occ (Chen et al., 2025a; 54.55). It is the best of the three on manmade and vegetation, the classes that depend most on height; Stage I learns this height from street-view supervision.

(2) Multi-view consistency. All cameras sample one world-space state, so overlapping views receive identical voxel features by construction (App. F.1). Against LiDAR labels projected into each camera, the rendered semantics reach 59.60–64.11 mIoU.

Table R2. Per-class IoU (%).

MethodInputDriveableOther flatSidewalkTerrainManmadeVegetationmIoU
FlashOccsix cameras78.838.048.752.537.932.248.02
SA-Occsix cameras and satellite82.643.854.058.547.041.454.55
Loc2Drive Stage Isatellite only77.9247.1152.0753.1048.4942.7153.57

W3/Q3. Temporal gap between videos and imagery.

Every result in the paper, including the 29.7% FVD₁₂₈ reduction noted by the reviewer, is already measured under this six-to-seven-year gap.

(1) Prior work uses the same pairing. SA-Occ, whose satellite data pipeline we inherit, pairs 2025 imagery with the same 2019 nuScenes cameras and still raises mIoU from 32.08 to 39.05.

(2) Our design goes further. Unlike SA-Occ, we use satellite imagery as the only input and street views only as supervision, so the state learns what both sources still share. Background purification removes movable classes (Tab. 3a; FVD₁₆ 33.64 → 29.77), and pose-aware temporal consistency aligns adjacent states (App. E.2 G; 33.01 → 29.77).

(3) We test the gap directly (App. E.2 H). Validation uses 2026 imagery. Training on 2025 imagery alone gives 30.31 FVD₁₆, close to training on 2026 alone (30.19) and far below UniMLVG (36.11); mixing both years is best (29.77).

W4/Q4. HD-map supervision and unseen cities.

The HD-map labels only supervise the Stage-I road branch, and Loc2Drive depends little on them.

(1) Reliance on HD-map labels. Without the labels, FVD₁₆ rises only from 29.77 to 30.57; without the whole road branch, it rises to 31.88 (Tab. 3a).

(2) Without HD-map supervision (Table R3). Even with no HD map in training or inference, Loc2Drive outperforms UniMLVG given the HD map: 62.66 against 88.65 FVD₁₂₈, and 71.20 against 70.81 road mIoU.

(3) Unseen cities (Table R4). Trained only on Boston and Singapore and tested in six new cities in the strict setting of Section 4.6 (no references, HD map, boxes, or text), the state reaches 59.4% road IoU against OpenStreetMap, and the generated videos 56.9%.

Table R3. HD-map labels and the external HD map.

ModelHD-map labelsExternal HD mapFVD₁₆ ↓FVD₁₂₈ ↓Road mIoU ↑
UniMLVG–yes36.1188.6570.81
Loc2Driveyesyes29.7761.4471.41
Labels offnoyes30.5761.8871.36
Labels and map offnono31.1762.6671.20

Table R4. Road IoU (%) against drivable OpenStreetMap roads, strict setting. OSM roads are centerlines buffered to estimated widths, so the reference is coarse and the IoU conservative.

CityStage-I state ↑Generated video ↑
London60.158.0
New York57.956.2
Shanghai58.057.1
Dubai62.359.9
Paris59.455.8
Tokyo58.654.6
Mean59.456.9

We hope these additional experiments address the reviewer's concerns and merit a reconsideration of the rating.

AuthorsOpenReview ↗

Discussion invitation: a brief summary of our reply

Dear n6cs, thanks for taking the time and effort to help us improve this work.

We understand that long replies take time to read, so we put a brief summary below. You can skim it first. If anything is still a concern, you can find the details in our point-by-point reply above. We included these details because we hoped to address your concerns as fully as possible.

A brief summary

  • Overhead of the dense grid and 10 Hz refresh. The full state pass, including the 89-sample projection, takes 0.0366 s and 1.52 GB per frame, so refreshing it at 10 Hz adds only 0.13% to each window's generation time.
  • Geometric accuracy of the state. From satellite imagery alone, the state reaches 53.57 occupancy mIoU, above a six-camera baseline with the same backbone (48.02), and the largest gains are on manmade and vegetation, the classes that depend most on height.
  • Temporal gap. SA-Occ, whose pipeline we inherit, already gains from pairing 2025 imagery with the 2019 cameras (32.08 → 39.05 mIoU). We go further: satellite imagery is our only input, and background purification removes movable classes (FVD₁₆ 33.64 → 29.77). Training on 2025 imagery and testing on 2026 imagery gives 30.31 FVD₁₆, close to 30.19 with matched years.
  • Reliance on HD-map labels. Removing them changes FVD₁₆ only from 29.77 to 30.57.
  • Unseen cities. With no reference frames or HD map, the generated videos reach 56.9% road IoU against OpenStreetMap in six unseen cities.

We'd be happy to hear your thoughts or discuss any remaining concerns.

Comments on the paper

4 comments
AuthorsOpenReview ↗

Ethics considerations in response to the ethics flag: licensing, privacy, and misuse

Reviewer JY56 flagged the submission for ethics review without stating a reason. We summarize the relevant facts below and will add an Ethics Statement to the revision.

  • Data and licensing. The driving data come from nuScenes (Caesar et al., 2020), released under CC BY-NC-SA 4.0 and the nuScenes Terms of Use, together with the Occ3D-nuScenes occupancy labels built on it (Tian et al., 2023); we use them for non-commercial research only. The satellite imagery is the georeferenced Google Earth imagery of the SA-Occ / Occ3D-nuScenes-SatExt data pipeline (Chen et al., 2025a), which we inherit. We will not redistribute raw imagery. The release will contain code, route and query metadata, alignment and preprocessing scripts, and redistributable derived annotations; access to the imagery itself remains subject to the provider's terms, as stated in the Reproducibility Statement.
  • Privacy. At 0.2 m/pixel, the overhead imagery cannot resolve faces or license plates, and the nuScenes camera images are released with faces and license plates blurred by the dataset provider. The static background is generated from overhead evidence, and dynamic agents come only from user-specified 3D boxes, so the model does not reproduce particular people or vehicles from the queried locations.
  • Potential misuse. Location-addressable generation could be used to fabricate realistic-looking videos of real places. The outputs are synthetic rather than faithful records of those places: facades, illumination, and dynamics are generated. We will (i) state this explicitly in the paper and (ii) release the models under a research-only license whose usage policy prohibits deceptive use. The statement will also specify the intended use: offline simulation and data generation for autonomous-driving research.

As the flag does not state a specific concern, we would welcome clarification from the reviewer or the ethics reviewers, and are happy to provide further details.

Dear Reviewers,

Thank you for your thoughtful reviews and contributions. The authors have now submitted their rebuttals. Please take some time to review their responses and assess whether your concerns have been adequately addressed. If appropriate, please update your comments and ratings accordingly.

Thank you for your time and continued efforts.

Best regards,

Your AC

AuthorsOpenReview ↗

Summary of the revised PDF

We have uploaded a revised PDF. The main text still fits in nine pages. All changes are listed below, together with the threads they answer.

1. New Appendix G: Additional Results from the Discussion Period. It collects every experiment from the discussion under the same table labels as in the threads (R1–R10):

  • G.1 Computational cost (R1; n6cs W1/Q1)
  • G.2 Accuracy of the reconstructed state, including newly revealed voxels (R2, R5; n6cs W2/Q2, JY56 W1/Q1, W3/Q3)
  • G.3 Reliance on HD-map supervision (R3; n6cs W4/Q4)
  • G.4 Road IoU in the six unseen cities (R4; JY56 W4/Q3, n6cs W4/Q4)
  • G.5 Straight versus turning clips (R6; JY56 W1/Q1)
  • G.6 Re-query frequency and error growth (R7, R9; JY56 W2/Q2, aPJA W1/Q1)
  • G.7 Cross-view consistency of the generated videos (R8; JY56 W3/Q3, aPJA Q3)
  • G.8 Rendered RGB on long rollouts (R10; aPJA W2/Q2)
  • G.9 Robustness to query misregistration (aPJA Q3)

2. Ethics Statement. As promised in our ethics comment, we added an Ethics Statement after the Reproducibility Statement, outside the page limit. It covers data licensing, privacy, potential misuse, intended use, and the research-only release.

3. Evaluation clips (Sec. 4.3; App. C.4, C.5). The text now states that all methods and horizons use the same 150 validation clips, one per scene starting at its first frame, so longer horizons extend the same rollouts (aPJA W1/Q1). The reported numbers are unchanged.

4. Clarifications in the appendix.

  • App. C.1: Loc2Drive without the external HD map is a separately trained Stage-II checkpoint that uses no external HD map in either training or inference (n6cs W4/Q4).
  • App. E.1, Control C: the 2D baseline shares the satellite encoder and training schedule and lacks only Stage I (aPJA W2/Q2).
  • App. E.2, H: all imagery settings are validated on 2026 imagery, so the 2025-only row is a cross-snapshot test (n6cs W3/Q3).
  • App. D.3, F.1, and F.2 now point to the road-IoU and consistency results in Appendix G.

5. References. We added VGGT (Wang et al., 2025) and FlashOcc (Yu et al., 2023).

No other text, figure, or result has changed.

AuthorsOpenReview ↗

Summary of the discussion for the AC

We thank the reviewers and the AC.

Core contribution and findings. Long-horizon errors in driving-video generation are usually blamed on autoregressive accumulation. We identify an earlier source, a future-scene observability deficit: turns and disocclusions reveal static content the visual history never captured (the references miss 83–87% of the occupied voxels over the next 8 s), so the generator must guess it. What the cameras have not seen, a satellite has. As its name says, Loc2Drive turns a location into a driving video: given only the location and the dynamic agents to place, it re-queries satellite imagery as external, persistent memory and renders an explicit 3D state into six cameras as the starting point for generation, with no visual references, HD maps, or street-view imagery. This matters for scale: satellite imagery is inexpensive and already covers most of the world, whereas street-level references of a specific place must be collected on site. To our knowledge, no prior driving-video generator works from this input alone.

  • nuScenes. Without an inference-time HD map, FVD drops by 14.7% at 16 frames and 29.7% at 128 frames relative to UniMLVG given the map, and by 29.1% from the location and 3D boxes alone.
  • Re-querying curbs error growth. FVD grows 2.44× over 128 frames with one-time querying, like UniMLVG (2.45×), but only 2.02× with re-querying.
  • Unseen cities (exploratory). Generated roads match OpenStreetMap (56.9% IoU, vs. 17.4% for another location), consistent with the visual comparisons, suggesting a path toward generalizable driving-video generation.
ConcernRaised byWhat we addedWhere
Computational costn6csA state refresh takes 0.13% of a window; end to end, 6.1% slower than UniMLVG (R1)n6cs W1/Q1
Accuracy of the 3D staten6cs, JY56 ✓From satellite alone, 53.57 mIoU, above six cameras (48.02), near camera plus satellite (54.55); 59.60–64.11 mIoU per camera (R2)n6cs W2/Q2, JY56 W3/Q3
Cross-view consistencyJY56 ✓, aPJAViews share state features by construction; on VGGT (Wang et al., 2025) poses from generated pixels alone, Loc2Drive's lead over UniMLVG grows from 1.45 to 3.08 AUC@30 (R8)JY56 W3/Q3 and follow-up, aPJA Q3
Video–imagery temporal gapn6csSA-Occ gains from the same pairing (32.08 → 39.05 mIoU); 2025→2026 imagery gives 30.31 vs. 30.19 FVD₁₆ for matched years (App. E.2 H)n6cs W3/Q3
Reliance on HD mapsn6csWithout HD-map labels, FVD₁₆ rises only from 29.77 to 30.57; with no HD map in training or inference, 62.66 vs. 88.65 FVD₁₂₈ for UniMLVG given the map (R3)n6cs W4/Q4
Observed vs. newly revealed contentJY56 ✓The state is as accurate on unseen voxels (55.03 vs. 54.92 mIoU; R5), and the gain is larger on turns (15.45 vs. 7.37 FVD₁₆; R6)JY56 W1/Q1
Role of re-queryingJY56 ✓Re-querying supplies 82% of the 128-frame gain (R7)JY56 W2/Q2
Per-window prediction vs. propagationaPJAThe growth above is measured on the same 150 clips at every horizon (R9)aPJA W1/Q1
Attribution vs. SR-AD, SatDreamer360, and 2D fusionaPJAThe 2D baseline lacks only Stage I; the explicit state lowers FVD₁₆ from 35.13 to 29.77, and its lead over rendered RGB grows from 0.40 to 4.33 by 128 frames (R10)aPJA W2/Q2, Q2
Robustness to misregistrationaPJA2 m / 4 m query shifts or a 5° rotation give 31.19–31.56 FVD₁₆ (30.80 unperturbed; UniMLVG 36.11)aPJA Q3
Geographic correspondenceJY56 ✓, n6csOn nuScenes, a road probe on generated videos gives 71.27 mIoU vs. 70.81 for UniMLVG given the map; the unseen cities lack six-view ground truth, so OpenStreetMap is the only reference (R4)JY56 W4/Q3 and follow-up, n6cs W4/Q4
Ethics flagJY56Data licensing, privacy, misuse, and release planSeparate comment; Ethics Statement in the revision

✓ JY56's latest reply raises no remaining concern here; where n6cs or aPJA raised the same concern, the same evidence answers both.

Status of the discussion. In the initial review, Reviewer n6cs found the results promising and would reconsider the rating if the four concerns were addressed with additional experiments; we answered all four, but n6cs has not yet responded. Reviewer JY56 acknowledged our responses and experiments, and the first reply questioned only the consistency and geographic-fidelity evaluation (W3, W4). After our follow-up, the second reply thanked us for the clarifications and raised no remaining technical concern; it maintained the score citing overall writing quality, for which the review identifies no specific issue, and the lack of updates, which the revised PDF (Appendix G) now addresses. Reviewer aPJA has not responded yet.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.