- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Loc2Drive augments a pretrained six-camera driving-video generator with geographically indexed satellite evidence. At successive route positions, it predicts a local state comprising semantic occupancy, a height-aware feature volume, and HD-map-supervised BEV road semantics. Camera projection produces aligned RGB, depth, semantic, and pseudo-HD-map conditions. A modality-specific encoder and shared trunk produce satellite residuals, which Control–State Fusion integrates using the original control residual and the current DiT hidden state. Extensive experiments, including baseline comparisons, long-horizon autoregressive generation, evaluations with limited reference inputs, and component ablations on nuScenes, as well as qualitative cross-city analyses, demonstrate the effectiveness of the proposed method.
Critical strengths
Identify one or two critical strengths and explain why they support acceptance.
- A coherent way to supply persistent geographic evidence. The method combines occupancy-based visibility, lifted appearance features, and road semantics in a shared coordinate system before producing six-view conditions. Refreshing these conditions from satellite imagery provides a static information source that does not directly inherit errors in generated references. Separating this source from supplied dynamic-agent boxes is a useful design choice. The contribution is consequential as a practical integration of geographic reconstruction and controllable driving-video generation.
- Consistent gains in matched evaluations. With external HD-map control, standard-window FVD improves from 36.11 to 29.77, and FVD at 128 frames improves from 88.65 to 61.44 (Tables 1–2). The variant without an inference-time external HD map also improves standard FVD to 30.80. In the zero-reference setting, the full model reduces FVD from 106.01 to 73.96. These comparisons retain the same 3D-box controls and are matched within each setting. Together with the representation and integration ablations, they provide meaningful empirical support for the complete system.
Critical weaknesses
Identify one or two critical weaknesses and explain why they weigh against acceptance.
- The evidence does not isolate reduced autoregressive error propagation. Missing information about future static content and error propagation through generated history are related but distinct problems. A stronger condition can improve every prediction window without changing how existing errors propagate. Loc2Drive still reuses generated frames, and better rollout FVD alone does not separate these effects. Moreover, the eligible evaluation subset changes across horizons. Appendix C.2 acknowledges that the stress tests are not a causal decomposition. I therefore interpret the results as evidence for better long-horizon conditional generation, while the specific explanation in terms of error propagation remains insufficiently established. This limits the strength of the paper’s central motivation.
- The incremental contribution needs stronger attribution against close alternatives. SR-AD already uses future-route geographic retrieval, and SatDreamer360 already uses satellite-derived 3D tri-plane representations and ray-based querying. Loc2Drive’s distinction lies mainly in the explicit occupancy/appearance/road decomposition and its integration into a pretrained six-camera video model. The direct 2D satellite cross-attention baseline achieves an FID of 5.74, relatively close to the full method’s 5.31, although the full method provides a larger improvement in FVD (35.13 to 29.77). Since the comparison explicitly matches only Stage-II settings, clarifying the Stage-I supervision and total training resources would help assess how much of this improvement comes from the explicit 3D representation. In addition, rendered RGB alone with an expanded input reaches FVD 30.17, compared with 29.77 for the full method (Appendix E.1). This does not negate the 3D reconstruction, which supplies that RGB condition, but it leaves the added value of the complete multimodal fusion less compelling.
Questions for authors
Ask only questions whose answers could materially affect the assessment.
- Mechanism and scope. What evidence separates improved per-window prediction from reduced propagation of errors in generated history? A focused diagnostic on the same routes and targets, comparing generated-history conditioning with a real-history counterfactual, could help. If this separation is unavailable, can the central claim be restricted to improved generation under stronger static conditioning?
- Comparison and supervision. Does the direct 2D baseline receive supervision and training resources comparable to the complete Stage-I pipeline? Please clarify the exact differences. Do the advantages over the rendered-RGB-only variant persist in long rollouts, where the paper’s motivation is strongest?
- Robustness to alignment and reconstruction errors. Satellite-derived conditions face two sources of uncertainty: potential errors in geographic registration, and the limited information that overhead RGB imagery provides about building facades and vertical structure. Could these uncertainties introduce ambiguous or misleading structural cues, thereby compromising the geometric consistency of the generated videos across views or over time? Could the authors provide robustness analysis, failure cases, or quantitative evaluations of geometric consistency? For example, a pretrained model such as VGGT[1] could estimate camera poses from overlapping generated views or temporally adjacent frames. Comparing the recovered relative poses with the known conditioning poses could provide an additional measure of cross-view or temporal geometric consistency. [1] Jianyuan Wang et al. VGGT: Visual Geometry Grounded Transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.
2 comments
Follow-up on error propagation and the VGGT check
Dear aPJA, thank you for your positive assessment and for the precise questions, which helped us sharpen the paper.
Your main reservation was whether the long-horizon gain reflects reduced error propagation or only better per-window prediction. In our reply, querying the state only at the first frame separates the two: it improves the first window, yet FVD still grows 2.44× by 128 frames, like UniMLVG (2.45×), whereas re-querying at every frame slows the growth to 2.02×. We also ran the VGGT check you suggested: the generated views of Loc2Drive stay more consistent than UniMLVG's, with the lead growing from 1.45 to 3.08 AUC@30 over the rollout.
We would be grateful to know whether this addresses your concern about the central claim, and we are happy to clarify anything else before the discussion closes.
Response to Reviewer aPJA: per-window prediction vs. error propagation, attribution, long rollouts, robustness, and geometric consistency
W1/Q1. Per-window prediction and error propagation.
We thank the reviewer for the precise question. Loc2Drive improves per-window prediction and also reduces the propagation of errors in generated history. As the reviewer notes, its satellite state "does not directly inherit errors in generated references".
(1) Mechanism. A window's only history is the previous window's last three frames, which the GPS-indexed state never reads; re-queried along the route, the state reaches every window and improves every handoff.
(2) A better start alone does not slow propagation (Table R9). Querying the state only at the first frame separates the two effects. This variant improves the first window over the UniMLVG baseline (Chen et al., 2025c), yet its FVD still grows 2.44× by 128 frames, like UniMLVG (2.45×); re-querying at every frame limits the growth to 2.02×. A stronger start thus leaves propagation unchanged; re-querying slows it by re-anchoring every window's static scene to evidence free of the history's errors.
(3) One cohort for all horizons. Our description in the paper may have suggested that the subsets change with horizon; they do not. Every horizon is evaluated on the same 150 val clips, one per scene from its first frame; longer horizons roll the same clips further, so Table R9 measures growth on a fixed cohort.
Table R9. FVD growth on the same 150 val clips (three references).
W2/Q2. Attribution against close alternatives.
The gain comes from the explicit supervised state, which neither close alternative has.
(1) SR-AD (Jia et al., 2026) fuses satellite features and street views retrieved at the target location through 2D cross-attention. Its 5.60 FID / 29.97 FVD is self-reported under its own protocol; Loc2Drive reaches 5.31 / 29.77 without any target street views.
(2) SatDreamer360 (Ze et al., 2026) synthesizes ground views from a satellite tri-plane, without supervised occupancy and road semantics, box control, or route re-querying.
(3) The SR-AD-style 2D baseline is our controlled comparison. It uses the same satellite encoder, initialized from ImageNet, and is trained end to end in Stage II under the same schedule. It lacks only Stage I, whose occupancy, masked-RGB, depth, and perspective-semantic losses all require the lifted volume and its projection, so the comparison isolates this supervision. The explicit state lowers FVD₁₆ from 35.13 to 29.77 (Tab. 3b), and to 30.57 even without HD-map labels.
(4) Rendered RGB alone supports the explicit state. This RGB condition is rendered from the state (App. E.1 D) and already outperforms 2D fusion (30.17 against 35.13), so it tests how the state is injected, not whether it is needed.
Q2. Long rollouts (Table R10). Yes, the advantage persists and widens: the full design leads the rendered-RGB-only variant by 0.40, 2.21, and 4.33 FVD at 16, 64, and 128 frames.
Table R10. Three references, external HD map, clips as in Table 2a.
Q3. Registration, vertical structure, and geometric consistency.
Neither uncertainty breaks geometric consistency, which we now measure on the generated videos.
(1) Robustness to misregistration. Training pairs already contain the imagery's registration error. At inference, shifting the query by 2 m or 4 m, or rotating it by 5°, gives FVD₁₆ of 31.19, 31.56, and 31.41, against 30.80 unperturbed and 36.11 for UniMLVG.
(2) Vertical structure. The state spans −1.0 m to 5.4 m, the volume that shapes the drivable scene; higher facades are left to the video prior, so the overhead imagery cannot inject a misleading vertical cue.
(3) Geometric consistency (Table R8). All six cameras sample one state, so a registration error shifts every view together instead of making the views disagree. As the reviewer suggests, we use VGGT (Wang et al., 2025) to recover the poses of the six generated views and compare the relative rotations of overlapping cameras with the calibration in the first and last windows of the rollout. Loc2Drive stays closer to real video in both windows, and its lead over UniMLVG grows from 1.45 to 3.08.
Table R8. Cross-view consistency on 150 val clips (three references): AUC@30 of the VGGT relative rotations of the six overlapping camera pairs against the calibration.
We hope these results strengthen the reviewer's confidence in the paper.