- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes GeoSCope, a framework designed to improve the long-horizon stability and controllability of generative driving world models. The method first reconstructs a static 3D point cloud from a single monocular driving recording using a dynamics-aware SLAM pipeline. During generation, the reconstructed scene is rendered at the target camera poses and used as geometric conditioning for pretrained world models, helping preserve static scene structure while allowing the models to generate dynamic content. The proposed framework is evaluated on three driving world models across four datasets, demonstrating improvements in long-horizon geometric consistency and generalization to unseen datasets.
Critical strengths
-
This paper focus on an important problem in driving world models, where scene geometry can gradually drift during long video generation. Using reconstructed 3D geometry to keep the generated scenes consistent is a simple and effective idea that improves the reliability of long-horizon generation.
-
The paper provides clear explanations of the method and implementation details, including the SLAM reconstruction, point cloud filtering, and geometric conditioning mechanisms. The appendix also includes detailed configurations and training settings, making the approach easier to understand or reproduce.
Critical weaknesses
1. The claimed controllability and interactive simulation capabilities are not sufficiently supported. The method relies on rendering geometric references from a pre-reconstructed scene for each target frame, which limits its effectiveness when the vehicle moves into previously unobserved areas.
For example, if the original recording follows a left turn, it is unclear how the model would handle a counterfactual right turn where no reconstructed geometry is available. The current experiments only consider small lateral offsets, trajectory wiggles, and speed changes near the original route, without evaluating such meaningful route deviations. This limitation is not sufficiently discussed and significantly reduces the practical contribution of the proposed framework as an interactive driving world model.
Overall, I think this work is closer to scene completion within a reconstructed 3D environment than to a truly controllable and interactive driving world model.
2. The main evaluation relies on a highly favorable same-route reconstruction setting. Most of the main experiments rely on 3D point clouds reconstructed from the complete video of the same route being evaluated, including frames from the prediction horizon.
This provides the model with substantial additional information about the target scene, including both geometry and appearance, which is unavailable to the baselines. Therefore, the improvements in generation quality and consistency metrics are somewhat expected, and it is unclear how much of the gain comes from the proposed model design rather than simply having access to more information.
Combined with the limitations discussed in Weakness 1, this raises concerns about both the practical applicability of the method and the significance of its reported improvements.
Questions for authors
-
How does the proposed method handle counterfactual trajectories that go beyond the reconstructed region? (in Weakness 1)
-
How should the performance improvements be interpreted given the same-route reconstruction setting?(in Weakness 2)
-
Could the authors provide a fairer comparison that isolates the effect of geometric conditioning from additional fine-tuning? According to Section 3.5, all learned GeoSCope variants fine-tune the entire pretrained backbone together with the newly introduced conditioning modules.. However, the main experiments compare these models against the original pretrained checkpoints without equivalent additional fine-tuning.
-
Why does GeoSCope lead to worse video quality on Cosmos 2.5?
3 comments
Response to authors
Thank you for the detailed response and additional experiments. I appreciate the authors' efforts to address my concerns.
However, I still have concerns about the practical value of the proposed method. As the authors acknowledge, GeoSCope is mainly effective within the reconstructed region, and its performance approaches that of the base model once the vehicle moves beyond it. Given this limitation, I do not think the rebuttal sufficiently addresses my concerns about the claimed controllability and interactive simulation capabilities. Moreover, reconstructing a 3D point cloud introduces additional computational and deployment overhead, which further limits the practical applicability of the proposed world model.
Considering these limitations, along with the concerns raised by other reviewers, I have decided to maintain my original rating of 2.
Thank you for engaging with our response. We briefly clarify two points for the record.
Scope. GeoSCope targets re-simulation of recorded drives, e.g. evaluating a policy with counterfactual ego motion on a logged route. This is the same setting as reconstruction-based simulators (e.g. NAVSIM v2) and industrial closed-loop simulators (e.g. GAIA-4), and reconstruction-based methods typically break down outright beyond the reconstructed region whereas GeoSCope falls back to the base world model. GeoSCope substantially reduces drift within this setting and never performs worse than the base model beyond it (Table R5), so grounding is a strict improvement rather than a trade-off.
Overhead. Map reconstruction is a one-time offline step per route from a single monocular recording, reused across all rollouts. Rendering the point cloud at each pose is cheap compared with the world model's denoising steps. Compared with methods that require dense geo-registered imagery, this is a lightweight requirement.
We thank the reviewer for the detailed and helpful review.
W1 + Q1: Controllability and trajectories beyond the reconstructed region. We agree that the map only provides geometry where the original drive observed the scene, and we now state this scope explicitly in the introduction: GeoSCope anchors generation within and near the recorded route, and beyond it the model falls back to its learned prior. To quantify what happens when the ego leaves the map, we built maps from only the first L = 1, 2, 4 or 8 s of each clip and froze them, so the rollout drives out of the map (new Appendix J.9). Inside the map, performance follows the full-map curve. Outside it, performance degrades smoothly toward the base model without falling below it (Table R5).
Table R5. nuPlan-Turns 172-clip subset, 10–20 s window (outside every frozen map).
W2 + Q2: Same-route setting. The same-route setting is an intended use case: re-simulating a recorded drive with counterfactual ego motion, as reconstruction-based simulators and recent industrial closed-loop simulators (e.g. GAIA-4; Wayve, 2026) do. Here the map is an input, comparable to an HD map for a map-conditioned planner. Our contribution is to show that a sparse map from a single monocular pass is sufficient, and that it need not come from the evaluated video:
(i) On Ann-Arbor-City-Bench (Table 3), the map is reconstructed from a different drive on a different day, so the map contains no frame of the evaluated video. GeoSCope, never trained on multi-traversal data, outperforms the Seoul World Model (FVD 487 vs. 640, mPSNR 18.6 vs. 15.2 dB).
(ii) In Table R5, the map is built only from frames before second L, yet GeoSCope still outperforms the base model beyond it.
(iii) The model does not copy the map: the rendered map alone scores far worse (FID 201 vs. 39, FVD 2298 vs. 487; Table 3). In new cross-traversal pairs where the map drive and the target drive differ in weather or season, GeoSCope keeps the target's appearance and improves LPIPS and keypoints in every condition (reply to Reviewer rwS6, Table R2).
(iv) The gains do not come from the additional fine-tuning on nuPlan either (Q3).
Q3: Fine-tuning control. To isolate the conditioning signal, we fine-tuned Orbis v1 with exactly the GeoSCope recipe but without the geometric reference.
Table R6. Average over the four datasets (protocol of Table 2 with short / long = 0–8 s / 8–16 s).
Averaged over the four datasets, the control is not better than the released checkpoint, while GeoSCope, trained with the same data and budget, is best on every metric on every dataset. Without the geo-reference, fine-tuning helps in-domain (nuPlan long FVD 440 → 270) but hurts transfer, drifting toward the training domain. We have added this control to Table 2.
Q4: Video quality on Cosmos 2.5. We agree that the Cosmos variant had worse FVD than the base model, although it improved LPIPS, keypoints and depth. We found two problems:
(i) Rollouts drift in appearance: contrast and saturation rise chunk by chunk, because classifier-free guidance on the map signal compounds over the autoregressive rollout.
(ii) The gates were unstable during training due to their initialization.
We fixed (i) by lowering the guidance weight from 3 to 1.5 and (ii) by initialising the gate open with a zero-initialised output projection. Averaged over the four datasets, FVD drops by about 30% compared with the submitted variant and is 10–12% lower than the base model. LPIPS and FVD slope improve as well, and long-horizon keypoints remain more than twice the base model's (Table R7). We have updated Tables 2 and 15.
Table R7. Cosmos 2.5, average over the four datasets (Table 2 protocol with short / long = 0–8 / 8–16 s; all models at 480p).
If these clarifications and additional results address your concerns, we would be grateful if you would consider raising your score.