acceptodds
Under review as a conference paper at ICLR 2027

GeoState: Learning View-Decoupled Geometry States for 3D World Modeling

Abstract

Recent 3D world models increasingly use features from geometry foundation models (GFMs) as predictive world states, benefiting from their rich geometric priors. However, the view dependence of these features entangles scene dynamics with camera-induced variation, making the resulting state evolution harder to predict. We propose GeoState, a view-decoupled geometry representation for predictive 3D world modeling that separates scene-state evolution from camera-dependent observation. Specifically, a scene tokenizer aggregates multi-view GFM features into a view-decoupled scene state, which a geometry decoder queries using camera rays. An autoregressive world model then predicts future scene states, while camera motion is modeled separately. The predicted states can therefore be decoded under either predicted or externally specified cameras, while retaining a single context-derived scale throughout the rollout without per-frame scale alignment. Experiments show that GeoState outperforms prior 3D world models on Waymo, supports geometrically consistent cross-view querying, and generalizes zero-shot to KITTI and nuScenes.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
World Models, 3D Representation Learning, Geometry Foundation Models, View-Decoupled Representation, Predictive Scene Modeling

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_a82hOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

GeoState constructs fixed-length scene tokens from frozen VGGT-Ω features using source geometry and camera cues, then decodes depth with query-camera rays. A temporal model predicts future scene tokens while a separate model predicts camera motion; metric scale is calibrated once from an observed LiDAR frame. The authors argue that this factorization reduces camera-induced variation in the predictive state. Results on Waymo, KITTI, and nuScenes support useful depth forecasting, and the cross-query experiment supports consistency among nearby views decoded from one state.

Critical strengths
  1. The method provides a useful operational separation between forecasting a scene state and choosing a camera for decoding it. Equations (7)–(10) permit the same predicted state to be queried from different cameras, and Figure 3 shows geometric agreement across nearby query views. This makes the predicted representation more flexible than a forecast tied to one output view.
  2. The forecasting gains are consistent across the reported datasets and horizons. Tables 1–2 show improvements over camera-matched Copy-Last on Waymo, KITTI, and nuScenes, while Table 10 shows larger gains on dynamic Waymo windows. These results support the value of predicting scene evolution beyond re-querying the last observed state, with metric scale fixed from the observed context.
Critical weaknesses

The mechanism used to explain the representation ablation is not isolated. Fixed scene tokens with ray-conditioned decoding have a close precedent in 3DRAE, while FR3D already separates ego-motion and scene prediction; GeoState’s distinctive claim is that this representation makes physical state evolution less sensitive to input-camera variation. However, source-camera poses and rays enter the encoder in Equations (4)–(5), Figure 3 appropriately measures local consistency across queries of one state; it does not measure how the state changes with its input views. Section 4.5 interprets Table 4 as suggesting that the view-decoupled state is easier to predict: the two methods have similar reconstruction quality, while GeoState improves predicted-depth AbsRel from 0.144 to 0.133 at 2.0 s. However, the compressed-feature baseline has a 2.11M-parameter compressor, whereas GeoState’s tokenizer and decoder have about 206M parameters together and use a different, ray-conditioned decoding path. The reported gains establish the utility of the full system, but do not determine how much comes from suppressing input-view variation.

Questions for authors
  1. Equations (4)–(5) provide source-camera poses and rays to the tokenizer; what in the method encourages it to use these cues to align views rather than retain camera-specific variation in the predictive state?
  2. The paper argues that GeoState reduces changes in the predictive state caused by camera motion. Could the authors test this directly? For the same scene and coordinate frame, encode the scene from different input views with similar coverage, then decode both resulting states from the same output camera. How much do the decoded depths and subsequent forecasts differ?
  3. Can the Table 4 comparison be repeated with a compressed-feature baseline whose encoder and camera-conditioned decoder have comparable capacity, while retaining the same state size and world model? If this is infeasible, what evidence rules out encoder or decoder design as the main cause of the forecasting gap?
4 comments
AuthorsOpenReview ↗

Response to Reviewer a82h (Part 1/2)

We thank the reviewer a82h for the detailed and constructive feedback. We address these concerns below with additional controlled experiments and analyses.

To Q1:

View decoupling is encouraged by two complementary training mechanisms:

  1. Representation learning: The tokenizer-decoder is trained to reconstruct geometry from varying query viewpoints, encouraging the scene state to capture geometry that generalizes across views rather than source-camera-specific information.
  2. World-model training: Autoregressive predictions are directly supervised through decoded future geometry, without latent-state matching during rollout. This encourages the predicted states to preserve geometrically meaningful information rather than reproduce camera-dependent feature variations.

To Q2:

We appreciate the reviewer's suggestion. To directly evaluate source-view sensitivity, we construct conditions that share the same static scene, reference frame, and output camera, but differ in their source observations. We examine both the resulting representations and decoded depths, and additionally test how changes in historical observations affect subsequent forecasts. We will incorporate these additional experiments and analyses into the revised manuscript.

Specifically, we select 100 static Waymo validation windows (seed 42). Three shared anchor images at time tt establish the coordinate frame. Condition A uses three source views at t+0.2t+0.2 s, while condition B uses three source views at t+0.6t+0.6 s. Under each condition, VGGT-Ω\Omega jointly processes the shared anchors and the corresponding source views, and the Scene Tokenizer reads only the source FRONT tokens. Both conditions are evaluated at the same GT FRONT camera at t+0.4t+0.4 s, without query-time RGB inputs. Metric scale is calibrated from the shared anchors, without output-depth alignment.

(a) Common-query depth consistency. GeoState directly decodes depth at the common query camera, while VGGT-Ω\Omega reprojects its calibrated source FRONT depth into that view. We measure the symmetric relative depth discrepancy between A/B and accuracy against GT LiDAR on identical LiDAR-verified commonly visible pixels. This shared evaluation region focuses the comparison on geometry visible under both source conditions.

Table R1. Common-query depth discrepancy and accuracy (↓).

MethodA/B discrepancyGT AbsRel, AGT AbsRel, B
VGGT-Ω\Omega (reprojection)0.048660.090490.08567
GeoState (decoding)0.043900.084500.08612

GeoState reduces depth discrepancy by 9.8% while maintaining comparable accuracy against GT LiDAR. This indicates that GeoState produces more consistent geometry under changing source observations without sacrificing geometric fidelity, providing output-level evidence of reduced source-view sensitivity.

(b) Representation stability. We further measure relative L2 and cosine distances between representations under the same A/B conditions:

E_{\cos} = 1-\cos(\operatorname{vec}X_A,\operatorname{vec}X_B).$$ For VGGT-$\Omega$, we directly compare the complete source FRONT patch-token grids at matching spatial indices. For GeoState, we compare corresponding latent slots. Both distances are computed on flattened representations and averaged over the same static windows. **Table R2. Representation variation under source-view changes (↓).** | Representation | Relative L2 | Cosine distance | |---|---:|---:| | VGGT-$\Omega$, layer 4 | 0.58516 | 0.17326 | | VGGT-$\Omega$, layer 11 | 0.73070 | 0.26903 | | VGGT-$\Omega$, layer 17 | 1.05183 | 0.55474 | | VGGT-$\Omega$, layer 23 | 1.04999 | 0.55316 | | GeoState latent state | **0.20729** | **0.02250** | GeoState exhibits substantially smaller normalized representation changes under different source observations. This suggests reduced camera-induced variation in the latent states used for temporal prediction, complementing the decoded-depth consistency results in Table R1 and supporting the motivation for view-decoupled world modeling.
AuthorsOpenReview ↗

Response to Reviewer a82h (Part 2/2)

(c) Subsequent forecast consistency. To test whether this robustness also holds during temporal prediction, we conduct a separate diagnostic on 100 static Waymo validation windows. Within each model, conditions A/B share the same final observed state StS_t to maintain a common forecasting origin, together with the same coordinate anchor and metric scale, while the preceding three observations are shifted:

H_B=[S_{t-0.6}^{B},S_{t-0.4}^{B},S_{t-0.2}^{B},S_t].$$ Each timestamp supplies three synchronized views. We compare GeoState with the View-coupled baseline introduced in Table R4. Both methods use GT source-camera conditioning and undergo the same history perturbation. Each model autoregressively predicts future states from its A/B histories. At each horizon, GeoState is decoded with the same GT FRONT query camera for both conditions; the baseline produces its native FRONT depth. We measure A/B depth discrepancy on identical LiDAR-verified common regions. This static-scene diagnostic tests sensitivity to shifted historical observations, including their temporal sampling phase, while keeping the final observed state fixed. **Table R3. A/B forecast depth discrepancy (AbsRel↓).** | Representation | +0.6 s | +1.0 s | +2.0 s | |---|---:|---:|---:| | View-coupled | 0.01096 | 0.01254 | 0.01527 | | GeoState | **0.00711** | **0.00864** | **0.01311** | | Relative reduction | **35.1%** | **31.1%** | **14.2%** | GeoState has lower mean forecast discrepancy at all reported horizons, indicating more stable subsequent predictions under the tested history perturbations. These results complement the direct source-view test by examining the effect of changing observations on autoregressive prediction. ### **To Q3:** Our original feature-compression baseline was designed under the consideration that the Scene Tokenizer–Decoder is trained solely with depth supervision, so comparable reconstruction accuracy suggested sufficient capacity for this task. We agree that a controlled comparison with matched architectures and model capacity more directly tests the role of view decoupling, and therefore add a view-coupled counterpart with the same encoder–decoder architectures, parameter count, latent-state size, inputs, and world-model architecture, using comparable training budgets. The key difference lies in the representation's reference frame: GeoState encodes scene states in a shared coordinate system and decodes depth using predicted camera-conditioned queries, whereas the view-coupled variant represents each state in its corresponding FRONT-camera coordinate frame and decodes depth using an identity-pose query. Both retain the same ray-conditioned decoder architecture; the query pose is expressed according to each representation's coordinate frame. **Table R4. Capacity-matched comparison on Waymo (AbsRel ↓ / $\delta_1$ ↑).** | Representation | Recon. (Avg. 3 Steps) | t+0.6s | t+1.0s | t+2.0s | |---|---|---|---|---| | View-coupled | 0.073 / 0.930 | 0.094 / 0.902 | 0.112 / 0.875 | 0.149 / 0.814 | | GeoState(Pred. camera) | 0.072 / 0.931 | **0.079 / 0.921** | **0.091 / 0.902** | **0.126 / 0.848** | Under comparable reconstruction performance, GeoState consistently outperforms the view-coupled variant across all reported forecasting horizons. With encoder–decoder capacity and decoder structure controlled, these results support our central hypothesis that separating camera-induced variation from scene dynamics facilitates more effective world modeling. We will update the corresponding table in the revised manuscript with this controlled ablation. ### **To Weaknesses:** We thank the reviewer for identifying these limitations. We address the weaknesses in our responses to Q1–Q3. Regarding sensitivity to input-camera variation and the insufficiency of the original evidence, we provide **a mechanism-level explanation in Q1** and **experimental validation in Q2 (Tables R1–R3)**. Q1 explains how the training objectives encourage states to preserve geometrically meaningful information rather than reproduce camera-dependent feature variations. Q2 shows smaller normalized representation changes and lower common-query depth discrepancies than VGGT-Ω under changing source observations, as well as more stable subsequent forecasts than the view-coupled baseline under history perturbations. Regarding the capacity and decoder differences in the original Table 4, **Q3 (Table R4)** adds a controlled ablation. The results suggest that with architectures and capacity matched, GeoState retains its forecasting advantage at comparable reconstruction quality. This supports our hypothesis that separating camera-induced variation from scene dynamics facilitates temporal prediction.
Reviewer_a82hOpenReview ↗

Response to Rebuttal

Thank you for the additional experiments. The response substantially addresses my main concern, particularly through the capacity-matched comparison in Table R4. Matching the encoder–decoder architecture, parameter count, latent-state size, and world model, while obtaining similar reconstruction accuracy, makes the forecasting improvements considerably more informative than the original compressor comparison. As described, this experiment substantially alleviates my concern about capacity and decoder-architecture differences. Table R1 also addresses an important distinction in my review: it changes the source observations while holding the output camera fixed, rather than querying a single state from different cameras.

Overall, I find the new evidence sufficient to substantially alleviate my original concern and am inclined to raise my recommendation to Weak Acceptance.

AuthorsOpenReview ↗

Thank You for Your Constructive Feedback

We sincerely thank the reviewer a82h for the thoughtful follow-up and positive assessment of the additional evidence. Your suggestions helped us strengthen the controlled comparison and more directly evaluate sensitivity to source-view changes, making the evidence for our central claim clearer. We have incorporated these additional experiments and clarifications into the revised manuscript and uploaded the updated PDF. We greatly appreciate your time and constructive feedback throughout the review process.

Reviewer_HpxuOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

GeoState studies view-decoupled scene representations for future geometry forecasting. It uses a frozen VGGT-Ω backbone to extract multi-view features and geometric information, which a learned Scene Tokenizer combines and compresses into a view-decoupled scene state. A geometry decoder queries this state using a specified camera viewpoint to predict the corresponding depth. Evaluations on Waymo, KITTI, and nuScenes support the effectiveness of GeoState for world modeling.

Critical strengths
  1. The paper identifies a limitation of geometry foundation models: even when the scene is static, adding new views can disturb the features of an existing view. GeoState addresses this issue through view-decoupled world modeling.
  2. The reported results show that GeoState outperforms the evaluated baselines on Waymo, KITTI, and nuScenes. Comparisons with Copy-Last and other ablation studies also provide useful empirical insights.
Critical weaknesses
  1. The paper does not clearly establish GeoState's advantages in geometric accuracy and computational efficiency over (1) world models based on video generation and (2) approaches that reconstruct a scene with 3D Gaussians and render novel views.
  2. Evaluation is limited on road scenes (Waymo, KITTI, nuScenes). The applicability of the proposed general geometry state to indoor scenes and outdoor environments beyond roads remains unverified.
Questions for authors
  1. How does GeoState compare with competing methods in computational efficiency? Number of trainable parameters, peak GPU memory usage, inference latency, FLOPs, etc.
  2. Could you provide quantitative or qualitative comparisons with competing methods on pointmap forecasting?
  3. For depth forecasting, how competing methods perform given GT cameras?
5 comments
AuthorsOpenReview ↗

Response to Reviewer Hpxu (Part 1/2)

We sincerely thank the reviewer Hpxu for the constructive feedback. We address the concerns with additional experiments and clarifications.

To W1 & Q1:

(1) Computational efficiency.

We thank the reviewer for this suggestion. We compare temporal prediction, scene construction, and depth querying in terms of parameters, FLOPs, peak GPU memory, and latency. These results will be included in the revised manuscript.

Table R1. Stage-wise computational costs. M: measured; C: calculated from released code; R: reported.

StageMethod / componentParameters (M)Computational cost
Temporal predictionGeoState (M)256.5513.81 TFLOPs; 315 ms / 10 outputs; 8.252 GiB
DINO-Foresight (M)257.7646.09 TFLOPs; 1,146 ms / 10 outputs; 3.161 GiB
VGGT-World flow (C)432.622,240.42 TFLOPs / 10 outputs; 7.285 GiB
FR3D forecast module (R)Not reported545 ms / 10 outputs; A100; 5.75 GB
Vista denoiser (C)1,648.081,572.11 TFLOPs / 10 joint frames; 11.60 GB
Epona temporal/diffusion modules (C)2,591.76842.18 TFLOPs / 10 future frames
ConstructionGeoState tokenizer + frozen GFM (M)70.46 + 1,143.81 frozen5.585 TFLOPs; 255.76 ms / one state; 5.178 GiB
MVSplat (M)11.962.415 TFLOPs; 254.89 ms / one scene; 4.748 GiB
DepthSplat-S (M)38.352.344 TFLOPs; 259.01 ms / one scene; 5.115 GiB
Depth queryingGeoState decoder + frozen dense head (M)135.55 + 32.33 frozen669.900 GFLOPs; 43.94 ms / query; 0.991 GiB
MVSplat rasterization (M/C†)No neural decoderApproximately 1.823 GFLOPs†; 1.37 ms / query; 0.150 GiB
DepthSplat-S rasterization (M/C†)No neural decoderApproximately 2.387 GFLOPs†; 1.58 ms / query; 0.158 GiB

Construction and querying use identical two-image 448×672 FRONT inputs, source poses/intrinsics, and four GT query cameras at matching depth resolution. Measurements use one Waymo window, FP32 with TF32 disabled, and the same RTX 5090. Query latency averages four five-run camera medians, excluding model loading/offloading. †Gaussian FLOPs are estimated from native-render fragment counters, excluding special functions, integer sorting, and memory operations.

GeoState requires fewer FLOPs and achieves lower inference latency than DINO-Foresight at comparable forecaster capacity, though with higher peak memory. Its temporal computation is also substantially lower than VGGT-World, Vista, and Epona under their respective inference configurations. Compared with Gaussian methods, GeoState has comparable construction latency, while Gaussian rasterization provides faster depth queries.

(2) Geometric accuracy.

We evaluate video-based forecasting on 100 Waymo validation windows and fixed-scene novel-view geometry on their static subset. Results average four horizons (+0.2, +0.6, +1.0, and +2.0 s), using identical LiDAR points within each comparison and anchor-only calibration.

For forecasting, we evaluate all 100 windows. Vista generates future RGB frames, converted to depth using frozen VGGT-Ω\Omega. GeoState uses VGGT-estimated historical poses and predicted future camera poses. Each method retains its original input configuration.

For Gaussian novel-view geometry, we use the existing 21 Static windows with identical two FRONT images, GT source/query cameras, and 448×672 resolution. GeoState queries a fixed encoded state; Gaussian methods use opacity-normalized rendered depth. We compare released RE10K-pretrained Gaussian checkpoints with pretrained and Waymo-adapted GeoState representations.

Table R2. Geometric accuracy. EPE denotes depth-derived 3D endpoint error on GT-calibrated target rays.

TaskMethodAbsRel ↓δ1\delta_1 ↑EPE (m) ↓Within 1 m (%) ↑
Forecasting (100 windows)Vista → VGGT-Ω\Omega0.22700.72937.64929.56
GeoState (Pred. camera)0.10410.88213.51544.40
Fixed-scene NVS (21 windows)MVSplat0.35700.410612.81511.45
DepthSplat-S0.23480.60468.26418.06
GeoState (pretrained)0.13260.82044.99330.47
GeoState (Waymo-adapted)0.08230.90773.01951.24

GeoState achieves more accurate future geometry than the evaluated Vista–VGGT pipeline across all reported metrics. For fixed-scene NVS, both GeoState representations outperform the evaluated Gaussian checkpoints, demonstrating more accurate novel-view depth even before Waymo adaptation under these evaluation settings. These results will be included in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer Hpxu (Part 2/2)

To Q2:

We thank the reviewer for this suggestion and add a quantitative comparison on the same Waymo validation windows. GeoState uses predicted future camera poses to avoid introducing GT future camera information. For evaluation, all methods’ predicted depths are back-projected along GT-calibrated target-image rays and compared on identical commonly visible LiDAR points. Native resolutions and anchor-only calibration are retained, without target-frame alignment.

Table R3. Depth-derived pointmap accuracy: 3D endpoint error (m ↓) / points within 1 m (% ↑).

Method+0.2 s+0.6 s+1.0 s+2.0 sMean
VGGT-World3.565 / 28.424.126 / 35.875.513 / 21.377.888 / 16.725.273 / 25.60
DINO-Foresight5.760 / 29.456.972 / 26.888.147 / 25.1210.856 / 18.807.933 / 25.06
GeoState2.626 / 52.853.081 / 48.733.535 / 44.054.820 / 31.993.515 / 44.40

GeoState improves both metrics at every horizon, reducing mean endpoint error by 33.3% over VGGT-World and 55.7% over DINO-Foresight. These results demonstrate that GeoState's improvements extend beyond depth prediction to more accurate 3D geometric forecasting. Unfortunately, FR3D's code is not publicly available, and its paper does not report corresponding pointmap forecasting metrics, preventing a direct comparison under the same evaluation protocol. We will incorporate these additional experiments and analyses into the revised manuscript.

To Q3:

We thank the reviewer for this question. The ability to query predicted scene geometry from arbitrary camera viewpoints is a distinctive capability enabled by GeoState's view-decoupled formulation. By separating scene-state prediction from camera-motion prediction, GeoState allows the same predicted scene state to be queried using either predicted or GT cameras, without re-predicting the scene state. This enables direct evaluation under GT camera trajectories while keeping the predicted scene geometry unchanged.

In contrast, DINO-Foresight and VGGT-World use view-coupled representations that jointly encode scene geometry and camera viewpoints. Similarly, FR3D predicts CUT3R's latent camera representations rather than explicitly conditioning on camera parameters. Therefore, GT cameras cannot be directly substituted into these methods' original forecasting pipelines without modifications.

Although GeoState supports GT-camera querying as an additional capability, our main comparisons use predicted future cameras rather than GT cameras. GeoState still outperforms the compared methods under this setting across all reported horizons (Table 1), achieving a 40.4% reduction in AbsRel over VGGT-World at +2.0 s without per-frame scale alignment, highlighting the effectiveness of our view-decoupled formulation.

To W2:

We thank the reviewer for highlighting this direction. To assess generalization beyond driving environments, we add an ETH3D evaluation on 100 windows from two indoor and three non-road outdoor scenes. We use the pretrained Scene Tokenizer and its paired Decoder, without fine-tuning either component on Waymo or ETH3D. The last observed state is held fixed and queried at ten subsequent viewpoints. Predicted-camera GeoState uses VGGT-estimated historical poses and our Waymo-trained camera predictor, applied zero-shot to ETH3D; the GT-camera row provides a dataset-pose oracle. All methods retain native preprocessing and use anchor-only calibration, with target-view RGB withheld and no target-frame depth alignment.

Table R4. ETH3D depth accuracy, averaged over ten subsequent viewpoints (AbsRel ↓ / δ₁ ↑).

MethodFuture cameraIndoorOutdoorOverall
DINO-ForesightPred.0.2236 / 0.74780.6790 / 0.51940.4968 / 0.6107
VGGT-WorldPred.0.2820 / 0.66340.5390 / 0.51450.4362 / 0.5741
GeoStatePred.0.2163 / 0.75620.4760 / 0.60610.3721 / 0.6662
GeoStateGT oracle0.1155 / 0.93210.1678 / 0.86360.1469 / 0.8910

These results demonstrate that GeoState's learned geometric representation generalizes beyond driving environments to both indoor and non-road outdoor scenes. Without any target-domain adaptation, GeoState consistently outperforms DINO-Foresight and VGGT-World across both scene categories using predicted cameras, reducing overall AbsRel by 25.1% and 14.7%, respectively. Moreover, the substantially improved accuracy under GT cameras (0.147 AbsRel) highlights the geometric information preserved in the fixed scene state and its ability to support accurate querying from novel viewpoints. These findings further support the generalizability and geometric fidelity of our view-decoupled representation beyond driving scenarios. We will incorporate this additional experiment and analysis into the revised manuscript.

AuthorsOpenReview ↗

Follow-up to Reviewer Hpxu

Dear Reviewer Hpxu,

Thank you again for your careful evaluation and constructive feedback. We have provided additional experiments and clarifications to address your concerns, summarized briefly below.

In [W1 & Q1], we provide detailed computational comparisons covering parameters, FLOPs, GPU memory, and latency, together with geometric accuracy evaluations against video-generation and Gaussian-based methods, demonstrating GeoState's advantages in temporal forecasting efficiency and geometric accuracy. In [Q2], we introduce quantitative pointmap forecasting comparisons, showing consistently lower 3D endpoint errors across all prediction horizons. In [Q3], we clarify GeoState's ability to query the same predicted scene state using either predicted or GT cameras, a capability not directly supported by other view-coupled baselines. In [W2], we add ETH3D experiments on indoor and non-road outdoor scenes, demonstrating generalization beyond driving environments without target-domain adaptation. The supporting experiments and analyses are provided directly in our responses and have also been incorporated into the revised manuscript.

When convenient, we would be grateful if you could review our responses and let us know whether any concerns remain unclear or insufficiently addressed. If you find these additional results and clarifications satisfactory, we would sincerely appreciate your consideration of updating your assessment.

Thank you again for your time and thoughtful feedback.

Best regards,

The Authors

Reviewer_DMpyOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper argues that features from geometry foundation models (GFMs) are suboptimal world states because they entangle scene dynamics with camera-induced variation. GeoState addresses this with three components. A Perceiver-style Scene Tokenizer aggregates multi-view VGGT-Ω features, augmented with explicit point/pose/ray cues, into a fixed 512×512 scene state in an anchor-defined coordinate frame. A ray-conditioned decoder queries this state from arbitrary cameras through the frozen VGGT-Ω DPT head. A residual masked factorized transformer (adopted from FR3D) autoregressively predicts future states, while camera motion is handled by a separate predictor. Metric scale is calibrated once from the context. Validation includes depth forecasting on Waymo up to 2.0 s, zero-shot transfer to KITTI/nuScenes, Copy-Last decompositions, cross-query reprojection consistency, and ablations on the representation and on state supervision.

Critical strengths

A queryable scene state is a clean and well-evidenced design for world modeling. In addition, the world model’s gains concentrate on dynamic windows (Table 10). Together, these indicate that the state absorbs camera-induced change and that temporal prediction can focus on scene change.

Critical weaknesses
  1. Attribution of gains and comparison fairness. Copy-Last (Pred) already outperforms all external baselines at most horizons, so most of the margin over prior work comes from re-querying and the camera predictor rather than learned dynamics. The world model’s gain over Copy-Last (Pred) is modest and shrinks with horizon (12.2% at 0.2 s → 3.8% at 2.0 s; ~3% on KITTI at 2.0 s). Conditions are also unequal across methods. GeoState uses ground-truth historical metric ego-poses, a camera predictor fine-tuned through the decoder (A.2.2), and anchor-frame LiDAR for scale. DINO-Foresight and VGGT-World take no camera input, yet are labeled “Pred.” The headline “40.4% reduction” therefore overstates the contribution of the proposed modeling.

  2. The central claim, that view-decoupling makes dynamics easier to predict, is only indirectly supported. The tokenizer explicitly consumes source poses and rays, the frame is anchored to a camera, and no objective enforces invariance to the choice of source views. Cross-query consistency (Fig. 3) is a necessary but not sufficient condition for view-decoupling. A camera-anchored representation—e.g., anchor-view geometry plus pose, decoded by a learned reprojection—would also yield low E_sym under the tested perturbations, since reprojections of a single depth map are mutually consistent by construction. The two hypotheses differ only in disoccluded regions, which the ≤1.5 m / ≤10° range rarely exposes. Accuracy against LiDAR in disoccluded pixels at large baselines, or a direct invariance test (Q3/Q6), is needed to support the claim.

  3. The key ablation (Table 4) is confounded: the compressor baseline has ~2M vs. ~206M parameters, lacks geometric cues, and receives no future camera at decoding, so the gap cannot be attributed to view-decoupling.

Questions for authors
  1. What does “Pred.” mean for DINO-Foresight and VGGT-World? Can all methods be evaluated on the same windows under the same scale protocol (e.g., anchor-only calibration for all, and per-frame alignment for GeoState), including a re-run of FR3D?
  2. In Table 4, how large is the gap if the compressor-based world model is also conditioned on the camera trajectory (FR3D-style), and/or with matched capacity? Was Table 4 “Pred.” evaluated with GT or predicted cameras?
  3. Can you directly test view-decoupling? For example: (a) encode the same static scene from different source-view subsets and report state distance and decoded-depth differences, and (b) in static windows, compare state change over time against raw GFM feature change.
  4. What is the quantitative depth accuracy when predicted states are queried at future FRONT_LEFT/FRONT_RIGHT cameras with projected LiDAR?
  5. Please answer to the critical weaknesses mentioned above.

Minor

  1. Table 3 and Table 4 (Rec./Ours) report identical numbers despite apparently different input settings; please clarify.
  2. Reconstruction outperforming VGGT-Ω (Table 3) likely reflects Waymo fine-tuning and should be framed as such. The notation G_Ω (frozen GFM) vs. G_ψ (decoder) is easily confused.
  3. Related work could discuss generative approaches that decouple time and camera at the conditioning level (e.g., BulletTime, 4DiM) and contrast them with state-level decoupling.

Overall: My current rating reflects that the central claim is not yet demonstrated, not a judgment that the approach is unsound. If the rebuttal provides a direct invariance test across source-view subsets and anchor choice, and these support the view-decoupling claim, I would like to raise my score.

6 comments
AuthorsOpenReview ↗

Response to Reviewer DMpy (Part 1/4)

We sincerely thank reviewer DMpy for the careful and constructive review. We address the concerns below with additional experiments and clarifications.

To Q1:

For DINO-Foresight and VGGT-World, “Pred.” indicates that future viewpoint changes and scene dynamics are jointly predicted within their view-coupled representations by the world model. In contrast, GeoState explicitly decouples scene dynamics from camera motion, enabling predicted scene states to be queried from arbitrary camera viewpoints.

Regarding evaluation fairness, our original comparison follows each baseline's reported scale alignment protocol, whereas GeoState calibrates scale only once from the observed context and maintains it throughout the forecasting rollout. Our anchor-only calibration is more challenging yet practically realistic, as future GT depth is unavailable at inference time for per-frame scale correction. However, we agree that evaluation under a unified scale protocol enables a more direct comparison. Following the reviewer's suggestion, we additionally evaluate DINO-Foresight and VGGT-World on the same Waymo windows and LiDAR masks using GeoState's anchor-only calibration, without target-frame alignment. We also replace GeoState's GT historical poses with VGGT-Ω\Omega estimates from observed RGB frames, removing access to GT historical poses at inference and addressing the input-information difference identified in Weakness 1. Results are in Table R1.

Table R1. Depth forecasting comparison on the same Waymo windows with anchor-only scale calibration (AbsRel ↓ / δ₁↑).

Method+0.2s (AbsRel↓ / δ₁↑)+0.6s (AbsRel↓ / δ₁↑)+1.0s (AbsRel↓ / δ₁↑)+2.0s (AbsRel↓ / δ₁↑)
DINO-Foresight0.1355 / 0.84580.1618 / 0.81370.1897 / 0.78160.2760 / 0.7163
VGGT-World0.1057 / 0.90730.1257 / 0.85270.1767 / 0.77590.2660 / 0.6403
GeoState0.0657 / 0.93880.0813 / 0.92170.0980 / 0.89790.1465 / 0.8104

Unfortunately, FR3D's code is not public, preventing a re-run under the same protocol. However, Table 14 of the FR3D paper explicitly documents scale drift, with its relative depth scale decreasing from 1.54 at 0.2 s to 1.18 at 2.0 s, while the oracle scale remains approximately 1.67, indicating that per-frame scale alignment can substantially improve its reported forecasting performance. Overall, these results demonstrate that GeoState consistently outperforms both baselines under matched evaluation conditions.

To Q2 :

Our original feature-compression baseline was designed under the consideration that the Scene Tokenizer–Decoder is trained solely with depth supervision, so comparable reconstruction accuracy suggested sufficient capacity for this task. We agree that a controlled comparison with matched architectures and model capacity more directly tests the role of view decoupling, and therefore add a view-coupled counterpart with the same encoder–decoder architectures, parameter count, latent-state size, inputs, and world-model architecture, using comparable training budgets. The key difference lies in the representation's reference frame: GeoState encodes scene states in a shared coordinate system and decodes depth using predicted camera-conditioned queries, whereas the view-coupled variant represents each state in its corresponding FRONT-camera coordinate frame and decodes depth using an identity-pose query. Starting from the trained trajectory-free world model, we add a lightweight trajectory encoder and further train with GT camera trajectories. At inference, this variant uses predicted camera trajectories just as GeoState.

Table R2. View-coupled baselines with matched representation capacity (AbsRel ↓ / δ₁ ↑).

RepresentationRecon. (Avg. 3 Steps)t+0.6st+1.0st+2.0s
View-coupled0.073 / 0.9300.094 / 0.9020.112 / 0.8750.149 / 0.814
View-coupled + trajectory (Pred. camera)0.073 / 0.9300.094 / 0.9030.110 / 0.8760.147 / 0.816
GeoState (Pred. camera)0.072 / 0.9310.079 / 0.9210.091 / 0.9020.126 / 0.848

With comparable reconstruction accuracy, GeoState consistently outperforms both capacity-matched view-coupled baselines. We also observe that supplying encoded predicted camera trajectories to the world model at inference yields only limited improvement in this comparison. These results support that separating camera-induced variations from scene dynamics facilitates more effective world modeling. We will update the corresponding table in the revised manuscript with this ablation.

Regarding the camera setting in Table 4, “Pred.” denotes evaluation with predicted cameras. Since view-coupled world models jointly predict scene dynamics and viewpoint changes, we also use predicted cameras by a camera prediction model for GeoState to ensure a comparable evaluation setting without introducing an advantage from GT future camera information.

AuthorsOpenReview ↗

Response to Reviewer DMpy (Part 2/4)

To Q3:

We conduct a controlled experiment on 100 static Waymo validation windows from static segments (seed 42) to evaluate source-view sensitivity at both the decoded-depth and representation levels. Two conditions share the same three anchor images at time tt, reference frame, and query camera, but use three source views at t+0.2t+0.2 s (A) and t+0.6t+0.6 s (B), respectively. Under each condition, VGGT-Ω\Omega processes the shared anchor images together with the corresponding source views, and GeoState's Scene Tokenizer extracts the state using only the source FRONT tokens. Both methods thus use the same input observations under each condition. Both conditions are evaluated at the same GT FRONT camera at t+0.4t+0.4 s without query-time RGB inputs. Metric scale is calibrated from the shared anchors in a common physical scale, without per-frame depth alignment.

(a) Decoded-depth consistency. GeoState directly decodes depth at the common query camera, while VGGT-Ω\Omega reprojects its calibrated source FRONT depth into the same view. We evaluate the symmetric relative depth discrepancy between A/B and AbsRel against GT LiDAR on identical LiDAR-verified commonly visible pixels.

Table R3. Common-query depth consistency and accuracy (↓).

MethodDepth discrepancy AbsRelGT AbsRel, AGT AbsRel, B
VGGT-Ω\Omega (reprojection)0.048660.090490.08567
GeoState (decoding)0.043900.084500.08612

GeoState reduces depth discrepancy by 9.8% while maintaining comparable accuracy against GT LiDAR. This indicates that GeoState produces more consistent geometry under changing source observations without sacrificing geometric fidelity, providing output-level evidence of reduced source-view sensitivity.

(b) Representation stability. We further measure relative L2 and cosine distances between representations under conditions A/B:

E_{\cos} = 1-\cos(\operatorname{vec}X_A,\operatorname{vec}X_B).$$ For VGGT-$\Omega$, we compare the complete source FRONT patch-token grids at matching spatial indices; for GeoState, we compare corresponding latent slots. Both metrics are computed on flattened representations and averaged over the same static windows. **Table R4. Representation variation under source-view changes (↓).** | Representation | Relative L2 | Cosine distance | |---|---:|---:| | VGGT-$\Omega$, layer 4 | 0.58516 | 0.17326 | | VGGT-$\Omega$, layer 11 | 0.73070 | 0.26903 | | VGGT-$\Omega$, layer 17 | 1.05183 | 0.55474 | | VGGT-$\Omega$, layer 23 | 1.04999 | 0.55316 | | GeoState latent state | **0.20729** | **0.02250** | GeoState exhibits substantially smaller normalized representation changes under different source observations. This suggests reduced camera-induced variation in the latent states used for temporal prediction, complementing the decoded-depth consistency results in Table R3 and supporting the motivation for view-decoupled world modeling. We will incorporate these additional experiments and analyses into the revised manuscript. ### **To Q4:** We thank the reviewer for this valuable suggestion, which we had also considered during our evaluation design. However, Waymo's FRONT and FRONT_LEFT/FRONT_RIGHT cameras have only **5.4° of nominal angular overlap**, corresponding to **approximately only 10% of the image area** based on actual camera calibration. Consequently, cross-view depth forecasting also involves substantial novel-view geometry completion, making it difficult to isolate temporal forecasting performance from geometric generalization. Nevertheless, we additionally evaluate predicted GeoStates at future GT FRONT_LEFT/FRONT_RIGHT cameras using projected LiDAR depth. **Table R5. Cross-view depth forecasting on Waymo (AbsRel ↓ / δ1 ↑).** | Query Camera | +0.2 s | +0.6 s | +1.0 s | +2.0 s | |---|---|---|---|---| | FRONT_LEFT | 0.1803 / 0.7963 | 0.1777 / 0.7910 | 0.2023 / 0.7761 | 0.2402 / 0.7330 | | FRONT_RIGHT | 0.2496 / 0.7534 | 0.2745 / 0.7297 | 0.2949 / 0.7165 | 0.3400 / 0.6352 | These results demonstrate that predicted GeoStates support depth forecasting across different camera viewpoints, despite the additional challenges introduced by limited view overlap and novel-view geometry completion.
AuthorsOpenReview ↗

Response to Reviewer DMpy (Part 3/4)

To W1:

We thank the reviewer for this observation. Copy-Last performs strongly because moving objects occupy only a limited portion of the image, while much of the static scene geometry can be preserved by re-querying the last observed state at the future camera. Consequently, whole-frame depth metrics can understate improvements concentrated in motion-related regions.

Our original results already provide evidence for this contribution. Figure 2 qualitatively illustrates the prediction of changing scene geometry, while the motion-stratified evaluation in Appendix A.8.2 (Table 10) shows substantially larger gains on Dynamic than Static windows: 13.10% versus 4.92% with GT query cameras, and 9.63% versus 2.17% with predicted cameras. The consistent pattern under GT cameras further supports the contribution of scene-state prediction independently of future-camera prediction errors.

To examine this contribution more directly, we extend the window-level analysis to a region-wise evaluation. On the same Waymo validation windows, we compare GeoState with camera-matched Copy-Last using identical GT query cameras. Motion-influenced regions are defined by the union of GT moving-object box projections from the anchor through each target timestamp. Background pixels exclude this union, current annotated objects, and a two-pixel boundary buffer.

Table R6. Regional AbsRel across ten forecasting horizons (↓), with paired 95% confidence intervals.

RegionCopy-LastGeoStateImprovement [95% CI]
Motion-influenced0.230200.1884418.14% [11.38, 25.22]
Background0.066970.064403.84% [0.70, 7.22]

GeoState achieves an 18.14% reduction in motion-influenced regions, compared with 3.84% in the background. The difference is 14.30 percentage points, with a paired 95% confidence interval of [7.88, 20.89] percentage points. This region-wise analysis strengthens the original qualitative and motion-stratified evidence that learned dynamics provide additional benefits concentrated in motion-related geometry. We will incorporate this experiment and analysis into the revised manuscript.

Regarding comparison fairness, we address historical poses and scale alignment in Table R1 of our response to Q1. Although DINO-Foresight and VGGT-World do not take explicit camera parameters as input, their view-coupled representations encode both scene geometry and viewpoint information. Predicting these future representations therefore implicitly predicts viewpoint changes together with scene dynamics.

To W2:

Our original cross-query consistency experiment decodes the same scene state from different camera viewpoints and measures geometric agreement between the resulting depth maps (Fig. 3). We agree that varying the source observations provides a complementary test that more directly examines whether the state is robust to input-view changes. We therefore add two experiments to evaluate source-view sensitivity and the forecasting benefit of view decoupling:

  1. Direct source-view sensitivity test (Q3, Tables R3–R4). Under different source observations of the same static scene, GeoState achieves 9.8% lower common-query depth discrepancy than VGGT-Ω\Omega reprojection, with comparable LiDAR accuracy. Its latent representations also exhibit low variation across source observations.

  2. Controlled view-decoupling ablation (Q2, Table R2). With matched architectures, model capacity, and comparable training budgets, GeoState consistently outperforms the view-coupled variant in future prediction despite comparable reconstruction accuracy, supporting that separating camera-induced variations from scene dynamics facilitates more effective world modeling.

To W3:

We address these factors with the capacity-matched view-coupled counterpart reported in Q2, Table R2, using the same encoder–decoder architectures, geometric input cues, latent-state size, and world-model backbone, with comparable training budgets. We also evaluate a trajectory-conditioned variant using predicted camera trajectories at inference. GeoState retains its forecasting advantage over both variants despite similar reconstruction accuracy, supporting the benefit of the view-decoupled formulation beyond differences in model capacity or access to geometric cues.

Reviewer_YqsLOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

GeoState compresses frozen VGGT-Ω features into a fixed-length scene state and decodes depth using camera-ray queries. A world model predicts future states from four historical states, while a separate model predicts camera motion. Training uses state and depth supervision for one-step initialization, followed by depth supervision for autoregressive rollouts. Experiments cover depth forecasting on Waymo, cross-view geometric consistency, and zero-shot transfer to KITTI and nuScenes.

Critical strengths
  • The motivation is clear. Features useful for geometry reconstruction may contain camera-dependent variation that complicates temporal prediction. The proposed scene state separates forecasting from camera queries, allowing the same predicted state to be decoded from different viewpoints. Retaining an observation-derived scale throughout the rollout also avoids target-frame alignment.
  • The internal comparisons are informative. Camera-matched Copy-Last holds the scene state fixed while changing the query camera, helping isolate the benefit of state prediction. GeoState improves on this baseline with both GT and predicted cameras; average AbsRel reductions on dynamic windows are 13.10% and 9.63%, respectively. The state-supervision ablation also provides useful evidence for the two-stage training procedure.
Critical weaknesses
  • The experiments do not isolate the benefit of view decoupling. Separating scene and camera prediction and learning queryable scene latents have precedents in FR3D and 3DRAE, making the temporal prediction benefit central to this paper's contribution. Yet Table 4 compares a 2.11M linear compressor with a 70.46M tokenizer and a 135.55M geometry decoder, with different geometry inputs. Similar reconstruction errors do not exclude capacity or decoder effects on forecasting. Figure 3 tests consistency across query cameras, while Appendix A.7 examines the original GFM; neither directly verifies reduced sensitivity to source-view changes. The results therefore support the complete system more clearly than the claimed explanation for its improved predictability.
  • The comparison with FR3D does not establish an advantage under matched conditions. GeoState uses four historical metric poses in addition to RGB for camera prediction, whereas FR3D predicts camera motion from images. The FRONT-only experiment retains this extra input. FR3D scores are also quoted from its paper, without establishing that they use GeoState's 100 Waymo windows, anchors, and evaluation masks. Although target-frame alignment favors the external baselines, the combined protocol differences leave the relative performance unclear. At least one comparison on the same samples with comparable inputs is needed.
Questions for authors
  • Does the forecasting advantage remain against a view-coupled representation with comparable capacity and decoder structure, using the same world model and a similar training budget? On static scenes with a fixed coordinate frame and shared visible regions, is GeoState's output at a common query less sensitive to changes in source views?
  • Table 10 groups windows by the presence of moving objects but evaluates whole-frame depth. With GT query cameras, how much does GeoState improve over Copy-Last on moving-object regions and static background separately? Please include paired confidence intervals across evaluation windows. This would help distinguish motion prediction from background completion or geometry correction.
  • Are the historical metric poses ground truth, odometry estimates, or image-based estimates? Do the quoted FR3D scores correspond to the same evaluation windows and masks? Could one close baseline be evaluated on the same samples with matched pose inputs, or GeoState be evaluated using image-estimated historical poses? Please also clarify each method's access to LiDAR calibration and target-frame alignment.
4 comments
AuthorsOpenReview ↗

Response to Reviewer YqsL (Part 1/2)

We sincerely thank the reviewer YqsL for the detailed feedback and for recognizing the motivation and effectiveness of our method. We address the concerns below with additional experiments and clarifications.

To W1 & Q1:

We appreciate the suggestion and conduct additional experiments to directly evaluate view decoupling. These results will be included in the revised manuscript.

(1) Controlled Ablation of View Decoupling.

Our original feature-compression baseline was designed under the consideration that the Scene Tokenizer–Decoder is trained solely with depth supervision, so comparable reconstruction accuracy suggested sufficient capacity for this task. We agree that a controlled comparison with matched architectures and model capacity more directly tests the role of view decoupling, and therefore add a view-coupled counterpart with the same encoder–decoder architectures, parameter count, latent-state size, inputs, and world-model architecture, using comparable training budgets. The key difference lies in the representation's reference frame: GeoState encodes scene states in a shared coordinate system and decodes depth using predicted camera-conditioned queries, whereas the view-coupled variant represents each state in its corresponding FRONT-camera coordinate frame and decodes depth using an identity-pose query.

Table R1. Controlled ablation of view decoupling on Waymo (AbsRel ↓ / δ₁ ↑).

RepresentationRecon. (Avg. 3 Steps)t+0.6st+1.0st+2.0s
View-coupled0.073 / 0.9300.094 / 0.9020.112 / 0.8750.149 / 0.814
GeoState(Pred. camera)0.072 / 0.9310.079 / 0.9210.091 / 0.9020.126 / 0.848

Under comparable reconstruction performance, GeoState consistently outperforms the view-coupled variant across all forecasting horizons. These results provide further evidence that separating camera-induced variations from scene dynamics improves the predictability of geometry states.

(2) Sensitivity to Source-View Changes.

To directly test source-view sensitivity, we construct two conditions sharing the same scene, reference frame, and query camera, but differing in source observations. We evaluate sensitivity at both the decoded-depth and representation levels.

Specifically, we select 100 static Waymo validation windows from 36 segments (seed 42). For each window, three shared anchor images at time tt establish a fixed coordinate frame. Condition A uses three source views at t+0.2t+0.2 s, while condition B uses views at t+0.6t+0.6 s. For each condition, VGGT-Ω\Omega processes the shared anchors together with the corresponding three source views, and the Scene Tokenizer extracts the state using only the source FRONT tokens. We then evaluate both conditions at the same GT FRONT query camera at t+0.4t+0.4 s, without using query-time RGB images. Both conditions use anchor-based metric calibration in a common physical scale, without output-depth alignment.

(a) Decoded depth consistency. GeoState directly decodes depth at the common query camera, while VGGT-Ω\Omega reprojects its calibrated source FRONT depth into that view. On identical LiDAR-verified commonly visible pixels, we measure the symmetric relative depth discrepancy between A/B and the AbsRel against GT LiDAR.

Table R2. Common-query depth consistency and accuracy (↓).

MethodDepth discrepancy AbsRelGT AbsRel, AGT AbsRel, B
VGGT-Ω\Omega (reprojection)0.048660.090490.08567
GeoState (decoding)0.043900.084500.08612

GeoState reduces depth discrepancy by 9.8% with comparable GT LiDAR accuracy. This indicates that GeoState produces more consistent geometry under changing source observations without sacrificing geometric fidelity, providing output-level evidence of reduced source-view sensitivity.

(b) Representation stability. We further measure relative L2 and cosine distances between representations under A/B. For VGGT-Ω\Omega, we directly compare the complete source FRONT patch-token grids at matching spatial indices. For GeoState, we compare matching latent slots. Both distances are computed on flattened representations and averaged over the same static Waymo validation windows.

Table R3. Representation variation under source-view changes.

RepresentationRelative L2Cosine distance
VGGT-Ω\Omega, layer 40.585160.17326
VGGT-Ω\Omega, layer 110.730700.26903
VGGT-Ω\Omega, layer 171.051830.55474
VGGT-Ω\Omega, layer 231.049990.55316
GeoState latent state0.207290.02250

GeoState exhibits substantially smaller normalized changes across source observations. This suggests reduced camera-induced variation in the latent states used for temporal prediction, complementing the decoded-depth consistency results in Table R2 and supporting the motivation for view-decoupled world modeling.

AuthorsOpenReview ↗

Response to Reviewer YqsL (Part 2/2)

To Q2:

We thank the reviewer for this suggestion. To distinguish motion prediction from background completion or geometry correction, we evaluate GeoState against camera-matched Copy-Last on the same Waymo 100 windows using GT query cameras. The motion-influenced region is the union of GT moving-object box projections from the anchor through each target timestamp. Background pixels outside this region exclude current annotated objects and a two-pixel boundary buffer.

Table R4. Regional AbsRel over ten forecasting horizons. Improvements include paired 95% confidence intervals.

RegionCopy-LastGeoStateImprovement [95% CI]
Motion-influenced region0.230200.1884418.14% [11.38, 25.22]
Background outside motion influence0.066970.064403.84% [0.70, 7.22]

GeoState's relative improvement is 14.30 percentage points larger [7.88, 20.89] in motion-influenced regions, supporting gains concentrated in motion-related geometry. Together with the qualitative results in Figure 2 and the motion-stratified evaluation in Appendix A.8.2 (Table 10), these findings support GeoState's ability to model scene dynamics. We will incorporate this additional experiment and analysis into the revised manuscript.

To W2 & Q3:

Our original evaluation uses GT historical poses to establish reliable geometric reference frames, leveraging the ability of our view-decoupled formulation to explicitly incorporate camera information, which is not explicitly supported by the compared view-coupled baselines. Nevertheless, we agree that evaluating GeoState with image-estimated historical poses enables a more direct comparison with the image-based baselines.

To address this concern, we replace GeoState's GT historical poses with VGGT-Ω\Omega estimates from observed RGB frames, keeping the same checkpoints. We further re-evaluate DINO-Foresight and VGGT-World on the same Waymo Random100 windows and LiDAR masks under both their original per-frame alignment protocols and unified anchor-only calibration. Since FR3D's code is not publicly available, we retain its reported results for reference but cannot establish a matched comparison.

Table R5. Depth forecasting with image-estimated historical poses under different scale alignment protocols (AbsRel ↓ / δ1\delta_1 ↑).

MethodScale Alignment+0.2 s+0.6 s+1.0 s+2.0 s
DINO-ForesightPer-frame0.119 / 0.8410.138 / 0.8250.151 / 0.7980.243 / 0.663
DINO-ForesightAnchor-only0.136 / 0.8460.162 / 0.8140.190 / 0.7820.276 / 0.716
VGGT-WorldPer-frame0.073 / 0.9310.112 / 0.8920.143 / 0.8260.213 / 0.731
VGGT-WorldAnchor-only0.106 / 0.9070.126 / 0.8530.177 / 0.7760.266 / 0.640
FR3D (reported)Per-frame0.110 / 0.8790.121 / 0.8580.131 / 0.8400.152 / 0.800
GeoState (estimated poses)Anchor-only0.066 / 0.9390.081 / 0.9220.098 / 0.8980.146 / 0.810

The results suggest that even under the estimated historical poses, GeoState consistently outperforms DINO-Foresight and VGGT-World across all horizons. These results further support the effectiveness and robustness of our view-decoupled formulation.

Thank you for the additional experiments. The ablation with matched capacity largely resolves my concern about the source of the forecasting gains, and the regional analysis shows that the benefits are concentrated in regions affected by motion. Using estimated historical poses also improves comparison fairness, although a matched comparison with FR3D remains unavailable. Overall, the new evidence addresses my main concerns sufficiently to raise my rating to 3. Please incorporate these results and clarify the remaining comparison limitations in the revision.

AuthorsOpenReview ↗

Thank You for Your Constructive Feedback

We sincerely thank the reviewer YqsL for the thoughtful reassessment and recognition of the additional evidence. Your suggestions have helped us better substantiate the forecasting gains and improve the fairness and clarity of our evaluation. We have incorporated the additional experiments and clarified the remaining comparison limitations in the revised manuscript, and have uploaded the updated PDF. We greatly appreciate your time and constructive feedback.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal and Manuscript Revisions

We sincerely thank all four reviewers for their time, thoughtful feedback, and constructive suggestions. Their comments have helped us strengthen the experimental evidence, clarify our contributions, and improve the manuscript.

A central question was whether the forecasting gains could be attributed to view decoupling given differences in model capacity. To address this, we revised the main representation ablation to compare GeoState with a view-coupled baseline matched in encoder–decoder architecture, parameter count, latent-state size, and world-model architecture (Table 5). GeoState retains its forecasting advantage despite comparable reconstruction accuracy, strengthening the evidence for the benefit of view decoupling. We also corrected the original compressor comparison to use the intended FRONT-only tokenizer input setting and moved it to the appendix (Table 10). To directly assess source-view sensitivity, we varied the source observations while holding the query camera fixed (Table 3) and examined how perturbations to the observation history affect subsequent forecasts (Table 17).

Reviewers also asked how much scene-state prediction contributes beyond re-querying the last observed state. We therefore expanded the camera-matched Copy-Last analysis to evaluate motion-influenced regions and background separately, reporting paired 95% confidence intervals (Table 16). The results show larger improvements in motion-influenced regions, supporting the contribution of learned scene dynamics. To address evaluation fairness, we re-evaluated GeoState, DINO-Foresight, and VGGT-World on the same evaluation windows and LiDAR masks under a shared anchor-only calibration protocol, using image-estimated historical poses for GeoState (Table 9).

Finally, we expanded the appendix with geometric accuracy comparisons, including pointmap forecasting, stage-wise computational cost analysis, and zero-shot evaluation of the geometric representation on ETH3D indoor and non-road outdoor scenes. We also revised the manuscript to improve the clarity and consistency of its presentation, terminology, and notation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.