- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SOM4D, a feed-forward framework for 4D driving scene reconstruction based on VGGT-Omega.
It introduces three modules:
1 Pose-Time Relational Adaptation (PTRA) to model camera and time relations,
2 Hierarchical Observation Consolidation (HOC) combine multi-level features,
3 Corrective Gaussian Head (CGH) refine dynamic Gaussians using source observations.
The paper also studies mixed-data training with Waymo, nuScenes and Argoverse 2. Experiments mainly on Waymo show improvements in rendering quality and depth accuracy, with ablation studies evaluating each module.
Critical strengths
None
Critical weaknesses
The technical novelty appears limited.
The method builds on a strong VGGT-Omega backbone and combines relatively familiar attention, feature fusion, and refinement designs. Although Table 3 shows improvements, removing PTRA also removes its feature stream, so the ablation does not isolate the benefit of the proposed relational design.
A simpler baseline would help clarify the contribution beyond the pretrained model. This limits my confidence in the significance of the architectural contributions.
Questions for authors
1 Does PTRA improve performance because of the camera–time relations, or simply because it adds more attention layers? Can you compare it with the same module without the relational biases?
2 How fast is the method, and how much GPU memory does it use compared with STORM and SLARM?
3 The shared context only improves full-image PSNR by 0.01 dB in Table 3. How much extra computation does it add, and is the small gain worth the cost?
Overall, I think this is a well-developed and solid piece of work. I will consider raising my score after reading the other reviewers' comments and the authors' response.
6 comments
Response to Q2: Runtime and GPU memory.
Thank you for asking. We report this comparison in Appendix D.2, Table 13, summarized below. Compared with STORM, SOM4D’s mean forward latency is 99.6 ms higher, and its peak GPU memory is 4.85 GiB higher. Compared with SLARM, the corresponding increases are 115.8 ms and 3.43 GiB.
| Method | Mean forward latency (ms) | Peak GPU memory (GiB) |
|---|---|---|
| STORM | 127.0 | 3.23 |
| SLARM | 110.8 | 4.65 |
| SOM4D | 226.6 | 8.08 |
While using more resources, SOM4D improves non-sky/dynamic NVS by 0.49/0.70 dB and REC by 1.10 dB, and reduces non-sky depth RMSE from 4.95 to 4.11 m, relative to our matched Waymo-only SLARM rerun (Table 1). For reconstruction from fixed sparse observations, we consider this a reasonable quality–resource trade-off. Reducing latency and memory remains an important direction for future work.
Response to Q3: Shared-context computation and benefit.
Thank you for raising this question. Under the same counting convention, Shared Context increases model FLOPs by 1.09% and peak GPU memory by approximately 0.04 GiB. The submitted joint context/subset ablation reports +0.10 dB dynamic NVS (Table 3). Given the limited additional computation and memory, we consider this an acceptable trade-off in the evaluated setting. Future work will further streamline context aggregation and readback to improve efficiency. (Added in Appendix E.4.)
Dear Reviewer zVEX,
We sincerely appreciate your thoughtful feedback and the time you have devoted to reviewing our work. We have provided further experimental evidence and uploaded a revised manuscript with changes highlighted in blue.
As the November 18 (AoE) deadline approaches, we would welcome any remaining questions. We hope our additional results help address your concerns and would be grateful if you could reconsider your evaluation.
Thank you again for your time and consideration.
Response to W1 & Q1: Relational biases and contributions beyond Omega.
Thank you for raising these questions. As described in Sec. 3.3, PTRA aims to adapt frozen Omega features to sparse multi-view temporal observations through structured camera–time relational modeling. Learned relative biases are one component of this adaptation.
To examine the role of relational biases beyond additional attention layers, we compare PTRA with the same attention module but without learned relative biases. To evaluate the contribution beyond the pretrained backbone, we also construct a simpler baseline using the same frozen Omega + Native DPT + Basic Gaussian Head. The results are as follows (Waymo NVS PSNR, dB):
For relational modeling, removing the biases while retaining the attention layers yields 26.82/22.47 dB, compared with 26.86/22.51 dB for Full—a reduction of 0.04 dB in each metric. The submitted Table 3 further shows that removing the complete PTRA pathway and its feature stream reduces non-sky/dynamic NVS by 0.67/0.97 dB, supporting the contribution of the integrated camera–time adaptation.
For the backbone attribution, the simpler baseline achieves 25.41/21.11 dB with the same frozen Omega, substantially below Full SOM4D, which improves non-sky/dynamic NVS by 1.45/1.40 dB, respectively. This demonstrates that the proposed adaptation and reconstruction architecture provides measurable benefits beyond the pretrained backbone, consistent with our paper's contribution. (Revised in Sec. 4.4 and Appendices E.1–E.2.)