- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper revisits some of the design choices in LVSMs, specifically how "scene reconstruction" and "scene rendering" interact with each other. The resulting architecture is a decoder-only transformer with unidirectional cross-attention from rendering tokens to scene tokens. Several minor changes, including the use of DINO-v3 feature, alternating attention blocks, further improve performance.
Critical strengths
- The paper explored several possible combinations inside the design space of a regression-based view-synthesis model (a.k.a. LVSM). The resulting weight-sharing model is supported by solid experiments.
- The evaluation covers sparse and dense inputs, higher resolutions, and multiple zero-shot datasets.
- Multiple components further improve the overall quality (with DINO-v3 features and alternative attention) and rendering speed (with KV-Cache).
Critical weaknesses
The main weakness is the generalizability of the paper's conclusion. The architectural conclusion may not hold under different data and model scales. The overall contributions are also somewhat incremental, as the final architecture is basically a decoder-only LVSM (with target->seen cross-attn.) plus VGGT's alternative attention. While the empirical improvements are notable, the architectural novelty and the extent to which these conclusions generalize remain limited.
Questions for authors
none
2 comments
Thanks for the rebuttal. I will retain my score.
We thank the reviewer for recognizing the empirical improvements and the solid experimental support for our weight-sharing model.
W1. Conclusion scope
We agree that our main conclusion about the benefit of sharing decoder weights between reconstruction and rendering may not hold at other model or training-data scales. We have only tested a common LVSM model size and training recipe, and now state this limitation. Our ps8 model outperforms the larger LaCT and LagerNVS models on the OOD benchmarks (Table 4), despite LagerNVS benefiting from VGGT pretraining and broader training data. These comparisons support the generalization of our final model, but do not establish that the weight-sharing benefit holds at other scales.