acceptodds
Under review as a conference paper at ICLR 2027

Raster-Free View Synthesis with Point-Anchored Geometric Embeddings

Abstract

Geometry foundation models (GFMs) such as VGGT estimate camera poses and depth from multi-view images in a single forward pass, and recent feed-forward novel-view synthesis (NVS) methods build on this geometry by rasterizing it into the target view. However, the estimated geometry inevitably contains small errors, and rasterization turns them directly into visible artifacts, such as objects appearing as overlapping copies. We propose RaViS, a raster-free view synthesis model built on a simple principle of attending to the geometry rather than rasterizing it. RaViS synthesizes the target view with a transformer and uses the estimated geometry only to tell attention how image patches are related in 3D. To this end, we introduce a point-anchored geometric embedding that places each patch at its own estimated 3D point, so that attention between two patches depends on their relative 3D position and camera rotation. Since the geometry only steers attention and never enters the output image, its errors merely shift where the model attends instead of appearing as artifacts. To further inject 3D structure and semantics, we feed the synthesis transformer with the latent features that the GFM already computes while estimating the geometry. Using only GFM-estimated camera poses, RaViS outperforms prior feed-forward methods on RealEstate10K and DL3DV and in zero-shot generalization. On RealEstate10K, it even outperforms similarly sized baselines that have access to ground-truth camera poses.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
novel view synthesis, multi-view vision

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_yQDuOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

RaViS proposes a transformer for novel-view synthesis that uses features and geometry from a frozen VGGT-Ω model. Its main contribution is Point-Anchored Geometric Embedding (PAGE), which extends camera-relative attention by anchoring each patch at an estimated 3D point. Target anchors are obtained by projecting context geometry into the target view. The method achieves strong reported results on RealEstate10K, DL3DV, and zero-shot transfer datasets.

Critical strengths
  1. PAGE is a clear and technically motivated extension of existing geometric attention. Table 4 provides direct evidence for point anchoring: replacing camera-centered anchors with 3D-point anchors improves PSNR from 23.80 to 24.44 dB with the other inputs and training settings fixed.
  2. The evaluation spans different overlap regimes, context-view counts, and unseen datasets. Retraining the DL3DV baselines under a shared data and training setup is a useful effort toward controlled comparison.
  3. Reusing appearance and geometry features from a frozen GFM is an effective design, and Table 5 separately examines their contribution. The manuscript also acknowledges approximate target anchors and the fallback used in uncovered regions.
Critical weaknesses

The proposed mechanism does not establish the claimed separation between geometry errors and image artifacts. The motivation argues that geometry errors merely change where the model attends, instead of appearing as artifacts, because the geometry is not rasterized into RGB content. However, PAGE still directly couples estimated point-cloud geometry to the synthesis transformer. In Eq. 5, the relative geometric transformation affects both the attention logits and the transformed values, and this operation is repeated at every layer. Target anchors also depend on projected context geometry. There therefore remains a direct pathway by which depth, pose, or anchor errors can affect the synthesized image.

Using geometry to guide attention is a reasonable design. My concern is the unsupported inference that this interface prevents geometric errors from becoming image artifacts. This distinction is particularly relevant to PVSM, whose projected image conditions a learned synthesizer rather than forcing the final output to reproduce the projection. The PAGE ablation supports its effectiveness, but does not isolate robustness to geometry errors. Controlled perturbations with latent features held fixed, or a narrower explanation of the empirical results, would strengthen the paper.

The behavior under incomplete target geometry remains insufficiently characterized. Empty target patches fall back to camera-centered anchors, but the supplied results do not identify these regions or quantify their prevalence and reconstruction quality. Results with fallback masks, coverage statistics, and smooth camera trajectories would clarify whether synthesis remains reliable when anchors are missing or switch between valid geometry and fallback.

Questions for authors
  1. Could you explain the differences between the GT-camera baseline results in Table 1 and the overlap results reported in PVSM’s Table 2 as well as Less3D [1] Table 3?
  2. If possible, please provide video results rather than still images. My experiences on view synthesis models, especially when conditioned or based on geometry inputs, contain cross-frame inconsistencies. RaViS avoids rasterizing geometry into RGB content, but still projects context points to construct target anchors. Changes in visibility, patch membership, or the transition between a valid anchor and the camera-center fallback could introduce abrupt changes in the conditioning. Sparse projection alone does not prove flickering, and the static image results do not resolve this question.

Please provide smooth camera-path renderings with a fixed context set and fixed GFM predictions, ideally alongside projected depth and an anchor-validity mask. This would help assess behavior around thin structures, occlusion boundaries, and uncovered regions. The nearest-context selection used in zero-shot evaluation can be tested separately, since switching context views introduces another possible source of discontinuity. I regard this as an open robustness question rather than an observed failure.

[1] https://pku-vcl-geometry.github.io/Less3Depend/

Reviewer_EeFYOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This work proposes Raster-Free View Synthesis (RaViS), which incorporates geometry predicted by a feed-forward model into attention. Previous methods use predicted geometry for rasterization or to construct 3D representations. The authors argue that these methods are more susceptible to geometry prediction errors, whereas RaViS is less sensitive to such errors because it uses the predicted geometry only to guide attention. RaViS outperforms prior methods on novel view synthesis tasks.

Critical strengths

Simple and effective integration of geometry into attention. Prior methods such as GTA and PRoPE established ways to incorporate camera geometry into attention. RaViS naturally extends this approach from camera-level geometry to token-level geometry. In a setting of leveraging geometry foundation models, it is reasonable to expect RaViS to outperform previous rasterization-based methods.

Critical weaknesses

1. Continued dependence on the geometry model. RaViS remains affected by errors in the geometry model’s predictions. The accuracy of the geometry model may therefore limit RaViS’s achievable performance.

2. Insufficient analysis of sensitivity to geometry prediction errors. More evidence would help establish that RaViS is less sensitive to these errors. In particular, it would be useful to show cases where a feed-forward geometry model such as VGGT produces inaccurate predictions—for example, outliers in the predicted point map—and RaViS mitigates their effects. Figure 4 could, for instance, include rasterization results to enable a more direct comparison.

Questions for authors
  1. Regarding Weakness 1, what strategies could mitigate RaViS’s dependence on the geometry model’s prediction errors? For example, could geometry perturbations during training help, although this may not be an optimal solution?
Reviewer_j3yHOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents a method for NVS leveraging GFM as a prior in 2 different ways. First the dense scene geometry and camera poses are used to formulate a positional embedding (PAGE) that anchors every patch to its 3D position via an SE(3) invariant formulation. In addition, latent features from various layers of the GFM are extracted and fused via DPT decoder. Finally a decoder only transformer feeds the fused features of the context views, the plucker rays of the target views. All context and target tokens use PAGE through this transformer that regresses the target view RGB.

Critical strengths
  • Leveraging the GFM prior via PAGE and feature fusion (both clearly ablated with clear benefits!) is well motivated and delivers good results.
  • The PAGE formulation is an interesting idea that can have applications beyond NVS.
  • The paper is well written and easy to follow, with nice qualitative results.
Critical weaknesses
  • The paper lacks an indepth analysis on PAGE, which is the core novelty in this paper. For example how sensitive/robust is it to wrong predicted geometry? How well does it cope with occlusions? How are patches covering depth discontinuities in the scene affected? Occlusions and unobserved regions are modelled by a simple fallback to the camera center, so it will be good to see some analysis on how far this can hold.
  • A direct ablation/comparison with RayRope is missing. This can provide a greater insight into the differences and trade-offs between PAGE and RayRope.
Questions for authors
  • Re the extracted feats, it is clear where to tap in for the geometric feats based on the DPT residual connections from VGGT-\omega. Can you elaborate on your choice of early feats for RGB as this one is less clear?
  • Can you elaborate more on the LagerNVS setup in 4.1 and 4.2? In 4.2 does it use VGGT-\omega or just VGGT? Why freeze the encoder unlike their original proposed recipe? It will be good to see the original recipe and maybe a modified one that is more fair for a complete comparison.
  • Thanks for providing the qualitative results, your results look nicer than the baselines indeed. I am curious as to why does your method seem to particularly struggle with foliage such as in fig 8 in the appendix, while it seems to suffer less with other high frequency scene components such as bricks. You also seem to make a claim that you keep them sharp, which is not very true.
  • Can you please provide some runtime analysis of your method? And how does it scale with the number of views?
  • Do you use the intrinsics predicted by VGGT-\omega or the GT provided one?
1 comment

Thanks for the additional experiments and clarifications. I find the additional analysis on PAGE to be compelling. Given that and the overall quality of the results/paper, I will be raising my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.