- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
PointWeave presents a feed-forward novel-view synthesis method that combines explicit sparse 3D points with a learned image renderer. The method predicts points and appearance features from the input views, uses geometry-aware ray-to-point attention to gather features for each target ray, retrieves RGB evidence from context images, and decodes the final target image. The main idea is to use geometry for correspondence while leaving fine appearance modeling to learned features and the image decoder. The results are strong overall. On RE10K, PointWeave reaches 31.38 dB PSNR with only 8,192 points, clearly outperforming the tested 3DGS baselines and matching LagerNVS in PSNR, while rendering faster. The paper also reports good cross-dataset generalization and robustness to camera transformations.
Critical strengths
-
The method is relatively simple and practical. The predicted point set can be reused across target views, and the ray-to-point formulation gives the renderer an explicit geometric interface.
-
The quantitative results are quite strong. In particular, the gap over 3DGS-based methods on RE10K and DL3DV is substantial, while using far fewer primitives.
-
The evaluation is broader than a standard NVS paper. Besides in-domain results, the authors include zero-shot transfer, camera transformation tests, runtime comparisons, scene editing, and additional point-density experiments.
Critical weaknesses
-
My main concern is novelty. Explicit point geometry plus learned point features and a neural image decoder has a fairly long history. Neural Point-Based Graphics already combines point descriptors with a neural renderer. Pointersect also performs local point-to-ray reasoning using ray-centered geometry. More importantly, FWD predicts depth-based point clouds, attaches latent and view-dependent features, fuses multi-view information, and applies a neural refinement renderer. I could not find a discussion of FWD in the paper. The specific PointWeave design is different, but the paper needs a much sharper explanation of what is fundamentally new beyond these earlier point-based neural rendering pipelines.
-
The ablation study does not sufficiently isolate the main technical contribution. The most important component is ray-to-point attention, but there is no convincing comparison against simple projection/splatting, distance-based weighting, or another lightweight point aggregation scheme. I would also like ablations of the ray-relative coordinates, view-dependent feature basis (A_j), background token, and the five confidence/reliability inputs.
-
There is some backbone confounding. PointWeave starts from a strong ReSplat/DepthSplat reconstruction backbone rather than learning the representation from scratch. A controlled experiment with the same point backbone but different rendering heads would make it much clearer where the gain comes from.
-
The “few primitives” argument is not fully established by the 131k-point experiment. The authors themselves note that the dense variant changes the neighborhoods seen by a backbone pretrained with sparse points. So this experiment does not cleanly show that dense points are unnecessary; it may simply show that the pretrained backbone does not transfer well to a different sampling density.
Questions for authors
-
How do the authors distinguish PointWeave from FWD and earlier neural point rendering methods beyond the use of ray-to-point attention? A direct discussion or experimental comparison would help a lot.
-
What happens if the proposed attention is replaced by simple kNN weighting, projected feature splatting, or an MLP over the nearest point? This seems like the most important missing ablation.
-
Can the authors separately ablate the view-dependent feature basis, ray-relative coordinates, background slot, five reliability features, and context RGB retrieval?
4 comments
Dear Reviewer,
We would like to kindly follow up on our rebuttal and additional experiments. As the discussion period is approaching its end, we would greatly appreciate it if you could let us know whether our responses have addressed your concerns or if any questions remain.
Thank you for your time and consideration!
Thanks for detailed responses. My concerns are well addressed. I raise my score from weak reject to weak accept.
Thank you for your positive feedback and for raising your score! We are glad that our responses have addressed your concerns, and we sincerely appreciate your time and consideration.
Thank you for the review! We are glad you find our method practical and our results strong with far fewer primitives. We address your concerns below.
Q1. Comparison with other neural point rendering methods
Thank you for pointing us to FWD, we will cite and discuss it in the revision. We do not claim "points + learned features + neural decoder" as new (Sec. 2 places PointWeave in this family). The differences are where the points come from and how pixel colours are predicted from them (table below). In FWD, a fixed rasterizer decides which points reach a pixel and with what weight. In our method, these weights are a learned, non-linear function of each point's position relative to the ray. As shown below, ours achieves better rendering quality than these baselines.
We evaluate these baselines on DTU with FWD's protocol. NPBG requires per-scene fitting, so we fit one model per test scene on its three input views. We also include NPBG++, which is a follow-up to NPBG but does generalize in a feedforward manner, for a fairer comparison. The three FWD variants are trained on DTU, while Pointersect and our zero-shot model are not. We also report our model fine-tuned on DTU for 10k steps with batch size = 4 for a fair comparison.
Q2.Additional ablation studies
We show the ablation results below, following the same ablation protocol in Table 6. We will add the ablation studies in the camera-ready version.
Q3. Ablation of different rendering heads
The rendering head consists of the ray-to-point attention and the decoder. Table 6 in the main paper compares different decoders, and the table below compares alternatives to the attention, with the decoder fixed.
Q4. On the number of primitives
We would like to clarify. The "fewer points" claim refers to Table 1: with 8,192 points we obtain better quality than 3DGS-based methods that use 47k–393k Gaussians. The 131k-point run in the appendix only asks whether more points help our method further. There is no density mismatch: ReSplat and our method both predict depth at the same resolution and run the point transformer at the same point density. ReSplat ends with 131k Gaussians, 16 times our count, because it predicts 16 Gaussians per output point from the point transformer, while we use the output points directly.