- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes a sparse-view reconstruction framework. The method first generates multiple novel-view sequences using pre-trained SEVA, and then conduct initial selection on generated views through DINO feature. During scene level optimization, the views are further progressively filtered based on designed strategy. After optimization pipeline, the framework is expected to have better reconstruction quality with sparse inputs such as 2 images.
Critical strengths
For sparse-view reconstruction, the best-of-N selection of diffusion generated novel views is reasonable. While most sparse-view reconstruction methods now require diffusion prior, making this qualitative selection automatic is timely.
Critical weaknesses
However, this paper does not seem to actually solve this problem. The biggest concern is, while the baseline for comparing should be 2DGS + SEVA, in Figure. 3 the proposed method shows no qualitative improvements (or even worse) comparing with 2DGS + SEVA. This leads to another concern that the quantitative results in Table. 1 does not correspond with qualitative results. I wonder why OracleGS has the highest SSIM and S2C-3D has the highest PSNR appear to be quite bad in Figure. 3. I can only imagine the reasons be: 1. Qualitative results are extremely selected cases. 2. The quantitative results are not correctly evaluated. 3. The metrics are not representative under such scenario. Either reason makes the experimental results not persuasive.
Questions for authors
While there might be certain misunderstanding, it would be good if the authors could address my concerns stated in the weaknesses section.
4 comments
Selection of the qualitative view
Fig. 3 of the main manuscript shows the same DTU view, view 25, for every scene and every method. Table C3 gives the object-masked PSNR at this view.
Table C3. Object-masked PSNR at the view shown in Fig. 3 of the main manuscript, for the five scenes shown in the figure.
| method | scene 21 | scene 24 | scene 110 | scene 114 | scene 118 |
|---|---|---|---|---|---|
| Ours | 8.40 | 11.79 | 14.32 | 12.80 | 14.99 |
| OracleGS | 8.43 | 10.61 | 14.38 | 9.11 | 15.94 |
| 2DGS+SEVA | 9.31 | 11.20 | 13.12 | 10.14 | 15.60 |
| S2C-3D | 8.95 | 10.94 | 13.65 | 17.64 | 17.77 |
| GeoQuery | 9.66 | 10.48 | 13.47 | 16.88 | 17.90 |
| Difix3D | 7.82 | 12.04 | 12.72 | 10.14 | 14.70 |
Table C3 shows that our method does not achieve the highest PSNR at this view on any of the five scenes, and compared with 2DGS+SEVA it is higher on some scenes and lower on others. The displayed view is therefore not one where our method renders best, although this does not by itself rule out every form of selection bias.
Comparison with 2DGS+SEVA
We agree that the RGB renderings of our method and 2DGS+SEVA look similar, and Fig. 3 of the main manuscript shows only RGB. Rendering metrics and surface metrics measure different properties. PSNR, SSIM and LPIPS compare images, while Chamfer distance and surface F1 compare the reconstructed surface with the ground-truth scan, which is the main objective of our paper.
Table C4. Geometry over all eleven scenes.
| method | All F1 | Obs F1 | Unobs F1 | All Chamfer |
|---|---|---|---|---|
| Ours | 0.818 | 0.807 | 0.819 | 22.12 |
| 2DGS+SEVA | 0.798 | 0.773 | 0.817 | 23.10 |
| OracleGS | 0.776 | 0.743 | 0.829 | 24.09 |
| S2C-3D | 0.719 | 0.667 | 0.789 | 24.76 |
| Difix3D | 0.649 | 0.595 | 0.709 | 30.07 |
| 2DGS | 0.620 | 0.598 | 0.653 | 43.05 |
| GeoQuery | 0.493 | 0.445 | 0.526 | 42.77 |
Table C5. All F1 per scene for Ours and 2DGS+SEVA.
| scene | 21 | 24 | 34 | 37 | 38 | 40 | 82 | 106 | 110 | 114 | 118 |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Ours | 0.950 | 0.846 | 0.802 | 0.534 | 0.841 | 0.737 | 0.603 | 0.832 | 0.931 | 0.967 | 0.959 |
| 2DGS+SEVA | 0.931 | 0.724 | 0.885 | 0.517 | 0.841 | 0.791 | 0.544 | 0.900 | 0.821 | 0.868 | 0.951 |
Tables C4 and C5 show that our method improves on 2DGS+SEVA in overall F1, Observed F1 and Chamfer, while the two perform comparably on unobserved surfaces. The improvement is modest, mainly on observed surfaces, and varies between scenes.
The 2DGS+SEVA baseline uses the same selected SEVA sequence as our method, with all 16 generated views and its own reconstruction settings.
We will state in the paper that Fig. 3 of the main manuscript compares appearance, and that the geometric comparison is given by the surface metrics.
As the discussion period is coming to an end, we would be sincerely grateful if the reviewers could kindly take a moment to consider our rebuttal and the additional experiments. We welcome further discussion and hope the new evidence can be considered in the final evaluation.
We thank the reviewer for pointing out the gap between the rendering metrics and the visual results. Investigating it, we found an error in how our rendering evaluation is described in the paper. We correct it below, report new object-masked rendering metrics, and then address the comparison with 2DGS+SEVA, which concerns geometry rather than appearance.
Correction of the rendering evaluation
Section 4 of the main manuscript and Appendix A.6 of the technical supplements state that PSNR, SSIM and LPIPS are computed with object masks on the sixteen PixelNeRF test views. This description is wrong, and we apologize for the confusion. The values reported in Table 1 of the main manuscript were computed on full images without object masks, over 25 DTU views, one of which is an input view. We recomputed all rendering metrics with object masks and report them below. We will replace the rendering columns of Table 1 of the main manuscript with these values and correct the text.
The new evaluation uses the same renders and the same DTU object masks for every method, over the twenty-four held-out views. PSNR is computed over object pixels, and SSIM and LPIPS are computed with the background set to zero.
Table C1. Rendering metrics on the five scenes of Table 1 of the main manuscript. The first three columns reproduce the values reported in Table 1. The last three columns are the corrected object-masked metrics.
Table C2. Corrected object-masked rendering metrics over all eleven scenes.
Tables C1 and C2 show that, with object masks, our method achieves the strongest SSIM on both scene sets, while GeoQuery and S2C-3D achieve a higher PSNR and a better LPIPS. The corrected evaluation therefore indicates competitive rendering quality on the object rather than the best rendering on every measure. OracleGS has a high SSIM as reported in Table 1 of the main manuscript but a weaker object-masked SSIM and LPIPS, which is consistent with the impression of the reviewer that some methods with high reported metrics look worse. We do not claim that the corrected metrics explain every visual difference in Fig. 3 of the main manuscript.