- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a differentiable mesh renderer using analytical anti-aliasing to improve exports of opaque meshes from neural fields or 3DGS.
Given posed multi-view images, the proposed approach first extracts a mesh using GOF, then optimizes the mesh using differentiable rendering with an analytical anti-aliasing technique.
There is substantial related work in the field of analytical anti-aliasing (cone tracing, diff rasterization, warped-area sampling, MC edge sampling, ...), and it is not clear to me what the novel contribution is in the proposed AA formulation. This could be clarified.
My main criticism is that geometry and appearance is entangled in the evaluation. The novel contribution here is improved geometric fidelity, so I suggest removing the appearance bit from the evaluation and focus on geometric quality of the extractions. Given that the appearance model does not match the reference anyway (Blender use a lit PBR material, while the exported meshes uses vertex colors AFAIU), there is already an appearance mismatch, which obstructs the evaluation of geometric quality.
Overall, the extracted meshes seem to be of high quality, and this is a very useful tool to have. However, I would have prefered the validation to focus entirely on geometry quality, with rendered views with normal map and/or diffuse shading instead of baked emissive color.
I am on the fence here. In this setting, given that the geometry quality improves over GOF, I am leaning positive, but if the main advantage over the GOF baseline is improved appearance (represented as emissive color/SH DC component, not full PBR), then I am less convinced.
Critical strengths
Improved mesh quality via differentiable rendering with analytical antialising.Using an analytical anti-aliasing approach to improve the extracted mesh makes sense to me.
Critical weaknesses
While I like the approach overall, I would prefer if the validation focus on geometry quality, using Chamfer distance and renderings with normal map shading and/or diffuse shading, to highlight the geometry quality. Baked emissive color is not a very useful appearance representation, I see it as a stop-gap. Similar to GOF and FlexiCubes (https://research.nvidia.com/labs/toronto-ai/flexicubes/), I would recommend to visualize the extracted meshes using normal map shading or diffuse shading, to more clearly see the geometry detail in the extracted meshes.
The only geometry evaluation I found was in Table 13 and Table 23. In Table 13, SPIRE has higher CD than GOF, which is used to initialize SPIRE's optimization. This is slightly concerning. Is SPIRE optimization reducing the geometry quality from the GOP initialization for this dataset? This could be discussed in more detail. Is this only for DTU or also true for NeRF synthetic? Also, other approaches, like MILo performs better than SPIRE here.
This statement at l377: "GOF retains sharper geometry but yields mottled vertex colors" also hints that the difference is not in geometric quality, but exported appearance.
Questions for authors
In Table 14, why is the GOF initialization training time for SPIRE larger than the GOF training time? I thought GOF was used to initialize SPIRE?
Let DΠ be the rendered median-depth image: How is this computed?
This statement at l377: "GOF retains sharper geometry but yields mottled vertex colors" also hints that the difference is not in geometric quality, but exported appearance. Is this generally the case, that GOF retains sharper geometry, and the proposed approach mostly improves appearance (the SH DC component)?
Some citations: Ray Tracing with Cones John Amanatides http://www.cs.yorku.ca/~amana/research/cones.pdf
Analytic Anti-Aliasing of Linear Functions on Polytopes Auzinger et al.
Unbiased Warped-Area Sampling for Differentiable Rendering, Bangaru et al.
Differentiable monte carlo ray tracing through edge sampling. Li et al.
7 comments
Official comment for reviewer Gipe (part 2/4: W2)
[W2] Does refinement improve the actual geometry?
SPIRE improves visible geometry from the same initial mesh. SPIRE combines visible face portions to better represent fine or curved scene structures within the given coarse topology. Chamfer distance (CD) also measures hidden portions, so it does not fully reflect this improvement. We thus evaluate geometry seen by standard mesh renderers. We clarify (1) DTU initialization and show that SPIRE (2) matches object silhouettes more closely (Table G1) while (3) placing their boundaries more accurately in 3D (Table G2), and (4) recovers more depth boundaries between overlapping object parts (Table G3).
(1) DTU comparison. As described in Appendix C, SPIRE starts from a mesh constructed by TSDF fusion of GOF-rendered depth maps. Table 13 evaluates a separately extracted GOF mesh. GOF's lower Chamfer distance therefore does not mean that SPIRE made its initial geometry less accurate. We examine the change directly on NeRF-Synthetic by comparing the same initial mesh before and after optimization in Tables G1-G3.
(2) Visible shape (Table G1). The color-only control fixes the initial geometry, so changes in silhouette require moving the faces. SPIRE improves coverage and outline alignment in held-out views. This directly shows geometric improvement from the initialization, beyond fitting colors.
Table G1. NeRF-Synthetic silhouettes and contours. Held-out views use a common ray caster and reference image alpha masks. Intersection-over-union (IoU) measures mask overlap; contour F-score measures outline alignment at each pixel tolerance. RaDe-GS retains its largest connected component; unfiltered TSDF retains all.
| Method | Silhouette IoU ↑ | Contour F@1px ↑ | Contour F@2px ↑ | Contour F@4px ↑ |
|---|---|---|---|---|
| GOF | 0.978 | 0.936 | 0.956 | 0.971 |
| MeshSplatting | 0.982 | 0.934 | 0.960 | 0.980 |
| 2DGS | 0.648 | 0.329 | 0.443 | 0.531 |
| nvdiffrast | 0.974 | 0.916 | 0.939 | 0.958 |
| MILo | 0.963 | 0.826 | 0.917 | 0.963 |
| RaDe-GS | 0.701 | 0.404 | 0.518 | 0.655 |
| RaDe-GS unfiltered TSDF | 0.929 | 0.537 | 0.698 | 0.877 |
| Initial mesh / color-only | 0.976 | 0.917 | 0.944 | 0.963 |
| SPIRE | 0.982 | 0.958 | 0.972 | 0.983 |
(3) Boundary depth (Table G2). We next check whether these outlines lie at the correct depth. SPIRE improves both precision and recall over the initial mesh and baselines in 3D. Thus, the visible boundary is more accurately placed in space, beyond matching its image projection.
Table G2. Outer-boundary 3D accuracy on NeRF-Synthetic. Mesh and original-scene silhouettes are backprojected using first-hit depths from identical rays. Points match within 0.02 scene units in the same view. Precision measures matched predictions; recall measures recovered reference points; F1 balances both. Eight-scene means (%).
| Method | Outer precision ↑ | Outer recall ↑ | Outer F1 ↑ |
|---|---|---|---|
| GOF | 94.48 | 93.01 | 93.59 |
| MeshSplatting | 94.79 | 93.17 | 93.79 |
| 2DGS | 47.85 | 45.03 | 43.47 |
| nvdiffrast | 92.09 | 90.66 | 91.10 |
| MILo | 92.15 | 92.48 | 91.85 |
| RaDe-GS | 75.65 | 53.70 | 58.64 |
| RaDe-GS unfiltered TSDF | 81.11 | 85.87 | 82.72 |
| Initial mesh / color-only | 94.09 | 91.19 | 92.36 |
| SPIRE | 94.89 | 94.28 | 94.43 |
(4) Internal structure (Table G3). Inside the outline, one object part can lie in front of another, creating a depth boundary. SPIRE recovers more of these reference boundaries in 3D at all three tolerances. The geometry updates therefore recover visible internal structure as well as the outer shape.
Table G3. Internal 3D boundary recovery on NeRF-Synthetic. Neighboring foreground depth changes above 2% of the nearer depth are backprojected from mesh and original-scene first-hit maps, excluding the silhouette band. Recall is the fraction of reference points recovered within each 3D distance in the same view. Distances are in scene units; eight-scene means (%).
| Method | [email protected] ↑ | [email protected] ↑ | [email protected] ↑ |
|---|---|---|---|
| GOF | 69.17 | 80.92 | 91.46 |
| MeshSplatting | 61.53 | 73.10 | 81.72 |
| 2DGS | 36.95 | 51.40 | 65.58 |
| nvdiffrast | 64.88 | 76.61 | 85.64 |
| MILo | 62.65 | 75.09 | 84.35 |
| RaDe-GS | 35.71 | 44.06 | 51.85 |
| RaDe-GS unfiltered TSDF | 54.49 | 70.41 | 83.22 |
| Initial mesh / color-only | 66.49 | 78.83 | 90.79 |
| SPIRE | 73.27 | 84.61 | 93.44 |
These visible-shape controls complement the DTU whole-surface comparison, including MILo's 0.68 mm CD. We will add Tables G1-G3 and clarify the initialization in Appendix D.5.
Official comment for reviewer Gipe (part 3/4: W3)
[W3] The wording about sharper GOF geometry
By "sharper geometry," we meant that GOF produces a more continuous-looking surface than MeshSplatting in this example. We explain (1) why this visual difference does not necessarily mean more accurate surface positions, and (2) how SPIRE improves visible geometry beyond GOF (Tables G1-G3).
(1) What the visual comparison means. MeshSplatting transitions from semitransparent to opaque faces, which can make previously less visible triangles stand out. GOF instead extracts a level set of its opacity field, producing a more continuous-looking surface in this example, but this level set can still be displaced from the true surface. Since GOF evaluates colors at the extracted vertex positions, this displacement can also lead to incorrect vertex colors. This provides a possible link between surface placement and mottled appearance, motivating direct measurements of the geometry as well as visual inspection.
(2) How SPIRE improves the geometry. Tables G1-G3 show better silhouette alignment, outer-boundary 3D accuracy, and internal-boundary recall than GOF. These tests assess the shape and position of the surface seen by the cameras. Together, they show that SPIRE improves visible geometry alongside exported appearance, addressing the concern that the difference is only in vertex colors. We will clarify the original comparison and connect it to these measurements in Section 4.2.
Official comment for reviewer Gipe (part 1/4: W1)
Thank you for recognizing the quality and usefulness of the extracted meshes and the value of analytic anti-aliasing for mesh optimization. We also appreciate your questions about separating geometry from appearance, interpreting the GOF comparison, and clarifying the relation to prior work. We will incorporate the clarifications, results, and revisions addressing all your points into the camera-ready version. We answer Critical Weaknesses W1-W3 and Questions Q1-Q4 in review order; Q4 addresses your suggested references.
[W1] Seeing the geometry without baked colors
We evaluate both the appearance preserved after mesh export and the underlying geometry. (1) We render exported meshes with a common emission material to measure how well they preserve the observed image appearance. (2) We will add the requested normal-map and diffuse-shading comparisons to show geometry without baked colors. These will complement the paper's Chamfer-distance and depth evaluations, with further quantitative evidence from silhouettes and 3D boundaries in Tables G1-G3.
(1) Why we evaluate exported appearance. A central goal of SPIRE is to produce meshes that preserve image fidelity when used in standard graphics software. Rendering the exported vertex colors with an emission material measures this transfer directly. It tests whether the mesh remains a faithful representation of the observed scene after leaving the training renderer. This is why baked-color comparisons are part of the evaluation alongside geometric accuracy.
(2) How we assess geometry separately. The paper already reports DTU Chamfer distance in Tables 13/23 and depth-map visualizations in Table 13. Your suggested normal-map and diffuse-shading views make local shape easier to inspect without baked colors, and we will include these comparisons with matched views and lighting. Tables G1-G3 further measure silhouettes and the 3D locations of visible boundaries. Comparing these with the same initial mesh isolates the change in geometry, directly addressing whether the gain goes beyond appearance. We will add these evaluations to Appendix D.5.