- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes a reference-free diagnostic for symmetry violations in 3D generative models. The symmetry defect is defined as the KL divergence between a model and its group-averaged distribution. It further yields a population lower bound on divergence to invariant targets and quantifies the improvement from randomized rotation. It distinguishs exact subgroup averaging from biased continuous-group estimation. Experiments cover synthetic targets, ShapeNet point clouds, pretrained generators, and empirical model selection, with explicit discussion of estimation limitations.
Critical strengths
The main derivation appears correct. The controlled experiments help check whether the score behaves as expected. Comparing models that preserve symmetry with those that do not is also useful. The authors clearly explain the limits of their results, including why the PointFlow experiment does not demonstrate better overall shape quality.
Critical weaknesses
The main contribution of this paper is applying the KL divergence to a model and its rotated version. The main mathematical ideas, including group averaging, the KL result, and random symmetrization, come from earlier work, as the authors acknowledge. The paper mainly applies these ideas to 3D generators. It needs to explain more clearly what this application lets users do beyond existing methods and provide stronger experiments showing that added value.
The method evaluates likelihood many times for each generated sample. For diffusion models, each evaluation may require solving an ODE, so the complete test could be expensive. Actual runtime, memory usage, and comparisons with other diagnostics using the same computational budget would make the claim that this is a cheap tool more convincing.
The pretrained shape-generation test uses PointFlow (2019), which is quiet old model. Tests on newer shape generators with other non point cloud based 3D representation such as TRELLIS (https://arxiv.org/abs/2412.01506) would help establish whether the diagnostic remains useful and practical for higher-quality 3D generation.
The shape experiments only use point clouds. It is unclear how the method would work for meshes or implicit fields, especially how to compute the required likelihoods for these outputs. Testing these representations would strengthen the paper. A CAD-generation experiment on ABC dataset https://arxiv.org/abs/1812.06216 could also help demonstrate its usefulness in tasks where geometric accuracy matters. The main experiment is conducted on limited category of the shapeNet which has limited diversity and does not prove the conclusion of the paper at larger range.
Sampling rotations can overestimate the true score. Even exact averaging over a finite subgroup leaves uncertainty from the limited number of generated samples. The authors provide bias corrections, but a reliable lower confidence bound that accounts for both sampling error and numerical errors in likelihood computation is still missing.
Questions for authors
What does this deployment formulation enable beyond established group-invariance diagnostics and symmetrization methods?
Response to Reviewer b9jx (3/3)
Following up on W2–W5, we ran the audit on a recent pretrained mesh generator. Details are in the new App. J.5 of the revised PDF (Tables 22–24, plus a row in Table 21).
W3/W4: a newer generator with a mesh representation. MeshXL (NeurIPS 2024) is a 350M-parameter autoregressive transformer that writes a triangle mesh as a sequence of quantized vertex coordinates, so its likelihood is exact. This is the class of mesh/CAD generators that App. I placed in scope; we now measure it instead of only arguing for it. We use the public unconditional base model unchanged and score real meshes with the model's own serialization. Our implementation of that serialization reproduces the released token sequences of all 1,754 validation meshes exactly. The 24 cube rotations and the three axis mirrors act on the continuous mesh before serialization. The left-right axis (x) was fixed from the data alone, before any model was scored. The up-down and front-back mirrors serve as contrasts the data do not have.
W5: categories. We use all 54 ShapeNet categories in MeshXL's release: 1,754 validation meshes, held out from pretraining (this is the validation split listed in the MeshXL paper).
Result: held-out sign test, ΔH=Etest[logqˉH−logq] (mean and 95% t-interval over meshes). "NLL removed" is the sum of Δ over the sum of NLL across all test meshes:
Limits (all in App. J.5).
W2: cost. One density call on a mesh (4.0k tokens on average) takes 0.32 s on a shared L40 GPU, with 5.4 GB peak memory. The octahedral audit of one mesh therefore costs about 8 s, against about 100 s to sample one mesh (22 GB, batch 8). The full held-out audit, 1,754 meshes × 27 transformations, took 4.3 GPU-hours. Rescoring 240 sequences in fp64 changes logq by at most 9×10−4 nats, so numerical error is negligible here.
W3, TRELLIS. Our earlier answer still holds: TRELLIS decodes through a non-invertible decoder and has no output density. MeshXL represents the mesh generators that do have one.