acceptodds
Under review as a conference paper at ICLR 2027

A Reference-Free Lower Bound on the Symmetry Error of 3D Generative Models

Abstract

Scalable 3D generative models increasingly use free-form backbones even when the data or downstream task has a known symmetry, learning rotation and permutation invariance from augmentation rather than enforcing it architecturally. Augmentation teaches symmetry only approximately, so deployed models carry a residual symmetry-breaking error that is hard to measure: the reference distribution is unavailable after training, and existing equivariance diagnostics quantify the network's mismatch with the group action rather than the distance between the generated distribution and the target. We define the symmetry defect as the Kullback–Leibler divergence between a model and its own group-orbit average, computable from the model likelihood alone. A classical information-geometric Pythagorean identity makes this model-only quantity a lower bound on the error to any symmetric target, and the same identity is constructive: applying one uniformly random group element to each sample removes exactly the certified component without retraining. We separate the population quantity that carries the guarantee from its finite Monte-Carlo estimate, and exact enumeration of a finite subgroup, itself a valid lower bound, from sampled continuous-group averaging with its Jensen bias. Across known-density targets, a free-form diffusion model with a probability-flow likelihood, real ShapeNet point clouds, and pretrained conformer and point-cloud generators, finite-sample estimates respect the population bound, non-equivariant backbones read a measurable defect while equivariant counterparts read near zero, symmetrization markedly improves likelihoods, and the score ranks candidate models in strong agreement with hidden reference orderings. Because the defect is a lower bound, a near-zero reading certifies near-invariance rather than closeness to the target: the certificate falsifies rather than validates.

Primary area:
generative models
Keywords:
3D generative models, equivariance, symmetry, point clouds, KL divergence, information geometry, orbit averaging, model auditing, diffusion models, normalizing flows

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_b9jxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes a reference-free diagnostic for symmetry violations in 3D generative models. The symmetry defect is defined as the KL divergence between a model and its group-averaged distribution. It further yields a population lower bound on divergence to invariant targets and quantifies the improvement from randomized rotation. It distinguishs exact subgroup averaging from biased continuous-group estimation. Experiments cover synthetic targets, ShapeNet point clouds, pretrained generators, and empirical model selection, with explicit discussion of estimation limitations.

Critical strengths

The main derivation appears correct. The controlled experiments help check whether the score behaves as expected. Comparing models that preserve symmetry with those that do not is also useful. The authors clearly explain the limits of their results, including why the PointFlow experiment does not demonstrate better overall shape quality.

Critical weaknesses

The main contribution of this paper is applying the KL divergence to a model and its rotated version. The main mathematical ideas, including group averaging, the KL result, and random symmetrization, come from earlier work, as the authors acknowledge. The paper mainly applies these ideas to 3D generators. It needs to explain more clearly what this application lets users do beyond existing methods and provide stronger experiments showing that added value.

The method evaluates likelihood many times for each generated sample. For diffusion models, each evaluation may require solving an ODE, so the complete test could be expensive. Actual runtime, memory usage, and comparisons with other diagnostics using the same computational budget would make the claim that this is a cheap tool more convincing.

The pretrained shape-generation test uses PointFlow (2019), which is quiet old model. Tests on newer shape generators with other non point cloud based 3D representation such as TRELLIS (https://arxiv.org/abs/2412.01506) would help establish whether the diagnostic remains useful and practical for higher-quality 3D generation.

The shape experiments only use point clouds. It is unclear how the method would work for meshes or implicit fields, especially how to compute the required likelihoods for these outputs. Testing these representations would strengthen the paper. A CAD-generation experiment on ABC dataset https://arxiv.org/abs/1812.06216 could also help demonstrate its usefulness in tasks where geometric accuracy matters. The main experiment is conducted on limited category of the shapeNet which has limited diversity and does not prove the conclusion of the paper at larger range.

Sampling rotations can overestimate the true score. Even exact averaging over a finite subgroup leaves uncertainty from the limited number of generated samples. The authors provide bias corrections, but a reliable lower confidence bound that accounts for both sampling error and numerical errors in likelihood computation is still missing.

Questions for authors

What does this deployment formulation enable beyond established group-invariance diagnostics and symmetrization methods?

1 comment
AuthorsOpenReview ↗

Response to Reviewer b9jx (3/3)

Following up on W2–W5, we ran the audit on a recent pretrained mesh generator. Details are in the new App. J.5 of the revised PDF (Tables 22–24, plus a row in Table 21).

W3/W4: a newer generator with a mesh representation. MeshXL (NeurIPS 2024) is a 350M-parameter autoregressive transformer that writes a triangle mesh as a sequence of quantized vertex coordinates, so its likelihood is exact. This is the class of mesh/CAD generators that App. I placed in scope; we now measure it instead of only arguing for it. We use the public unconditional base model unchanged and score real meshes with the model's own serialization. Our implementation of that serialization reproduces the released token sequences of all 1,754 validation meshes exactly. The 24 cube rotations and the three axis mirrors act on the continuous mesh before serialization. The left-right axis (xx) was fixed from the data alone, before any model was scored. The up-down and front-back mirrors serve as contrasts the data do not have.

W5: categories. We use all 54 ShapeNet categories in MeshXL's release: 1,754 validation meshes, held out from pretraining (this is the validation split listed in the MeshXL paper).

Result: held-out sign test, ΔH=Etest[log⁡qˉH−log⁡q]\Delta_H=\mathbb E_{\rm test}[\log\bar q_H-\log q] (mean and 95% t-interval over meshes). "NLL removed" is the sum of Δ\Delta over the sum of NLL across all test meshes:

HHΔ\Delta (nats/mesh)meshes with Δ>0\Delta>0held-out NLL removedcategories with mean Δ>0\Delta>0
OO (24 rotations)+24.0 [22.0, 25.9]69%3.5%54/54
C4C_4 about the up axis (chosen after seeing OO)+22.8 [21.4, 24.3]73%3.3%54/54
left-right mirror+0.5 [0.2, 0.8]6%0.07%28/54
up-down mirror (contrast)−0.4 [−0.6, −0.2]1%−0.06%5/54
front-back mirror (contrast)−0.4 [−0.5, −0.3]2%−0.06%7/54
  • Averaging a pretrained mesh model's likelihood over the cube rotations lowers held-out NLL on real, canonically posed data, in every category and with no retraining.
  • The model keeps the up direction but is unsure of the yaw. For 96% of meshes its preferred rotation keeps the up axis, but only 25% of those keep the original yaw. Averaging over the four rotations about the up axis alone removes 3.3% of the NLL, 95% of the octahedral gain, at a sixth of the cost. The MeshXL paper augments pretraining with 0/90/180/270° rotations and does not name the axis.
  • The left-right mirror gain is statistically detectable but negligible (0.07% of the NLL), not worth doubling the density cost. The two contrast mirrors are significantly negative. So on one model the same test says: symmetrize over yaw, not over mirrors.
  • For every mirror the median Δ\Delta is −log⁡2-\log 2: for most meshes, the model gives the re-serialized mirror image almost no probability. An orientation-preserving mirror (reflect, then reverse each face's vertex order), added after the first results, gives the same picture. We do not separate geometry from serialization effects here.

Limits (all in App. J.5).

  • The fine-tuned chair/table/bench/lamp checkpoints memorize their training split. On 40 training vs. 40 validation chairs, the chair checkpoint has NLL 0.010 vs. 0.782 nats/token; the base model has 0.164 vs. 0.186. On these checkpoints every transformation raises held-out likelihood, the contrast mirrors included. There the test detects overfitting, not symmetry. We report them as a supplement and draw no symmetry conclusion from them.
  • A mesh has many token sequences, and the sampler mostly emits sequences outside the canonical order (16% canonical for the base model). Sample-side certificates under one canonical serialization are therefore only a proxy. We report them as a description, not for decisions.
  • We sample by pure ancestral sampling (temperature 1, no top-k/top-p), because MeshXL's default top-k/top-p sampler draws from a different distribution than its likelihood.

W2: cost. One density call on a mesh (4.0k tokens on average) takes 0.32 s on a shared L40 GPU, with 5.4 GB peak memory. The octahedral audit of one mesh therefore costs about 8 s, against about 100 s to sample one mesh (22 GB, batch 8). The full held-out audit, 1,754 meshes × 27 transformations, took 4.3 GPU-hours. Rescoring 240 sequences in fp64 changes log⁡q\log q by at most 9×10−49\times10^{-4} nats, so numerical error is negligible here.

W3, TRELLIS. Our earlier answer still holds: TRELLIS decodes through a non-invertible decoder and has no output density. MeshXL represents the mesh generators that do have one.

Reviewer_etByOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
1
Presentation
3
Soundness
2(+1)
Read review
Paper summary

The authors study 3D generative models that are built from models that are not equivariant by design. They propose 1) a measure to audit and measure how "symmetric" a generative model is, which consists in comparing the model's likelihood with its group-orbit-average likelihood; 2) a corrector sampling algorithm, which consists in applying randomly an operation (e.g. a rotation) after sampling. Then, they audit several models and apply their correction sampler.

Critical strengths
  • Propose an auditing pipeline for the symmetry error of 3D generative models

  • Analyse symmetry error of several recent 3D generative models in different settings.

  • The paper is well written, with a clear explanation of the audit procedure.

Critical weaknesses
  • Very limited novelty, as acknowledged by the authors (appendix H): "We do not claim novelty for orbit averaging, the KL projection, the ($q/\bar q$) statistic, the entropy identity, or randomized symmetrized sampling." The proposed auditing pipeline contains very few novelty. It seems that the paper is mainly a study of existing 3D generative models using technical tools already developed.

  • If the symmetry error of a model can be put to zero thanks to a simple randomized symmetrized sampling (or by computing group-orbit-average if we are interested in the likelihood), then what is the purpose of auditing the symmetry error of generative models? Meaning that the problem has simple, principled, and effective solutions.

Questions for authors
  • For diffusion models, does the likelihood of a model exactly reflect its generated distribution? It actually depends on what is the sampling algorithm, e.g. is classifier-free guidance used, is stochastic sampling used, etc. See e.g. "Maximum Likelihood Training for Score-Based Diffusion ODEs" from Lu et al. for a discussion on the topic. Could the authors clarify that point on the setting used in the paper and, generally, what it implies on their auditing mechanism?
1 comment
AuthorsOpenReview ↗

Response to Reviewer etBy (3/3)

Thank you very much for reading our rebuttal so carefully and for raising your score. We appreciate that you reconsidered the role of auditing alongside randomized symmetrization.

Your remaining concern, limited technical novelty and impact, was also raised by the other reviewers, so we answer it here together with their related questions. On novelty we do not claim more than App. H states. On impact, we ran one more experiment that also answers Reviewer nYV1's request for a gain on real held-out data and Reviewer b9jx's requests for a newer generator, a mesh representation and more categories. Details are in the new App. J.5 of the revised PDF (Tables 22–24).

Setup. MeshXL (NeurIPS 2024) is a pretrained 350M-parameter autoregressive transformer that writes a triangle mesh as a sequence of quantized coordinates, so its likelihood is exact. We use the public base model unchanged and score the 1,754 held-out ShapeNet validation meshes (all 54 categories) in their original canonical pose, with the model's own serialization. Nothing is retrained and nothing in the evaluation is rotated. The quantity is the held-out sign test of Sec. 7, ΔH=Etest[log⁡qˉH−log⁡q]\Delta_H=\mathbb E_{\rm test}[\log\bar q_H-\log q], which is positive exactly when the symmetrized model fits the real test data better.

HHΔ\Delta (nats/mesh, 95% CI)held-out NLL removedcategories with mean Δ>0\Delta>0
24 cube rotations+24.0 [22.0, 25.9]3.5%54/54
4 rotations about the up axis (chosen after seeing the row above)+22.8 [21.4, 24.3]3.3%54/54
left-right mirror+0.5 [0.2, 0.8]0.07%28/54
up-down mirror (contrast)−0.4 [−0.6, −0.2]−0.06%5/54
front-back mirror (contrast)−0.4 [−0.5, −0.3]−0.06%7/54

W2: why audit when randomized symmetrization exists. This model shows the point more directly than our earlier examples. Symmetrization is cheap to apply, but which group to apply it over is not known in advance. On MeshXL the same test gives three different answers. Rotations about the up axis help (3.3% of the NLL, 4 density calls per mesh). The left-right mirror is statistically detectable but negligible, so doubling the cost buys nothing. The up-down and front-back mirrors make held-out likelihood worse. The audit also located the axis: the MeshXL paper says pretraining rotates meshes by multiples of 90° but does not name the axis, and the model's own likelihoods show that it keeps the up direction and is unsure only of the yaw.

Your likelihood–sampler question. MeshXL is a concrete case of the issue you raised. Its default sampler uses top-k and top-p truncation, so its samples do not follow the distribution whose likelihood the model reports. For the sample-side audit we therefore sample by pure ancestral sampling (temperature 1, no truncation) and say so in App. J.5. The held-out test above does not use the sampler at all.

Cost. Auditing one mesh over the 24 rotations takes about 8 s on one GPU, against about 100 s to sample one mesh. The full held-out audit took 4.3 GPU-hours.

Limits, also stated in App. J.5. The fine-tuned per-category MeshXL checkpoints memorize their training split, so on them the test detects overfitting rather than symmetry; we report them only as a supplement. A mesh has many token sequences and the sampler mostly emits non-canonical ones, so the sample-side numbers are only a proxy and we do not base decisions on them. We measured held-out likelihood, not a downstream task.

Reviewer_nYV1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary
  • The authors study symmetry in the context of (likelihood-based) 3D generative modelling, focusing on failures of modern models that do not provide guarantees regarding it. They focus on the properties of the probability distribution itself, rather than at the network level.
  • They devise the "symmetry defect", which measures (lower bounds) the "error" in terms of symmetry of a generative model. They provide extensive theoretical analysis and also suggest a simple (data-free, training-free) way to remove this error: simply sample a random group element when sampling.
Critical strengths
  • The authors are right in that the many techniques developed for enforcing symmetry have fallen by the wayside as simple architectures + scale has proven more impactful than complex architectures. I also think theoretical or architectural approaches to constraints, such as symmetry, remain important to investigate, rather than always relying on scale.
  • The theoretical characterization is elegant, simple, and thorough.
  • The authors show that the metric can be easily computed and does measure what we expect it to, on a few examples. Their approach can improve experimental likelihoods in one tested case, and can help with model ranking when symmetry is a concern. I appreciated that they did not attempt to overstate their results.
Critical weaknesses
  1. My primary concern is that the practical utility of the method seems too limited; specifically, what real-world tasks could this approach assist in? The paper need not necessarily perform such tasks, but it should be clearer what tasks and problems they could impact. For instance, is the asymmetry of existing models actually a problem, wrt permutation or rotation? Are there applications that suffer due to it missing?

In the case of orientation, for instance, we might prefer canonically posed 3D shapes, not truly randomly rotated ones, though this may depend on the application.

  1. In addition, the experimental results themselves are on real models, but the current approach does not show improvements on a real-world task with them. The improved likelihoods are interesting, but on a somewhat contrived evaluation distribution that prefers uniform rotation by construction. For the molecular conformers, the metric does correctly detect a failure in symmetry, but it is unclear what downstream impacts this may have.

  2. Finally, while the theory is interesting and novel in its application to 3D shapes, it seems to be largely an application of existing theory. In this sense, the novelty and impact may be somewhat limited.

Minor notes:

  • I would have found it more compelling if they could tackle symmetries related to the shape itself, such as (approximate) reflection symmetries or articulation-invariant (isometric) symmetry. (Though I realize these are symmetries of shapes themselves, not the generative distribution, so it may be outside the purview of the author's current work.)
  • Table 4 and 14, as well as appendix I, mention an error - this may have been mistakenly included?
Questions for authors
  1. Could the authors describe a concrete application in which the measured rotation or permutation asymmetry constitutes a meaningful failure, and explain how the diagnostic would inform a modeling or deployment decision? For rotational symmetry, what makes an invariant target appropriate for that application, rather than a canonical or condition-dependent orientation?

  2. What evidence connects the measured symmetry defect to adverse consequences beyond likelihood on a uniformly rotation-augmented target? E.g., what practical consequences should we infer from the DiTMC result, and is there evidence connecting to a downstream molecular computation? A concrete analytical argument or clarification of existing results would help.

  3. Can the authors discuss which of their theoretical contributions are the most novel and impactful, and why?

3 comments
AuthorsOpenReview ↗

Response to Reviewer nYV1 (4/4)

Thank you again for W2 and Q1. W2 noted that our likelihood gains were measured on an evaluation distribution rotated by construction. We have now run the held-out test on a real pretrained model with real, unrotated test data (new App. J.5 of the revised PDF).

W2: improvement on real held-out data. MeshXL (NeurIPS 2024) is a pretrained 350M-parameter autoregressive mesh generator with an exact likelihood. We score the 1,754 ShapeNet validation meshes (all 54 categories) in their canonical pose. They are held out from pretraining, and we score them with the model's own serialization and no retraining. The quantity is ΔH=Etest[log⁡qˉH−log⁡q]=KL(p∥q)−KL(p∥qˉH)\Delta_H=\mathbb E_{\rm test}[\log\bar q_H-\log q]=\mathrm{KL}(p\Vert q)-\mathrm{KL}(p\Vert\bar q_H). Here qq is MeshXL's likelihood of its own serialization, which is not normalized over meshes; orbit averaging preserves the total mass, so the identity holds after normalizing qq and qˉH\bar q_H by the same constant. It holds for any test distribution pp, so a positive Δ\Delta means the symmetrized model is closer to the real test distribution.

  • Octahedral averaging: Δ=+24.0\Delta=+24.0 nats/mesh [22.0, 25.9] (95% t-interval), which removes 3.5% of the total held-out NLL. The mean is positive in 54/54 categories, and Δ>0\Delta>0 for 69% of meshes.
  • The gain comes from yaw: the model keeps the up direction but is unsure of the yaw. Averaging over the 4 rotations about the up axis alone removes 3.3%, 95% of the octahedral gain, at 4 density calls per mesh.

Is this the same rotated-by-construction setting again? We think not, for two reasons.

  • Nothing in the evaluation is rotated. The held-out meshes are the original ShapeNet validation meshes in their canonical pose, and averaging over the 4 up-axis rotations still removes 3.3% of their NLL.
  • The rotation invariance comes from MeshXL's own training, not from us. The MeshXL paper says pretraining rotates meshes by 0/90/180/270° but does not name the axis. The audit found the axis (up) and the right subgroup (C4C_4, 95% of the octahedral gain) from the model's likelihood alone. We state openly that C4C_4 is a post-hoc analysis, added after seeing the octahedral result.

Q1: how the diagnostic informs a decision. The rule (Sec. 7) is to compute ΔH\Delta_H on held-out examples:

  • significantly positive: symmetrize over HH;
  • significantly negative: the target is canonical in that direction, so do not symmetrize;
  • positive but negligible: not worth ∣H∣|H| times the density cost.

On MeshXL all three outcomes appear in one model:

  • rotations about the up axis: 3.3% of the NLL removed, so symmetrize (4 density calls per mesh suffice);
  • left-right mirror: statistically detectable but negligible (0.07% of the NLL), so not worth doubling the cost;
  • up-down and front-back mirrors: significantly negative (−0.06% each), so symmetrizing over them would hurt.

This also answers your point that canonically posed shapes are often what one wants. On our canonical ShapeNet flows, the same test gives negative octahedral Δ\Delta (−0.58 to −1.00 nats/point, App. J.3), so those models should not be symmetrized, while MeshXL does gain. The test tells which case a given model is in. Any use of the model's likelihood (ranking or scoring shapes, a likelihood prior) inherits the yaw uncertainty. We measured held-out NLL only, not a downstream task.

Limits (App. J.5). The fine-tuned MeshXL checkpoints memorize their training split. On them every transformation raises held-out likelihood, so there the test detects overfitting, not symmetry, and we report them only as a supplement. For every mirror the median per-mesh Δ\Delta is −log⁡2-\log 2: for most meshes, the model gives the re-serialized mirror image almost no probability. We do not separate geometry from serialization effects. Sample-side certificates are only a proxy, because the sampler mostly emits sequences outside the model's canonical order.

Reviewer_nYV1OpenReview ↗

Acknowledgment

Thank you for the detailed response. I have also read the other reviews and rebuttals. I appreciate the clarifications and new experiments, but my overall assessment is that applicability is too limited (it can only act as a diagnostic, in particular on a limited model set, and only assess a source of error that is generally not the primary problem with existing models). I will therefore leave my score as is.

Thank you for reading our replies and the other reviews, and for telling us your final assessment. We understand the concern about scope. In the final version we will state plainly in the introduction which model classes the audit covers and which kind of error it measures, so that a reader can judge its applicability from the start. Thank you again for the careful review and the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.