acceptodds
Under review as a conference paper at ICLR 2027

Rethinking Diffusion Models with Symmetries through Canonicalization with Applications to Molecular Graph Generation

Abstract

Many generative tasks in chemistry and science involve distributions invariant to group symmetries (e.g., permutation and rotation). A common strategy enforces invariance and equivariance through architectural constraints such as equivariant denoisers and invariant priors. In this paper, we challenge this tradition through the alternative canonicalization perspective: first map each sample to an orbit representative with a canonical pose or order, train an unconstrained (non-equivariant) diffusion or flow model on the canonical slice, and finally recover the invariant distribution by sampling a random symmetry transform at generation time. Building on a formal quotient-space perspective, our work provides a comprehensive theory of canonical diffusion by proving: (i) the correctness, universality and superior expressivity of canonical generative models over invariant targets; (ii) canonicalization accelerates training by removing diffusion score complexity induced by group mixtures and reducing conditional variance in flow matching. We then show that aligned priors and optimal transport act complementarily with canonicalization and further improves training efficiency. We instantiate the framework for molecular graph generation under symmetries. By leveraging geometric spectra-based canonicalization and mild positional encodings, canonical diffusion significantly outperforms equivariant baselines in 3D molecule generation tasks, with similar or even less computation. Moreover, with a novel architecture Canon, CanonFlow achieves state-of-the-art performance on the challenging GEOM-DRUG dataset, and the advantage remains large in few-step generation.

Primary area:
generative models
Keywords:
Learning with symmetries, diffusion models, molecule generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_9WgmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The authors propose using a canonicalized representation for 3D molecule generation tasks. The method first projects each sample to a canonical representation and then trains diffusion (or flow) models on these canonical representations. This enables the use of non-equivariant models, which the authors claim leads to faster and more accurate training. This is backed by universality and error reduction theorems proved in the paper. The proposed methods is evaluated on QM9 and GEOM-DRUG benchmarks.

Critical strengths

The overall presentation of the paper is relatively clear, and Figure 2 makes the pipeline concrete. The central claims (i.e., theorems) appear correct, and the assumptions are clearly stated. Importantly, I was able to check technical details in the appendix when needed, including proofs, experimental details, ablation studies, and related work. The authors are upfront about the limited experimental scope of the paper (i.e., evaluation only on unconditional small molecule generation).

Critical weaknesses

(Reproducibility) While the paper includes important implementation details, I was unable to verify the code. Some baseline results seem to be taken from prior work, which appears to be the reason for the lack of standard deviations. The authors should clearly state the source of these numbers.

(Quality) The presented methods in Tables 1 and 2 are inconsistent: Table 1 does not include CanonFlow, whereas Table 2 does not include PCS results. The authors defer the entire related work section to the appendix, which makes the paper hard to place in context. To the authors' credit, they mention prior work on non-equivariant diffusion models, but do not include them in the baselines. While the authors mention in the introduction that equivariant models incur substantial architectural and computational overhead, the actual training time is similar (or longer) for the proposed method as detailed in Appendix E.1.3, which seems contradictory.

(Significance) I think the performance improvement is marginal. The core metrics (atom stability, molecule stability, and validity) are all validity-related, which may be unimportant for practical applications (since baselines already achieve over 95%, and rejection sampling can easily filter out invalid molecules). The experimental scope is currently quite limited. The paper would be significantly strengthened by additional evaluations, such as conditional generation tasks.

Questions for authors
  1. Do the authors plan to release the code and configuration files for reproducibility?
  2. Could the authors soften the phrase "similar or even less computation" in the abstract, given the training time observations reported in Appendix E.1.3?
  3. Could the authors provide the full results for CanonFlow in Table 1? If these results are already present in the appendix, please point to the specific table or section.
  4. Could the authors provide additional metrics or experiments (e.g., conditional generation or downstream tasks) that clearly demonstrate the practical advantages of the proposed method?
3 comments
AuthorsOpenReview ↗

Author Rebuttal

We appreciate the reviewer's feedback. We answer your questions below.

Weakness 1 & Question 1: Source code

Yes. We intend to release the code and configuration files upon acceptance. For now, we upload our code to this link \url{https://anonymous.4open.science/r/CanonFlow-Rebuttal-ICLR-E48F/}. All the experimental results can be fully reproduced. We indeed take some baseline results from previous papers, however all the settings are fully identical, and our reproduced results are basically the same.

Weakness 2 & Question 2 & Question 3

  • Efficiency

We will modify our phrase to avoid confusion. However, we respectfully emphasize that our reported timings support the description of similar computation. In the GEOM-DRUG comparison in Appendix E.1.3, SemlaFlow, canonicalized SemlaFlow, and CanonFlow take 60.8, 61.3, and 61.7 hours, respectively—differences of approximately 0.8% and 1.5% from the baseline. The other same-device comparison is 10 h 13 min versus 10 h 15 min, a difference of about 0.3%. These are essentially comparable training costs, consistent with retaining the base computational structure while improving generation quality.

In terms of "less compute", we aim to highlight the actual efficiency in both training and sampling. Figure 3 shows faster improvement in validation quality during training, especially with the aligned prior. Table 10 shows that CanonFlow at 50 NFE exceeds SemlaFlow at 100 NFE in both molecule stability and validity. Please note that we intentionally keep the training FLOPs almost the same as baselines, which is the standard "iso-FLOPs" comparison in modern AI papers: with controlled FLOPs and training time, our method significantly outperforms the baseline; to achieve the same performance, our method takes significantly less time than baseline. Thus, comparable fixed-budget training time and improved learning/sampling efficiency are consistent observations.

  • Performance of CanonFlow and non-equivariant models

We additionally evaluated CanonFlow, as well as a genuinely non-rotation-equivariant Transformer backbone on QM9:

MethodAtom stability ↑Mol. stability ↑Validity ↑Opt-RMSD ↓
Transformer99.8 ± 0.0%99.25 ± 0.03%98.94 ± 0.05%0.27 ± 0.00
CanonFlow99.9 ± 0.0%99.58 ± 0.05%98.32 ± 0.12%0.23 ± 0.00

CanonFlow is also slightly better than the equivariant SemlaFlow and much better than non-equivariant baseline without canonicalization. Again, the canonicalized SemlaFlow variant retains the underlying equivariant SemlaFlow architecture and adds canonical-rank conditioning, whereas CanonFlow uses the Canon architecture described in Section 4.2 which is internally non-equivariant. Hence, our method improves both equivariant and non-equivariant models, while retaining an established backbone lets us examine the canonicalization framework within an existing molecular generator.

As stated in the paper, PCS is only an optional sampling and has small effects on the performance in Table 2.

Weakness 3 & Question 4: Benchmarks and metrics

We agree that the the metrics and datasets are kind of saturated. However, these are standard and common settings in similar papers -- unfortunately, we cannot find alternative metrics or benchmarks in unconditional generation. Conditional generation is also beyond the scope of this paper, which involves more complicated design space and deserve a separate publication.

The GEOM-DRUG comparison is less saturated: SemlaFlow has 93.9% validity, versus 95.0% for canonicalized SemlaFlow and 95.9% for CanonFlow. CanonFlow also improves molecule stability from 97.3% to 98.4%. These are improvements of 2.0 and 1.1 percentage points, respectively.

A stronger result is the quality–NFE comparison in Table 10: CanonFlow at 50 NFE achieves 98.0±0.08% molecule stability and 95.4±0.09% validity, compared with 97.3±0.08% and 93.9±0.19% for SemlaFlow at 100 NFE. Against SemlaFlow at the same 50 NFE, the gains are 1.0 and 1.5 percentage points. This shows better reported quality with fewer neural evaluations, while leaving the end-to-end wall-clock advantage to be measured directly. We believe these contributions are already solid enough for a conference paper.

I thank the authors for their response. However, I am still concerned about the performance and the scope of the paper. The paper would benefit from more thorough experiments.

Thank you for the follow-up. We would like to clarify how the theory and experiments fit together again. Our theory establishes universality for invariant target distributions, characterizes the expressivity gained by modeling on a canonical slice, and explains why learning can be easier through simpler diffusion scores and reduced flow-matching target variance under group-aligned couplings. In unconditional 3D molecular generation, we show faster empirical convergence in validation quality, and show that CanonFlow achieves higher validity and molecule stability than SemlaFlow on GEOM-DRUG with fewer inference step. We welcome broader evaluation on conditional generation and downstream tasks, but we believe the theoretical and empirical contributions stand on their own. In light of these results, we hope you will consider raising your score.

Reviewer_gXN9OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a canonicalization-based framework for molecular generation. It combines canonical atom ordering, positional encodings, and a new Canon architecture with diffusion and flow matching models. The authors provide a conditional variance analysis and evaluate the approach on QM9 and GEOM-DRUG.

Critical strengths

1.The paper offers an interesting perspective on handling symmetries through canonicalization, supported by a theoretical analysis based on conditional variance decomposition.

2.The proposed approach can be integrated into SemlaFlow and improves several generation metrics on QM9 and GEOM-DRUG with limited additional computational overhead.

Critical weaknesses
  1. Insufficient empirical support for the main motivation:The paper questions traditional equivariant generative modeling but lacks direct evidence of the practical advantages of canonicalization. In particular, the best-performing CanonFlow still relies on equivariant message passing, leaving its ability to replace equivariant modeling unverified.

  2. Missing key ablations make the source of improvements unclear:The main experiments involve changes in canonicalization, OT strategies, and architecture. In Table 13, GEOM-DRUG validity improves only from 93.9% to 94.2%, with no further improvement from adding PE. A fair comparison between random ordering + PE and Fiedler ordering + canonical PE is also missing.

  3. The connection between theoretical benefits and generation performance is insufficiently established:Table 9 reports that permutation ambiguity accounts for 38.9% of the total conditional variance on GEOM-DRUG, while improvements in final generation metrics remain relatively modest. Despite the reduced training loss, the paper lacks direct evidence linking variance reduction to training efficiency and generation quality.

Questions for authors

1.What are the main practical limitations of existing equivariant generative models? How does canonicalization compare in generation quality, training efficiency, and stability?

2.How much improvement comes from canonicalization alone when the SemlaFlow architecture, OT strategy, prior, and training budget are kept unchanged?

3.Under matched model capacity and training budget, how does random atom ordering with standard PE compare with Fiedler ordering with canonical PE?

4.Since CanonFlow still uses equivariant message passing, what limitation of equivariant models does canonicalization actually address? Could the gains mainly come from additional positional information? Can the authors evaluate a genuinely non-equivariant backbone?

5.Why does the reported 39% symmetry-induced variance component lead to relatively modest generation improvements? Can the authors directly connect variance reduction to convergence speed and generation quality?

6.Why does joint permutation and rotation canonicalization not clearly outperform permutation-only canonicalization? Does this suggest limitations when handling different symmetry groups?

2 comments
AuthorsOpenReview ↗

Author Rebuttal (Part 1)

Thank you for your review. We now address your concerns below.

Weakness 1, Question 1 and Question 4 all center around the advantage of non-equivariant architectures. We now answer these questions.

Weakness 1

First, please note that the presence of equivariant message-passing layers does not imply that the complete CanonFlow denoiser remains permutation-equivariant. Canonical-rank PE assigns fixed identities to the canonical slots, so permuting the molecular state while holding these conditions fixed is no longer required to permute the output in the same way. Our framework is architecture-agnostic and permits the remaining symmetry groups to be handled by canonicalization or by retained architectural inductive biases.

Retaining the SemlaFlow geometric backbone is useful for this evaluation: it demonstrates that the canonical approach can improve an existing generator without requiring a wholesale replacement of its architecture. Using the same backbone also enables fair comparisons under same architecture and matched training and inference FLOPs, which is a common and standardized setting.

As a matter of fact, our models show significant improvements both with and without changing the architecture: canonicalized SemlaFlow raises GEOM-DRUG molecule stability from 97.3% to 98.1% and validity from 93.9% to 95.0% without changing the architecture; the novel Canon architecture further raises them to 98.4% and 95.9%.

Question 1

The limitation targeted by canonicalization is the ambiguity between symmetry-equivalent targets during denoising, together with the architectural constraint of treating equivalent atom permutations identically. Canonical ordering selects a consistent representative, and rank conditioning gives the network access to its atom positions. Theorem 3.5 formalizes the resulting change in regression ambiguity under the specified lift. This design allows the model to exploit canonical atom identities while retaining useful rotational equivariance. It offers a practical alternative to imposing both symmetry constraints throughout the denoiser.

Canonicalization improves baselines in multiple aspects. Quality and stability: on GEOM-DRUG, Table 2 improves validity from 93.9% for SemlaFlow to 95.0% for canonicalized SemlaFlow and 95.9% for CanonFlow. Our method also improves both atom and molecule stability metrics. Efficiency: Figure 3 shows faster validation-quality improvement (faster convergence), and Table 10 shows stronger molecule stability and validity with CanonFlow at 50 NFE than with SemlaFlow at 100 NFE (strong performance in few-step generation). The training times in Appendix E.1.3 remain essentially comparable: 60.8, 61.3, and 61.7 hours.

Question 4

The benefit is faster empirical convergence from learning in a consistent canonical representation. Figure 3 shows earlier improvement in QM9 validation validity and Opt-RMSD, particularly with the aligned prior. Table 9 reports lower EMA coordinate losses: 0.165 to 0.104 on QM9 and 0.211 to 0.095 on GEOM-DRUG. The generation result is also stronger at fewer steps: CanonFlow at 50 NFE achieves 98.0% molecule stability and 95.4% validity, exceeding SemlaFlow at 100 NFE (97.3% and 93.9%).

This benefit is compatible with retaining equivariant message-passing operations. As explained in W1, the fixed canonical conditions remove the requirement that the complete denoiser remain permutation-equivariant in the molecular state alone. The network can therefore exploit the chosen atom ordering instead of averaging over interchangeable atom slots.

The positional information is precisely how this canonical representation is communicated to the network. Canonicalization defines what each position means, and PE makes those positions distinguishable to the Semla-based backbone. As explained in our paper and the common response, PE is an essential part of canonicalization, which itself adds almost no additional parameters and FLOPs. Calling this “additional positional information” is therefore actually inaccurate. The Canon architecture further develops that mechanism through an explicit canonical-information stream.

We also evaluated a genuinely non-rotation-equivariant Transformer backbone and also the non-equivariant CanonFlow on QM9. Using 100 sampling steps, its results are:

MethodAtom stability ↑Mol. stability ↑Validity ↑Opt-RMSD ↓
Transformer99.8 ± 0.0%99.25 ± 0.03%98.94 ± 0.05%0.27 ± 0.00
CanonFlow99.9 ± 0.0%99.58 ± 0.05%98.32 ± 0.12%0.23 ± 0.00

Results are mean ± standard deviation over three sampling replicates of 10,000 molecules each, using one model trained for 300 epochs. Therefore, a genuinely non-equivariant backbone underperforms our non-equivariant CanonFlow with canonicalization, which is a strong evidence of our method's effectiveness.

AuthorsOpenReview ↗

Author Rebuttal (Part 2)

Weakness 2 & Question 3: Set up and comparison of ordering

We politely disagree with this opinion. Please note that, as explained in the paper and our general response, positional encoding is the essential mechanism to break equivariance in canonical models and let the backbone use the canonical positions, which has minimal increase of model parameters and FLOPs but maximal performance improvement. The main configurations are evaluated separately in Table 2: validity is 93.9% for SemlaFlow, 95.0% for canonicalized SemlaFlow, and 95.9% for CanonFlow. The canonicalized SemlaFlow result demonstrates a benefit on the base backbone, independently of adopting the new Canon architecture.

For Table 13, as we already stated in the caption and the texts around it, the red PE rows deliberately use a combination that fails to preserve the condition–noise alignment required at sampling. Adding PE without preserving that alignment is not the same intervention as introducing the intended canonical model. The absence of an additional gain in those rows, and their degradation at fewer steps, are negative ablations, supporting our theoretical claim that OT remains beneficial only when the induced coupling is compatible with the conditioning joint law. Therefore, we have to change the OT schedule in SemlaFlow baselines.

To answer Question 3, all our experiments already control capacity and training budget to be matched. It does not make sense to train a model with random ordering without canonicalization: the best case is to ignore the noisy ordering condition and matches the naive equivariant baseline. However, we still train this model and present the results as follows, which shows that Fiedler ordering is the best among all.

RecipeAtom stabilityMol. stabilityValidityOpt-RMSD (Å)
Atomic-number ordering99.9 ± 0.098.48 ± 0.0696.67 ± 0.300.26 ± 0.00
Random Ordering99.9 ± 0.098.97 ± 0.0898.49 ± 0.100.33 ± 0.00
Fiedler ordering99.9 ± 0.099.64 ± 0.0499.58 ± 0.060.17 ± 0.00

Weakness 3 and Question 5: Variance reduction and downstream performance

The 38.9% value quantifies a component of squared regression-target variance, which is a direct prove of Theorem 3.5. A lower variance suggests an easier denoising task for the model to learn: Table 9 reports corresponding reductions in achieved EMA coordinate loss, from 0.165 to 0.104 on QM9 and from 0.211 to 0.095 on GEOM-DRUG. Correspondingly, the optimization for canonical model is easier, and trajectories in Figure 3 show that the canonical models converge much faster than baseline, reaching better QM9 validity and Opt-RMSD earlier in training. Table 10 also shows that the resulting CanonFlow model exceeds SemlaFlow's 100-NFE molecule stability and validity using only 50 NFE, proving the advantage in few-step generation.

Please note that validity and molecule stability are discrete chemical acceptance metrics applied to the generated output, so their percentage changes need not track that variance reduction linearly. This is the very common non-linear evaluation metrics issue, for example, the downstream task performance metrics (e.g., accuracy) do not have a linear correspondence w.r.t. training loss or perplexity in LLMs. Near saturation, even a small absolute increase can correspond to a substantial decrease in failures: GEOM-DRUG validity increases from 93.9% to 95.9%, reducing the invalid fraction from 6.1% to 4.1%, which we believe is already a huge improvement.

Question 2: Controlled variables

As we explained above, Canon. SemlaFlow reported in our table already does not change architecture, training budget, and prior (whose change is completely optional) -- OT strategy change is also essential to keep the conditional joint law, and we empirically find it largely improves the performance. Therefore, we believe our evaluations are already pretty fair.

Question 6: Effect of SO(3)SO(3)

As has already been explained in our paper, Table 9 shows that permutation ambiguity accounts for 37.3%37.3\% of the ambient variance on QM9 and 38.9%38.9\% on GEOM-DRUG. Including rotations increases the ambiguity component by 0.520.52 and 0.340.34, respectively, corresponding to a further 4.5%4.5\% and 2.5%2.5\% of the ambient variance. Permutation ambiguity therefore accounts for most of the measured symmetry contribution in these settings. The smaller additional rotational component motivates evaluating SNS_N and SN×SO(3)S_N\times SO(3) separately, and we also attribute the slightly lower performance of SN×SO(3)S_N\times SO(3) to its instability when determining the axis. Therefore, we believe SNS_N alone is sufficient and could largely improve empirical performance.

Reviewer_bMgGOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper studies canonicalization for molecular diffusion and flow models. It uses spectral atom ordering, optional rotation alignment, and rank-conditioned denoising. It analyzes invariant sampling and conditional variance, and reports improvements on QM9 and GEOM-DRUG.

Critical strengths
  • The paper highlights the learning difficulties caused by symmetry-related ambiguity.

  • The full pipeline improves reported molecule stability and validity. The training curves and generation results at fewer sampling steps provide useful empirical evidence.

Critical weaknesses

Weaknesses

  1. The framework gives up continuity without resolving the resulting instability. The difficult part is not choosing an orientation or atom order. It is making that choice stable under small changes to the molecule. Once discontinuous choices are accepted, a PCA-based frame is already an alternative for rotational alignment on generic inputs. Simpler atom-ordering rules are also relevant. The paper does not establish what its spectral construction provides beyond another discontinuous convention. Rank noise and prediction may help the network tolerate that convention, but they do not make the canonicalizer stable.

  2. The theory does not establish the claimed learning advantages. Haar randomization restores invariance regardless of the canonicalizer. This guarantee therefore provides no justification for the proposed construction. Lower conditional variance does not prove faster optimization. Likewise, the expressivity separation uses discontinuous frame functions, not the score or velocity fields needed for generation. These results do not establish that the proposed representation makes the actual learning problem easier.

  3. The experiments do not isolate the contribution of spectral canonicalization. Table 14 changes both ordering and other generation components. The gains could therefore come from conditioning, prior changes, OT annealing, or sampling adjustments. The stability evidence is also incomplete: Table 8 excludes changed-order cases from its RMSD statistics, although small perturbations change the ordering in roughly 15% of cases. The paper needs controlled comparisons with simpler constructions and an analysis of these jumps. Otherwise, the evidence supports the combined training recipe, not the claimed benefit of its canonicalizer.

Questions for authors

I have explored this same direction before. The main obstacle I encountered was finding a consistent canonicalizer that remains stable near symmetric configurations. For the relevant point-cloud actions, globally continuous canonicalization is ruled out by Dym et al. (ICML 2024). Allowing measurable choices makes the construction possible, but leaves the stability problem unresolved.

  • How are repeated eigenvalues, tied ranks, and ambiguous frames handled? How large are the output jumps when the ordering changes in Table 8? These cases should be included when assessing robustness, especially for projected sampling.

  • What establishes that the resulting score or velocity field is easier to learn? The conditional-variance decomposition does not itself prove faster optimization. How do the authors separate a benefit of canonicalization from the effects of rank regularization and the other training changes?

  • Can spectral ordering be compared with simpler ordering rules while keeping the backbone, prior, coupling, and sampler fixed? PCA-based frames would also be a useful comparison for rotational alignment. Table 14 changes multiple components, so it does not isolate the proposed canonicalizer’s contribution.

5 comments
AuthorsOpenReview ↗

Author Rebuttal (Part 1)

We sincerely appreciate the reviewer's feedback. We address your concerns as below.

Weakness 1 & Question 1: Instability of canonicalization

While there are chances for tied eigenvalues theoretically, we now show that our canonicalization is empirically extremely stable. The theoretical instability issue has almost no effect on practical molecules or on our model performance.

  • Perfect symmetries rarely exist for practical 3D molecules.

We first want to clarify that in realistic QM9/GEOM conformers, the complete 3D geometry usually breaks graph-level substructure symmetries. In Table 7 of the Appendix, we already report measurements of spectral near-degeneracy.

Let γ=∣λ3−λ2∣/max⁡(∣λ2∣,∣λ3∣)\gamma=|\lambda_3-\lambda_2|/\max(|\lambda_2|,|\lambda_3|). On the full test splits, median γ\gamma is 0.224/0.469 for QM9/GEOM-drugs, while γ<10−3\gamma<10^{-3} occurs in only 5/12,929 (0.039%) and 1/10,000 (0.010%), respectively -- note that γ≈10−3\gamma \approx10^{-3} is still a fully recognizable eigengap value and does not imply degeneracy. This audit clearly shows that exact eigen-ties and perfect symmetries almost do not exist for practical 3D molecules.

  • Explicitly handling eigen-ties.

We explicitly handle degenerate cases as follows. The current implementation handles small eigengaps by expanding the retained Fiedler subspace and applying Maximal Axis Projection (MAP) through its projector UU⊤UU^\top, which removes dependence on the eigensolver’s arbitrary signs and basis whenever the projection summaries determine a canonical basis.

  • Canonicalization is consistent and stable.

In addition to Table 7 and Table 8 where we have already provided evidence of numerical stability, we further directly evaluate whether the same molecular geometry yields a consistent canonical representation under changes in orientation and atom indexing. For each test molecule, we independently apply random rotations (SO(3)SO(3)), atom permutations (SNS_N), and their combination. The original and transformed inputs are then canonicalized independently, including both spectral ordering and SO(3) frame construction. These results demonstrate high empirical consistency and numerical stability under rotations and atom relabeling across the evaluated benchmark conformers.

dataset and actionrank equivariantcanonical pose consistentmedian relative residual
QM9 (800), SO(3)SO(3)99.31%99.31%1.4×10−71.4\times10^{-7}
QM9 (800), SNS_N99.06%99.06%0
QM9 (800), SN×SO(3)S_N\times SO(3)99.12%99.12%1.4×10−71.4\times10^{-7}
GEOM (400), SO(3)SO(3)100.00%100.00%1.2×10−71.2\times10^{-7}
GEOM (400), SNS_N100.00%100.00%0
GEOM (400), SN×SO(3)S_N\times SO(3)100.00%100.00%1.2×10−71.2\times10^{-7}
  • Training and sampling techniques for robustness.

We further proposed multiple training and sampling techniques in the paper to make the model extremely robust to rank conditions. PCS, relaxed ranks, and self-estimation improves robustness of the ranks and mitigate downstream train-test rank mismatch. The output is rather stable under small perturbations of rank conditions. Our strong empirical results prove that all the spectral canonicalization is actually stable and the mild theoretical dis-continuous issue could be addressed by empirical learning methods.

AuthorsOpenReview ↗

Author Rebuttal (Part 2)

Weakness 2 & Question 2: Theoretical and practical benefit of the reduced variance

We have actually provided both theoretical and empirical results in the paper to establish the connection.

On the theory side, Theorem 3.5 decomposes the minimum ambient squared regression risk into the canonical risk and a nonnegative symmetry-ambiguity term under the group-aligned lift. Proposition C.12 shows that this construction includes standard independent isotropic Gaussian-noise training. The result identifies uncertainty arising from equivalent symmetry-related targets and explains how fixing the representation removes that component from the regression problem. Proposition C.4 characterizes when measurable gauge lifting strictly enlarges the class of frame-valued equivariant mappings beyond continuous equivariant constructions, under its non-extendability assumption. This establishes a representation-level distinction.

We also provide strong empirical evidences below. Table 9 shows a remarkable decrease in coordinate-loss during training: 0.165 to 0.104 on QM9 and 0.211 to 0.095 on GEOM-DRUG, suggesting that canonicalization enables a much easier denoising task and empirically makes the learning much easier. Correspondingly, significantly faster convergence is also supported empirically by Figure 3, which shows the actual learning dynamics. The theorem explains the regression mechanism, and the curves show the actual learning behavior. We believe these results already clearly link the decreased variance to practical learning advantage. If you are looking for a universal optimization-rate theorem, we believe it is not needed (nor possible in this or similar papers) to fully theoretically characterize the observed convergence improvement.

For the question about rank regularization, there are two different uses of rank. As noted in our general reponse, fixed rank PE is the architectural interface to the canonical representation: it allows the denoiser to use the atom positions defined by canonical ordering. Rank jitter and the auxiliary rank-prediction objective instead address robustness to uncertainty in that condition. This distinction explains why rank conditioning belongs to the definition of our canonical model, while regularization techniques are additional training choices.

Weakness 3:

We already keep backbone, prior, coupling and sampler fixed in Table 14. The only difference is the canonicalization method, so we believe the comparison is fair and strong enough to show the difference. Moreover, our paper aims to provide a general framework, so the concrete canonicalization method is agnostic to our contribution and can be fully compatible within our framework. Therefore, we believe that your claim "the evidence supports the combined training recipe" is correct and does not diminish the contribution of this paper at all -- it supports the universality and effectiveness of our contribution.

Thank you for the clarification. The new tests check rotations and relabelings of the same molecule. My concern is what happens when the geometry changes slightly. Table 8 excludes changed-order cases from its RMSD statistics. Those are precisely the cases that need further analysis.

A simple stress test would be to generate 50 points from N(0,I3)\mathcal{N}(0,I_3), add independent noise from N(0,0.03I3)\mathcal{N}(0,0.03I_3), and recompute the canonical frame. Report the angular change of the frame and the frequency of ordering changes. Repeat this across random point clouds and smaller noise levels.

Your statement that Table 14 changes only the ordering also conflicts with its caption, which says other generation components differ. Please clarify which description is correct.

I acknowledge the improvements in Figure 3. However, lower training loss can simply reflect a lower irreducible error floor. It does not establish that the model learns faster because of the proposed variance reduction.

Reviewer_zvXEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the generation of three-dimensional molecular graphs under permutation and rotation symmetry: first, the molecule is mapped to a canonical order or pose, and a diffusion or flow model is trained on this representation. After sampling, random symmetry transformations are applied. The theoretical part provides the decomposition of invariant distributions and the conditional variance decomposition under specific couplings; experiments are conducted on QM9 and GEOM-DRUG, using the normalized version of SemlaFlow and the new architecture CanonFlow to verify stability, effectiveness, and the performance of few-step sampling. The following judgments are based on the main text and appendices of the paper.

Critical strengths
  1. Meaningful results at matched sampling budgets. On GEOM-DRUG at 100 NFE, CanonFlow reports 98.4% molecule stability and 95.9% validity, versus 97.3% and 93.9% for SemlaFlow. An advantage remains at 50 NFE (Tables 2 and 10). This makes the approach practically promising.

  2. A useful theoretical framework with stated limits. The variance decomposition identifies a symmetry-ambiguity term under a specified coupling, while the paper distinguishes this term from the difficulty addressed by OT.

Critical weaknesses
  1. The experiments do not clearly isolate the effect of canonicalization. The main comparisons also change positional encoding, the OT schedule, or the model architecture. In Table 13, adding positional encoding lowers performance at 20 and 50 NFE under the reported OT setting. As a result, it is hard to tell how much of the main gain comes from canonicalization itself.

  2. Some evidence needs clearer reporting. Table 9 does not explain how its variance values were estimated or report their uncertainty. The QM9 Opt-RMSD average includes only samples for which geometry optimization succeeds, but the paper does not give the success rates. These gaps make the related claims harder to assess.

Questions for authors
  1. Could the authors report a matched SemlaFlow comparison on GEOM-DRUG with and without canonicalization, using the same backbone, prior, OT schedule, and sampling settings? A separate comparison with and without rank positional encoding would help clarify which part of the gain comes from canonical ordering.
  2. What estimator, coupling, and sample size were used for the variance results in Table 9? How uncertain are the reported values?
  3. What are the geometry-optimization success rates for each method? Results broken down by molecule size, especially for larger GEOM-DRUG molecules, would also help assess how broadly the gains hold.
1 comment
AuthorsOpenReview ↗

Author Rebuttal

Thanks for the reviewer's feedback. We now address your concerns as follows.

Weakness 1 & Question 1: Experiment setup

We politely disagree with this claim. Please note that, as explained in our general response, positional encoding is essential to break equivariance in canonical models, whihc has minimal increase of model parameters and FLOPs but maximal performance improvement. We do not change the model architecture for Canon. SemlaFlow. Moreover, the OT schedule change is essential: as explained in Section 3.2, a proper aligned algorithm version must satisfy the conditional start-time joint law π~1tr(C,Z~1)=π~1inf(C,Z~1)\tilde \pi_1^{tr}(C,\tilde Z_1) = \tilde \pi_1^{inf}(C,\tilde Z_1). As we already stated in the caption of Table 13, the red rows report a deliberately non-aligned variant that combines PE with OT without preserving the conditional start-time joint law as a negative ablation, whose weaker few-step performance aims to prove that a naive combination of PE, aligned prior, and OT does not explicitly preserve this conditional joint law: OT may improve endpoint matching, but it can simultaneously destroy the alignment between CC and Z~1\tilde Z_1 that the sampler relies on. The result supports our theoretical claim that OT remains beneficial only when the induced coupling is compatible with the conditioning joint law. Therefore, we have to change the OT schedule in SemlaFlow baselines -- remarkably, the performance of SemlaFlow will drastically drop when removing OT, while our method can retain strong performance.

Hence to answer question 1: we already reported matched comparison. SemlaFlow without canonicalization and without PE is just the baseline. Canon. SemlaFlow is the same model with canonicalization, which must be paired with PE as we have explained for several times. We believe every setting except those necessary change in our comparison has been fair enough and cannot be improved. The main results in our original tables are already aligned: SemlaFlow (w/o canonicalization) and Canon. SemlaFlow (w/ canonicalization) share every setting except the necessary change in prior or OT schedule to satisfy the conditional joint law π~1tr(C,Z~1)=π~1inf(C,Z~1)\tilde \pi_1^{tr}(C,\tilde Z_1) = \tilde \pi_1^{inf}(C,\tilde Z_1). The backbones are completely the same; the PCS sampler is optional and only adds marginal improvements, that being said, a normal sampler for Canon. SemlaFlow has almost the same strong performance, outperforming SemlaFlow w/o canonicalization.

The aligned prior also has a representation-specific meaning: it is estimated conditional on canonical rank. An atom index in the uncanonicalized baseline does not carry that same meaning. Applying the same rank-dependent statistics to arbitrary baseline indices would not preserve the alignment that defines this prior. Thus, comparing the complete canonical method with its baseline should be distinguished from a separate experiment designed to isolate ordering while using a common unstructured prior.

Weakness 2 & Question 2: Variance estimator in Table 9

The implementation uses a direct posterior Monte Carlo estimator of the coordinate conditional-variance decomposition, analytically marginalizing the continuous zero-center-of-mass Gaussian prior. It uses an independent data–prior product coupling with a shared group-aligned lift, without optimal transport. The documented primary configurations use up to 3,000 fixed-size molecules as a sampling pool, 256 data components and 32 group samples per replicate. Each configuration evaluates 8,192 queries at 21 time points, weighted by Beta(2,1), with three replicates. Hence the results are already an Monte Carlo average of a large amount of estimations.

The calculated variance is extremely certain and stable. Every single estimation is a Monte Carlo estimation of large samples. We further calculate standard deviation errors over 10 independent estimations: the stds for ambient, within-slice and symmetry ambiguity are 0.019, 0.009, 0.010 for SNS_N on QM9, and 0.012, 0.024, 0.022 for SNS_N on Geom-Drug, respectively. The stds for these estimations are very small, which proves that our estimation in Table 9 is very accurate and certain.

Weakness 2 & Question 3: Optimization success rate

The success rate for optimization on QM9 is 99.37$\pm$0.07 for SemlaFlow, and 99.42$\pm$0.12 for Canon. SemlaFlow. While our method is slightly better, and the main message here is that the success rate is universally high for all models, so it does not really affect the conclusion in Opt-RMSD, where our model gives a huge advantage over baseline. We do not find significant difference in success rate with respect to the size of molecules.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response to All Reviewers and Area Chair

We sincerely thank all reviewers for their thorough reading, thoughtful critiques, and constructive suggestions. We have carefully addressed each reviewer's comments individually, conducted all requested additional experiments and analyses, released our anonymous codebase (https://anonymous.4open.science/r/CanonFlow-Rebuttal-ICLR-E48F/), and updated the revised manuscript accordingly. Below, we briefly summarize the core contributions of our work and the main points clarified during the rebuttal period.

Summary of Core Contributions

  1. Theoretical Framework for Canonical Diffusion/Flow Matching: We establish a formal quotient-space theory showing that (i) modeling on a canonical slice followed by Haar randomization is sufficient and universal for any invariant target distribution, while symmetry breaking expands the effective hypothesis class for non-equivariant denoisers (Section 3.1, Proposition 3.3, Proposition C.4, Theorem C.6); and (ii) canonicalization simplifies training by eliminating diffusion score mixture complexity and removing the symmetry-ambiguity component of conditional flow-matching variance under group-aligned lifting (Section 3.2, Theorem 3.5, Proposition C.12). We further formalize how aligned priors and optimal transport (OT) act complementarily on the remaining within-slice variance, along with the start-time joint-law condition π~1tr(C,Z~1)=π~1inf(C,Z~1)\tilde{\pi}_1^{\mathrm{tr}}(C, \tilde{Z}_1) = \tilde{\pi}_1^{\mathrm{inf}}(C, \tilde{Z}_1) required for training–sampling consistency.
  2. Practical Pipeline & Canon Architecture: We instantiate this framework for 3D molecular generation under SN×SO(3)S_N \times SO(3) using geometric spectral canonicalization (Fiedler ordering and optional rank-anchored frames), canonical-rank positional encoding (PE), aligned priors, OT annealing, and robustness techniques (rank jitter, auxiliary rank prediction, and PCS). We also introduce Canon (CanonFlow), a three-stream architecture that explicitly refines canonical rank representations alongside coordinates and atom features.
  3. Strong Empirical Performance & Efficiency: Under matched backbones and iso-FLOPs/comparable wall-clock training budgets (~0.3%–1.5% overhead), canonicalization consistently improves both generation quality and efficiency:
    • QM9: Canon. SemlaFlow improves molecule stability (99.56% →\to 99.64%), validity (99.31% →\to 99.58%), and reduces Opt-RMSD by 26% (0.23 Å →\to 0.17 Å).
    • GEOM-DRUG: Without changing the backbone, Canon. SemlaFlow improves molecule stability (97.3% →\to 98.1%) and validity (93.9% →\to 95.0%). CanonFlow further achieves state-of-the-art 98.4% molecule stability (+1.1%) and 95.9% validity (+2.0%, reducing invalid molecules from 6.1% to 4.1%).
    • Training & Few-Step Efficiency: Canonicalization achieves higher QM9 validation validity in 10k steps than the baseline across 130k steps (~13$\times$ faster convergence, Figure 3), and CanonFlow at 50 NFE (98.0% Mol. Stab., 95.4% Valid.) outperforms SemlaFlow at 100 NFE (97.3%, 93.9%).

Summary of Key Points Addressed in the Rebuttal

  1. Isolating Canonicalization, Role of PE, and OT Compatibility (Reviewers zvXE, bMgG, gXN9):

    • We clarified that canonical-rank PE is the minimal, essential interface to break permutation equivariance and expose canonical ordering to the denoiser (with negligible parameter/FLOP overhead), while Canon. SemlaFlow keeps the exact same backbone as SemlaFlow to isolate the gain of canonicalization prior to introducing the Canon architecture.
    • We clarified why OT must be adjusted/annealed to preserve the conditional start-time joint law π~1tr(C,Z~1)=π~1inf(C,Z~1)\tilde{\pi}_1^{\mathrm{tr}}(C, \tilde{Z}_1) = \tilde{\pi}_1^{\mathrm{inf}}(C, \tilde{Z}_1), and why the red rows in Table 13 serve as a deliberate negative ablation validating this theoretical condition.
    • We clarified the controlled ordering ablations in Table 14 (and updated its caption in the revision), showing that geometric Fiedler ordering outperforms both atomic-number ordering and random ordering under matched capacity and training budgets.
  2. Canonicalization Stability & Perturbation Stress Tests (Reviewer bMgG):

    • We verified near-exact equivariance/consistency under random SO(3)SO(3), SNS_N, and SN×SO(3)S_N \times SO(3) transformations (99.06%–99.31% on QM9; 100.0% on GEOM-DRUG; median relative residuals ∼10−7\sim 10^{-7}).
    • In our follow-up response, we additionally ran the requested changed-order conformer analysis (covering every excluded case from Table 8) and a 1,000-trial 50-point Gaussian cloud stress test across noise scales τ∈[10−8,0.03]\tau \in [10^{-8}, 0.03]. On GEOM-DRUG at σ=10−3 A˚\sigma = 10^{-3}\text{ Å}, 99.875% of all conformers (799/800) and 99.12% of changed-order cases (113/114) retain frame changes below 5∘5^\circ, with 102/114 ordering changes being single adjacent atom swaps.
  3. Non-Equivariant Backbones & CanonFlow Evaluation on QM9 (Reviewers gXN9, 9Wgm):

    • We clarified that canonical-rank conditioning breaks permutation equivariance at the model level even on a Semla trunk, and CanonFlow is internally non-permutation-equivariant.
    • We added new QM9 evaluations for CanonFlow (99.58% Mol. Stab., 98.32% Valid., 0.23 Å Opt-RMSD) and a genuinely non-rotation-equivariant Transformer (99.25% Mol. Stab., 98.94% Valid., 0.27 Å Opt-RMSD), demonstrating the generality and advantage of our approach across architectures.
  4. Variance Decomposition, Convergence, and Reporting Details (Reviewers zvXE, bMgG, gXN9):

    • We provided full details of the direct posterior Monte Carlo estimator in Table 9 and confirmed its low uncertainty across 10 independent runs (standard deviations ≤0.024\le 0.024). We also explained why permutation ambiguity (SNS_N: 37.3%/38.9% of ambient variance) dominates over rotation (SO(3)SO(3): +4.5%/+2.5%).
    • We reported QM9 geometry-optimization success rates (99.42 ±\pm 0.12% for Canon. SemlaFlow vs. 99.37 ±\pm 0.07% for SemlaFlow), confirming that the 26% Opt-RMSD improvement is not affected by optimization failures.
    • We emphasized that Figure 3 tracks downstream validation validity (not only training loss), directly evidencing ~13$\times$ faster convergence alongside lower coordinate regression loss (0.165 →\to 0.104 on QM9; 0.211 →\to 0.095 on GEOM-DRUG).
  5. Reproducibility & Manuscript Revisions (Reviewer 9Wgm, all reviewers):

    • We uploaded the full anonymous codebase and configuration files for reproducibility, softened/clarified the computation phrasing to reflect iso-FLOPs training with improved convergence and few-step sampling efficiency, and revised the manuscript tables, captions, and diagnostics for clarity.

We have made every effort to address all theoretical, experimental, and presentation concerns thoroughly. As the discussion period is drawing to a close, we kindly invite the reviewers to review our individual responses and the revised manuscript. We would greatly appreciate any feedback, and we respectfully hope you will consider updating your scores if your concerns have been resolved.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.