AnchorSet: Structure-Anchored Set-Valued Canonicalization for Non-Equivariant 3D Molecular Generation

Equivariant diffusion models have achieved strong performance in three-dimensional molecular generation by encoding Euclidean symmetries directly into the denoising architecture. Their specialized geometric operations, however, can increase computational cost and constrain the scalability and flexibility of the generative backbone. Existing canonicalization can support non-equivariant generation, but it can become unstable when molecular geometry does not define a unique orientation, replacing rotational redundancy with an arbitrary or perturbation-sensitive frame. We introduce AnchorSet, a structure-anchored, set-valued canonicalization framework that combines chemically informed pose estimation with explicit preservation of unresolved frame ambiguity. AnchorSet estimates molecular orientation from structure-conditioned anchors that combine a chemical-scaffold prior with learnable atom-type weights, biasing pose estimation toward stable molecular cores rather than flexible peripheral atoms. When the geometry does not identify a unique orientation, it preserves a compact set of symmetry-equivalent frames related by proper signed permutations, enabling a standard non-equivariant generator to operate in canonical coordinates. Across QM9 and GEOM-Drugs, AnchorSet outperforms prior non-equivariant approaches and matches or surpasses equivariant diffusion models in sample quality. Its stability under global rotations and non-anchor perturbations demonstrates that structure-aware set-valued canonicalization is a practical, scalable alternative to built-in equivariance.