acceptodds
Under review as a conference paper at ICLR 2027

Uni²: Unified Training of Latent Diffusion over Unified Modalities for De Novo Binder Design

Abstract

Structure-based binder design asks a model to generate a peptide, antibody, or small molecule that binds a given target. One family of approaches handles this task with latent diffusion: an autoencoder compresses full-atom complexes into a learned latent space, and a diffusion model generates there. That latent space is usually obtained in two steps: an autoencoder is fit to reconstruct complexes, then frozen, and a diffusion model is trained on top of it. The autoencoder therefore meets the target only as data to be compressed, never as a constraint on what the generator will later have to produce, and a space that reconstructs cleanly can still hand the diffusion model a geometry that is difficult to occupy. We formulate autoencoding and latent diffusion as a unified optimization problem, jointly learning the latent representation and its generative dynamics. By allowing diffusion gradients to flow into the autoencoder, the diffusion objective directly shapes the latent manifold, aligning representation learning with the generative task while retaining the structural information required for reconstruction. On peptide, antibody, and small molecule benchmarks, our training strategy improves structural accuracy, physical plausibility, and binding energy together. These gains persist when the three modalities are trained together in one model. The change is visible in the latent space itself. Measured with a standardized sliced Wasserstein distance, the gap between posterior latents and generated latents narrows once training is unified. Finally, an ablation that treats and separately measures what unified training contributes through each channel: accounts for the gain in geometry, accounts for the gain in chemistry and interaction, and only when both receive gradients does the model become Pareto-optimal.

Primary area:
generative models
Keywords:
Latent Diffusion, Unified Training, De Novo Binder Design

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_PnumOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3(+1)
Presentation
3
Soundness
2
Read review
Paper summary

The main contribution of Uni² is a joint VAE–diffusion training strategy for biomolecular binder design on the UniMoMo backbone. This strategy replaces the frozen latent interface of two-stage training, allowing diffusion gradients to update the encoder alongside reconstruction objectives. The reported experiments on peptides, antibody CDRs, and pocket-conditioned small molecules support improvements in several primary metrics or aggregate scores.

Critical strengths
  1. The method addresses the mismatch between reconstruction-oriented representations and latent generative modeling through a simple joint objective applicable across multiple all-atom binder-design tasks.

  2. The method incorporates generative feedback into representation learning, thereby allowing latent representations to adapt to the generative objective. Consistent with this adaptation, latent-distribution analyses indicate improved alignment between generated latents and the encoder’s aggregated posterior.

Critical weaknesses
  1. Limited coverage of recent baselines, particularly for antibody design. Beyond UniMoMo, the antibody comparisons in Table 2 rely mainly on methods from 2022–2024, leaving competitiveness against more recent antibody/CDR design approaches unclear. Comparisons should prioritize recent methods that support the same task. Selected broader approaches, such as BoltzGen [1], PXDesign [2], RFdiffusion3 [3], and Proteina-Complexa [4], would also be informative where their tasks and conditioning are compatible.

  2. The mechanism underlying the training gains needs further clarification. The shared-backbone comparison and gradient-routing ablations provide useful evidence, but improvements in the fully detached variant indicate that the shared training schedule also contributes to the gains. Meanwhile, Figure 2 and Table 6 support improved latent alignment but do not independently establish improved representation learnability. A clearer explanation connecting the training schedule, diffusion feedback, and latent-space changes to the observed generation gains would strengthen the mechanistic interpretation.

  3. The connection between latent alignment and decoded structural quality remains undercharacterized. Reconstruction fidelity, component-wise validity failures, and sensitivity to latent perturbations or iterative decoding are insufficiently analyzed.Table 7 reports lower PB-valid fractions for antibody and small-molecule reconstructions, leaving the relationship between improved latent alignment and decoded structural validity unresolved.

[1] Hannes Stark, Felix Faltings, MinGyu Choi, et al. BoltzGen: Toward Universal Binder Design. bioRxiv preprint, 2025.

[2] Protenix Team, Milong Ren, Jinyuan Sun, et al. PXDesign: Fast, Modular, and Accurate De Novo Design of Protein Binders. bioRxiv preprint, 2025.

[3] Jasper Butcher, Rohith Krishna, Raktim Mitra, et al. De novo Design of All-atom Biomolecular Interactions with RFdiffusion3. bioRxiv preprint, 2025.

[4] Kieran Didi, Zuobai Zhang, Guoqing Zhou, et al. Scaling Atomistic Protein Binder Design with Generative Pretraining and Test-Time Compute. In International Conference on Learning Representations (ICLR), 2026.

Questions for authors
  1. What methodological advances distinguish this approach from existing end-to-end VAE–flow/score models, particularly regarding gradient routing and training stability?

  2. How competitive is the method against recent task-compatible baselines under matched data, conditioning, and inference budgets?

  3. How do the latent-distribution changes in Figure 2 and Table 6 translate into reconstruction fidelity, decoded geometry, and physical validity, and how sensitive are these effects to the decoder?

6 comments

We thank the reviewer for the thoughtful comments. Blow is our responses point by point. All new text and experiments are included in the revised PDF and marked in blue.

W1. Comparison with recent antibody and binder design models

We added BoltzGen [1], PXDesign [2], RFdiffusion3 [3] and Proteina-Complexa [4] on antibody design, using the same test complexes and the same number of candidates n as in Table 2. Results are below (AAR and IMP in %; clash in %; RMSD in Å; L-RMSD is the binder RMSD).

Modeln=1 AARn=1 L-RMSDn=1 IMPn=10 AARn=10 L-RMSDn=10 IMPn=100 AARn=100 L-RMSDn=100 IMPClash_inClash_outJSD_bbJSD_sc
PXDesign [2]13.133.703.3925.412.931.0234.352.501.7114.116.810.2590.246
BoltzGen [1]33.310.9447.4643.460.7050.0052.710.4648.930.340.120.2090.312
RFdiffusion3 [3]14.271.8211.8630.820.7416.9543.750.5217.060.150.120.1800.300
Proteina-Complexa [4]25.982.0215.2543.291.3419.8352.620.7719.732.000.320.1930.297
UniMoMo (all)21.442.5213.3342.051.4441.6752.341.0465.000.180.030.2240.221
Uni² (all)19.321.9523.3340.111.1843.3350.320.8568.330.120.030.2160.217

Where Uni² stands. Uni² is one model for peptides, antibodies and small molecules, and it is competitive with these recent general-purpose binder design models. At n=100 it has the highest IMP (68.33% versus 48.93% for the best baseline), the lowest clash rates (0.12% and 0.03%), and the best side-chain JSD (0.217). Its AAR (50.32%) is close to BoltzGen and Proteina-Complexa (52.71% and 52.62%) and higher than RFdiffusion3 and PXDesign. Its L-RMSD (0.85) is lower than PXDesign (2.50), while BoltzGen, RFdiffusion3 and Proteina-Complexa (0.46, 0.52, 0.77) are lower. These models were designed for a broad range of targets; our contribution is a training recipe that improves a given model (Uni² over UniMoMo here), so it is complementary to scaling up those models.

W2. Mechanism: how the training schedule, diffusion feedback and latent changes lead to the gains

We separate the gain into three links, each supported by a measurement.

  1. The shared schedule removes most of the sequential-training mismatch. The fully detached variant (w/o grad h-x) keeps one schedule but blocks the diffusion gradient. On peptides it already improves L-RMSD from 1.48 to 0.90, ΔG from -28.72 to -32.61, IMP from 29.03% to 36.56%, and Clash_out from 0.94% to 0.16% (Section 5).

  2. Diffusion feedback adds a further gain on top of the schedule. Moving from the detached variant to the full model improves every row shown below.

SettingMetricTwo-stageShared schedule only (w/o grad h-x)Uni² (full)
Peptide, singleΔG ↓-28.72-32.61-33.91
Peptide, singleIMP ↑29.03%36.56%38.71%
Peptide, multi-dataL-RMSD ↓1.270.850.70
Peptide, multi-dataIMP ↑40.86%43.01%44.09%
Antibody, multi-data (n=100)RMSD ↓1.040.900.85
Antibody, multi-data (n=100)IMP ↑65.00%65.00%68.33%
Small molecule, singleWeighted score ↑1.582.052.37
  1. Training dynamics of the VAE and LDM (Section 6, "Training dynamics and stability"; Figure 2; Appendix G, Figure 7).
  • The VAE and LDM are trained together from the first step, and the latent settles early while the LDM keeps refining. The distance between the latent distributions of consecutive checkpoints falls from 0.29 to about 0.06 within the first 45k steps and then stays there (see Figure 2, panel (a) in the paper), while the KL per block and the reconstruction accuracy stay stable throughout training (see Figure 7, panels (a) and (d) in the paper). The denoising loss drops fastest over the same period (see Figure 2, panel (d) in the paper) and then continues to improve slowly on this nearly fixed latent space.
  • The diffusion loss does not interfere with reconstruction and mainly makes the latent space easier to learn. Its gradient is nearly orthogonal to the reconstruction gradient (see Figure 7, panel (e) in the paper) and is smaller in magnitude (see Figure 2, panel (c) in the paper), so reconstruction still determines what the latent encodes. The diffusion loss mainly optimizes the latent space so that it is easier to learn. As a result, the generated latents lie closer to the posterior.

W3. Latent alignment, reconstruction fidelity, and decoded structural quality

To make the link between reconstruction and generation explicit, we added the following analyses (new Appendix L).

A. Reconstruction fidelity. Reconstruction of the test set by the two-stage VAE (UniMoMo) and by the jointly trained Uni² encoder and decoder. JSD compares the distribution of reconstructed geometry (backbone and side-chain dihedrals for peptides and antibodies, bond lengths and bond angles for small molecules) with that of the ground truth. PB-valid is the PoseBusters-valid fraction of reconstructions.

ModalityMetricUniMoMo VAEUni²
PeptideJSD_bb ↓0.1660.164
JSD_sc ↓0.1900.195
PB-valid ↑0.5700.591
AntibodyJSD_bb ↓0.2290.227
JSD_sc ↓0.2890.286
PB-valid ↑0.4500.417
Small moleculeJSD_BL ↓0.3480.374
JSD_BA ↓0.5850.587
PB-valid ↑0.5700.530

Joint training leaves reconstruction fidelity essentially unchanged: the geometric JSD differs by at most 0.005 for peptides and antibodies and by at most 0.026 for small molecules, and PB-valid of reconstructions improves for peptides. Since generation quality is higher in Uni² (main paper), the gain in generation does not come from a better autoencoder but from a better-aligned latent space.

B. Sensitivity to latent perturbation. We add Gaussian noise (standard deviation σ = 0.1, 0.25, 0.5 in standardized units) to the latent before decoding, and measure the geometric JSD of the decoded structures against the ground truth (lower is better).

σModelPep JSD_bbPep JSD_scAb JSD_bbAb JSD_scMol JSD_BLMol JSD_BA
0.1UniMoMo0.3340.2720.4770.3910.5410.684
Uni²0.3550.2910.4940.3990.5530.693
0.25UniMoMo0.4990.3010.6730.3650.6100.686
Uni²0.5550.3370.6940.3830.6290.705
0.5UniMoMo0.6750.3450.6970.4240.6260.673
Uni²0.6660.3680.6820.4380.6410.707

Adding noise degrades the reconstruction of both models, and the two-stage VAE and Uni² behave similarly: at any noise level their JSD differs by at most 0.06 (by 0.02 or less in 12 of 18 entries), and the rise from σ = 0.1 to 0.5 differs by at most 0.03. The decoder of Uni² therefore reacts to latent noise in a similar way, with a slightly higher JSD in 16 of 18 entries (Appendix L, Table 24).

C. Sensitivity to the number of reconstruction iterations. The decoder reconstructs the structure by N rounds of iterative refinement (N = 10 by default). We vary N for the Uni² decoder and measure the geometric JSD of the reconstructions on the test set (lower is better); the N = 10 row is the Uni² result in the table of part A.

NPep JSD_bbPep JSD_scAb JSD_bbAb JSD_scMol JSD_BLMol JSD_BA
10.2860.3500.3260.4310.7240.730
30.1990.2460.2620.3490.5670.666
50.1900.2300.2570.3350.4910.655
100.1640.1950.2270.2860.3740.587

JSD decreases monotonically with N for every modality and metric. Most of the gain comes from the first three iterations for peptides and antibodies (for example, peptide JSD_bb falls from 0.286 to 0.199), and further iterations keep refining the geometry, which is why we use N = 10 (Appendix L, Table 25).

D. Alignment versus decoded quality. We use the seven unified-training runs of the KL sweep (Appendix I, peptides). For each run we compute the standardized Sliced W1 (SW1) between generated and posterior latents, which measures alignment, and the JSD_bb and JSD_sc of the generated structures, which measure decoded structure quality (Appendix L, Table 26; Figure 8).

KL multiplier10^-310^-210^-1.510^-110^-0.510^0.510
SW10.02570.02160.03170.04050.05330.07760.1045
JSD_bb0.1750.1750.1770.1790.1800.1660.160
JSD_sc0.2030.2000.1990.2080.2020.1950.192

A linear fit of each JSD against SW1 (see Figure 8 in Appendix L, part D, of the revised paper) gives r = -0.827 and R^2 = 0.684 for JSD_bb, and r = -0.724 and R^2 = 0.524 for JSD_sc. A lower SW1 (better alignment) therefore comes with a higher JSD (worse decoded structure), so alignment and decoded structure quality are negatively correlated, and keeping them balanced is a trade-off.

Q1. Methodological differences from existing end-to-end VAE and flow/score models

Uni² differs from existing end-to-end VAE and flow/score models in three ways.

  1. Joint training from scratch. Some end-to-end models, such as LSGM [5], first pretrain the VAE and only then train it jointly with the diffusion model, so the latent space is already formed before the diffusion loss acts on it. In Uni², the autoencoder and the denoiser are randomly initialized and optimized together from the first step, so the diffusion loss shapes the latent space throughout training.
  2. Gradient flow through two latent channels. Uni² has an equivariant geometric latent z_x and an invariant content latent z_h, and the diffusion gradient flows through each channel separately (w/o grad h, w/o grad x, w/o grad h-x).
  3. Stable training. Existing end-to-end VAE and flow/score models are often unstable: the latent space has no fixed anchor and can drift, and the score-based objective grows without bound at small noise levels, so they rely on extra stabilizers or a pretrained VAE. Uni² avoids both issues. The KL term keeps the latent close to a standard Gaussian, and the epsilon-prediction objective has the same scale at every noise level, so Uni² trains stably from scratch without extra stabilizers. We show this in Appendix F, and the training curves confirm it: the drift of the latent decays while the denoising loss decreases smoothly throughout training (Figure 2, panels (a) and (d) in the paper).

Q2. Competitiveness under matched data, conditioning and inference budgets

Matched data. The four recent antibody baselines (PXDesign, BoltzGen, RFdiffusion3 and Proteina-Complexa) are large-scale pretrained models with their own, larger training data, so we report them as reference points that are matched in evaluation but not in training data. All other antibody baselines in Table 2 are trained and tested on the same dataset as Uni².

Matched conditioning. All antibody baselines in Table 2, including the four recent models, are evaluated on exactly the same test complexes, the same designed region and the same antigen and framework conditioning as Uni².

Matched inference budget. Every antibody method generates the same number of samples per target (n = 100), and no reranking or filtering is applied to any method. The accuracy metrics (AAR, L-RMSD, IMP) and the rationality metrics (clash, JSD) of all antibody models are reported at n = 1, 10 and 100 in Table 2 of the paper.

Result. Even though it is trained only on the shared dataset, Uni² reaches the highest IMP at n=100 (68.33% versus 48.93% for the best of the four), the lowest Clash_in and Clash_out, and the best JSD_sc in Table 2. BoltzGen, RFdiffusion3 and Proteina-Complexa reach a lower RMSD, consistent with their larger models and training data, whereas Uni² covers peptides, antibodies and small molecules with one model.

Q3. From latent alignment to reconstruction and decoded geometry

See W3. Experiment A gives reconstruction fidelity, and experiments B and C measure sensitivity to latent perturbation and to the number of decoding iterations. Experiment D relates latent alignment to decoded structure quality across KL strengths.

References

[1] Hannes Stark, Felix Faltings, MinGyu Choi, et al. BoltzGen: Toward Universal Binder Design. bioRxiv, 2025.

[2] Protenix Team, Milong Ren, Jinyuan Sun, et al. PXDesign: Fast, Modular, and Accurate De Novo Design of Protein Binders. bioRxiv, 2025.

[3] Jasper Butcher, Rohith Krishna, Raktim Mitra, et al. De novo Design of All-atom Biomolecular Interactions with RFdiffusion3. bioRxiv, 2025.

[4] Kieran Didi, Zuobai Zhang, Guoqing Zhou, et al. Scaling Atomistic Protein Binder Design with Generative Pretraining and Test-Time Compute. ICLR, 2026.

[5] Arash Vahdat, Karsten Kreis, Jan Kautz. Score-based Generative Modeling in Latent Space. NeurIPS, 2021.

Reviewer_Cem5OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Most papers that train latent diffusion models for design within structural biology rely on a two step approach: train a VAE, freeze it, and train a latent diffusion model on the frozen VAE representations. This work explores different variants for training both jointly, arguing that joint training might be beneficial for final performance.

Critical strengths

Many of the current design sota methods build on latent diffusion models in structural biology (also for images, videos, and other data domains).

Paper tackles a question that's therefore relevant, how to better find latent spaces that not only favor reconstruction but also yield better latent diffusion models.

Critical weaknesses

Solution proposed: propagate gradients from the diffusion loss to the encoder, train everything jointly. This is not new in this work. One of the first latent diffusion model papers (Score-based Generative Modeling in Latent Space) already explored this. Later work moved towards two-stage training, showing better scalability and more stable training.

One more ablation would be nice: freeze the autoencoder, instead of training it jointly with the diffusion module. Right now, iiuc, even when the diffusion loss gradients are stopped and do not reach the autoencoder, both components are still trained jointly. The latent space is therefore a moving target for the latent diffusion, which might affect final performance. While UniMomo is trained like this, having this additional run trained using exactly the same training framework could be informative.

Given the limited methodological novelty of propagating diffusion loss gradients to the encoder, I would need to see a stronger and more convincing empirical evaluation to recommend acceptance.

Questions for authors

A few relevant citations appear to be missing, ProteinZen and LaProteina, which build exactly on latent diffusion for protein design.

RMSD and AAR metrics reported against native binders. In the design task, are these metrics informative? We're interested in producing novel proteins, and not so much about their similarity to the native one.

Would be interesting to see how this works for models that have been scaled further, such as LaProteina or Proteina-Complexa. Though this might require a different compute scale.

I wonder, depending on the regularization strength, could this lead to pathological solutions? For instance, if not regularized enough, one might imagine that the latents for each sample could be placed very far in the latent space, which would lead to good reconstruction and low diffusion loss, albeit might struggle with generalization. If the reconstruction loss is underweighted, without strong regularization, the latent representations might collapse to the same point, pretty much making the diffusion loss zero everywhere. Did you observe any of such effects during different training runs? Additionally, these simple cases make me wonder whether the diffusion loss is the right loss to shape the latent space.

6 comments

We thank the reviewer for the careful reading and the constructive comments. Below is our responses point by point.

W1. Joint training is not new (LSGM), and later work moved to two-stage training

We thank the reviewer for pointing to LSGM [1], which we now cite and discuss in Section 2. While LSGM also explores joint training, Uni² differs in that it trains from scratch on biomolecular complexes without LSGM's stabilizers and consistently outperforms the two-stage recipe; we explain below why it trains stably.

(a) The training protocol differs from LSGM

LSGM is not trained jointly from scratch. Its first stage trains an NVAE with a standard Normal prior to convergence (Table 7 of [1]). The second stage adds a randomly initialized score network, and the VAE continues from those stage-one weights. In our setting, the autoencoder and the denoiser both start from random weights and are trained together in a single stage (Section 3), so the denoiser's target moves from the very first step. LSGM also needs several tricks to stay stable (importance sampling of t, a mixed score parameterization, KL annealing), and even with them it still pretrains the VAE first. Later latent diffusion models therefore pretrain the VAE and freeze it.

(b) Why our joint training is stable

Joint training from scratch is stable in our setting for three reasons (statements and proofs in the new Appendix F).

  1. Bounded latent (Propositions 1 and 2). The KL term to N(0, I) keeps the latent in a bounded region: the average squared mean is at most 2K_j, and the W2 distance between the encoder's latent distributions at any two training states is at most 2 sqrt(2K), where K is the average KL. In our peptide run this radius is 2.8 for z_h and 1.2 for z_x, while the measured W2 between consecutive checkpoints is at most 0.29 and 0.009 and falls to about 0.06 within 40k steps, after which the latent stays nearly fixed (Figure 2a).

  2. Bounded target and gradient (Proposition 3). Our denoiser predicts epsilon, whose second moment is d for every t, so the regression target has constant scale and the diffusion gradient reaching the encoder is bounded. Measured on peptides, it is on average 0.2 times the reconstruction-plus-KL gradient (maximum 0.95 at one checkpoint; Figure 2c), and the two gradients are nearly orthogonal (cosine close to zero; Figure 7e), so reconstruction remains the dominant force and the diffusion loss reorganizes the latent without overwriting what it encodes. The epsilon loss stays at or below its target energy of about 1 per dimension at every t (Figure 2d and Figure 7f).

  3. Starting from scratch (Proposition 4, informal). At step 0 the denoiser output is zero, so no diffusion gradient reaches the encoder and the warmed-up KL term pulls the encoder toward N(0, I).† The denoiser then tracks a slowly moving target, which we describe as a two-time-scale stochastic approximation [3]. Empirically, the denoising loss decreases smoothly while the drift of its target decays (Figure 2a, d; per-panel readings in Appendix G).

(c) Scalability and cost

Joint training is also cheaper in our setting. Because the two stages run in one job, Uni² needs 172 versus 304 A100 GPU-hours on peptides, 104 versus 112 on antibodies, 272 versus 344 on small molecules, and 537 versus 792 in multi-data training (Section 4.5), at identical inference cost. The La-Proteina results (Q3) show that the gains carry over to a different model family.

W2. Additional ablation: freeze the autoencoder under the same training framework

We ran this control on the peptide dataset and in multi-data training. The autoencoder is trained alone and frozen, and the latent diffusion model is then trained on its fixed latents, using the code, optimizer, loss weights, batch size, and number of steps of the unified runs.

DataModelAAR ↑C-RMSD ↓L-RMSD ↓ΔG ↓IMP ↑Clash_in ↓Clash_out ↓JSD_bb ↓JSD_sc ↓Seq. div ↑Struct. div ↑
SingleUniMoMo (two-stage)37.59%2.481.48-28.7229.03%1.53%0.94%0.3900.3650.6260.629
SingleFrozen AE (our framework)37.41%2.441.52-29.4230.11%1.47%0.87%0.3160.3420.6320.621
SingleUni²38.91%1.920.85-33.9138.71%0.22%0.15%0.2760.2190.6380.502
AllUniMoMo (two-stage)39.45%2.191.27-34.3540.86%0.45%0.93%0.2050.1800.6170.573
AllFrozen AE (our framework)39.83%2.141.21-33.7539.48%0.48%0.94%0.2120.1820.6260.581
AllUni²39.57%1.780.70-36.1444.09%0.11%0.07%0.2460.1750.6230.460

The frozen-autoencoder control gives results close to those of UniMoMo and clearly below Uni². Therefore, the gain therefore comes from the unified training and not from differences in the training code.

Q1. Missing citations

Thank you. We added ProteinZen [4], La-Proteina [5], and Proteina-Complexa [6] to a new paragraph in Section 2 ("Latent generative models for proteins"), together with LSGM [1] and NVAE [2]. We also tested our paradigm on La-Proteina (Q3).

Q2. Are RMSD and AAR informative in a design task?

We fully agree that generating novel binders is the ultimate goal of design. RMSD and AAR are not intended to measure novelty; rather, they assess whether the model produces binders that are physically valid and consistent with the target, which is a prerequisite for meaningful design.

  • RMSD (C-RMSD and L-RMSD). Each benchmark target has an experimentally determined binder. C-RMSD (peptide placement relative to the target) and L-RMSD (peptide conformation) test whether the model conditions on the pocket and recovers a physically valid binding mode. A model that ignores the target or produces implausible poses scores badly: RFDiffusion has a C-RMSD of 4.69 Å against 1.92 Å for Uni² (single-task), although both are generative models. These metrics also separate models that look similar on other axes.
  • AAR. We state in the paper that AAR is a weak proxy of design quality [7] and report it only for completeness. For peptides we draw no conclusion from small AAR differences. Recovery metrics are the standard in antibody CDR design [8, 9], where the epitope strongly constrains the CDR.
  • We do not rely on them. The conclusions rest on native-independent metrics: Rosetta binding energy (ΔG; -33.91 kcal/mol for Uni² versus -28.72 for UniMoMo), IMP (the share of targets where the generated peptide beats the native binder: 38.71% versus 29.03%), clash rates, dihedral JSD to real peptides, and diversity. For small molecules there is no similarity to a native ligand at all (Vina, QED, SA, geometry JSD).

Q3. Models that are scaled further

We applied our paradigm to La-Proteina [5], an autoencoder plus latent generative model for protein design, as the reviewer suggested. We trained on proteins of up to 256 residues, compared the original two-stage recipe with unified training under the same architecture, and sampled lengths n = 100, 150, and 200.

SettingModelAll-atom co-designability (%)Cα co-designability (%)Seq+Str clustersMPNN-8 (%)MPNN-1 (%)
n = 100La-Proteina (two-stage)92.093.065100.095.0
n = 100La-Proteina + Uni²96.096.071100.095.0
n = 150La-Proteina (two-stage)83.084.05299.090.0
n = 150La-Proteina + Uni²84.089.058100.094.0
n = 200La-Proteina (two-stage)70.073.04789.081.0
n = 200La-Proteina + Uni²81.084.05499.095.0
All lengthsLa-Proteina (two-stage)81.783.316496.088.7
All lengthsLa-Proteina + Uni²87.089.718399.794.7

Unified training improves co-designability at every length, with the largest gain on the longest proteins (70.0% to 81.0% at n = 200), and gives more clusters at every length. The effect carries over to a different architecture, data domain, and generative objective (Section 4.4 and Appendix E). We did not train Proteina-Complexa [6]: it extends this family of latent protein architectures, but its pretraining requires a much larger compute budget than we have, and we state this as a limitation. La-Proteina is the closest member of that family that we could train.

Q4. Could weak regularization lead to pathological solutions? Is the diffusion loss the right loss to shape the latent space?

The reviewer describes two pathologies. We agree that both are possible in principle: latents placed far apart lower the diffusion loss and help reconstruction, and a collapsed latent makes the diffusion loss zero. Both directions are controlled in our objective, and we checked them.

Latents placed far apart. This is bounded by the KL term (Propositions 1 and 2): the average squared mean is at most 2K_j and the aggregate posterior stays within W2 distance sqrt(2K) of N(0, I). In our peptide run, no z_h entry exceeds 3 standard deviations, 0.2% of z_x entries do, and the largest entries are 2.9 (z_h) and 3.7 (z_x) throughout training (Appendix G, Figure 7). The latents of the antibody, small-molecule, and multi-data runs remain similarly bounded throughout training.

Sweep over the KL strength (peptides, one-stage versus two-stage). We scaled both KL weights from 10^-3 to 10 and trained both recipes with the same architecture (new Appendix I, Table 19).

× KL (λ_h/λ_x)TrainingKL z_hKL z_xVal. block acc.Val. diff. lossEff. rank/d z_hmax abs z_hmax abs z_xJSD_bbJSD_sc
10^-3 (0.0006/0.0008)1-stage7.299.041.0000.2080.1273.9211.13.175.203
2-stage7.8820.491.0004.9120.8959.9420.52.223.282
10^-2 (0.006/0.008)1-stage5.874.161.0000.1970.1312.067.97.175.200
2-stage4.494.490.9994.0160.9135.3712.53.217.272
10^-1.5 (0.019/0.025)1-stage4.831.981.0000.1720.1801.504.79.177.199
2-stage3.411.831.0003.8180.9515.496.43.218.258
10^-1 (0.06/0.08)1-stage3.721.191.0000.1850.3101.532.93.179.208
2-stage2.460.851.0003.3710.8494.674.65.201.235
10^-0.5 (0.19/0.25)1-stage2.400.550.9990.2110.5042.393.35.180.202
2-stage1.110.730.9992.0260.4954.183.48.173.205
10^0.5 (1.9/2.5)1-stage0.420.070.9700.3400.2512.091.19.166.195
2-stage0.410.070.9721.1360.3392.481.31.161.189
10 (6.0/8.0)1-stage0.310.010.8330.3470.2252.170.50.160.192
2-stage0.320.010.8490.8480.1251.920.43.155.180

The sweep answers both parts of the question:

  • No far-apart latents under weak KL. When the KL is weakened, the two-stage latent inflates (at 10^-3, max abs z_x = 20.5 and KL z_x = 20.5). Under joint training the same setting gives 11.1 and 9.0, and max abs z_h stays between 1.5 and 2.4 from 10^-2 to 10^0.5. The diffusion loss itself therefore regularizes the latent scale, so the pathological solution is not reached even with the KL nearly switched off.
  • The diffusion loss is a good signal for shaping the latent. At every KL strength, joint training gives a 2.4 to 24 times lower validation diffusion loss (0.17 to 0.35 versus 0.85 to 4.91) at matched block accuracy, and the loss is nearly flat across KL weights, whereas the two-stage loss only becomes small when a strong KL forces the latent toward N(0, I). Under weak KL, the diffusion loss leads the encoder to a compact z_h code (effective rank/d 0.13 to 0.31) that still reconstructs perfectly (block accuracy 1.000), and z_x keeps full rank (above 0.995) in every run, so no latent collapses.
  • Joint training is insensitive to the KL weight, two-stage training is sensitive. Across the sweep, the JSD of one-stage training varies little (JSD_bb 0.160 to 0.180, JSD_sc 0.192 to 0.208), whereas two-stage training varies three to six times more (JSD_bb 0.155 to 0.223, JSD_sc 0.180 to 0.282). Under weak KL (10^-3 to 10^-1), joint training gives a much lower JSD (at 10^-3: 0.175 versus 0.223 for JSD_bb, 0.203 versus 0.282 for JSD_sc). As the KL strengthens, the two-stage latent approaches N(0, I) and the two recipes become close (gap at most 0.012). Joint training therefore removes the need to tune the KL weight, which two-stage training must do to obtain a learnable latent.
  • Strong KL hurts reconstruction for both recipes (block accuracy 0.83 to 0.97 at 10^0.5 and 10); our default weight sits between the two regimes.
Reviewer_kcQJOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Uni², which jointly trains the autoencoder and latent diffusion model within the UniMoMo framework, allowing diffusion gradients to reach the encoder through the chemical latent (z_h) and geometric latent (z_x). The method is evaluated on peptide, antibody, and small-molecule design under both single- and multi-modality training. Experiments demonstrate that the proposed training strategy can improve structural accuracy, physical plausibility, and binding energy.

Critical strengths
  1. Uni² enables diffusion-aware latent representation learning without modifying the architecture or introducing additional inference-time overhead. This is a practical contribution and offers a promising direction for unified molecular generation.
  2. This paper conducts extensive experiments covering diverse scenarios with clear explanations and detailed ablations.
Critical weaknesses
  1. The technical novelty is incremental as the model framework is mostly based on UniMoMo.
  2. The improvement is marginal or overestimated. The weighted score in small-molecule design improves only marginally from 11.53 to 11.57 (Table 3). Besides, I am concerned that the improvement in peptide design is merely due to the model generating more structurally similar, higher-quality candidates, as the structural diversity of Uni² is the worst among the baselines.
Questions for authors
  1. Could the authors provide a more specific and clearer introduction to the training details? It would be better to make a comparison between unified training and two-stage baselines regarding the computational budgets.
5 comments

We thank the reviewer for the careful reading and constructive comments. Below we respond point by point. All new experiments and text are included in the revised PDF and marked in blue.

W1. The technical novelty is incremental, as the framework is mostly based on UniMoMo.

We agree that we deliberately keep the UniMoMo backbone. This is a design choice that lets us attribute every gain to the training paradigm and to nothing else. Our contribution is the paradigm itself: we replace the frozen interface between the autoencoder and the latent diffusion model with a bidirectional one, so that diffusion gradients shape the latent manifold (Eq. 3). The paper also contributes a channel-wise gradient analysis of the invariant latent z_h and the equivariant latent z_x, which shows that z_x drives geometry and z_h drives chemistry and interaction (Section 5), and a latent-space analysis based on sliced Wasserstein distance (Section 6).

To show that the paradigm is not specific to UniMoMo, we applied it to a different latent generative model, La-Proteina (autoencoder plus latent generative model for protein design) [1]. We trained on proteins of up to 256 residues, compared the original two-stage recipe with unified training under the same architecture, and sampled proteins of length n = 100, 150, and 200.

SettingModelAll-atom co-designability (%)Cα co-designability (%)Diversity, Seq+Str clustersMPNN-8 (%)MPNN-1 (%)
n = 100La-Proteina (two-stage)92.093.065100.095.0
n = 100La-Proteina + Uni²96.096.071100.095.0
n = 150La-Proteina (two-stage)83.084.05299.090.0
n = 150La-Proteina + Uni²84.089.058100.094.0
n = 200La-Proteina (two-stage)70.073.04789.081.0
n = 200La-Proteina + Uni²81.084.05499.095.0
All three lengthsLa-Proteina (two-stage)81.783.316496.088.7
All three lengthsLa-Proteina + Uni²87.089.718399.794.7

(The full table with pLDDT and structure and sequence cluster counts is Table 16 in Appendix E of the revised paper.)

Unified training improves co-designability at every length, with the largest gain on the longest proteins (all-atom co-designability from 70.0% to 81.0% at n = 200). It also yields more clusters at every length, so the gain does not come at the cost of diversity. The effect therefore carries over to a different architecture, data domain, and generative objective, which supports our claim that the contribution is the training paradigm and not a UniMoMo-specific trick. We added this experiment as Section 4.4 of the revised paper.

W2. The improvement is marginal or overestimated.

(a) Small molecule: 11.53 to 11.57

We agree that the all-task weighted score changes little, and we explain why, and add evidence from a second dataset.

  1. The single-task gain is clear. On CrossDocked2020 with single-modality training, the weighted score improves from 9.58 (UniMoMo) to 10.32 (Uni²), and the rank from 6th to 5th.
  2. The all-task setting has little headroom. The all-task model is trained on three datasets together, and this training gain already lifts both UniMoMo and Uni² to a high level (11.53 is already 2nd in the leaderboard). The weighted score is also derived from ranks, so it records whether a model overtakes another and not the size of the improvement in the underlying continuous metrics. Uni² still moves from 2nd to 1st.
  3. A second small molecule dataset. We trained and evaluated UniMoMo and Uni² on the PDBbind data used in DiffDock [2]. Uni² is better on all seven metrics (Table 17 in Appendix E of the revised paper):
ModelQED ↑SA ↑JSD_BL ↓JSD_BA ↓Vina Score ↓Vina Min ↓Vina Dock ↓
UniMoMo (single)0.430.690.35520.4766-6.08-6.72-7.93
Uni² (single)0.450.700.31380.4117-6.75-7.09-8.15

The improvement of the Vina scores by 0.37 to 0.67 kcal/mol and of both JSD values by about 0.04 to 0.07 shows that the gain is clear on two different small molecule datasets.

(b) Peptide: are the gains due to lower diversity?

We note that the lower structure diversity does not indicate mode collapse. Sequence diversity is maintained (0.638 versus 0.626) and the dihedral JSD to real peptides drops, indicating that the model still covers the distribution of real peptides. In addition, the structure diversity metric clusters samples at a 2 Å RMSD threshold, so a more accurate model naturally places more candidates in the same native-like cluster; it is therefore partly coupled to accuracy and is not a pure measure of design quality.

Q1. Training details and a computational budget comparison with the two-stage baselines

Setup. UniMoMo and Uni² are trained with the same hardware (8 GPUs with 80 GB of memory each) and the same batch size. UniMoMo trains the VAE until convergence, freezes it, and then trains the latent diffusion model. Uni² initializes the autoencoder and the denoiser together and optimizes the combined objective (Eq. 3) in one run. Loss weights and hyperparameters are listed in Tables 20 and 21 of Appendix J, and we added a paragraph on the training procedure and compute to Appendix J.

Cost. The table reports A100 GPU-hours. The UniMoMo total is the sum of the VAE and LDM stages.

PeptideAntibodySmall moleculeMulti-data
UniMoMo (VAE)20064240528
UniMoMo (LDM)10448104264
UniMoMo (total)304112344792
Uni²172104272537
Saving43%7%21%32%

Uni² needs fewer GPU-hours on every modality. On peptides it uses 172 versus 304 GPU-hours (43% fewer), and in the multi-data setting 537 versus 792 (32% fewer, about two thirds of the two-stage budget). The inference cost is unchanged. Unified training therefore improves quality and also lowers the training cost. This table is now Table 4 in Section 4.5 of the revised paper.

References

[1] Geffner et al. La-Proteina: Atomistic Protein Generation via Partially Latent Flow Matching. ICLR 2026. [2] Corso et al. DiffDock: Diffusion Steps, Twists, and Turns for Molecular Docking. ICLR 2023. [3] Kynkäänniemi et al. Improved Precision and Recall Metric for Assessing Generative Models. NeurIPS 2019. [4] Dhariwal and Nichol. Diffusion Models Beat GANs on Image Synthesis. NeurIPS 2021. [5] Watson et al. De novo design of protein structure and function with RFdiffusion. Nature 2023. [6] Geffner et al. Proteína: Scaling Flow-based Protein Structure Generative Models. ICLR 2025.

Dear Reviewer kcQJ,

Thank you for the constructive comments. In our response we have added experiments and analyses, which we hope address the concerns raised. We would be grateful for any feedback, and we are glad to provide further clarification if needed. Thank you for your time.

Best regards,

Authors,

Comments on the paper

1 comment
Area_Chair_yeVdOpenReview ↗

Authors: please respond to your reviewers

To the paper authors: Please acknowledge and respond to the thoughtful replies by reviewers. They have raised some going points about strengths and weaknesses of your paper, as well as questions that merit replies.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.