acceptodds
Under review as a conference paper at ICLR 2027

UNAAGI: Atom-Level Diffusion for Generating Non-Canonical Amino Acid Substitutions

Abstract

A central challenge in structural biology is identifying beneficial amino acid substitutions, a problem that underpins both mutational effect prediction and protein engineering. Recent inverse-folding models, trained to reconstruct sequences from structure, have shown considerable promise for identifying functional mutations. However, current approaches are constrained to designing sequences composed exclusively of canonical amino acids (CAAs). Non-canonical amino acids (NCAAs) offer greater chemical diversity and are frequently used for protein engineering in vivo, yet they remain largely inaccessible to current variant effect prediction methods. To address this gap, we introduce UNAAGI, a diffusion-based generative model that reconstructs residue identities from atomic-level structure using an E(3)-equivariant framework. By modeling side chains in full atomic detail rather than as discrete tokens, UNAAGI enables the exploration of canonical and non-canonical amino acid substitutions within a unified generative paradigm. We evaluate UNAAGI zero-shot, without mutational-effect supervision or task-specific fine-tuning, on experimental mutational-effect benchmarks and demonstrate substantially improved performance on NCAA substitutions relative to existing methods. Furthermore, our results suggest a shared methodological foundation between protein engineering and structure-based drug design, motivating a unified training framework across these domains.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Protein Engineering, Molecular Diffusion, Non-canonical Amino Acids

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_EZdDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes UNAAGI, an atom-level diffusion framework for generating canonical and non-canonical amino acid (NCAA) substitutions and predicting their mutational effects. Unlike conventional inverse-folding models that predict residue identities from a fixed vocabulary, UNAAGI uses an E(3)-equivariant diffusion model to generate side-chain molecular graphs conditioned on local protein structures. The generated graphs are mapped to amino acid identities, and their sampling frequencies are used to estimate mutation likelihoods and derive zero-shot mutational-effect scores.

Critical strengths

Replacing fixed residue identities with atom-level molecular graph generation is a reasonable and interesting approach to non-canonical amino acid design. By directly modeling atomic composition, chemical bonds, and 3D conformations, UNAAGI can represent a broader range of side-chain chemistries without requiring a predefined output class for every amino acid. This provides a natural connection between molecular generative modeling and protein inverse folding, and offers a promising direction for extending protein design beyond the canonical amino acid vocabulary.

Critical weaknesses
  1. The main motivation of UNAAGI is to overcome the fixed vocabulary of conventional inverse-folding models. However, after generating atomic-level molecular graphs, the authors still map them to amino acid identities using a reference library. This raises a question about whether atom-level diffusion is really necessary for the evaluated task. For example, could a conventional inverse-folding model simply extend its output vocabulary to include the NCAAs in the reference library? While UNAAGI can theoretically generate molecular structures outside this library, the paper does not clearly demonstrate the practical advantage of this capability. Also, the model is already trained on substantial NCAA structural data, including isolated NCAA molecules and NCAA-containing protein structures. Therefore, its ability to generate NCAAs does not necessarily demonstrate generalization to previously unseen amino acids. Although the authors conduct held-out experiments, they only exclude four canonical amino acids rather than actual NCAAs. Moreover, these experiments report coverage, which only measures whether an amino acid appears at least once among many generated samples. With 1,000 samples, even an amino acid assigned a probability of just 0.5% has over a 99% chance of appearing at least once. Thus, high coverage does not necessarily indicate that the model has learned accurate residue probabilities or meaningful preferences.

  2. The main experimental results show positive Spearman correlations between UNAAGI's mutation scores and experimental measurements. However, it is unclear whether these correlations reflect the model's ability to distinguish different NCAA substitutions or simply its ability to identify positions that are generally sensitive to mutations. For example, consider a position where the model frequently generates the wild-type amino acid but never generates any of the tested NCAAs. Because the authors assign the same pseudocount to all unsampled amino acids, every NCAA substitution at this position receives exactly the same mutation score. The model therefore cannot distinguish which NCAA would be better or worse at that position. However, different positions may have different wild-type probabilities. If the model can identify which positions are generally more tolerant of mutations, it could still achieve a positive Spearman correlation when all substitutions across all positions are evaluated together. In other words, a positive overall correlation does not necessarily demonstrate that the model has learned the chemical differences between NCAA substitutions. I think this is particularly important because the paper's central goal is to extend mutational-effect prediction beyond canonical amino acids. The authors should evaluate whether UNAAGI can rank different NCAA substitutions at the same position, and compare against simple baselines that only capture position-level mutation sensitivity. In addition, the training-data ablations are only evaluated on canonical ProteinGym benchmarks, rather than the NCAA benchmarks that form the main contribution. Finally, estimating mutation probabilities requires 1,000 diffusion samples per position, each involving 20 denoising steps. Given this substantial computational cost, the paper would benefit from runtime comparisons and stronger evidence that atom-level diffusion provides advantages over simpler prediction methods.

Questions for authors
  1. Have the authors considered a leave-one-NCAA-out experiment to directly evaluate generalization to unseen amino acid chemistries? Additionally, given that generated molecular graphs are ultimately mapped to a reference library, what advantages does UNAAGI offer over simply extending the vocabulary of a conventional inverse-folding model?
  2. Could the authors clarify whether the reported Spearman correlations reflect the ability to distinguish different NCAA substitutions at the same position, rather than simply identifying positions that are generally sensitive to mutations? In particular, how frequently are NCAA substitutions never sampled and therefore assigned identical pseudocount-based scores? A within-site ranking evaluation or a simple site-level baseline would help establish whether the model actually captures NCAA-specific mutational effects.
4 comments
AuthorsOpenReview ↗

Response to Reviewer EZdD (1/2)

We thank the reviewer for asking what atom-level generation adds and whether UNAAGI can distinguish NCAA substitutions at the same site. These questions helped us clarify the rationale for our approach and strengthen its evaluation. We report within-position correlations, quantify overlap between the training data and the benchmarks, and evaluate the contribution of each training source on CP2 and PUMA.

1. Ranking alternative substitutions at the same position (W2, Q2)

To assess whether UNAAGI can distinguish among NCAA substitutions at the same site, we added a within-position analysis in Figure D3 of the revised manuscript. At each position, we compute Spearman’s ρ between predicted scores and measured mutational effects, then average the correlations across positions.

BenchmarkMean within-position ρ, all substitutionsMean within-position ρ, NCAA only
CP20.1540.280
PUMA0.1440.177

At a given position, the wild-type term in ΔNLL is shared by all candidates and therefore leaves their ranking unchanged. The positive mean NCAA-only correlations indicate that UNAAGI captures differences in mutational effects among NCAA substitutions at the same site. Figure D4 complements this analysis by comparing predicted and measured effects for each residue identity across positions, where the wild-type term can vary.

In Figure D2, cells in the NCAA columns mark substitutions that UNAAGI sampled at least once at the corresponding position in each of the five runs.

2. Why atom-level generation for open-ended NCAA design? (W1, Q1)

UNAAGI generates side-chain atoms, bonds, and three-dimensional coordinates conditioned on the local protein environment. This allows it to propose NCAA structures beyond the benchmark panel. Section 4.7 and Figure E1 illustrate the range of sampled chemistries. We use the reference library after generation to identify sampled residues and estimate their frequencies for substitutions with experimental measurements.

Extending a residue-token model such as ProteinMPNN introduces a separate output class for each added NCAA, many of which have few training examples. UNAAGI shares atom and bond representations across residue identities, allowing examples of one side chain to inform predictions for chemically related NCAAs. The same representation can express new combinations of atoms and bonds as the set of candidate chemistries grows. We clarify this rationale in Appendix F of the revised manuscript.

3. Interpretation of held-out recovery (W1)

We count a site as covered if UNAAGI generates the held-out canonical residue at least once in each of five runs of 1,000 draws. Recovery rates were significantly higher at experimentally favorable sites than at harmful sites (Table 1 and Appendix D.1). This suggests that UNAAGI can recover site-specific preferences for a residue identity withheld during training.

4. Training overlap (W1, Q1)

Table 3 of the revised manuscript, reproduced below, reports how many records in each of the two NCAA-specific training sources match the heavy-atom molecular graph of each audited NCAA.

Benchmark NCAAIsolated-NCAA training recordsNCAA-containing PDB training records
2NP011
2TH00
3TH00
ABU10286
AHP00
AIB0756
AOC02
BZT00
CHA068
CPA01
HSM100
MEA061
MEB100
MEF061
MEG0145
NLE10204
NVA1041
TBU00
TME00
YME05
Residue graphs represented5/2012/20
Total matching records501,641

Across the two NCAA-specific training sources, 14 of the 20 audited residue graphs have a match. The six unmatched graphs are 2TH, 3TH, AHP, BZT, TBU, and TME. Our PDBBind training examples contain small-molecule ligands and provide no additional NCAA residue examples. Figure D2 shows coverage at each position in CP2 and PUMA, including positions where UNAAGI generates AHP and TBU despite their absence as NCAA training examples. Figure D4 includes across-position correlations for both AHP and TBU.

AuthorsOpenReview ↗

Response to Reviewer EZdD (2/2)

5. Ablations on the NCAA benchmarks (W2)

To assess the contribution of each training source, we evaluated the ablated models on CP2 and PUMA over five sampling runs. The table reports Spearman’s ρ for all evaluated substitutions and for the NCAA-only subset. Table 5 of the revised manuscript is reproduced below.

Training configurationCP2, allCP2, NCAA onlyPUMA, allPUMA, NCAA only
Full UNAAGI0.3241 ± 0.0220.4283 ± 0.0090.3455 ± 0.0270.3216 ± 0.029
Without NCAA-containing PDB data0.2244 ± 0.0520.3298 ± 0.0530.2729 ± 0.0400.2297 ± 0.032
Without PDBBind0.2814 ± 0.0480.3795 ± 0.0550.3235 ± 0.0470.3195 ± 0.043
Without isolated-NCAA data0.2402 ± 0.0550.3577 ± 0.0500.2597 ± 0.0280.2399 ± 0.035
Canonical PDB data only0.1851 ± 0.0680.2589 ± 0.0740.2132 ± 0.0320.2244 ± 0.033

6. Sensitivity to the pseudocount (Q2)

Before computing ΔNLL, we add the same pseudocount k to every residue count. Table 4 reports Spearman correlations on the full evaluation sets for five values of k and includes a separate analysis restricted to substitutions sampled by UNAAGI.

Pseudocount k or evaluation subsetCP2, allCP2, NCAA onlyPUMA, allPUMA, NCAA only
0.10.2697 ± 0.0290.3891 ± 0.0090.2890 ± 0.0250.2941 ± 0.032
1.0 (reported)0.3241 ± 0.0220.4283 ± 0.0090.3455 ± 0.0270.3216 ± 0.029
5.00.3492 ± 0.0180.4400 ± 0.0100.3717 ± 0.0290.3301 ± 0.028
10.00.3536 ± 0.0170.4396 ± 0.0090.3766 ± 0.0300.3295 ± 0.029
20.00.3528 ± 0.0190.4384 ± 0.0100.3696 ± 0.0300.3293 ± 0.030
Sampled substitutions only0.3300 ± 0.0370.4712 ± 0.0470.3655 ± 0.0240.3035 ± 0.057

Thank you for the detailed response and additional experiments. I appreciate the new within-position analysis and NCAA-specific ablations, which address several of my concerns.

However, I am still not fully convinced of the necessity of atom-level diffusion. The authors argue that extending the vocabulary of conventional inverse-folding models would suffer from data sparsity, while atom-level generation enables knowledge sharing across different NCAAs. Although reasonable, this argument remains largely conceptual and is not supported by direct comparisons. Simpler alternatives, such as extended-vocabulary models or structure-conditioned predictors using molecular graph embeddings, could potentially achieve similar benefits.

Moreover, the substantial inference cost of estimating mutation probabilities through 1,000 diffusion samples per position remains unaddressed. Without comparisons against simpler NCAA-aware models or an analysis of the sampling budget, it is difficult to assess whether the additional complexity of atom-level diffusion is justified.

Dear reviewer EZdD,

Thanks for your comments.

We just want to clarify one important aspect of our paper, which has perhaps not been sufficiently clear: We agree with the reviewer that extended-vocabulary models might work well in principle. But in practice, we do not have the data to train them. As illustrated in our new Table 3, many of the amino acids that are experimentally interesting do not appear a single time in the PDB. If we train an inverse folding model with an extended vocubulary, there would therefore be no signal to update the logit-dimensions corresponding to these NCAAs. The mixing-in of isolated NCAAs works in our model because transfer of information between different amino acids is possible, but in an extended-vocabulary model, there is nothing that would couple NCAAs to their closest canonical AAs. To clarify this point in the paper, we now included a discussion on the contrast to extended vocabulary methods in Section F.1 of the paper.

We agree that training a structure-conditional model to predict small molecule embeddings is another potential solution to this problem. Actually, we followed this approach in parallel to the diffusion model presented here - but were never able to get competitive performance out of that approach. We agree that it would be reasonable to further investigate the potential of structure conditioned embedding-prediction in the future, but we believe that this is out of scope for this current paper.

Our main objective with this paper was to demonstrate that generalization from canonical to nearby non-canonical amino acids was possible, and that this can lead to useful levels of correlation to experimental data. We agree that further gains can probably be achieved in this area - but as we demonstrate in the paper, no other methods can currently explore NCAA space reliably, and we therefore believe that already this current version has real value for peptide exploration in a pharmaceutical context.

Reviewer_rfYiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces UNAAGI, an atom-level diffusion model for proposing canonical and non-canonical amino acid substitutions in a fixed local structural environment. The model jointly generates side-chain coordinates, atom features, and bond types, using virtual nodes to accommodate variable atom counts. Generated molecular graphs are mapped to residue identities, and sampling frequencies are converted into mutation scores through a mutant-versus-wild-type log-probability ratio. The authors evaluate recovery of held-out canonical residues, zero-shot mutational-effect prediction on the CP2 and PUMA benchmarks, canonical-substitution prediction on 25 ProteinGym assays, and training-data ablations. The results show positive correlations with experimental NCAA mutational effects on CP2 and PUMA, while reliable generation is largely restricted to NCAAs chemically similar to canonical residues.

Critical strengths
  1. A meaningful formulation beyond fixed residue vocabularies. Generating side-chain molecular graphs instead of selecting among predefined amino acid tokens addresses a genuine limitation of conventional inverse-folding models. The framework provides a unified atom-level representation for canonical and non-canonical side chains without requiring a predefined vocabulary of residue identities.

  2. Validation using experimental mutational-effect measurements. The paper evaluates predictions against measured effects rather than relying exclusively on computational structural scores. Positive NCAA-only correlations on CP2 and PUMA support the presence of predictive information beyond canonical substitutions. The controlled held-out-residue experiments and explicit reporting of limited chemical coverage are also valuable.

Critical weaknesses
  1. The scope and source of NCAA generalization remain insufficiently established. The full model is trained on isolated NCAAs, NCAA-containing protein structures, and protein–ligand complexes, but the overlap between benchmark NCAA identities and these training sources is not clearly quantified. Zero-shot evaluation without mutational-effect labels is legitimate, but does not by itself demonstrate generalization to unseen chemical identities. Moreover, Table 2 evaluates training-data ablations on canonical ProteinGym substitutions rather than on the central NCAA benchmarks. Consequently, the current evidence does not clearly separate the contribution of canonical-residue training from that of supplementary chemical data, or establish performance on genuinely unseen NCAA identities. This distinction is important for interpreting the claimed generalization beyond the canonical vocabulary.

  2. The practical value of NCAA substitution ranking remains insufficiently validated. The principal evidence comes from two peptide–protein systems, with NCAA-only Spearman correlations of approximately 0.3, but no direct evaluation of whether highly ranked substitutions are enriched for experimentally beneficial variants. Unsampled residues receive pseudocount-based scores, yet the reported pseudocount robustness analysis concerns ProteinGym rather than the NCAA benchmarks. The baseline comparisons are informative but do not fully establish an advantage in mutational-effect prediction, given the differences in scoring objectives and structural inputs. The current results do not sufficiently establish that UNAAGI can reliably prioritize beneficial NCAA substitutions for experimental testing.

Questions for authors
  1. Training–evaluation overlap. Appendix F acknowledges that benchmark proteins may occur in the structural training corpus. Could the authors quantify this overlap specifically for CP2 and PUMA, including their complexes and closely related structural environments? Please also identify which benchmark NCAA identities occur in each NCAA training source and report performance separately for seen and unseen identities, where feasible.

  2. Ablations on the core task. Could the authors evaluate the Table 2 ablation models on CP2 and PUMA, particularly for NCAA-only substitutions? This would directly establish which training sources support the claimed NCAA predictive capability.

  3. Definition of held-out recovery. How is a site counted as successfully generating the held-out residue in Table 1? Does success require at least one occurrence among a fixed number of samples? Please specify the sampling budget, any frequency threshold, and how repeated runs are aggregated. Comparing the probabilities or ranks assigned to held-out residues at experimentally favorable and harmful sites would help distinguish chemical recoverability from site-specific preference.

  4. Coverage and score robustness. For what fraction of evaluated NCAA substitutions is the mutant residue never generated at the corresponding site? How sensitive are the correlations to the sampling budget and pseudocount choice? Please report sampling coverage and correlations on the sampled subset alongside the full-benchmark results, with the retained sample sizes clearly specified.

  5. Practical design utility and sample validity. Can the authors report the enrichment or precision of experimentally beneficial substitutions among top-ranked NCAA candidates at a fixed selection budget? Please also clarify whether samples that fail geometric validity checks contribute to residue-frequency estimates and whether excluding these samples materially changes predictive performance.

3 comments
AuthorsOpenReview ↗

Response to Reviewer rfYi

Thank you for your careful review and for raising questions about UNAAGI’s practical utility. We address these questions by quantifying overlap between the training data and the benchmarks, evaluating the contribution of each training source, and assessing how well UNAAGI ranks NCAA substitutions.

1. Training overlap and generalization (W1, Q1)

NCAA overlap. Table 3 of the revised manuscript reports the number of records in each of the two NCAA-specific training sources that match each of the 20 audited residue graphs. We match heavy-atom molecular graphs to account for different names for the same structure.

Benchmark NCAAIsolated-NCAA training recordsNCAA-containing PDB training records
2NP011
2TH00
3TH00
ABU10286
AHP00
AIB0756
AOC02
BZT00
CHA068
CPA01
HSM100
MEA061
MEB100
MEF061
MEG0145
NLE10204
NVA1041
TBU00
TME00
YME05
Identities represented5/2012/20
Total matching records501,641

Together, these sources contain 14 of the 20 audited residue graphs. The six absent from both sources are 2TH, 3TH, AHP, BZT, TBU, and TME. Our PDBBind training examples contain small-molecule ligands and provide no additional NCAA residue examples. Figure D2 shows coverage by residue and position, including examples of NCAAs that UNAAGI generated without NCAA training examples for those identities.

Receptor overlap. We searched for related receptor chains using Foldseek in TM-align mode (TM-score ≥ 0.5) and BLOSUM62-based sequence alignments (sequence identity ≥ 30%). Tracing hits from either search to PDB entries that contributed graphs to the NCAA-containing training source identified four KDM4A-related entries and 13 MCL-1-related entries. The table reports sequence identities from the sequence alignments and structural scores from Foldseek.

ReceptorPDB IDSequence identity (BLOSUM62 alignment)Foldseek TM-score
CP2 (KDM4A, 5LY1)2OX0100%1.00
2OQ6100%0.99
5FPA67%0.87
5FP967%0.87
PUMA (MCL-1, 2ROC)5W8F89%0.83
6UA388%0.87
5C3F88%0.86
6VBX86%0.82
4G3585%0.76
5W8985%0.79
5VAX35%0.76
9O1435%0.74
9O1635%0.74
9O1533%0.77
4YK929%0.74
5VWZ26%0.78
5VX026%0.78

The zero-shot evaluation remains valid in the presence of receptor overlap. The benchmark asks whether UNAAGI can predict the effects of substitutions in held-out peptides that bind to known receptors. The assayed peptides and their exact benchmark complexes, 5LY1 and 2ROC, were absent from training, and UNAAGI was trained without experimental mutational-effect labels (Appendix F). The evaluation therefore tests zero-shot prediction of peptide mutational effects in structurally familiar receptor environments.

2. Training-data ablations on CP2 and PUMA (W1, Q2)

We evaluated the ablated models on CP2 and PUMA, including the NCAA-only subsets. Table 5 of the revised manuscript reports Spearman correlations across five sampling runs and is reproduced below.

Training configurationCP2: allCP2: NCAA onlyPUMA: allPUMA: NCAA only
Full UNAAGI0.3241 ± 0.0220.4283 ± 0.0090.3455 ± 0.0270.3216 ± 0.029
Without NCAA-containing PDB data0.2244 ± 0.0520.3298 ± 0.0530.2729 ± 0.0400.2297 ± 0.032
Without PDBBind0.2814 ± 0.0480.3795 ± 0.0550.3235 ± 0.0470.3195 ± 0.043
Without isolated-NCAA data0.2402 ± 0.0550.3577 ± 0.0500.2597 ± 0.0280.2399 ± 0.035
Canonical PDB data only0.1851 ± 0.0680.2589 ± 0.0740.2132 ± 0.0320.2244 ± 0.033

Removing NCAA-containing PDB data reduces the NCAA-only mean correlation from 0.4283 to 0.3298 on CP2 and from 0.3216 to 0.2297 on PUMA. Training on canonical PDB data alone yields positive NCAA-only correlations of 0.2589 and 0.2244, respectively. The full model achieves higher mean correlations on both assays, while the PUMA NCAA-only mean changes little when PDBBind is removed (0.3216 to 0.3195).

3. Held-out recovery (Q3)

We draw 1,000 samples per site in each of five runs. A site counts as covered if the held-out residue appears at least once in each of the five runs. The recovery rate is the fraction of sites meeting this criterion. Table 1 compares recovery rates at favorable and harmful sites, with favorable-to-harmful ratios ranging from 1.10× to 1.38×. Appendix D.1 gives the corresponding site counts.

AuthorsOpenReview ↗

Response to Reviewer rfYi (2/2)

4. Within-position ranking, coverage, and scoring sensitivity (W2, Q4)

Within-position ranking. To assess whether UNAAGI ranks alternative NCAA substitutions at the same site, we added Figure D3 to the revised manuscript. We compute a Spearman correlation at each position and average the correlations across positions. The wild-type term in the mutation score is constant across candidates at the same site, so this analysis measures how well UNAAGI ranks substitutions at that site.

BenchmarkMean within-position ρ: all substitutionsMean within-position ρ: NCAA only
CP20.1540.280
PUMA0.1440.177

The positive NCAA-only means show that the ranking signal persists within positions. Performance varies across sites, and unsampled candidates share a score at each site. Figure D4 holds the candidate amino acid fixed and compares scores with experimental measurements across positions. Marker area indicates the fraction of evaluated positions at which UNAAGI generated that amino acid.

Coverage and pseudocount sensitivity. In Figure D2, the NCAA columns show which substitutions meet the five-run coverage criterion at each position. The row totals give the number of covered identities among the 39 plotted candidates. Table 4 evaluates five values of the pseudocount k, which we add to every residue count before scoring. The final row reports correlations restricted to sampled substitutions. Values are means ± standard deviations across five sampling runs of 1,000 draws per position.

Pseudocount k or evaluation subsetCP2, allCP2, NCAA onlyPUMA, allPUMA, NCAA only
0.10.2697 ± 0.0290.3891 ± 0.0090.2890 ± 0.0250.2941 ± 0.032
1.0 (reported)0.3241 ± 0.0220.4283 ± 0.0090.3455 ± 0.0270.3216 ± 0.029
5.00.3492 ± 0.0180.4400 ± 0.0100.3717 ± 0.0290.3301 ± 0.028
10.00.3536 ± 0.0170.4396 ± 0.0090.3766 ± 0.0300.3295 ± 0.029
20.00.3528 ± 0.0190.4384 ± 0.0100.3696 ± 0.0300.3293 ± 0.030
Sampled substitutions only0.3300 ± 0.0370.4712 ± 0.0470.3655 ± 0.0240.3035 ± 0.057

Correlations remain positive on both benchmarks across the tested pseudocounts, including on the NCAA-only subsets.

Sampling budget. We also evaluated smaller numbers of draws per position over five runs. NCAA-only correlations remain positive at 100 draws per position and increase through 1,000 draws on both assays. The revised manuscript reports these results in Table 6.

Benchmark and evaluation set100 draws200 draws500 draws1,000 draws
CP2: all+0.324 ± 0.041+0.342 ± 0.028+0.337 ± 0.014+0.3241 ± 0.022
CP2: NCAA only+0.381 ± 0.041+0.412 ± 0.027+0.427 ± 0.013+0.4283 ± 0.009
PUMA: all+0.322 ± 0.033+0.340 ± 0.025+0.349 ± 0.012+0.3455 ± 0.027
PUMA: NCAA only+0.279 ± 0.032+0.296 ± 0.026+0.313 ± 0.013+0.3216 ± 0.029

5. Candidate prioritization and geometric validity (Q5)

Precision among top-ranked substitutions. Table 7 of the revised manuscript reports Precision@K. We classify experimentally benign substitutions by ΔΔG ≤ 0. We average the residue counts across the five sampling runs before scoring and ranking candidates. For each assay and candidate set, K is the number of experimentally benign substitutions in that set. The expected precision under random selection is the fraction of benign substitutions in the candidate set. Lift is the ratio of the reported precision to this reference. For CP2 NCAA, the reported uncertainty reflects random tie-breaking at the selection cutoff.

BenchmarkCandidate setKPrecision@KRandom-selection referenceLift
CP2All substitutions510.235 ± 0.0150.1152.05×
CP2NCAA only320.342 ± 0.020.1402.44×
PUMAAll substitutions2020.327 ± 0.0240.2181.50×
PUMANCAA only1210.364 ± 0.0230.2551.43×

The top-ranked NCAA candidates show 2.44× enrichment on CP2 and 1.43× on PUMA over random selection at these benchmark-defined budgets.

Geometric validity. The default frequency estimates include graph-matched samples that fail geometric validity checks. Appendix E.3 and Figure E5 report PoseBusters pass rates for CP2 and PUMA. Excluding samples that fail PoseBusters yields the following correlations:

Evaluation setCP2: PB-filtered ρPUMA: PB-filtered ρ
All substitutions+0.3173 ± 0.03+0.3462 ± 0.023
NCAA substitutions only+0.4290 ± 0.01+0.3302 ± 0.04

The NCAA-only means after filtering are close to the corresponding unfiltered means on CP2 (0.4290 vs. 0.4283) and PUMA (0.3302 vs. 0.3216).

Reviewer_rfYiOpenReview ↗

Official Comment by Reviewer rfYi

Thank you for the rebuttal and additional experiments, which address several of my questions. However, the practical ranking performance remains modest: the mean NCAA-only within-position correlations are 0.280 on CP2 and 0.177 on PUMA. It is therefore still unclear whether the method can reliably prioritize beneficial NCAA substitutions in broader settings. I will keep my current rating. I did not use any AI tools to write this review.

Reviewer_buKUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper focuses on side-chain generation and mutational effect prediction for NCAAs. Existing inverse folding models typically represent amino acids as a fixed vocabulary of 20 CAA tokens, making it difficult to extend to NCAAs. The authors propose UNAAGI, an E(3)-equivariant diffusion model that directly generates side-chain atomic coordinates, atom types, and bond types, while using virtual nodes to support variable-size side-chain generation. Sampling frequencies are used to estimate the probabilities of different amino acid substitutions for zero-shot mutational effect prediction.

Experiments include held-out CAA generalization, ProteinGym, and two NCAA DMS benchmarks. The results show that UNAAGI can generate a subset of chemically proximal NCAAs and outperforms major baselines on NCAA mutational effect prediction.

Critical strengths
  • The research problem is practically meaningful, and the modeling approach is well motivated. By extending amino acid representations from fixed categorical tokens to atom-level molecular graphs, the model can explore side-chain chemistry beyond the canonical vocabulary, providing a feasible approach for NCAA generation and mutational effect prediction.
  • The experimental design provides useful evidence. Held-out canonical amino acid experiments demonstrate some generalization ability to unseen residue chemistry. On NCAA DMS benchmarks, UNAAGI shows better predictive performance than PepINVENT, NCFlow, and AF3 iPTM.
Critical weaknesses
  • EQGAT-diff, categorical diffusion, and virtual-node padding are existing molecular diffusion techniques. UNAAGI applies these techniques to NCAA side-chain generation, but lacks a controlled comparison with residue-level prediction methods using the same all-atom structural context, making it difficult to establish the advantages of atom-level generation itself.
  • Although NCAA-only prediction achieves a Spearman correlation of approximately 0.3 and outperforms the baselines, the absolute predictive performance is still relatively limited.
  • The training data include NCAA structures and NCAA-containing PDB complexes, but the paper does not sufficiently distinguish between learning NCAA chemistry already encountered during training and generalizing to genuinely unseen NCAAs.
  • It is unclear whether stereochemical information can be represented or preserved in the adopted E(3)-equivariant architecture, e.g., D/L amino acids.
Questions for authors
  • Could the authors provide generation coverage and mutational effect prediction performance for different NCAA classes? In particular, distinguishing between training-seen and unseen NCAA chemistries would help further validate the model's generalization ability.
  • Some test structures in the NCAA DMS benchmarks may overlap with the structural training corpus. Could the authors analyze the structural overlap between training and test data and evaluate its impact on zero-shot prediction?
2 comments
AuthorsOpenReview ↗

Response to Reviewer buKU (1/2)

Response to Reviewer buKU

We thank the reviewer for their assessment of our work. Below, we address the concerns raised and clarify several aspects of our approach, referring to additional analyses included in our revised manuscript.

1. Predictive performance of UNAAGI (W2)

We acknowledge that a Spearman correlation of approximately 0.3 may appear limited in absolute terms. Nevertheless, from a practical perspective, when designing larger libraries of peptide variants, even a correlation of 0.3 provides a useful signal for ruling out unlikely mutations and identifying pools of promising candidates for experimental validation. As illustrated in Figure D1, existing methods on canonical amino acids are often just 0.1-0.15 Spearmann rho points better than our procedure and these methods are routinely used in library design.

To further demonstrate the practical utility of UNAAGI, we have added a Precision@K analysis in Table 7 of the revised manuscript, evaluating its ability to prioritize experimentally favorable NCAA substitutions.

For NCAA substitutions, UNAAGI achieves a Precision@K of 0.342 on CP2 and 0.364 on PUMA, corresponding to 2.44-fold and 1.43-fold enrichment over random selection, respectively.

These results provide additional evidence that UNAAGI can effectively prioritize promising NCAA substitutions, despite the remaining room for improvement in absolute predictive accuracy.

2. Coverage of NCAAs in the training set (W3, Q1)

We have updated our manuscript to address the reviewer's concerns regarding the performance of UNAAGI on NCAA chemistries that it has not directly encountered during training.

First, we quantified the occurrence of benchmark NCAAs in our training data. The results are reported in Table 3 of the revised manuscript and reproduced below.

BenchmarkFormulaIsolated NCAA — recordsNCAA in PDB — records
2NPC13NO—11
2THC7NOS——
3THC7NOS——
ABUC4NO10286
AHPC7NO2——
AIBC4NO—756
AOCC8NO—2
BZTC11NOS——
CHAC9NO—68
CPAC8NO—1
HSMC5NO210—
MEAC4NO—61
MEBC5NO10—
MEFC10NO—61
MEGC3NO—145
NLEC6NO10204
NVAC5NO1041
TBUC7NO——
TMEC5NO2——
YMEC10NO2—5
Covered5 / 20 (50)12 / 20 (1,641)

As shown in Table 3, six benchmark NCAAs, namely 2TH, 3TH, AHP, BZT, TBU, and TME, have no matching molecular graphs in either of our NCAA-specific training sources.

Importantly, Figure D2 of the revised manuscript shows that UNAAGI can generate NCAA structures that have never appeared in these training sources. This demonstrates that UNAAGI is not restricted to reproducing amino acid identities encountered during training.

Furthermore, Figure D4 reports the predictive correlations for individual amino acid identities across different positions. The results show that UNAAGI can achieve meaningful predictive correlations for some NCAA identities absent from the training data, providing further evidence of generalization to previously unseen chemistry.

3. Concerns regarding stereochemistry (W4)

We acknowledge that stereochemical discrimination is a limitation of our current framework. Although our E(3)-equivariant architecture generates three-dimensional molecular structures, it does not explicitly enforce L-stereochemistry or guarantee reliable discrimination between enantiomers.

Nevertheless, we investigated the proportions of D- and L-amino acids among the generated samples at each position of our NCAA benchmarks.

We have added the results as Figure E8, together with a brief analysis in Appendix E.5 of the revised manuscript.

Interestingly, we observe that the vast majority of stereochemically determinate samples are L-amino acids, with pooled D-amino-acid fractions of only 6.8% on CP2 and 3.4% on PUMA. This suggests that UNAAGI empirically captures stereochemical preferences from the training data and surrounding structural environment, even without explicitly enforcing chirality.

We also observe that positions occupied by glycine in the wild-type sequence have higher fractions of D-amino-acid samples, potentially reflecting weaker stereochemical preferences in their local environments.

We acknowledge, however, that this empirical preference does not establish reliable stereoisomer discrimination, which remains a limitation of the current framework.

AuthorsOpenReview ↗

Response to Reviewer buKU (2/2)

4. Structural overlap between training and evaluation (Q2)

We have analyzed the structural overlap between our training data and the NCAA DMS benchmarks.

Specifically, we searched for related receptor chains using Foldseek in TM-align mode (TM-score ≥ 0.5) and BLOSUM62-based sequence alignments (sequence identity ≥ 30%).

Tracing hits from either search to PDB entries contributing to our NCAA-containing structural training data identified four KDM4A-related entries and 13 MCL-1-related entries.

The table below reports the corresponding sequence identities and structural similarity scores.

ReceptorPDB IDSequence identity (BLOSUM62 alignment)Foldseek TM-score
CP2 (KDM4A, 5LY1)2OX0100%1.00
2OQ6100%0.99
5FPA67%0.87
5FP967%0.87
PUMA (MCL-1, 2ROC)5W8F89%0.83
6UA388%0.87
5C3F88%0.86
6VBX86%0.82
4G3585%0.76
5W8985%0.79
5VAX35%0.76
9O1435%0.74
9O1635%0.74
9O1533%0.77
4YK929%0.74
5VWZ26%0.78
5VX026%0.78

The zero-shot evaluation remains valid in the presence of receptor overlap.

Our benchmarks evaluate whether UNAAGI can predict the effects of substitutions in held-out peptides that bind to known receptors.

The assayed peptides and their exact benchmark complexes, 5LY1 and 2ROC, were absent from training, and UNAAGI was trained without experimental mutational-effect labels (Appendix F).

Therefore, our evaluation measures zero-shot prediction of peptide mutational effects in structurally familiar receptor environments, rather than generalization to entirely unseen receptor structures.

5. Controlled comparison with residue-level prediction models (W1)

Finally, the reviewer raises the concern that we lack a controlled comparison with residue-level prediction methods. This point is similar to that made by EZdD about an extended vocabulary approach. We agree that this is a potential alternative in theory – but we wish to stress that in practice there is simply not enough training data exists to train such a method. There are many experimentally relevant NCAAs that do not appear a single time in the available structural training data (the PDB).

If we were to train a conventional residue-level predictor with an extended output vocabulary, these unseen NCAA classes would receive no positive class-specific supervision during training. Consequently, simply extending the output vocabulary would not provide a mechanism for learning meaningful context-dependent probabilities for these previously unseen amino acid identities.

This ability to transfer knowledge across related chemical structures is precisely one of the central motivations for our approach, and we apologize that this point was not clearer in our original submission. We now include a table showing (lack of) overlap between the training source (PDB) and the experimental test scenario (Table 3) and updated Section F.1 of the revised manuscript to clarify this point.

Our work does not claim that atom-level diffusion is the only viable approach to NCAA modeling. Rather, we demonstrate that it provides an effective route beyond fixed residue vocabularies, with experimentally validated predictive performance that substantially improves upon the existing methods evaluated in our study.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.