acceptodds
Under review as a conference paper at ICLR 2027

PepSAGE: Hierarchical All-Atom Peptide Generation with Continuous Local-Geometry States

Abstract

Peptides can engage protein surfaces that are difficult to target with small molecules, making pocket-conditioned peptide design a valuable yet challenging therapeutic goal. Generating such peptides at all-atom resolution requires the joint modeling of global backbone pose, amino-acid identity, and residue-internal atomic conformation. Existing representations face a persistent tension between expressive local detail and geometric validity: Cartesian coordinates expose rigid-body redundancy, torsion variables are chemically grounded but less convenient as a uniform learned interface across residue types, and discrete structural tokens may discard fine conformational variation. We introduce a geometry-state generation formulation for pocket-conditioned all-atom peptide design, instantiated as PepSAGE (Peptide Structure-Aware Generation). Rather than directly generating unconstrained atom coordinates, relying on residue-specific torsions, or discretizing local structure into tokens, PepSAGE represents residue-internal conformation with a shared six-dimensional continuous state (), while modeling residue pose and amino-acid identity through separate channels. This state is learned through a reconstruction interface that is frozen during generation, allowing PepSAGE to generate continuous pre-quantization local geometry and recover all-atom structures through a geometry-constrained reconstruction pathway. Under a unified training budget and leakage-controlled evaluation, PepSAGE achieves the best backbone, all-atom, and side-chain RMSD, the lowest clash, and the highest rotamer recovery among competitive baselines, with additional analyses showing a tunable quality–diversity trade-off. Controlled ablations show that the continuous local-geometry channel drives side-chain recovery, while geometry-constrained reconstruction improves geometric validity, supporting continuous local geometry as an effective internal state for structured all-atom peptide generation.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Deep Generative Models, Biomolecular Design, Representation Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ArPVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposed PepSAGE target at pocket-conditioned all-atom peptide design. In this method, peptide residues are decoupled into translations, rotations, amino sequence and a 6-dimentional continuous local geometry states. FSQ and Mamba are used in auto encoder, and BFN is used in generative model. And PepSAGE outperformed PepFlow, PepGLAD and PepBFN in RMSD, atom collision rate and recovery rate of dihedral angles of side chains in 120-target strict filtered test set.

Critical strengths
  1. In traditional all-atom representation of peptide, inconsistent dimensions of side chain dihedrals remains as a problem, PepSAGE resolved this problem by encoding it into a 6-dimensional continuous state, which significantly reduced atomic clashes without introducing complex auxiliary loss functions.
  2. Data leakage exists in previous benchmarks. Test set is strictly filtered in this work and all models are re-trained in same data pipeline and computational budget. The experiment results have high reliability and reproducibility with 10000 pairwise bootstrap statistical tests.
Critical weaknesses
  1. The method claims that generating a continuous state before FSQ can preserve information lost from discrete tokens, but generated tokens are rounded before decoder refers to S6, S7 and S18. Meanwhile, decoder received information from main network hidden representation. It is hard to discern causal relationships regarding the source of continuous details.
  2. The paper relies on metrics with rmsd or Rotamer recovery rate compared to conformation in dataset, inhibiting the generation of diverse peptides. The paper also lacks physical functions like rosetta metrics, or co-folding metrics like scRMSD.
Questions for authors
  1. How does continuous conformational information transmitted to the final atomic coordinates in S7? If we cut input of h_i^L (only use D_\psi(c_i) to decode), does the advantage of "generating continuous pre-quantization state" regarding "generating discrete tokens" still exists?
  2. In real-world medical chemistry scenarios, specific interactions of short peptides are often severely broken at the 6.0 \AA all-atom RMSD level, can the authors provide more metrics like rosetta or co-folding metrics.
  3. In tab.1, in side-chain RMSD, PepBFN is 8.509, PepSAGE reaches 6.900. While in tab.S5, -2.61 [-2.98, -2.24] vs PepBFN is filled (I think it should be -1.61), how would you explain this numerical gap? Furthermore, in tab.s6, three loss function components weights are non-zero, but only two are mentioned in S8, which one is actually used in downstream training?
2 comments

We thank the reviewer for the careful reading. We conducted targeted controls and record audits to clarify the continuous state, FSQ, hidden representation, and the reported numerical details.

  1. Continuous state, FSQ, and hiLh_i^L. The BFN generates the continuous pre-quantization state z^i\hat z_i. During denoising, this state contributes to the backbone representation hiLh_i^L; at the endpoint, z^i\hat z_i is mapped by the frozen FSQ projection and a linear map into geometry code cic_i. The coordinate head receives [hiL;ci][h_i^L;c_i]. Thus, FSQ is not bypassed: it is the shared, constrained local-geometry decoding interface, while continuous generation affects both the pre-quantization state and its backbone context. The main model and post-quant control both use FSQ before coordinate decoding. They differ in the generation target: the main model generates continuous pre-quantization states, while post-quant directly generates coarsened quantized states. Accordingly, continuous generation improves χ1\chi_1 recovery from 0.294 to 0.383, χ1:2\chi_{1:2} recovery from 0.105 to 0.213, and BB-SC RMSD from 5.206 Å to 5.050 Å. We further trained a strict code_only control: hiLh_i^L is zeroed at the coordinate head and decoding uses only Dψ(ci)D_\psi(c_i). Continuous and discrete-token generators used the same data, budget, 120 targets, 64 samples per target, and 100 steps. Continuous generation obtains fixed Cα\alpha RMSD of 5.197 Å versus 7.191 Å for discrete tokens, paired difference −1.994-1.994 Å, 95% CI [−2.441,−1.537][-2.441,-1.537]. Fixed-backbone RMSD is 5.129 Å versus 7.163 Å, difference −2.035-2.035 Å, 95% CI [−2.482,−1.586][-2.482,-1.586]. Therefore, continuous pre-quantization generation remains significantly better even without direct hiLh_i^L input to the coordinate head. Matched code_only, hidden_only, and both models obtain χ1:2\chi_{1:2} recoveries of 2.21%, 3.97%, and 4.76%, respectively. hidden_only improves rotamer recovery over code_only by 3.15 percentage points, 95% CI [2.00,4.28][2.00,4.28]. These controls support complementary roles for backbone context and FSQ-derived local geometry. We will rename the original “z=0z=0” control as “w/o FSQ-code input,” since it removes the code entering the coordinate head while retaining ziz_i in the BFN backbone.
  2. Biophysical and co-folding evaluation. The 6.060 Å all-atom RMSD measures end-to-end deviation from the native peptide, including sequence, backbone, pocket placement, and side-chain errors; it is not a direct interface-quality measure. Current results include contact recovery, clash score, bond geometry, BB-SC RMSD, and rotamer recovery. PepSAGE obtains BB-SC RMSD of 5.050 Å and joint χ1\chi_1/χ1:2\chi_{1:2} recovery of 0.438/0.305 on sequence-matched residues. We will add Rosetta interface scores and complex co-folding/self-consistency evaluation, alongside the existing structural and diversity metrics.
  3. PepBFN values and tokenizer losses. Thank you for identifying the inconsistency. Table 1 contains transcription errors for PepBFN: correct Backbone, All-atom, and Side-chain RMSDs are 6.319, 8.683, and 9.510 Å. Hence, 6.900−9.510=−2.6106.900-9.510=-2.610 Å, consistent with Table S5’s −2.61[−2.98,−2.24]-2.61[-2.98,-2.24]. We will correct the table and release target-level summaries. The tokenizer checkpoint uses local and global reconstruction losses with weights 1.0 and 1.0; distance, bond, angle, and clash auxiliary weights are zero. The nonzero distance weight in Table S6 came from a different default configuration. Eq. S8 correctly reflects the actual two-loss training objective. We will correct Table S6 and provide checkpoint hashes and complete hyperparameters.

Thank you for your candid response, the table clarifications, and the new code_only ablation. The added experiments helpfully demonstrate the benefit of continuous pre-quantization states over discrete tokens, while also showing the substantial role of hiLh_i^L; accordingly, refining the naming and discussion around the original (z=0) control will certainly improve clarity in the revised text.

My primary remaining reservation is that biophysical and interface evaluations (such as Rosetta interface scores and complex co-folding self-consistency), which are important for assessing binding quality around 6 Å RMSD, are currently planned for a future revision rather than available during the rebuttal. Since these results and other promised analyses cannot yet be evaluated, I will maintain my current rating for now. Incorporating them will certainly make the manuscript much stronger.

Reviewer_7nP6OpenReview ↗
Rating
3(+1)
Confidence
2(−1)
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents PepSAGE, a hierarchical framework for peptide generation. The key idea is to represent residue-internal conformation with a shared six-dimensional continuous local-geometry state, whose space is learned beforehand by a coordinate-only reconstruction interface with finite scalar quantization and then frozen. Bayesian Flow Network (BFN) jointly generates residue translations, rotations, amino-acid identities, and the continuous pre-quantization value of this state; the generated state is then passed through the frozen FSQ projection and a trainable atom-coordinate head, and converted into atom14 coordinates via torsion extraction and rigid-group reconstruction. Evaluated compared to the three retrained baselines, PepSAGE reports the best RMSDs, the lowest clash, and the highest rotamer recovery, while trailing on diversity and novelty.

Critical strengths
  1. Well-motivated and meaningful representation design. The paper addresses an important representation challenge in all-atom peptide generation by introducing a compact, shared continuous local-geometry state. The hierarchical decomposition of residue pose, amino-acid identity, and internal conformation provides a meaningful alternative to explicit torsion-based and unconstrained latent representations.

  2. Strong improvements in side-chain conformation accuracy. PepSAGE demonstrates notable improvements in structural accuracy, particularly in side-chain conformation recovery.

  3. Careful comparison with baselines. The authors report retraining three baselines under a unified training budget and evaluation protocol, with leakage-controlled testing, paired bootstrap confidence intervals, and three-seed robustness analysis.

Critical weaknesses
  1. Insufficient validation of the continuous local-geometry representation. The paper emphasizes generating continuous pre-quantization states, but Eq. S18 shows that these states are quantized through FSQ before entering the atom-coordinate head. Although continuous information may still influence the generation process through hidden representations, its contribution beyond the quantized representation remains unclear. The paper also lacks an evaluation of the frozen geometry interface's reconstruction accuracy before and after quantization, making it difficult to assess the effectiveness of the proposed representation.

  2. Missing diversity-matched accuracy comparisons. PepSAGE achieves lower RMSD than the baselines at its default sampling temperature but exhibits substantially lower diversity. Table S3 indicates that the RMSD advantage becomes smaller at higher sampling temperatures, where diversity approaches that of the baselines. However, the paper does not provide controlled comparisons at matched diversity levels.

Questions for authors
  1. Reconstruction fidelity of the geometry interface. Could the authors report the reconstruction accuracy (e.g., the all-atom and side-chain RMSD) of the frozen geometry interface on ground-truth structures before and after FSQ quantization? This would be helpful to quantify the information loss introduced by the representation bottleneck.

  2. Contribution of continuous-state generation. Since the main model already applies FSQ before coordinate decoding, could the authors clarify how the post-quantization ablation differs from the main model? Additionally, given that the coordinate head receives both Transformer hidden features and quantized geometry states, could the authors provide a controlled ablation to isolate the contribution of the continuous local-geometry state?

  3. Diversity-matched evaluation. Could the authors provide accuracy comparisons at matched diversity levels, along with paired bootstrap confidence intervals, to determine whether PepSAGE maintains its structural accuracy advantage under comparable sampling diversity?

  4. Sampling speed. Could the authors report the inference time per peptide? Since PepSAGE requires 100 BFN steps update, an analysis of the trade-off between sampling steps and generation quality would also be helpful.

I would be willing to reconsider my rating if the authors can adequately address these questions.

2 comments
AuthorsOpenReview ↗

Author Response to Reviewer Comments

1. Fidelity of the frozen geometry interface.
We evaluated the frozen coordinate-only atom-level geometry interface on 120 clean ground-truth structures, comparing reconstruction before and after FSQ. Ground-truth local structures are fixed in this test, so it isolates interface fidelity from BFN sampling, sequence prediction, and global backbone errors. Before FSQ, all-atom and side-chain RMSD are 0.0574 Å and 0.0762 Å; after FSQ, they are 0.0814 Å and 0.1104 Å. Thus, quantization changes reconstruction by only 0.0240 Å for all atoms and 0.0342 Å for side chains. The frozen FSQ interface therefore retains high local coordinate fidelity, with post-quantization error remaining near 0.1 Å. It provides a constrained and shared local-geometry decoding space, while PepSAGE generates continuous pre-quantization states to retain continuous generative expressiveness.

2. Continuous pre-quantization states versus post-quant.
Both the main model and post-quant control use frozen FSQ before coordinate reconstruction. Their difference is the generation target: the main model generates a continuous pre-quantization endpoint z^i\hat z_i, which is projected by FSQ only when constructing the geometry feature cic_i; the coordinate head receives [hiL;ci][h_i^L;c_i]. The post-quant control directly generates the coarsened quantized representation.

Accordingly, continuous generation achieves χ1\chi_1 recovery of 0.383 versus 0.294 for post-quant, χ1:2\chi_{1:2} recovery of 0.213 versus 0.105, and BB-SC RMSD of 5.050 Å versus 5.206 Å.

We further trained a strict code_only control, which zeros hiLh_i^L at the coordinate head and decodes only from Dψ(ci)D_\psi(c_i). Continuous and discrete-token models used identical data, budget, 120 targets, 64 samples per target, and 100 steps. Continuous generation achieves fixed Cα\alpha RMSD of 5.197 Å versus 7.191 Å for discrete tokens, paired difference −1.994-1.994 Å, 95% CI [−2.441,−1.537][-2.441,-1.537]. Fixed-backbone RMSD is 5.129 Å versus 7.163 Å, difference −2.035-2.035 Å, 95% CI [−2.482,−1.586][-2.482,-1.586]. Thus, continuous pre-quantization generation remains significantly better even without direct hiLh_i^L input to the coordinate head.

Matched code_only, hidden_only, and both models obtain χ1:2\chi_{1:2} recovery of 2.21%, 3.97%, and 4.76%, respectively. hidden_only improves rotamer recovery over code_only by 3.15 percentage points, 95% CI [2.00,4.28][2.00,4.28]. These results support complementary roles for the backbone context and FSQ-derived geometry code. We will rename the original “z=0z=0” control as “w/o FSQ-code input,” since it zeros the code at the coordinate head while retaining ziz_i in the BFN backbone.

3. Diversity-matched comparison.
PepSAGE reaches diversity 0.519 and 0.546 at temperatures 1.5 and 2.0, close to PepFlow’s default diversity of 0.534; all-atom RMSD is 6.352 Å and 6.751 Å, respectively. We will add temperature sweeps for all methods and compare RMSD, rotamer recovery, and clash rate at matched diversity targets with paired bootstrap confidence intervals.

4. Sampling speed.
We will report inference time per peptide and evaluate step counts below and above the current 100 BFN updates, using the same structural-quality and diversity metrics to characterize the speed-quality trade-off.

The authors' rebuttal addresses my concerns to a certain extent. I have decided to increase my score to 3.

Reviewer_cAcjOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes PepSAGE, a method for generating all-atom peptides conditioned on a protein binding pocket. Rather than directly predicting atom coordinates or side-chain torsion angles, the model represents each residue by its backbone pose, amino acid type, and a compact latent vector learned by a frozen autoencoder. A Bayesian Flow Network generates these variables jointly, and the final all-atom structure is rebuilt using rigid-group templates to keep bond lengths and angles valid. The authors evaluate the model on a filtered subset of PepBDB and Q-BioLiP complexes against retrained peptide-design baselines (PepFlow, PepGLAD, and PepBFN), alongside ablations on the latent state and reconstruction step.

Critical strengths
  • The paper addresses a well-motivated problem in pocket-conditioned all-atom peptide generation, where representing variable-sized side chains without inducing steric clashes or bond distortions is a known challenge.

  • The framework also jointly generates peptide backbone pose, sequence identity, and all-atom side-chain conformation in a single generative model instead of requiring separate inverse-folding and side-chain packing steps.

  • The manuscript is generally well-organized and easy to read, with a clear high-level overview of the pipeline.

Critical weaknesses

I find that the experimental section of PepSAGE has major flaws and is missing key experiments and baselines:

  • Missing details and unfair training budget for retrained baselines: In section 4.2 the authors simply say that “We retrain all three methods on the same processed dataset using the same comparison budget”, without providing additional details. Looking at PepGLAD however, the authors mention that they need to do a 2-step training, with the VAE first, followed by the diffusion model as an “informative latent space is necessary for the consecutive training of the diffusion model.” Can the authors clarify the retraining procedure they used for their baselines such as PepGLAD? They also restricted the training to 180 epochs, which seems to be significantly lower than the 500 epochs used to train PepGLAD. This will clearly undermine the performance of this model.

  • Missing details about the training procedure and datasets:

    • Firstly, In section 4.1 the authors mention constructing “the dataset from experimentally resolved peptide–protein complexes collected by PepBDB.” However, they do not mention any further details regarding this, such as the size of their training set. Additionally in the appendix they mention that “The continuous local-geometry interface is trained separately and then frozen for PepSAGE generation.” Again, details are missing about the train set size, or the number of epochs used for training this local-geometry interface.
    • Secondly, in Section 4.1, the paper describe how they use a 'leakage-controlled evaluation' that removes the overlapping PDB chain identifiers from the target pool. However, as described in prior benchmarks, for example PepGLAD, this doesn’t filter homologous pockets. I believe the standard practice is to enforce a receptor sequence-identity cutoff. Can the authors clarify their datasets and filtering?
  • Missing major baselines: One of the main issues in this paper is that unlike prior works, such as PepGLAD and PepFlow, this paper omits general-purpose protein design and structure baselines. For example, PepGLAD contains comparisons against RFdiffusion. To demonstrate the competitiveness of PepSAGE, the authors should demonstrate performance against sequence-structure co-design models, such as BoltzGen and DISCO

  • Omission of biophysical binding, interface, sequence recovery, and co-folding metrics: Although PepSAGE co-designs both AA identities and the and 3D structures, Section 4.3 evaluates designs only on their internal peptide geometry and coordinate RMSD "over matched heavy atoms” (alongside a coarse 10 Å C-alpha contact metric.) First, the paper doesn’t disclose the number of matched heavy atoms which can clearly bias the metrics. Secondly, the paper fails to include all standard interface and sequence metrics reported in PepGLAD and PepFlow, such as Rosetta binding energy (ΔG<0\Delta G < 0), DockQ, and complex co-folding self-consistency. Additionally, Amino Acid Recovery is only included in the appendix and according to Table S3, it is only 20.7% at temperature 1.0. This would significantly affect the matched heavy atom count RMSD that the authors report in the main results. 


  • In table S2, the authors show that a DDPM variant is actually outperforming the model on most all-atom and sidechain metrics. Given this, I am not sure I understand the motivation behind the BFN version the authors choose to present in the main text. Can they clarify?

Questions for authors

See above.

1 comment
AuthorsOpenReview ↗

Author Response to Reviewer Comments

We thank the reviewer for the detailed and constructive assessment. We agree that the training protocol, filtering, evaluation coverage, and reproducibility details require clearer reporting. We re-audited the training records, target-level statistics, and tokenizer checkpoint, and will revise the manuscript accordingly.

Training budget and baselines. The 180-epoch setting was designed as a common-budget comparison, not as the best attainable setting for every baseline. In particular, PepGLAD uses two stages: VAE training followed by diffusion training. We will report the pretraining and main-training stages, update counts, GPU time, validation checkpoint selection, and learning curves for every baseline. We will also add results using each method’s recommended training procedure or convergence criterion. The current results will be described as common-budget results, while fully trained baselines will support the final performance comparison.

Dataset and leakage control. The main generator and tokenizer use the same training-data source. The tokenizer was configured for up to 100 epochs, and the checkpoint used here was saved at epoch 79. The reproduced training index contains 9,853 unique IDs. We will add complete preprocessing, split, tokenizer, generator, and checkpoint-selection details.

Training uses a validation pool of 187 targets; final reporting uses the clean-120 subset obtained with the pre-specified PDB-chain rule. All methods use the same 120 targets. The clean-120 set has zero exact PDB-chain overlap with the training cache. We agree that this rule does not fully control receptor homology. Our additional audit found 16 targets with pocket sequences identical to training examples. We will disclose this result and add receptor/pocket sequence-similarity stratified analyses.

Broader baselines. PepFlow, PepGLAD, and PepBFN were selected because they can be directly retrained under the same pocket-conditioned short-peptide setting. We agree that broader sequence-structure co-design models would strengthen the study. We will assess BoltzGen, DISCO, and related methods, and evaluate compatible methods using their recommended procedures. For methods that cannot directly support the same conditioning or output format, we will state the required adaptation explicitly.

Matched atoms and biophysical evaluation. All-atom and side-chain RMSD are computed over matched heavy atoms after residue-position matching; thus, atom coverage and amino-acid recovery affect their interpretation. PepSAGE has AAR 20.7% at T=1.0T=1.0. We will move AAR to the main table, report matched-atom counts and coverage for every method, and add fixed-atom-set, sequence-matched, and coverage-stratified RMSD analyses.

Current metrics include contact recovery, clash score, bond geometry, side-chain RMSD, and rotamer recovery. We agree that they do not fully characterize interface biophysics. We will add Rosetta interface/binding scores, DockQ or equivalent interface metrics, and complex co-folding/self-consistency evaluation, including success rates. We will correspondingly limit the current claims to pocket-conditioned structural generation and local geometric validity.

BFN versus DDPM. With the same local-geometry interface, DDPM is slightly better on several metrics: all-atom RMSD 5.972 versus 6.060 Å, side-chain RMSD 6.868 versus 6.900 Å, and rotamer recovery 0.178 versus 0.163. This supports that the representation and reconstruction interface can work with multiple continuous generators. BFN was the primary framework for the controlled comparison because of its endpoint prediction and joint-state generation. We agree that it should not be presented as the uniquely best generator, and will elevate the DDPM result accordingly.

Corrections. Table 1 contains a transcription error for PepBFN: the correct Backbone, All-atom, and Side-chain RMSD values are 6.319, 8.683, and 9.510 Å. Therefore, 6.900−9.510=−2.6106.900-9.510=-2.610 Å, consistent with Table S5’s −2.61[−2.98,−2.24]-2.61[-2.98,-2.24]. We will correct the table and release target-level summaries. We also verified that the tokenizer checkpoint used only local and global reconstruction losses, both with weight 1.0; the nonzero distance weight in Table S6 came from another default configuration. We will correct Table S6 and release configurations, checkpoint hashes, filtering lists, scripts, and bootstrap inputs.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.