acceptodds
Withdrawn from ICLR 2027

Physio-DPO: Aligning Protein Language Models with Local Energy Gaps to Mitigate Structural Hallucinations

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Protein language models generate sequences that score well under their own likelihood yet fold into structures an empirical energy function rates as unstable. Direct Preference Optimization cannot separate such a case from a marginal one, because a binary label discards how large the violation is. We propose Physio-DPO, which recovers that magnitude. For a backbone-matched pair consisting of a native structure and a physics-perturbed decoy of the same fold, a thresholded and bounded function of the local energy gap scales the DPO update, so severe stereochemical violations drive larger updates than ambiguous ones. Pairs of unrelated folds keep the ordinary binary objective, so no energy magnitude is ever compared across folds. We build PhysioPref-1M, one million preference pairs that combine generation-mined preferences with physics-perturbed hard negatives, and verify the labels by blinded expert review. Across three seeds Physio-DPO reaches an sc-RMSD of  \AA and a foldability of , ahead of SFT, PPO, DPO, and the strongest energy-aware baseline, and it keeps that lead under folding and scoring oracles unseen during training. The gains reach experimental data as well: on ProteinGym deep mutational scanning assays Physio-DPO raises the average Spearman correlation from to .

Primary area:
reinforcement learning
Keywords:
LLM, LLM agents, DPO, Structural Hallucinations, Protein Energy

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_EH1TOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Physio-DPO, an energy-aware post-training method for autoregressive protein language models. Starting from ProGen2-XL, the method constructs preference data using predicted structures and empirical energy scores, and modifies DPO by weighting physically comparable preference pairs according to their local energy gaps. The paper reports improvements over standard DPO and several energy-aware baselines on structural metrics, cross-oracle evaluation, and ProteinGym fitness prediction.

Critical strengths
  1. Physio-DPO incorporates local energy-gap magnitude only for structurally comparable pairs, avoiding direct energy comparisons across unrelated folds.

  2. Table 2 shows substantial improvements over standard DPO and energy-aware baselines, and the paper further evaluates generalization with unseen computational oracles and ProteinGym.

Critical weaknesses
  1. Appendix D constructs hard negatives through side-chain repacking, backbone torsion perturbations, and chirality inversion, which appear to modify structure rather than sequence. Since Eqs. 4-5 optimize sequence likelihoods, it is unclear how these perturbations produce distinct ywy_w and yly_l. If the sequence is unchanged.

  2. Eq. 6 still contains σ(−βΔθ)\sigma(-\beta\Delta_\theta), which decays as the preference becomes confidently satisfied. Therefore, the claim in Section 4.3 that the gate supplies gradient independently of model confidence appears inconsistent with the stated objective.

Questions for authors

How are the PyRosetta-perturbed structures in DphysD_{\rm phys} converted into distinct sequence-level ywy_w and yly_l required by the DPO objective?

Reviewer_WmMYOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes aligning protein language models with local energy gaps to mitigate structural hallucinations. It also introduces PhysioPref-1M, a dataset comprising one million preference pairs, to achieve the alignment.

Critical strengths
  1. The paper points out a significant problem for protein language models.
  2. The idea to utilize structural preferences for the alignment is reasonable.
Critical weaknesses

The fundamental error in the paper makes it difficult to assess the correctness and validity of the proposed work.

  1. The paper exists fundamental inconsistency. It denotes yy as structure (e.g., line 143) but also sequence (e.g., line 661) in the paper. Thus, it is unclear whether the preference pairs are constructed in terms of structure or sequence.
  2. There already exists a related work [1], which also aligns with the protein stability objective and contributes to a large amount of preference dataset. The paper misses it and thus does not well position the contribution of the work.
Questions for authors

See the weakness.

Reviewer_iXd9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduced Physio-DPO, which is a preference optimization method to align PLMs with structural energy information. The authors construct PhysioPref-1M, a dataset of preference pairs combining generation mined examples and physics perturbed structural decoys. The proposed algorithm applies a sigmoid weight based on the energy gap between the preferred and dispreferred structures for only backbone matched pairs. The experiments showed improvements over baselines in predicted structural quality and predicted foldability, and zero shot fitness prediction on ProteinGym.

Critical strengths

The author built a novel dataset PhysioPref-1M. It incorporates physical information into sequence level preference learning through negatives with high predicted folding confidence but unfavorable energy. This large scaled dataset with the reported expert validations is a useful contribution to the community.

Critical weaknesses
  1. As the paper is currently written, there seems to be an inconsistency between the construction of physics perturbed pairs and sequence level DPO training. The authors generate physics perturbed pairs from the same native structure, which would modify atomic coordinates but do not change the amino acid sequence. However, the DPO objective in Eq 4 operates on sequence likelihoods. If the preferred and dispreferred structures have identical sequences, their log likelihood ratio difference is zero and the corresponding DPO gradient is zero regardless of the energy gap. The paper does not explain how these structural decoys are converted into distinct sequence level preference pairs. Was any inverse folding models like ProteinMPNN used?

  2. The evaluation metrics are not convincing for showing improved physical quality. The main metrics rely heavily on structure prediction models (scRMSD an pLDDT). The refolding evaluation evaluates whether the predicted structure can be recovered after sequence redesign, rather than directly evaluating the original PLM generated sequence. The refolding evaluation is more suited to evaluating generated backbones than generated sequences. I am concerned that, since the preference labels are also constructed using predicted structures, the improvements are reflecting alignment with the biases of the folding oracles. In addition, the ProteinGym experiments evaluate correlations between model likelihoods and existing experimental fitness measurements. However, fitness in ProteinGym can reflect biophysical stability or other functional properties, depending on the assay. Could the authors clarify which ProteinGym assays were used and whether the observed improvements reflect better physical stability rather than other aspects of protein function?

Questions for authors

See Critical Weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.