acceptodds
Under review as a conference paper at ICLR 2027

Bridging Structure to Sequence via Local Geometries for Protein Inverse Folding

Abstract

Inverse folding—designing amino acid sequences for a given protein structure—is a core task in protein engineering. Recently, masked diffusion protein language models (PLMs) have shown great potential, typically decoding residues in a confidence-first order. While decoding order has drawn growing attention in diffusion large language models (dLLMs), it has remained largely unexplored in protein language models. In this work, we evaluate decoding orders through a unified sequence–structure pass@K metric, assessing a model's ability to generate sequences that are both native-like and designable. Unlike in natural language dLLMs, where decoding order reshapes sampling coverage, we find that in inverse folding it leaves coverage unchanged but steers a trade-off: confidence-first decoding yields highly native-like but poorly designable sequences. Entropy analyses trace this to premature commitment: high-uncertainty positions are deferred and resolved by sequence statistics rather than structural geometry. We show the trade-off can be addressed at two levels. At inference time, a local-geometry guided decoding order resolves structurally prioritized regions first while sampling freely within them. At training time, upweighting high-uncertainty positions during fine-tuning biases the model toward structural constraints at exactly those positions. On CATH, the resulting model improves designability and sequence recovery simultaneously over its confidence-decoded counterpart, with consistent gains on the out-of-distribution TS50 and TS500 benchmarks and strong performance on motif-scaffolding. This plug-and-play framework enables both inference-time control and training-time enhancement.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Protein language model, Protein sequence design, Inference

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_9eUTOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper investigates the effect of decoding order in masked-diffusion models for protein inverse folding. The authors compare several decoding strategies, including confidence-first, autoregressive, random, and secondary-structure-guided decoding, under a controlled sampling budget. They observe that confidence-first decoding tends to favor sequence recovery, whereas structure-guided decoding achieves a better balance between sequence similarity and structural designability. The paper further analyzes high-entropy “fork” positions, where prediction uncertainty and sequence mismatches are more concentrated. Based on these observations, the authors propose two interventions: a structure-guided decoding strategy at inference time and an entropy-reweighted training objective that assigns larger weights to uncertain positions. These components are integrated into Hier-IF and evaluated on CATH, TS50/TS500, and motif-scaffolding benchmarks.

Critical strengths
  1. The problem is interesting and relevant. Decoding order is an important degree of freedom in masked-diffusion models, while its role in protein inverse folding has received relatively limited attention.

  2. The paper is generally well organized and easy to follow. The overall presentation follows a clear progression from empirical observation to analysis of uncertain positions. The motivation and main experimental findings are presented in a reasonably manner.

Critical weaknesses
  1. The analysis is relatively shallow and the proposed methods are mostly heuristic. My main concern is that the paper does not reach the expected depth of an ICLR contribution. It identifies an interesting empirical phenomenon that decoding order matters, but the subsequent analysis does not lead to a sufficiently general or principled understanding of this phenomenon. The proposed structure-guided order is manually designed, while the fork analysis mainly shows correlation between uncertainty and prediction errors without establishing its claimed connection to foldability. I think the paper would benefit from a deeper analysis of these observations, rather than mainly comparing several heuristic decoding strategies.

  2. The robustness of the decoding-order analysis is not sufficiently established. The dual pass@K metric relies on fixed recovery and scTM thresholds, but these choices are not well justified and no sensitivity analysis is provided. It is unclear whether the reported trade-off is robust to different thresholds.

Questions for authors
  1. How robust is the decoding-order conclusion to the threshold choices in dual pass@K? The main analysis relies on fixed thresholds for sequence recovery and scTM. Could the authors provide sensitivity analysis to show that the reported advantage of structure-guided decoding is not specific to the selected thresholds?
  2. How does the fork analysis explain the effectiveness of different decoding orders? Section 3.3 shows that high-entropy positions contribute more prediction errors, but the connection to the decoding-order results in Section 3.2 is not very clear. For example, does SS-first decoding reduce the prediction entropy or errors at these fork positions compared with confidence-first decoding? Also, are these fork positions mainly located in loop regions, which are decoded later under SS-first? I think such analysis would help explain why the proposed decoding order works, rather than only showing the performance differences between several decoding strategies.
1 comment
AuthorsOpenReview ↗

Response to Reviewer 9eUT

Thank you for the review. We add fork diagnostics, a label-shuffle control, and threshold-sensitivity tests.

W1 / Q2 — Heuristic methods and limited analysis

The ss-first policy was motivated by a context-building hypothesis: resolving helices/strands first may provide useful sequence context for uncertain loop positions. We do not claim a universally optimal order.

Forks are the top quartile by reveal-time predictive entropy under a fixed AR reference. Table 1 reports 42.2% of entropy mass and 40.1% of all reference mismatches in these positions (1.7×/1.6× enrichment). The latter is a share of all mismatches, not a within-fork mismatch rate. Predictive entropy measures residue uncertainty, not conformational entropy; reference mismatches do not imply structural failure.

Fixed-fork diagnostics. We compare the same AR-defined positions using temperature-0.6 trajectories with T=L steps (L=chain length). Diagnostics use coordinate-valid positions. Entropy uses renormalized probabilities over the 20 standard amino acids; reference NLL is the reference residue's negative log-probability. Both use temperature 1 and natural logarithms (nats). Relative reveal step is fill step/L; larger means later.

OrderPredictive entropyReference NLLMismatch (%)Relative reveal step
AR2.28412.093785.460.6643
Conf2.32062.242683.070.6518
Rand2.21802.128380.330.7045
Prior2.22971.978577.920.7318

Loops comprise 63.54% of forks versus 38.36% of all positions (approximately 1.66× enrichment). Forks are predominantly in loops, but not confined to them. Prior reveals the same forks later than Conf (0.7318 versus 0.6518), with lower entropy, NLL, and mismatch rate. These observations are consistent with the context-building rationale, without establishing that entropy reduction causes improved foldability.

Label-shuffle control. Within each chain, we preserve the sizes of the early and late groups but randomly reassign positions to them, breaking their correspondence with true secondary structure. The same group-priority decoding rule is then applied. Shuffling gives scTM 0.6740 and structure pass@128 67.34%, versus 0.7718 and 76.56% for Prior: gains of 0.0978 scTM and 9.22 percentage points for true grouping. Here scTM compares ESMFold predictions with input target backbones; structure pass@128 requires at least one candidate with scTM≥0.8.

Together, the fork diagnostics and label-shuffle control provide a clearer empirical understanding of how structural organization and decoding context influence prediction at uncertain positions. The gains in scTM and structure pass@128 over the group-size-preserving label-shuffle control support the value of aligning generation priority with true secondary structure for improving predicted foldability. We will emphasize these structural insights and designability gains alongside sequence-level performance in the revised manuscript.

W2 / Q1 — Robustness to threshold choices

Recovery (AAR) is the reference-residue match fraction. Dual pass@K requires at least one candidate to satisfy both thresholds on the same sequence. Recovery≥0.44 uses the ProteinMPNN AAR reference level; scTM≥0.8 is an operational structural cutoff. Neither is a universal validity boundary.

We add sensitivity tests using existing pools with five seeds, temperature 0.6, and T=L. These are additional controls, separate from reproducing the manuscript's original curves. Below we report dual pass@128 (%).

Varying recovery at scTM≥0.8:

Recovery thresholdARConfRandPrior
0.3073.4472.6672.4173.44
0.4068.7569.5369.5369.75
0.5057.0361.2861.7263.72
0.6040.4743.3141.2545.47

Prior ranks first or jointly first at every displayed recovery threshold, including the stricter 0.50 and 0.60 cutoffs.

Varying scTM at Recovery≥0.44:

scTM thresholdARConfRandPrior
0.5075.0075.7875.7876.00
0.6073.4473.2275.7874.44
0.7071.8871.8872.6672.66
0.7570.3169.0970.3171.53
0.8067.1966.9767.1967.41
0.8563.2864.0664.8465.06
0.9056.2560.1658.5961.38

Prior ranks first or jointly first at six of seven structural thresholds; Rand leads at 0.60. Margins vary and are small at the original cutoff pair: Prior exceeds AR/Rand by 0.22 percentage points. These point estimates do not establish statistical significance or universal dominance.

The full sweep covers K=1,2,4,8,16,32,64,128. We will report the full sweep and complementary structure-only success, making the dependence on decoder, budget, and evaluation criteria explicit.

Reviewer_rF5HOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the impact of decoding order on masked diffusion models for protein inverse folding, aiming to improve both sequence recovery and structural designability. The authors first conduct a systematic comparison of different decoding strategies and introduce a dual sequence–structure pass@K metric to jointly evaluate sequence similarity and structural consistency. Their analysis reveals that confidence-first decoding favors native-sequence recovery but can compromise designability, particularly at high-uncertainty positions. To address this issue, they propose Hier-IF, a structure-conditioned masked diffusion framework that combines local-geometry-guided decoding with online entropy-reweighted fine-tuning to prioritize structurally informative regions and improve predictions at uncertain positions. Experiments on CATH 4.2/4.3 and the out-of-distribution TS50/TS500 benchmarks demonstrate competitive or superior sequence recovery and perplexity compared with existing inverse-folding methods. Hier-IF also achieves a TM-score of 0.85 in computational folding validation and solves 19 out of 24 motif-scaffolding tasks, outperforming the evaluated baselines. Ablation studies further suggest that structure-guided decoding and entropy-based training provide complementary improvements. Overall, the work highlights decoding order and uncertainty-aware optimization as effective mechanisms for improving diffusion-based protein inverse folding.

Critical strengths

The paper systematically investigates how different decoding strategies affect masked diffusion models for protein inverse folding. The finding that confidence-first decoding may compromise structural designability provides an interesting perspective on protein sequence generation.

Critical weaknesses

Overemphasis on sequence recovery as an evaluation metric. The paper relies heavily on sequence recovery to demonstrate improvements in protein inverse folding. However, given the one-to-many nature of the inverse folding problem, sequence recovery is no longer a sufficiently informative metric for evaluating protein design quality. A designed sequence can differ substantially from the native sequence while still folding correctly into the target structure. I strongly recommend adopting the evaluation setting of ProtInvTree (NeurIPS 2025), which emphasizes structural self-consistency (e.g., sc-TM and sc-RMSD) and evaluates designability rather than native-sequence similarity. In particular, the authors should investigate whether the proposed decoding-order strategies and entropy-weighted training consistently improve structural designability under this setting. This would provide more convincing evidence of the practical value of the proposed approach.

Questions for authors

See Weaknesses.

1 comment
AuthorsOpenReview ↗

Response to Reviewer rF5H

Thank you for the review and the recommendation to strengthen structural-designability evaluation.

W1 / Questions — Overemphasis on sequence recovery as an evaluation metric

We agree that inverse folding is one-to-many and recovery alone is insufficient to assess design quality. We evaluate decoding order and entropy-weighted training under the structural self-consistency setting recommended by the reviewer, following ProtInvTree.

Evaluation protocol. This evaluates freely generated sequences, rather than reference-conditioned reconstruction. Following Appendix B of ProtInvTree, we compute scTM and scRMSD between ESMFold predictions of generated and reference sequences under identical folding settings. Diversity is the mean normalized pairwise Hamming distance among generated sequences, measured over full sequences. Novelty is 1−Recovery against the reference sequence, evaluated over valid target-coordinate positions. We include all candidates and weight targets equally.

Decoding-order comparison. We compare AR, Conf, Rand, and Prior using the same fork-weighted checkpoint and CATH 4.3 test chains, with five seeds, temperature 0.6, and K=128 candidates per chain. All orders use T=L decoding steps, start fully masked, and keep filled positions fixed.

OrderscTM ↑scRMSD (Å) ↓Diversity (%) ↑Novelty (%) ↑
AR0.7912.40548.8546.34
Conf0.7922.13442.6642.66
Rand0.7741.87249.2245.94
Prior0.8011.98850.7141.02

Prior achieves the highest mean scTM and diversity (50.71%), but the lowest novelty. Random achieves the lowest scRMSD. Compared with Conf, Prior improves both structural metrics and diversity, while producing sequences closer to the reference. Thus, it does not dominate all orders across every metric.

Matched training comparison. We compare unweighted continuation, which uses unit position weights, with Full, which uses fork weighting. Both continue from the same Base checkpoint for 6,000 matched updates and are evaluated with Conf and Prior under the same sampling and self-consistency protocol above.

ConfigurationscTM ↑scRMSD (Å) ↓Diversity (%) ↑Novelty (%) ↑
Unweighted continuation + Conf0.75353.29647.9950.99
Unweighted continuation + Prior0.76062.12848.5348.53
Full + Conf0.79202.13442.6642.66
Full + Prior0.80101.98850.7141.02

Under both orders, fork weighting increases mean scTM and reduces scRMSD. Novelty decreases, indicating higher recovery. Its effect on diversity depends on the order: diversity decreases with Conf (47.99% → 42.66%) but increases with Prior (48.53% → 50.71%). Within each training configuration, Prior improves both structural metrics and diversity relative to Conf, with lower novelty.

These results provide structural evidence for the benefits of the proposed interventions beyond recovery alone, while identifying trade-offs involving diversity and reference-sequence novelty. We will report structural consistency, recovery, diversity, and novelty jointly in the revised manuscript.

Reviewer_DPubOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies decoding order in structure-conditioned masked diffusion models for protein inverse folding. Built on a frozen DPLM 650M backbone, Hier-IF combines secondary-structure-guided decoding with online entropy-weighted fine-tuning. The paper evaluates joint sequence recovery and predicted structural consistency using dual pass@K, and reports results on CATH, TS50/TS500, and motif scaffolding. Its main empirical finding is that secondary-structure-guided ordering improves dual pass@K under the irreversible-commitment sampler, while the full configuration improves several reported sequence-design metrics.

Critical strengths
  1. A useful empirical study of decoding order and joint sequence–structure quality. Comparing orders at a fixed denoiser-evaluation budget, across two structure encoders, provides evidence that decoding policy matters for inverse folding. Evaluating recovery together with predicted structural consistency is more informative than recovery alone, and the ss-first advantage under ancestral decoding persists through the evaluated sampling budgets (Figs. 4 and 7).

  2. Simple interventions with promising benchmark results. The ordering policy requires no model retraining, and entropy weighting reuses the training forward pass. The four-way component ablation reports complementary gains, while the full model achieves 66.42% recovery on TS500 versus 64.30% for LM-Design (Fig. 6; Table 3). These results make the approach potentially useful if the attribution and evaluation issues below are resolved.

Critical weaknesses
  1. The evidence does not sufficiently isolate the proposed mechanism or the source of the training gains. Table 1 establishes an association between uncertainty and native-sequence mismatches, rather than a causal effect on foldability; Appendix E.2 also leaves agreement between online and reveal-time entropy unmeasured. Figure 6 provides a useful component ablation, but Sec. 3.4 starts weighted fine-tuning from an existing checkpoint without clearly identifying an unweighted continuation control with the same additional updates. Moreover, Eq. (10) can increase the overall decoder-loss scale relative to the auxiliary head loss. Without a matched training comparison accounting for these factors, the improvements cannot be confidently attributed to preferentially training the proposed fork positions.

  2. Key evaluation protocols are insufficiently specified to interpret central results. Figure 6 reports a perplexity improvement from 4.09 to 4.03 for an inference-only change, but does not define whether perplexity is measured on fixed reference contexts or order-dependent generated contexts. The distinction matters for the conditional-likelihood claim and comparisons with other models. In addition, Sec. 4.3 describes sequence-only motif scaffolding without explaining how the structure-conditioned model handles unavailable scaffold coordinates, including whether its structural components are disabled or supplied with additional information. These are substantive ambiguities about what is being evaluated and whether comparisons use equivalent information, rather than merely missing implementation details.

Questions for authors
  1. Attribution: Does entropy weighting still improve recovery and structural consistency over unweighted continuation from the same checkpoint, with matched training updates and controlled decoder-loss scale? What measured evidence links its online entropy signal to the reveal-time fork positions?

  2. Evaluation: What exact perplexity estimator and conditioning contexts are used in Fig. 6 and the benchmark tables? For motif scaffolding, what inputs and model components are active when scaffold coordinates are unavailable, and are the conditioning information and sampling budgets matched to the baselines?

1 comment
AuthorsOpenReview ↗

Response to Reviewer DPub

Thank you for highlighting these attribution questions.

W1 / Q1 — Attribution of the training gains and evidence for the entropy-based mechanism

We agree that the error enrichment in Table 1 alone does not establish a causal effect on foldability. We therefore add matched training controls and direct measurements of the entropy signal.

Matched continuation controls. We compare three 6,000-step continuations from the same Base checkpoint, with matched training data, baseline loss formulation, and optimizer schedules. All are evaluated with the same inference policy. Unweighted continuation uses unit position weights. Full divides each masked position’s online entropy by the within-chain masked-position mean and clips the ratio to [1, 3]. Mean-normalized Full divides these weights by their mean, preserving relative emphasis with mean weight 1.

These controls use reference-conditioned reconstruction. Structural metrics compare ESMFold predictions of reconstructed sequences with input target backbones; structure pass@128 requires at least one candidate with scTM≥0.8.

Training configurationRecovery ↑scTM ↑scRMSD (Å) ↓Structure pass@128 ↑
Unweighted continuation0.540.82021.9750.81
Mean-normalized Full0.590.84021.4740.84
Full0.600.85011.3600.85

Full and mean-normalized Full improve all four metrics over unweighted continuation. The normalized variant retains most observed gains, supporting relative position weighting beyond additional training steps or uniform weight scaling. These controls complement Fig. 6.

Agreement between online and reveal-time entropy. We compare position-wise online entropy, averaged over 128 training-style masking draws per chain, with reveal-time entropy measured immediately before each position is filled along Base’s fixed greedy left-to-right AR trajectory. For Full, the mean within-chain Spearman rank correlation is 0.897, and the top-quartile overlap is 72.2%, compared with a random expectation of 25.3%. This provides direct evidence that the online uncertainty signal aligns with the reveal-time uncertainty used to identify fork positions.

Predictive uncertainty after training. We evaluate all checkpoints on identical AR contexts from Base and fix the fork set to Base’s highest-entropy quartile. Predictive entropy is computed over the 20 standard amino acids after renormalizing their probabilities; higher values indicate greater predictive uncertainty. Reference NLL is the negative log-probability assigned to the reference residue; lower values indicate better reference-residue prediction. Both metrics use natural logarithms (nats) and are evaluated at temperature 1. Reference NLL and training weights use the full vocabulary. We will correct the vocabulary description in Appendix C.3 accordingly.

CheckpointMean predictive entropy ↓Predictive entropy at fixed forks ↓Reference NLL ↓
Base1.53622.42631.9785
Unweighted continuation1.48852.26211.8370
Full1.42562.14941.7216

Both continuations lower entropy and reference NLL relative to Base. Full further reduces fixed-fork entropy (2.2621 → 2.1494) and reference NLL (1.8370 → 1.7216) over unweighted continuation. These show reduced fork uncertainty and improved reference prediction under shared contexts. We will add these controls and clarify their scope.

W2 / Q2 — Clarification of evaluation protocols

Perplexity. We apologize for the ambiguity in the original description. All PPL results score the reference sequence, following the reference-sequence scoring convention of ProteinMPNN and PiFold. PPL is exp(mean residue-level NLL). For our model, the scoring order determines which reference residues are visible as context. Changing this order therefore changes the conditioning contexts even with a fixed checkpoint, explaining the inference-only PPL difference in Fig. 6. We will clarify the estimator and contexts. W1’s reference NLL is a separate diagnostic on shared Base-generated contexts.

Motif scaffolding. We use a separately fine-tuned sequence-only variant; the frozen-backbone setting in Sec. 3.1 applies to inverse folding. Motif residues are fixed, other positions start masked, and the structural encoder/adapter are not used. No coordinates enter generation; reference motif geometry is used only for evaluation. Following DPLM-2’s sequence-based evaluation, ESMFold predictions are assessed using overall pLDDT >70 and aligned motif RMSD <1 Å. We use 24 tasks and 100 designs/task, with benchmark length ranges and motif-segment order. The DPLM/EvoDiff reference results use the same sequence-only conditioning, candidate counts, and success criteria. We will clarify these task-specific settings in Sec. 4.3 and Fig. 5.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.