- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper investigates the effect of decoding order in masked-diffusion models for protein inverse folding. The authors compare several decoding strategies, including confidence-first, autoregressive, random, and secondary-structure-guided decoding, under a controlled sampling budget. They observe that confidence-first decoding tends to favor sequence recovery, whereas structure-guided decoding achieves a better balance between sequence similarity and structural designability. The paper further analyzes high-entropy “fork” positions, where prediction uncertainty and sequence mismatches are more concentrated. Based on these observations, the authors propose two interventions: a structure-guided decoding strategy at inference time and an entropy-reweighted training objective that assigns larger weights to uncertain positions. These components are integrated into Hier-IF and evaluated on CATH, TS50/TS500, and motif-scaffolding benchmarks.
Critical strengths
-
The problem is interesting and relevant. Decoding order is an important degree of freedom in masked-diffusion models, while its role in protein inverse folding has received relatively limited attention.
-
The paper is generally well organized and easy to follow. The overall presentation follows a clear progression from empirical observation to analysis of uncertain positions. The motivation and main experimental findings are presented in a reasonably manner.
Critical weaknesses
-
The analysis is relatively shallow and the proposed methods are mostly heuristic. My main concern is that the paper does not reach the expected depth of an ICLR contribution. It identifies an interesting empirical phenomenon that decoding order matters, but the subsequent analysis does not lead to a sufficiently general or principled understanding of this phenomenon. The proposed structure-guided order is manually designed, while the fork analysis mainly shows correlation between uncertainty and prediction errors without establishing its claimed connection to foldability. I think the paper would benefit from a deeper analysis of these observations, rather than mainly comparing several heuristic decoding strategies.
-
The robustness of the decoding-order analysis is not sufficiently established. The dual pass@K metric relies on fixed recovery and scTM thresholds, but these choices are not well justified and no sensitivity analysis is provided. It is unclear whether the reported trade-off is robust to different thresholds.
Questions for authors
- How robust is the decoding-order conclusion to the threshold choices in dual pass@K? The main analysis relies on fixed thresholds for sequence recovery and scTM. Could the authors provide sensitivity analysis to show that the reported advantage of structure-guided decoding is not specific to the selected thresholds?
- How does the fork analysis explain the effectiveness of different decoding orders? Section 3.3 shows that high-entropy positions contribute more prediction errors, but the connection to the decoding-order results in Section 3.2 is not very clear. For example, does SS-first decoding reduce the prediction entropy or errors at these fork positions compared with confidence-first decoding? Also, are these fork positions mainly located in loop regions, which are decoded later under SS-first? I think such analysis would help explain why the proposed decoding order works, rather than only showing the performance differences between several decoding strategies.
Response to Reviewer 9eUT
Thank you for the review. We add fork diagnostics, a label-shuffle control, and threshold-sensitivity tests.
W1 / Q2 — Heuristic methods and limited analysis
The ss-first policy was motivated by a context-building hypothesis: resolving helices/strands first may provide useful sequence context for uncertain loop positions. We do not claim a universally optimal order.
Forks are the top quartile by reveal-time predictive entropy under a fixed AR reference. Table 1 reports 42.2% of entropy mass and 40.1% of all reference mismatches in these positions (1.7×/1.6× enrichment). The latter is a share of all mismatches, not a within-fork mismatch rate. Predictive entropy measures residue uncertainty, not conformational entropy; reference mismatches do not imply structural failure.
Fixed-fork diagnostics. We compare the same AR-defined positions using temperature-0.6 trajectories with T=L steps (L=chain length). Diagnostics use coordinate-valid positions. Entropy uses renormalized probabilities over the 20 standard amino acids; reference NLL is the reference residue's negative log-probability. Both use temperature 1 and natural logarithms (nats). Relative reveal step is fill step/L; larger means later.
Loops comprise 63.54% of forks versus 38.36% of all positions (approximately 1.66× enrichment). Forks are predominantly in loops, but not confined to them. Prior reveals the same forks later than Conf (0.7318 versus 0.6518), with lower entropy, NLL, and mismatch rate. These observations are consistent with the context-building rationale, without establishing that entropy reduction causes improved foldability.
Label-shuffle control. Within each chain, we preserve the sizes of the early and late groups but randomly reassign positions to them, breaking their correspondence with true secondary structure. The same group-priority decoding rule is then applied. Shuffling gives scTM 0.6740 and structure pass@128 67.34%, versus 0.7718 and 76.56% for Prior: gains of 0.0978 scTM and 9.22 percentage points for true grouping. Here scTM compares ESMFold predictions with input target backbones; structure pass@128 requires at least one candidate with scTM≥0.8.
Together, the fork diagnostics and label-shuffle control provide a clearer empirical understanding of how structural organization and decoding context influence prediction at uncertain positions. The gains in scTM and structure pass@128 over the group-size-preserving label-shuffle control support the value of aligning generation priority with true secondary structure for improving predicted foldability. We will emphasize these structural insights and designability gains alongside sequence-level performance in the revised manuscript.
W2 / Q1 — Robustness to threshold choices
Recovery (AAR) is the reference-residue match fraction. Dual pass@K requires at least one candidate to satisfy both thresholds on the same sequence. Recovery≥0.44 uses the ProteinMPNN AAR reference level; scTM≥0.8 is an operational structural cutoff. Neither is a universal validity boundary.
We add sensitivity tests using existing pools with five seeds, temperature 0.6, and T=L. These are additional controls, separate from reproducing the manuscript's original curves. Below we report dual pass@128 (%).
Varying recovery at scTM≥0.8:
Prior ranks first or jointly first at every displayed recovery threshold, including the stricter 0.50 and 0.60 cutoffs.
Varying scTM at Recovery≥0.44:
Prior ranks first or jointly first at six of seven structural thresholds; Rand leads at 0.60. Margins vary and are small at the original cutoff pair: Prior exceeds AR/Rand by 0.22 percentage points. These point estimates do not establish statistical significance or universal dominance.
The full sweep covers K=1,2,4,8,16,32,64,128. We will report the full sweep and complementary structure-only success, making the dependence on decoder, budget, and evaluation criteria explicit.