acceptodds
Under review as a conference paper at ICLR 2027

HiGenIF: Hierarchical Structural Context Modeling for Protein Inverse Folding

Abstract

Protein inverse folding aims to design amino-acid sequences compatible with a target backbone. Protein function emerges from coordinated structural environments spanning local, regional, and protein-wide scales. However, regional relationships often remain implicit in residue-level representations, leaving the connection between structural organization and position-specific biochemical preferences insufficiently explicit. We propose HiGenIF, which explicitly organizes protein-wide, regional, and residue-level structural context to predict position-specific amino-acid preferences. The resulting complete sequence draft and continuous structural states jointly condition ESM2-650M, which incorporates structural information at an intermediate layer and predicts residual corrections to the initial logits. On the CATH 4.2 topology split, HiGenIF achieves 59.37% amino-acid recovery and 3.40 perplexity across 1,120 test chains, establishing the highest recovery within our focused comparison of inverse-folding methods that incorporate pretrained sequence knowledge. Controlled experiments further demonstrate the effectiveness of explicit structural-region modeling and pretrained sequence refinement.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Protein inverse folding, Structure-conditioned protein design, Hierarchical generative modeling, Geometric deep learning, Masked sequence modeling

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_bMkkOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents HiGenIF, a protein inverse folding framework that combines hierarchical structural representations with pretrained sequence modeling. The structural predictor uses backbone geometry and DSSP-defined regions to aggregate information at residue, regional, and protein-wide levels, producing an initial amino acid sequence and continuous structural features. ESM2-650M then takes the predicted sequence as input and incorporates the structural features through an intermediate-layer interface to refine the initial predictions. The authors evaluate the method on CATH 4.2, TS50, and TS500, reporting amino acid recovery and perplexity. Additional experiments examine the contributions of regional modeling and sequence refinement, as well as the combination of HiGenIF with external generators such as DPLM-2 Bit and ADFLIP.

Critical strengths
  1. The paper evaluates HiGenIF on CATH 4.2, TS50, and TS500, reporting amino acid recovery and perplexity under the respective evaluation protocols. The results provide evidence of the method's performance across several established inverse folding benchmarks.

  2. The authors include controlled experiments examining regional structural modeling and pretrained sequence refinement. The reported improvements of 4.13 and 7.48 percentage points in amino acid recovery, respectively, help establish the contribution of these components within the proposed framework.

Critical weaknesses
  1. The methodological novelty is limited. Most of the proposed framework consists of established techniques combined into a larger pipeline. The structural encoder uses an EGNN [1], while the DSSP-based grouping and subsequent regional message passing follow familiar hierarchical graph modeling ideas. Related approaches to multiscale protein representations and secondary-structure-based graph pooling have already been explored in SurfPro [2] and BioPhysPool [3]. The sequence refinement stage also closely resembles existing work. LM-Design [4] already combines structural representations with pretrained protein language models for inverse folding. SPURS [5] uses cross-attention to inject structure-derived features into intermediate ESM2 layers, which is particularly close to the proposed structural injection mechanism, although the downstream task differs. I find it difficult to identify a sufficiently distinct algorithmic contribution beyond the particular combination of these components. The authors should explain more clearly how their regional hierarchy differs from existing hierarchical protein representations and why their refinement mechanism is meaningfully different from previous structure-conditioned language model adapters.

  2. The claimed structure guided refinement is not strongly supported by the ablation results. The full model improves over the structure only predictor by 7.48 points, but removing the continuous structural state injection while keeping the same predicted draft and ESM2 reduces AAR by only 0.153 points. This suggests that most of the refinement gain comes from applying a large pretrained sequence model to the predicted sequence draft, while the proposed structural interface contributes only marginally. Since this interface is presented as one of the main methodological contributions, I find its practical role insufficiently established. The reliance on ESM2 also makes it important to clarify how much of the reported transfer performance may benefit from sequence knowledge acquired during large scale pretraining.

  3. The current regional ablation does not isolate the benefit of the proposed DSSP based hierarchy. The main control replaces all DSSP regions with a single group, which simultaneously removes meaningful between group communication, reduces the number of hierarchy tokens, and substantially changes the information pathways of the model. The resulting 4.13 point improvement therefore shows that multiple regional units are useful, but does not establish that the proposed structural partition itself is responsible for the gain. A capacity matched comparison against alternative partitions with a similar number and size of groups would be necessary to support the central claim of explicit structural region modeling.

[1] Satorras, V. G., Hoogeboom, E., & Welling, M. (2021). E(n) Equivariant Graph Neural Networks. ICML, 9323–9332.

[2] Song, Z., Huang, T., Li, L., & Jin, W. (2024). SurfPro: Functional Protein Design Based on Continuous Surface. ICML, 46074–46088.

[3] Gu, Y., Zhao, S., Liu, L., & Ding, Y. (2026). A Biophysical Prior-Based Hierarchical Graph Pooling Strategy for Protein Function Prediction. IEEE Transactions on Computational Biology and Bioinformatics, 23(4), 1813–1826.

[4] Zheng, Z., Deng, Y., Xue, D., Zhou, Y., Ye, F., & Gu, Q. (2023). Structure-informed Language Models Are Protein Designers. ICML, 42317–42338.

[5] Li, Z., & Luo, Y. (2026). Generalizable and scalable protein stability prediction with rewired protein generative models. Nature Communications, 17, 891.

Questions for authors
  1. The no-injection control reduces AAR by only 0.153 points while the full ESM2 refinement improves much more over the structure-only model. How do the authors interpret this gap? To what extent is the refinement gain coming from the predicted sequence draft and pretrained ESM2 rather than from the proposed continuous structural interface?

  2. The main regional control compares DSSP-defined regions with a single group for the whole chain. Since this also changes the number of group tokens and removes between-group communication, what evidence supports the claim that the gain specifically comes from the DSSP-based structural partition rather than simply introducing multiple intermediate groups?

  3. Several components of HiGenIF resemble existing hierarchical protein representations and structure-conditioned language-model refinement methods. Could the authors clarify what they consider to be the main algorithmic novelty beyond combining these existing ingredients, and how it differs technically from the closest prior approaches?

Reviewer_JKW3OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes HiGenIF, a two-stage protein inverse-folding framework. A structure-conditioned sequence predictor partitions chains into contiguous DSSP-defined regions, combines global, regional, and residue-level context, and conditions amino-acid prediction on six biochemical classes to produce a sequence draft and continuous structural states. ESM2-650M then processes the predicted draft, receives structural states at an intermediate layer, and predicts residual corrections to the initial logits. Experiments report AAR and perplexity on CATH 4.2, TS50, and TS500, distribution fusion with external generators, and controls for regional modeling and sequence refinement.

Critical strengths

Several training and evaluation procedures are clearly documented. The paper distinguishes checkpoint selection, fusion-parameter selection, and locked test evaluation. Refinement consistently uses predicted drafts rather than native sequence inputs, making the reported procedure easier to audit.

The complete system reports strong sequence-prediction results and includes useful controlled comparisons. CATH 4.2 AAR is 59.37%; the regional comparison improves from 54.95% to 59.08%, and adding ESM2 refinement improves from 51.89% to 59.37%. These results support the utility of components within the system, although they do not isolate DSSP semantics or establish structural design capability.

Critical weaknesses
  1. The motivation and incremental contribution are not adequately validated: the ablation shows that grouping may help, not why DSSP-defined structural grouping helps. The main architectural contribution is an explicit global–regional–residue hierarchy. However, Table 5 compares multiple DSSP regions with a single group per chain, simultaneously changing group count, pooling scale, and within-/between-group information flow. The 4.13-percentage-point gain therefore cannot be attributed specifically to meaningful secondary-structure organization. Controls such as random contiguous partitions or fixed windows with comparable group counts, length distributions, and compute are needed to connect the biological motivation to the improvement. Moreover, Table 8 shows that continuous-state injection adds only 0.153 percentage points of AAR when the predicted draft and initial logits are retained. This statistically significant but small gain indicates that most of the refinement benefit remains available without this injection, limiting the demonstrated incremental value of the particular interface. The paper needs to distinguish the contributions of structural-region semantics, generic multiscale computation, and pretrained sequence refinement. Although no-ESM2 and single-group controls are provided, the study lacks comparisons with established or simpler structural predictors receiving the same ESM2 refinement procedure. The incremental value over a generic PLM-refinement pipeline therefore remains insufficiently established. Testing another compatible pretrained protein language model under a comparable adaptation protocol would also clarify whether the benefit depends on ESM2.

  2. The evaluation does not establish structural design capability, and substantive issues remain in benchmark comparability and inference behavior. The design-quality evaluation reports sequence-based metrics, chiefly AAR/PPL, without refolding-based structural consistency or design-success evaluation. Consequently, it is unclear whether refinement and fusion improve compatibility with the target backbone rather than only native-sequence prediction. This concern does not require wet-lab validation; computational structural evaluation is the immediate missing evidence. In addition, Appendix A.4 defines residue-pooled AAR, whereas A.9 explicitly imports FAMPNN’s median chain recovery and retains literature evaluation protocols. Directly juxtaposing these numbers does not establish superiority; aggregation must be harmonized or comparisons clearly separated. Appendix A.2 further states that structural outputs depend on the other proteins in the preprocessing batch and change when chains are regrouped. Preserving cached cohorts reproduces one configuration but does not establish reliable single-chain or arbitrary-batch inference. Together, these limitations leave the main performance and practical-value claims insufficiently supported.

Questions for authors
  1. Structural evaluation: The current design-quality evaluation primarily relies on AAR/PPL. Could you provide refolding-based structural consistency and design-success results for both the structural stage and the refined model? This would directly establish whether improvements in sequence metrics translate into more reliable compatibility with the target backbone.

  2. Effectiveness and interpretability of DSSP grouping: The current regional-versus-single-group comparison does not isolate DSSP semantics from generic grouping. Could you include a non-DSSP partition with comparable group counts, length distributions, and computational budget? Analyzing gains by secondary-structure type, region boundary, or cross-region contact would also help explain where and how DSSP grouping provides benefits.

  3. EGNN pipeline and stage-level performance: Could you present a clearer pipeline from the backbone through EGNN inputs, hierarchical representations, and the draft to ESM2 refinement, specifying node/edge features, graph construction, normalization, and feature updates? The paper already reports some stage-level results; consolidating them into a common ablation table with corresponding structural metrics would make the contributions of the EGNN, hierarchical modeling, and refinement easier to assess.

Reviewer_N1m7OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents HiGenIF, a hierarchical structural modeling framework for protein inverse folding. It integrates global, regional, and residue-level structural information to improve amino acid sequence prediction from a given protein backbone.Continuous secondary-structure regions are identified using DSSP. Hierarchical message passing is then used to model interactions both within and across these regions. The model also predicts six categories of amino acid physicochemical properties to guide initial sequence generation.HiGenIF further incorporates the pretrained protein language model ESM2-650M. After layer 16, continuous structural features are introduced through cross-attention and positional alignment to refine the initial sequence. Experiments show that HiGenIF achieves a 59.37% amino acid recovery rate on CATH 4.2 and demonstrates generalization on the TS50 and TS500 datasets.

Critical strengths
  1. HiGenIF introduces a hierarchical structural modeling framework that explicitly integrates global, regional, and residue-level protein structure. Message passing within and between structural regions enables more effective modeling of multiscale structural relationships.
  2. The framework effectively incorporates a pretrained protein language model. By injecting continuous structural features into ESM2-650M and refining initial amino acid predictions through residual correction, the sequence recovery rate improves from 51.89% to 59.37%, highlighting the value of pretrained sequence knowledge for inverse folding.
  3. HiGenIF achieves strong sequence prediction performance and generalization. It reaches an amino acid recovery rate of 59.37% on CATH 4.2, as well as 64.83% and 67.96% on TS50 and TS500, respectively. Ablation studies confirm the contribution of the key components. DSSP-based regional structural modeling improves amino acid recovery by 4.13 percentage points, while ESM2-based sequence refinement provides a further 7.48-point gain. Injecting continuous structural features also improves performance, supporting the effectiveness of each module.
Critical weaknesses
  1. The comparison with state-of-the-art methods is incomplete. Although HiGenIF is evaluated against ProteinMPNN, PiFold, LM-Design, and Bridge-IF, it is not compared with other competitive inverse-folding methods, such as KW-Design, MapDiff, and DualMPNN. This makes it difficult to fully assess its advantage over current state-of-the-art approaches.
  2. The framework diagram is not sufficiently clear or concise. Figure 1 contains too much text and too many modules, making the information flow difficult to follow. Simplifying the figure and emphasizing the core components and their relationships would improve readability.
  3. The ablation study of key components is insufficient. The paper does not provide an independent ablation of the six amino acid physicochemical property prediction modules. In addition, Table 8 shows that continuous structural feature injection improves overall AAR by only 0.153 percentage points. Additional experiments should evaluate the effect of removing the physicochemical property prediction module, varying the number of ESM2 layers receiving structural information, and comparing alternative fusion strategies.
Questions for authors
  1. Effectiveness of key components: Could the authors provide an independent ablation study of the six amino acid physicochemical property prediction modules? In addition, why are structural features injected at layer 16 of ESM2? How do different injection layers and fusion strategies affect performance?
  2. The robustness of the ensemble results to hyperparameter choices remains unclear. In Table 3, combining HiGenIF with DPLM-2 Bit achieves an AAR of 68.15%, but this result depends on the temperature, fusion weights, and voting weights used for sequence generation. Although these parameters were tuned only on the validation set, the paper does not provide a hyperparameter sensitivity analysis or comparisons with simpler ensemble strategies. Reporting results across different parameter settings would help clarify whether the gain mainly comes from model complementarity or from careful tuning of the fusion parameters.
  3. Comparison with state-of-the-art methods: Could the authors compare HiGenIF with advanced inverse-folding methods such as KW-Design, MapDiff, and DualMPNN, and add experiments on the CATH 4.3 dataset to further evaluate its competitiveness and generalization ability?
  4. The framework diagram could be clearer. Figure 1 contains too much text and detail, resulting in a complex layout that makes the main components and information flow difficult to follow. Simplifying the text, improving the layout, and emphasizing the key design elements and overall workflow would improve readability.
  5. The ablation study on structural region partitioning is incomplete. The paper only compares DSSP-based regions with a single-region baseline, without a control using randomly assigned groups. Adding a random-grouping ablation would help determine whether the performance gain truly comes from secondary-structure information rather than simply introducing a grouping mechanism.
  6. The model’s ability to design multichain proteins has not been evaluated. The study focuses primarily on single-chain inverse folding and does not include experiments on protein complexes. Incorporating inter-chain interactions and evaluating the model on multichain inverse-folding tasks would help assess its applicability and generalization to more complex protein structures.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.