acceptodds
Withdrawn from ICLR 2027

Rate-Constrained Geometry-Aware Protein Inverse Folding with Synchronized Masking

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Structure-conditioned protein design requires bridging discrete one-dimensional sequence representations and continuous three-dimensional geometry while preserving evolutionary and structural constraints. We propose GeoSeqAlign, a probabilistic inverse-folding framework that aligns sequence and structure through a variational information bottleneck (VIB). To prevent trivial reliance on local cross-modal correspondences, we introduce synchronized structure–sequence corruption, which masks both modalities at the same residue positions. Combined with a bottleneck that balances reconstruction fidelity against latent information capacity, this mechanism encourages the model to capture predictive sequence–structure relationships while reducing sensitivity to geometric perturbations. To integrate geometric constraints with evolutionary priors, GeoSeqAlign injects -invariant pairwise distances and relative orientations as attention biases into a frozen protein language model, enabling geometry-aware sequence generation without coordinate-equivariant updates. Across CATH+ 4.3, TS50, and CASP15/TS45, GeoSeqAlign achieves the highest amino-acid recovery among the evaluated fixed-backbone baselines while tuning only 6.4% of its parameters. Controlled ablations support the complementary contributions of bottleneck regularization and synchronized corruption, with greater benefits under structural distribution shift and backbone perturbations. These results highlight information-constrained cross-modal learning as an effective approach to robust, parameter-efficient protein inverse folding. Source code is available at: https://anonymous.4open.science/r/GeoSeqAlign-17466.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
protein inverse folding, structure-conditioned protein design, variational information bottleneck, structure–sequence alignment, geometric representation learning, protein language models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ixjVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents GeoSeqAlign, a protein inverse folding framework that combines a frozen protein language model with geometry-aware conditioning. The method uses SE(3)-invariant pairwise geometric features to bias attention, synchronized masking of sequence and structure during training, and a variational information bottleneck to regularize the latent representation. An auxiliary geometry reconstruction loss is also used during training, while inference generates sequences from the target backbone in a single PLM decoding pass. The method is evaluated on CATH+ 4.3, TS50, and CASP15/TS45, with additional tests under coordinate noise, structural self-consistency, diversity, and computational cost. The experiments report improvements over the evaluated baselines and include ablations on synchronized masking, KL regularization, and geometric attention.

Critical strengths
  1. The evaluation is fairly broad for an inverse folding paper. In addition to CATH+ 4.3, the authors test zero shot transfer on TS50 and CASP15/TS45, robustness to coordinate noise, structural self consistency, sequence diversity, and computational cost. This makes it easier to judge the method beyond a single in distribution recovery number.

  2. The paper includes reasonably well controlled ablations for the two main training choices. In particular, synchronized masking is compared with independent masking under matched corruption rates, and the KL term is removed while keeping the stochastic encoder unchanged. These controls give useful evidence that both components contribute to the reported robustness gains, even if the proposed explanation for why synchronized masking helps is not fully established.

Critical weaknesses
  1. The motivation for synchronized masking is not fully justified. In fixed-backbone inverse folding, local backbone geometry is legitimate conditioning information rather than a label leak, and it does not uniquely determine the amino acid identity. The theoretical analysis only shows that synchronized masking removes a co-located information channel, while the appendix explicitly notes that this does not imply that the model learns global topology or avoids overfitting. The experiments therefore support synchronized masking as a useful regularizer, but not the stronger shortcut-removal mechanism used to motivate the method.

  2. The empirical comparisons are not fully controlled. GeoSeqAlign is trained on CATH+ 4.3, while several baselines rely on released checkpoints trained on earlier CATH variants or additional external data, so Table 1 does not correspond to a common training protocol. The robustness comparison is also favorable to GeoSeqAlign in a nontrivial way because coordinate noise is already part of its training corruption, whereas the competing checkpoints are not reported to receive matched noise augmentation. This makes it difficult to separate gains from the proposed modeling choices from gains due to the training setup.

  3. The experimental section is broad, but much of it is peripheral to the central claim. ProteinGym mutation ranking, Rosetta energy proxies, diversity analysis, and several folding-oracle diagnostics show that the model has useful downstream behavior, but they do not address the main question of whether synchronized masking actually changes the model from relying on local geometry to using broader structural context. A more direct analysis of local versus non-local evidence would be more informative than adding further auxiliary evaluations.

Questions for authors
  1. The main motivation argues that synchronized masking prevents the model from taking a local geometry shortcut. Since local backbone geometry is valid conditioning information in inverse folding, could the authors clarify what exactly is considered the undesirable shortcut here, and what evidence shows that independently masked models rely less on broader structural context?

  2. In Table 1, GeoSeqAlign is trained on CATH+ 4.3, while several baselines use released checkpoints trained on different CATH versions or additional external data. Could the authors clarify how comparable these training conditions are and whether any representative baselines were retrained on the same CATH+ 4.3 split?

  3. The robustness experiments evaluate Gaussian coordinate noise, which is also used during GeoSeqAlign training. Were any competing models trained or fine-tuned with matched coordinate-noise augmentation? If not, how should the robustness gains be interpreted relative to differences in training augmentation?

  4. The paper includes several recent baselines, but leaves ADFLIP and CoFlow outside the matched evaluation. Could the authors clarify why these methods were not included and whether their omission affects the claimed positioning of GeoSeqAlign among recent inverse-folding models?

Reviewer_EmsgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes GeoSeqAlign, an inverse-folding framework combining invariant geometric conditioning, a frozen protein language model, synchronized structure–sequence masking, and a variational information bottleneck. During training, sequence and geometry are masked at the same residue positions, with Gaussian noise added to visible coordinates. Relative distances and frame orientations provide attention biases; aligned representations parameterize a Gaussian latent that conditions a frozen PLM through prefix adapters. Training combines sequence prediction, auxiliary geometry reconstruction, and a KL penalty. Inference uses complete geometry, an all-mask sequence, and the posterior mean, followed by external folding evaluation. Experiments cover recovery, structural quality, noise robustness, diversity, computational cost, and retrospective mutation evaluation.

Critical strengths
  1. The evaluation addresses several relevant dimensions of design performance. Beyond AAR/PPL, the paper reports refolding-based structural consistency, coordinate-noise robustness, cross-sample diversity, structure-filtered evaluation, and generation cost. It explicitly identifies native-aware Best@8 as a diagnostic rather than a deployable selection rule and acknowledges quality–diversity trade-offs. This breadth is more informative than recovery alone.
  2. Meaningful component controls are provided. Table 6 compares synchronized, independent, and sequence-only corruption while keeping the model and optimization budget fixed, and separately removes KL regularization. Under synchronization, KL adds 1.3 CATH AAR points and 0.06 scTM under 2 Å input-coordinate noise; independent masks with KL yield 40.6 CASP AAR versus 42.4 for synchronization. These reported results support empirical benefits of the training components, even though they do not fully establish the proposed mechanism.
Critical weaknesses
  1. The implemented model is not specified clearly enough to establish its core computational pipeline. Equation (7) uses geometric biases to obtain aligned representations, Eq. (8) constructs a latent from them, and Eq. (9) maps that latent to prefixes for a frozen PLM. It is unclear whether these stages share a PLM, whether geometric biases enter a separate alignment module or particular PLM layers, how Query_N is constructed, and whether prefixes are input tokens or layerwise key/value conditioning. These details define the central computation rather than incidental implementation choices. Figure 1 labels components as “LLM Embedding & Prefix Layers” and “Frozen LLM Backbone,” whereas the main text generally uses PLM, without identifying the protein model or explaining the module boundary. Figure 4 explicitly separates a conceptual joint-output SLM from implemented fixed-backbone inference; the unresolved issue is the identity and connectivity of the PLM actually used, rather than whether joint structure generation was implemented. The frozen PLM and exact checkpoint, structural-encoder architecture, latent/prefix dimensions, and masking configuration are also incompletely specified. On the evaluation side, the construction and post-filtering sizes of CATH+ 4.3 are unclear, while the local paths and descriptive labels in Table 9 do not uniquely identify baseline weights. Consequently, the reported tables are difficult to independently verify, especially given the approximately 0.3-point clean CATH AAR margin over KW-Design.

  2. The overall framework has limited novelty, while the task-specific rationale for its added training mechanisms is insufficiently established. Conditioning a protein language model on structural representations through adapters is already established, for example in LM-Design (Zheng et al., 2023). The paper acknowledges an integration-based contribution, so its incremental value rests on reliable benefits from synchronized masking and the KL bottleneck over established conditioning pipelines, rather than on the structure-to-PLM framework itself. In particular, the rationale for synchronized masking and its training–inference input mismatch need direct validation. Inverse folding uses geometry to predict sequence; co-located geometry carrying sequence information is not inherently a harmful shortcut. Training removes sequence and geometry at the same positions, whereas inference combines complete geometry with an entirely masked sequence. This combination is not directly produced by strictly synchronized masking. The mismatch does not establish failure, but the paper needs to explain why withholding legitimate structural conditioning improves prediction from complete backbones. Appendix A.2.1 proves that an erased channel no longer transmits information, not that the channel is harmful or that erasing it promotes global reasoning. Table 6 already provides synchronized, independent, sequence-only, and KL controls supporting empirical benefits. However, these results do not clearly distinguish specific shortcut suppression from generic corruption regularization and adaptation to missing inputs. Direct motivating diagnostics and matched comparisons with training configurations closer to inference, or alternative corruption couplings, would be informative beyond simply repeating the existing sequence-only control. The interpretation of KL as filtering coordinate noise should likewise be separated from a general regularization benefit. Beyond controls within the custom architecture, comparison with an established structural-conditioning adapter under the same identified PLM and comparable training budget would help contextualize the added mechanisms.

Questions for authors
  1. Implemented model and configuration: Could you identify the actual PLM and checkpoint and clearly show how structural encoding, geometric attention, the latent, prefixes, and sequence outputs are connected? Please specify Query_N, bias-injection sites, and the main dimensions, and explain how Figure 1’s two LLM-labeled modules correspond to the PLM described in the text.

  2. Task-specific rationale for synchronized masking: Table 6 supports an empirical benefit from synchronized masking. However, when complete backbone geometry is legitimate conditioning at inference, why does withholding co-located structural information during training improve final prediction? Can the existing evidence distinguish the proposed shortcut-suppression explanation from general corruption regularization and explain how synchronized missing-view training supports complete-geometry inference?

  3. Core contribution and supporting evidence: Prior work already conditions protein language models on structural representations. Could you clarify the central advance over this established framework and identify the experiments that directly support it? In particular, how do the ablations establish the value of synchronized masking and the information bottleneck relative to a basic structure-conditioned model, beyond showing that removing components from the full system reduces performance?

Reviewer_ykAZOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents an interesting study that introduces the Information Bottleneck (IB) principle into the protein inverse folding task. Specifically, the authors employ a Variational Information Bottleneck (VIB) to filter spatial geometric information extracted from protein backbones, suppressing task-irrelevant noise while preserving information essential for sequence recovery. This strategy aims to reduce interference from irrelevant structural information and improve the model's generalization ability.

Furthermore, the authors propose a Synchronized Structure–Sequence Corruption training strategy, combined with sequence reconstruction and geometry reconstruction objectives, to enhance the consistency between sequence and structural representations. The effectiveness of this strategy is supported by the reported foldability results. Compared with the baseline methods, the proposed GeoSeqAlign achieves certain improvements in sequence recovery and foldability. The ablation studies also suggest that both the Information Bottleneck and Synchronized Structure–Sequence Corruption strategies contribute positively to the model's performance.

However, I have significant concerns regarding the experimental results and the validity of the baseline comparisons, which substantially affect my assessment of the proposed method. I would appreciate a detailed response from the authors to the following issues.

Critical strengths

This paper presents an interesting study that introduces the Information Bottleneck (IB) principle into the protein inverse folding task. Specifically, the authors employ a Variational Information Bottleneck (VIB) to filter spatial geometric information extracted from protein backbones, suppressing task-irrelevant noise while preserving information essential for sequence recovery. This strategy aims to reduce interference from irrelevant structural information and improve the model's generalization ability.

Furthermore, the authors propose a Synchronized Structure–Sequence Corruption training strategy, combined with sequence reconstruction and geometry reconstruction objectives, to enhance the consistency between sequence and structural representations. The effectiveness of this strategy is supported by the reported foldability results. Compared with the baseline methods, the proposed GeoSeqAlign achieves certain improvements in sequence recovery and foldability. The ablation studies also suggest that both the Information Bottleneck and Synchronized Structure–Sequence Corruption strategies contribute positively to the model's performance.

Critical weaknesses
  1. Significant discrepancies between the reported baseline results and those in the original publications on CATH4.3.

Protein inverse folding aims to recover amino acid sequences from the geometric features of protein backbones. Beyond purely structure-based approaches, a considerable number of studies have explored incorporating external knowledge, particularly evolutionary information from pretrained protein language models, to improve inverse folding performance. These methods include LM-Design [1], Bridge-IF [2], KW-Design [3], and VFN-ESM [4]. In addition, MapDiff introduces a pretrained Invariant Point Attention (IPA) module to promote sequence–structure consistency during the reverse generation process. However, the performance of baseline methods reported in GeoSeqAlign differs substantially from the results published in their original papers, as illustrated in the following table.

ModelDatasetRecovery (%)Source
LMDesignCATH4356.80ICML-2023
Bridge-IFCATH4358.93NeurIPS-2025
KW-DesignCATH4360.38ICLR-2024
VFN-ESMCATH4262.67ICLR-2024
MapDiffCATH4360.68NMI-2025

These discrepancies raise serious concerns about the reliability and fairness of the experimental comparisons. More importantly, when compared against the originally reported results of these methods, the 58.145% sequence recovery achieved by GeoSeqAlign on CATH4.3 appears considerably less competitive. Given that GeoSeqAlign also incorporates both structural and evolutionary information, its reported performance does not convincingly demonstrate the practical benefits of introducing the Information Bottleneck into protein inverse folding. I strongly encourage the authors to carefully verify the baseline results and clarify the reasons for these substantial discrepancies. In particular, the authors should explain whether the evaluation protocols, dataset splits, pretrained checkpoints, or inference configurations differ from those adopted in the original publications.

  1. Significant discrepancies in the baseline results on the TS50 benchmark.

Similar concerns arise regarding the experimental results on the TS50 benchmark. The performance of several baseline methods reported in the manuscript differs substantially from the results presented in their original publications, as shown below. The corresponding results can be found in the original publications of VFN [4] and GRADE-IF [5].

ModelDatasetRecovery (%)
ProteinMPNNTS5054.43
GRADE-IFTS5056.32
PiFoldTS5058.72
MapDiffTS5068.76
VFNTS5059.54

These inconsistencies further undermine confidence in the experimental comparisons. I strongly recommend that the authors re-examine the reported baseline performance and provide a convincing explanation for the observed discrepancies. More importantly, based on the results reported in the existing literature, the sequence recovery of 58.145% on CATH4.3 and 54.5% on TS50 does not convincingly establish the practical effectiveness of the Information Bottleneck strategy, particularly when both structural and evolutionary information are incorporated. Although the ablation studies indicate that the proposed components contribute to performance improvements within the proposed framework, such improvements alone are insufficient to establish the competitiveness and practical value of the approach relative to existing methods. Therefore, I believe that a more rigorous and fair comparison with previously published methods is essential to substantiate the claimed advantages of GeoSeqAlign.

References:

[1] Zheng Z, Deng Y, Xue D, et al. Structure-informed language models are protein designers[C]//International conference on machine learning. PMLR, 2023: 42317-42338.

[2] Zhu Y, Wu J, Li Q, et al. Bridge-if: Learning inverse protein folding with markov bridges[J]. Advances in Neural Information Processing Systems, 2024, 37: 39901-39922.

[3] Gao Z, Tan C, Chen X, et al. KW-design: Pushing the limit of protein design via knowledge refinement[C]//International Conference on Learning Representations. 2024, 2024: 34556-34574.

[4] Zhu M, Sun Z, Shen S, et al. De novo protein design using geometric vector field networks[C]//International Conference on Learning Representations. 2024, 2024: 14120-14143.

[5] Yi K, Zhou B, Shen Y, et al. Graph denoising diffusion for inverse protein folding[J]. Advances in Neural Information Processing Systems, 2023, 36: 10238-10257.

Questions for authors

While the idea of incorporating an Information Bottleneck into protein inverse folding is interesting, and the proposed training strategies demonstrate some benefits in the ablation studies, the substantial discrepancies in baseline performance raise serious concerns about the reliability of the experimental evaluation. In particular, the reported sequence recovery results do not convincingly demonstrate the practical advantages of GeoSeqAlign over existing approaches that also leverage structural and evolutionary information. I strongly encourage the authors to revisit the baseline comparisons, provide a detailed explanation for the discrepancies, and ensure that all methods are evaluated under consistent and fair experimental settings. Addressing these issues is crucial for establishing the validity of the empirical conclusions and the practical significance of the proposed method.

  1. Potential overlap between the results in Table 2 and Table 4. Some experimental results presented in Table 2 appear to overlap with those in Table 4. The authors should clarify the distinct purposes of these two tables and consider consolidating redundant results to improve the clarity of the experimental presentation.

  2. Inconsistency between the framework illustration and the implementation. The current framework figure does not clearly illustrate the logical relationships and information flow among the VIB, pretrained protein language model (PLM), and Geometric Adapter modules. More importantly, there appear to be substantial discrepancies between the architecture illustrated in the manuscript and the actual implementation in the released code. I encourage the authors to carefully revise the framework figure to accurately reflect the implementation and explicitly clarify how these components interact during training and inference.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.