acceptodds
Withdrawn from ICLR 2027

Retention Is Not Preservation: Budget-Matched Evaluation of Isofunctional Protein Rewriting

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Generative protein models are increasingly capable of controllable editing—such as shortening enzymes for viral delivery or diversifying sequences to evade antibodies—while preserving biological function. We introduce ISOFORM, a conditional generative model for constrained protein editing that jointly optimizes target length, sequence identity, and explicit functional constraints. ISOFORM achieves state-of-the-art performance, successfully retaining nearly all annotated functional motifs during severe length compression and identity-controlled divergence. However, through extensive experiments across our model and existing state-of-the-art systems, we uncover a fundamental dilemma hidden by aggregate evaluation metrics: a severe decoupling between functional motif retention and 3D structural preservation. Even when achieving SOTA retention of functional residues, current models yield significantly lower global structural similarity and severely distort the 3D spatial geometry of the retained active sites. Because aggregate scores mask this ubiquitous structure-function decoupling, we advocate for decoupled, structure-aware evaluation protocols. Our findings highlight the necessity of evaluating 3D motif geometry alongside sequence retention, establishing a rigorous foundation for controllable protein design.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Generative protein models, Isofunctional protein rewriting, Conditional edit model, Functional site retention

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_GQt2OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors investigate isofunctional protein rewriting, which modifies a protein sequence through compression, expansion, or paraphrasing while attempting to preserve its function. A non-autoregressive editing model, ISOFORM, is introduced and evaluated with operation-specific budgets and a set of computational proxies. The main finding is that retaining annotated functional sites does not necessarily imply preservation of the protein's structural organization or function. In addition, the paper reports that the composite rankings are sensitive to metric weights. The authors acknowledge that the evaluation relies on computational proxies rather than biochemical measurements.

Critical strengths
  1. The paper raises an evaluation question and provides an experimental framing. The comparison framework attempts to hold the editing budget fixed across methods and includes feasibility checks.
  2. The experimental results are reported, and several findings are discussed.
Critical weaknesses
  1. The current experimental results can not establish that ISOFORM reliably preserves function or consistently outperforms baselines. The primary compression comparison is mixed, the aggregate result is not statistically significant, and the evaluation depends heavily on proxy metrics whose relationship to biochemical function remains uncertain.
  2. The aggregate score utilize fixed weights, and the paper reports that changing these weights can reverse the ranking, which means the comparison depends on a choice of evaluation priorities that may not have a uniquely justified answer. It would be better to supplement the composite with a Pareto analysis, sensitivity plots over a broader range of weights, and a clear justification for normalization and weighting.
  3. The experimental findings demonstrate a multi-objective trade-off. Since the test set only have 40 held-out proteins, it important to describe the diversity.
Questions for authors

Please refer to the weaknesses.

Reviewer_nV54OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies protein design tasks where we are looking for a new sequence of a specified length that preserves the function of the original protein. In silico, function preservation can be predicted with a number of proxies such as TM score and annotated functional site preservation. The authors train a Levenshtein Transformer model to suggest edits to protein sequences, and observe that different proxies for function preservation can disagree.

Critical strengths

None

Critical weaknesses
  • It is not really clear what the main contribution from this paper should be. The insight about different functional preservation metrics disagreeing is not particularly surprising/new - this is why previous works (SCISOR, Raygun, etc.) validate multiple proxies. While the IsoForm method sounds interesting, the method does not get much attention and many implementation details are left vague.
Questions for authors
  • Are the comparisons against SCISOR and Raygun fair? For IsoForm, you use an oracle to select the top candidate from 8 sampled sequences, but the baselines don't seem to be afforded this step. Also, the noise and temperature parameters in these baselines may have a significant effect on the quality and diversity of generated sequeences, so I would have liked to see a more detailed evaluation of these.

  • What exactly is the training objective for IsoForm (base and oracle-stage steps)? You wrote "Base training runs for 200k steps followed by 20k oracle-stage steps; the rebalanced model uses the same 20k oracle-stage budget." The loss in (2) seems to expensive to be using for the full training run, as it involves structure prediction and alignment.

Reviewer_iu3ZOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces an evaluation protocol for protein rewriting at a target length or sequence identity, together with the conditional edit model ISOFORM. On 40 proteins compressed to 60% of their original length, ISOFORM retains more annotated sites than a contiguous wild type fragment at the expense of a slightly lower pTM-score.

Critical strengths
  1. Reporting site retention, structure and the weighted score separately reveals a useful difference between these measures. The paper clearly acknowledges the limits of predicted structures, shared predictors and the fragment control.
  2. The task is timely and practically relevant.
  3. The primary comparison uses paired proteins and reports uncertainty.
Critical weaknesses
  1. The reproducibility statement claims a release that is not provided. It states, “The release includes the evaluation CSV, run-log joins, and figure scripts,” but no release files or release link are provided with the submission. The manuscript also incompletely specifies the motif/property formulas and calibration, site-retention checks, fragment selection rule, and TM-score normalization. This is particularly important because the evaluation protocol is the paper’s central contribution.

  2. The main comparison has limited scope. The fragment loses 40% of annotated sites at 0.6L and is correctly described as a structural reference. The comparison shows a difference between the reported measures, but does not establish whether that difference also holds against a control that preserves the annotated sites.

  3. The site-geometry analysis is difficult to assess. Appendix E describes 31 pooled records without protein or budget identifiers. Matching the number of site pairs does not establish that the same pairs were compared.

Questions for authors

Does the retention/structure difference also hold against one simple deletion control that preserves the annotated sites, on proteins where this is possible at 0.6L?

Can the authors provide protein, budget and site-pair identifiers, exclusion counts, and the calculation used to combine the geometry records?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.