- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper tackles protein inverse folding with a test-time approach: given a backbone, design a sequence that folds into it. The idea is to optimize each target separately at test time, with no weight updates.
TTDesign works in two steps. Foldseek retrieves the ten most similar chains from the CATH training set, and their aligned residues are blended with ESM-3's probabilities to initialize sequence logits. Those logits are then optimized for 200 steps: sample a sequence with Gumbel-Softmax, fold it with a frozen ESMFold, compute FAPE and distogram loss against the target, and backpropagate. The best candidate by TM-score is returned.
Contributions as claimed: a test-time design paradigm needing no fine-tuning, a self-optimization loop driven by structural feedback, a retrieval mechanism for external structural priors, and best-in-class results.
Validation: CATH v4.2, CATH v4.3 and TS50 against several baselines, using scTM, RMSD, pLDDT and a success rate. Designed sequences are folded with ESMFold2 rather than the in-loop ESMFold. There is also a breakdown by retrieval quality and an ablation separating the two components. Sequence recovery, runtime and variance are not reported.
Critical strengths
-
The retrieval-quality analysis is useful and numerically consistent. Table 4 groups targets by retrieval quality, which is appropriate for this method. Its weighted averages match Table 1, and the reported percentages are correct. My concerns are about the evaluation design, not numerical reporting.
-
The ablation shows that optimization helps without retrieval. Optimization alone improves ESM-3 from 0.799 to 0.814 scTM. This is a relatively clean comparison using the same base model and initialization source, suggesting that structural feedback provides some benefit on its own.
Critical weaknesses
-
Both components lack novelty, follow much standard computational designing approach. E.g., these closely resemble prior work (that are also not cited): The optimization loop resembles AfDesign's fixed-backbone hallucination, including the same loss weights. The retrieval step is also similar to Refold, published seven months earlier. Neither is cited, and the paper does not clearly establish what is new or compare against these methods. It appears as careful engineering than actual methodological (or theoretical) novelty.
-
The evaluation does not directly test the main claim. The method selects the candidate with the best TM-score and then reports scTM, which is nearly the same metric. Also no baseline uses the same 200-sample budget and ranking method. So the gains may come from "best-of-200" selection rather than the proposed loop. The paper does not rule this out.
Questions for authors
- How does TTDesign compare to ESM-3 with best-of-200 sampling, ranked by the same ESMFold TM-score?
- How does the method differ from AfDesign fixed-backbone hallucination, and from the retrieval approaches in PRISM and Refold (both are recent, 2025-'26)?
- What is the sequence recovery on CATH v4.2, CATH v4.3 and TS50? How do they compare with the closely related baselines?
- Was ESMFold2 run in single-sequence or MSA-conditioned mode for the reported numbers? This matters a lot for the foldability scores, especally in retrieve+generation based approaches like this one.
- What is the wall-clock cost per target, and where does performance saturate as T increases? How does it compare with the closely related baselines without any test-time guided refinements?