- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes TC-NMD, a non-Markovian discrete diffusion framework for biological sequence design. It combines an independent-corruption forward process, a learned trajectory summary, and a biased corruption schedule so that intermediate decisions can be revisited. The framework is instantiated with both TC-SLM and TC-MDLM. With 5 sampling steps, the proposed models outperform their corresponding Markovian baselines on enhancer and promoter design tasks.
Critical strengths
-
The method is well motivated by a clearly identified problem. It preserves the marginal distribution at each noise level while changing the joint distribution over the trajectory. The same framework can also be applied to different backbones, including SLM and MDLM, suggesting a degree of generality.
-
The few-step experimental evaluation is relatively broad. The paper compares against the main experimental settings reported in the SLM paper and additionally includes motif-profile experiments.
-
The router is lightweight. It introduces few additional parameters and does not substantially increase the computational cost.
Critical weaknesses
-
Limited multi-step evaluation. Prior work generally reports performance in a multi-step regime, often using around 100 NFE, whereas this paper mainly compares TC-NMD and its reproduced baselines at 5 NFE. Consequently, the paper does not establish whether the observed gains persist when more sampling steps are allowed. Results across a broader range of NFE values, including 100 NFE, would make the comparison more complete.
-
Dataset- and backbone-specific tuning may impose a large inference-time tuning cost. The bias and the classifier-free guidance scale are selected separately for each dataset and backbone, and their optimal values appear to vary considerably across tasks and architectures. In addition, obtaining the best inference performance requires searching over the guidance weight , which may result in a substantial hyperparameter selection budget.
-
Computational-cost measurements are mainly reported for the SLM backbone. The paper provides detailed measurements for SLM, but gives limited or no corresponding information for the MDLM backbone. It is therefore unclear whether the claimed computational efficiency and memory overhead generalize to MDLM.
-
Minor: incorrect table references remain in the manuscript. Several references still contain placeholders such as “Table ??,” which reduces the clarity and polish of the presentation.
-
Minor: the legend in Fig. 4 is somewhat confusing. The melanoma curves are shown with dashed lines, but this line-style convention is not clearly explained in the legend. The figure remains interpretable, but the legend should explicitly identify the meaning of the dashed and solid lines.
Questions for authors
-
Is the simplex superscript in Eq. 10 intended to be ? Since outputs a probability distribution over , there are probability components and therefore degrees of freedom under the usual geometric definition of a simplex.
-
Could the authors report the performance of TC-NMD and the SLM/MDLM baselines across a wider range of sampling steps, including 100 NFE? The SLM baseline performs substantially better at 100 NFE than at 5 NFE. If TC-NMD does not retain a clear advantage in the multi-step regime, could the authors explain why its benefits are primarily observed in few-step sampling and clarify the intended application regime?
-
Could the authors specify precisely how is computed in the experiments in Section 3? Does this quantity require additional hyperparameter tuning? Please also report whether the experimental results are robust to different choices or parameterizations of , ideally through a sensitivity analysis or ablation study.
5 comments
Response to Reviewer ukxd part 2
W2:
We thank the reviewer for raising this concern. The inference-time hyperparameter tuning cost is relatively small. One validation FBD evaluation at 5 NFE takes approximately 15–30 seconds on a single H100/H200 GPU for both TC-SLM and TC-MDLM. Therefore, evaluating nine bias values takes around 3–5 minutes per setting. Our complete TC-MDLM hyperparameter sweep over 7 bias and 5 guidance values took approximately 15 minutes on an A100 GPU. We also found that the selected values are within a relatively narrow range: 0–0.07 for TC-SLM bias, 0.15–0.25 for TC-MDLM bias, and a guidance scale of 1 for all MDLM settings.
We also noticed that the guidance weights reported for the MDLM baselines had been tuned on an earlier version of our code, and these values were carried over after changes to the implementation. We have now repeated the tuning using the final version. On conditional Fly Brain, the MDLM baseline improved from 9.79 to 6.15, while TC-MDLM improved from 6.07 to 2.53 after tuning the inference bias. On conditional Melanoma, the baseline improved from 14.34 to 7.44, while TC-MDLM remained at 6.48. The updated results show that TC-MDLM still achieves lower FBD than MDLM on both datasets. On Fly Brain, TC-MDLM achieves 2.43× lower FBD, and the absolute improvement remains almost the same as in the original results. On Melanoma, the improvement is more modest after retuning (7.44 → 6.48, a 12.9% reduction). We will update these results in the revised paper and make a separate public comment on OpenReview to report the corrected comparisons to all reviewers and the area chair.
Q3:
Regarding the history weights, in Equation 12 is the corruption level obtained directly from the forward noise schedule at timestep . We use and , so these weights do not require any additional hyperparameter tuning.
We also conducted an additional ablation using the trained TC-SLM checkpoints with different fixed weights. The following table reports FBD at 5 NFE using the best checkpoint for each setting.
| Current / History weights | Melanoma Cond. | Melanoma Uncond. | Fly Brain Cond. | Fly Brain Uncond. |
|---|---|---|---|---|
| / (ours) | 4.71 | 5.31 | 1.61 | 4.64 |
| 1.5 / 0.5 | 4.76 | 5.39 | 1.61 | 4.84 |
| 1.2 / 0.8 | 4.61 | 5.32 | 1.51 | 4.38 |
| 1.0 / 1.0 | 4.66 | 5.44 | 1.59 | 4.32 |
| 0.8 / 1.2 | 4.68 | 5.48 | 1.60 | 4.05 |
| 0.5 / 1.5 | 5.06 | 5.91 | 2.48 | 5.06 |
| 2.0 / 0.0 (no history) | 30.42 | 33.62 | 23.01 | 29.87 |
The results show that our method is not highly sensitive to the exact choice of history weights within a reasonable range. Using different fixed weights gives similar results to our noise-dependent weighting, and in some cases slightly better results. However, removing the history contribution substantially increases FBD in all settings. These results show that the method benefits from the trajectory history without requiring careful tuning of its contribution.
W3:
Regarding the computational cost for MDLM, we conducted additional measurements using the same experimental setup as Table 6. The results are reported below.
| Metric | MDLM | TC-MDLM | Relative |
|---|---|---|---|
| Forward MACs (GMAC) | 95.08 | 95.21 | 1.0013× |
| Training time (ms/step) | 20.09 | 22.75 | 1.13× |
| Inference time (ms/batch) | 66.90 | 73.45 | 1.10× |
| Training peak memory (MB) | 1313.74 | 1804.06 | 1.37× |
| Inference peak memory at 5 NFE (MB) | 338.48 | 343.09 | 1.014× |
The results show that the computational overhead is similar to the SLM backbone. TC-MDLM adds only 0.07% additional parameters and increases the forward MACs by approximately 0.13%. The inference-time overhead is approximately 10%, while the peak inference memory increases by only 1.4% at 5 NFE. The training memory overhead is higher because the model uses multiple noisy views during training.
We hope that the additional experiments and clarifications adequately address the reviewer's concerns and provide stronger evidence for the contributions of our method. We would appreciate it if the reviewer could reconsider their rating.
Minor issues:
We will address the missing placeholders, and clarify the legend in figure 4.
Dear Reviewer,
Thank you again for your time and feedback. We have addressed your concerns in our rebuttal and updated the manuscript with additional experiments on higher sampling steps, computational cost, and further clarifications.
We would appreciate it if you could review our responses and let us know if you have any remaining concerns. We would be happy to discuss them further.
Thank you for your time and consideration.
Response to Reviewer ukxd part 1
We thank the reviewer for recognizing the motivation of our method, its applicability to different backbones, and its improvements in the few-step sampling regime. We address the reviewer’s concerns below:
Q1 — Simplex notation
We thank the reviewer for pointing this out. The reviewer is correct that the router outputs a probability distribution over T−t history states, and therefore the simplex has T−t−1 degrees of freedom. We will correct the notation in Equation 10 to ΔT−t−1.
Q2, W1 — Performance at higher NFEs
We thank the reviewer for raising this concern. We agree that our method is most useful in the low-sampling regime, and we will make this clear in the revised paper and discuss its performance at higher NFEs.
At 5 NFE, TC-SLM substantially improves FBD on Fly Brain (14.65 → 2.02 conditional, 19.57 → 4.79 unconditional) and Melanoma (9.95 → 4.64 conditional, 10.11 → 5.74 unconditional). To further demonstrate its practical efficiency, we measured generation time on an H100 GPU (batch size 64, sequence length 500). On conditional Fly Brain, SLM requires approximately 17 NFE to approach the quality of TC-SLM at 5 NFE. Generating 8,192 sequences takes 9.0 seconds with TC-SLM versus an estimated 28.2 seconds with SLM, giving approximately 3.1× faster generation, including the computational overhead of our method. For one million sequences, this corresponds to approximately 18.4 versus 57.4 minutes, extrapolated from measured throughput.
We also investigated performance at higher NFEs and evaluated an adjusted version of TC-SLM on Melanoma. The following table reports FBD (conditional / unconditional), with all adjustments selected on the validation split and results averaged over three seeds.
Values are conditional / unconditional means; SDs are omitted for brevity. The adjusted method uses validation-selected re-masking budget (k) and router temperature (τ).
At higher NFEs, applying re-masking at every step can repeatedly reintroduce excluded candidates and prevent progressive refinement. We therefore limit re-masking to a fixed budget (k=16 at 20–100 NFE), while also adjusting the router temperature where needed. The results show that re-masking remains beneficial compared with removing it entirely in high NFE settings.
At very high NFEs, SLM still performs better. One possible reason is that the router must select useful information from a much larger number of history states (up to 99 at 100 NFE, compared with 4 at 5 NFE). More advanced router architectures may improve performance in this setting, which we leave for future work.
Overall, these results support our main motivation of improving generation quality with fewer sampling steps. This is particularly useful for large-scale biological sequence design, where many candidates are generated and screened before experimental testing.
For example, Bennett et al. (2023) generated approximately 2 million protein designs per target and selected around 20,000 for experimental testing, explicitly considering computational efficiency in their design pipeline. Dai et al. (2026) further demonstrated that increasing the number of sampled protein candidates can improve the predicted quality of selected designs. These studies demonstrate the practical importance of efficient sampling in biological sequence design. We will include these additional experiments and clarify the low-sampling focus and higher-NFE limitations in the revised paper.
Bennett et al. Improving de novo protein binder design with deep learning.
Dai et al. Programmable design of functional proteins from natural language.