acceptodds
Under review as a conference paper at ICLR 2027

ToDNA: Preference-Programmable Test-Time Optimization for Regulatory DNA Design

Abstract

Restricting gene expression to target cell types is a central goal in regulatory DNA design for gene therapy and synthetic biology. While genomic foundation models capture rich sequence regularities, their pre-trained priors do not explicitly enforce the differential activity across cellular contexts that defines cellular selectivity. Although reward-based post-training can encode such context-specific preferences into model parameters, adapting to new design objectives often necessitates costly re-training. To address this limitation, we introduce , a novel test-time optimization framework that cleanly decouples a reusable regulatory sequence prior from cell-specific functional objectives. Specifically, ToDNA employs target-agnostic low-rank adaptation (LoRA) to specialize a base model (Evo 2) to specific regulatory-element classes, while leveraging a frozen multi-cell predictor to guide nucleotide-logit updates, regularized against a fixed local reference. This modular separation allows target-to-off-target preferences to be dynamically re-configured via reward functions without retraining the generative backbone. Evaluated across four enhancer and promoter design tasks, ToDNA demonstrates that candidate optimization substantially boosts enhancer design specificity, whereas promoter responsiveness exhibits more modest gains. Sequence-level analyses further reveal that stronger motif agreement does not necessarily align with higher predicted cell specificity. Crucially, we show that alternative reward configurations can redirect cell-activity preferences purely at inference time, though their effectiveness varies depending on the interplay between activity and sequence-level objectives. These findings establish the viability of reusable, inference-time control for regulatory DNA design, while highlighting objective formulation and predictor validity as key determinants of true cellular selectivity.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Regulatory DNA design, Test-time optimization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_njXCOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes ToDNA, a test-time optimization framework for designing cell-type-specific regulatory DNA sequences. It first adapts the pretrained Evo 2 genomic foundation model to enhancer and promoter sequences through LoRA fine-tuning. It then optimizes individual candidate sequences by updating nucleotide logits using a straight-through estimator. These updates are guided by a sequence-to-activity oracle trained on multiple cell types and regularized using position-wise categorical KL divergence from a local reference distribution. The authors evaluate ToDNA on an enhancer design benchmark covering three cell types and a promoter design benchmark covering one cell type.

Overall, I find this work interesting, but it needs a more rigorous evaluation, clearer motivation, presentation, and a better-supported claim of originality. I look forward to seeing these concerns (see below) addressed in the discussion or a revised submission.

Critical strengths
  1. The test-time optimization method is practically useful because it can be applied to multiple cell-type targets without additional fine-tuning for each target.
  2. The method preserves sequence composition and transcription factor motif profiles reasonably well, providing some evidence that optimization retains biologically relevant sequence features.
Critical weaknesses
  1. The choice of Evo 2 (Brixi et al., 2026) as the backbone is not sufficiently justified. The paper does not compare it with other DNA foundation models, such as Nucleotide Transformer v2 (Dalla-Torre et al., 2025), Nucleotide Transformer v3 (Boshar et al., 2025), Caduceus (Schiff et al., 2024), or HyenaDNA (Nguyen et al., 2023). Comparisons should account for differences in these models’ training objectives and generation procedures.
  2. In Table 2, the reported standard deviations for “Base logits” (I assume unadapted Evo 2) and ToDNA, overlap across metrics. This alone does not establish whether the differences are statistically significant and leaves the benefit of adaptation unclear. The authors should report uncertainty on the differences and discuss whether the gains justify the fine-tuning cost.
  3. The optimization procedure appears closely related to FastSeqProp (Linder & Seelig, 2021). Both use straight-through estimators to optimize discrete sequences through a differentiable predictor, and support regularization. The paper should cite this work where relevant and explain which technical differences account for the proposed method’s contribution. These differences should be motivated and tested through ablations.
  4. Ledidi (Schreiber et al., 2025) is another directly relevant baseline. It uses Gumbel–Softmax optimization and explicitly penalizes the number of edits to an initial sequence. A comparison with Ledidi, together with an ablation replacing KL regularization with an edit penalty, would help establish the value of the proposed formulation. Its omission also leaves the state-of-the-art claim insufficiently supported.
  5. Several aspects of the evaluation need clarification:

o The benchmark appears incomplete relative to the Ctrl-DNA study from which it is adapted (Chen et al., 2025). In particular, activity in individual off-target cell types and results for the remaining promoter targets are not fully reported.

o Some reproduced baseline values appear lower than previously published results on the same benchmark. The authors should explain these discrepancies and identify any differences in evaluation settings.

o Relevant baselines are missing, including Ledidi (Schreiber et al., 2025), DNA-CRAFT (Awasthi et al., 2026), GPA (Sarkar et al., 2026), and SVDD (Li et al., 2025).

o Metrics assessing output diversity or mode collapse are missing.

o The design and evaluation predictors share the Enformer architecture and pretraining, which may lead to correlated prediction errors. Evaluation with additional predictors, such as Borzoi (Linder et al., 2025) or AlphaGenome (Avsec et al., 2026), would help assess whether the gains transfer beyond the current predictors, provided suitable outputs and sequence contexts are available.

  1. The evaluation covers only three enhancer targets and one promoter target, using a single MPRA-derived activity prediction setup. Additional cell types and complementary assay modalities, such as chromatin accessibility, would provide stronger evidence of generality.
  2. The results do not show consistent improvements across targets. Ctrl-DNA achieves higher enhancer specificity for HepG2 and K562, while promoter specificity remains negative for every method. The paper acknowledges these limitations, but its broader performance claims should reflect them.

Minor comments:

  1. Several claims in the introduction need correction or addition of relevant citations.
  2. The notation is dense in places, and several symbols appear before they are defined.
  3. Table 1 reports both motif Pearson and motif Spearman correlations. These appear to provide largely overlapping information in the reported results. The authors should explain the value of reporting both or retain only one.
  4. The motivation for the different ToDNA variants, including the reward formulations in Appendix C, should be explained before the variants are introduced.
  5. MinGap is mentioned but never introduced.
Questions for authors
  1. In Table 5, the mean-difference objective achieves the best MinGap for some targets, while directly optimizing MinGap does not consistently produce the best score on that metric. Can the authors explain this pattern? In particular, were the objectives compared using matched optimization budgets and comparable hyperparameter tuning? I also find it surprising that optimzing for Target only still yields positive MinGap scores raising concerns about the accuracy of the reported results.
  2. Can the authors report evaluation metrics for the starting sequences sampled from the adapted prior, before any test-time optimization? This would provide a reference point for motif and composition metrics and help separate the contribution of optimization from that of the adapted prior.
  3. How is the number of edits determined? Does the method use an early-stopping criterion, or does it always run for a fixed number of iterations? Can the authors vary the KL regularization strength and iteration count to characterize the trade-off between sequence fidelity and predicted specificity?
  4. Can the authors compare KL regularization with a minimum-edit loss, as used in Ledidi (Schreiber et al., 2025)?
  5. What is the source of the enhancer and promoter sequences used to fine-tune Evo 2?
  6. Could the authors use a DNA diffusion model, such as D3 (Sarkar et al., 2024) or DNA-CRAFT (Awasthi et al., 2026), as the proposal model instead of Evo 2? What changes to the optimization framework would this require?
5 comments
AuthorsOpenReview ↗

Response to Reviewer njXC

Thank you for your detailed and insightful comments. We address each concern point by point below.

Q1: Why choose Evo2 over other DNA foundation models?

A1: Thank you for pointing this out. We agree that comparisons must respect model-specific objectives. Other DNA foundation models, including Nucleotide Transformer v2 (Dalla-Torre et al., 2025), Nucleotide Transformer v3 (Boshar et al., 2025), Caduceus (Schiff et al., 2024), and HyenaDNA (Nguyen et al., 2023), learn reusable representations or sequence priors through pretraining on DNA. To broaden our comparison across architectures and generative paradigms, we additionally included D3 (DNA Discrete Diffusion; Sarkar et al., 2024) as an alternative proposal generator. We compared D3 proposal-only with D3 proposals followed by ToDNA test-time optimization. As shown in Table 1, we observe that adding ToDNA increased the mean evaluation reward. This indicates that our strategy can generalize to other DNA foundation model architectures.

Table 1. D3 model on three enhancer tasks. The metric is the final normalized evaluation reward (×10⁻³) ↑. Bold marks the best mean in each row.

Target cell typeD3 proposal-onlyD3 + ToDNA_TTO
HepG21.061 ± 0.3341.151 ± 0.119
K5622.054 ± 0.5996.604 ± 1.548
SK-N-SH3.530 ± 0.7597.647 ± 2.375

Q2: Do the differences between ToDNA and Base logits in Table 2 justify the cost of adaptation?

A2: Thank you for highlighting this issue. We clarify that “Base logits” uses unadapted Evo2 for the initial logits and KL reference, while retaining the same adapted-model candidate pool. Therefore, this comparison evaluates adaptation in initialization and regularization. We observe that adaptation took about 40 minutes for enhancers and 39 minutes for promoters on one GPU each, once per element class.

Q3: How does ToDNA compare with FastSeqProp (Linder & Seelig, 2021) and Ledidi (Schreiber et al., 2025)?

A3: Thank you for pointing this out. We compared all three methods directly. As shown in Table 2, we observe that the ToDNA model continues to hold an advantage across most DNA design tasks.

Table 2. Comparison with Fast SeqProp and Ledidi. The metric is normalized mean margin on the evaluation predictor (↑). Bold marks the highest mean in each row.

TaskFast SeqPropLedidiToDNA
HepG2 enhancer0.353 ± 0.0070.333 ± 0.0060.349 ± 0.006
K562 enhancer0.541 ± 0.0110.543 ± 0.0170.546 ± 0.011
SK-N-SH enhancer0.242 ± 0.0270.214 ± 0.0090.247 ± 0.004
JURKAT promoter0.065 ± 0.0020.085 ± 0.0130.110 ± 0.015
K562 promoter−0.178 ± 0.009−0.115 ± 0.003−0.120 ± 0.005
THP-1 promoter0.189 ± 0.0020.190 ± 0.0050.184 ± 0.004
AuthorsOpenReview ↗

Response to Reviewer njXC

Q4: The benchmark appears incomplete relative to the Ctrl-DNA study from which it is adapted (Chen et al., 2025). Some reproduced baseline values appear lower than previously published results on the same benchmark.

**A4:**Thank you for pointing this out. We agree that the original presentation was incomplete. We have completed the results for the two omitted promoter targets, JURKAT and THP-1. As shown in Table 3, our model ToDNA continues to hold an advantage across most metrics. Regarding the lower baseline scores, our experiments adapt the benchmark to a common evaluation protocol. We use separately trained Design and Evaluation predictors, different sequence-group partitions and starting pools, and three runs with 256 candidates locked before independent evaluation. (Chen et al., 2025) report five runs and metrics over sequences generated in the final round. These differences limit direct numerical comparisons with the published results.

Table 3. The results for JURKAT and THP-1. Bold marks the highest mean in each row.

Promoter targetMetricCtrl-DNAToDNA
JURKATMean specificity margin ↑0.069 ± 0.0140.039 ± 0.002
JURKATMinGap ↑−0.035 ± 0.019−0.075 ± 0.002
JURKATMotif Pearson ↑0.136 ± 0.0540.857 ± 0.011
JURKATMotif Spearman ↑0.717 ± 0.0380.884 ± 0.004
JURKAT3-mer Pearson ↑0.449 ± 0.0600.891 ± 0.005
JURKATPositional entropy (bits) ↑1.872 ± 0.0111.779 ± 0.002
JURKATMean pairwise Hamming distance ↑0.710 ± 0.0030.680 ± 0.001
THP-1Mean specificity margin ↑0.215 ± 0.0130.186 ± 0.001
THP-1MinGap ↑0.148 ± 0.0130.127 ± 0.002
THP-1Motif Pearson ↑0.470 ± 0.1740.868 ± 0.018
THP-1Motif Spearman ↑0.789 ± 0.0540.699 ± 0.061
THP-13-mer Pearson ↑0.409 ± 0.1070.718 ± 0.151
THP-1Positional entropy (bits) ↑1.557 ± 0.1121.928 ± 0.017
THP-1Mean pairwise Hamming distance ↑0.595 ± 0.0580.729 ± 0.006

Q5: Relevant baselines are missing, including Ledidi (Schreiber et al., 2025), DNA-CRAFT (Awasthi et al., 2026), GPA (Sarkar et al., 2026), and SVDD (Li et al., 2025).

A5: Thank you for pointing this out. We completed a direct comparison with Ledidi using the same starting sequences, predictors, and search-query budget. As shown in Table 4, ToDNA achieved a higher mean specificity score on four of the six tasks, while Ledidi performed better on the K562 and THP-1 promoter tasks. Complete matched comparisons with DNA-CRAFT, GPA, and SVDD have not yet been completed due to time constraints**.**

Table 4. Comparison with Ledidi. The metric is normalized mean specificity margin on the Evaluation predictor (↑). Bold marks the higher mean in each row.

TaskLedidiToDNA
HepG2 enhancer0.333 ± 0.0060.349 ± 0.006
K562 enhancer0.543 ± 0.0170.546 ± 0.011
SK-N-SH enhancer0.214 ± 0.0090.247 ± 0.004
JURKAT promoter0.085 ± 0.0130.110 ± 0.015
K562 promoter−0.115 ± 0.003−0.120 ± 0.005
THP-1 promoter0.190 ± 0.0050.184 ± 0.004

Q6: Design and evaluation predictors share the Enformer architecture and pretraining, which may lead to correlated prediction errors. Evaluation with additional predictors, such as Borzoi (Linder et al., 2025) or AlphaGenome (Avsec et al., 2026), would help assess whether the gains transfer beyond the current predictors, provided suitable outputs and sequence contexts are available.

**A6:**Thank you for highlighting this concern. We conducted an exploratory evaluation with AlphaGenome, obtaining ATAC, CAGE, and RNA-seq predictions for ToDNA and Ledidi candidates across three enhancer targets. As shown in Table 5, we interpret them as exploratory cross-predictor assessments and do not claim that they establish transfer of cell-specific gains.

Table 5. Exploratory AlphaGenome evaluation of ToDNA and Ledidi(Schreiber et al., 2025). All scores are expressed in ×10⁻³. “Design target” identifies the intended target of sequence optimization, not a cell-specific AlphaGenome output track. Bold indicates the higher mean signal within each target and output.

Design targetMethodATACCAGERNA-seq
HepG2Ledidi56.365 ± 18.9935.495 ± 0.2460.363 ± 0.086
HepG2ToDNA53.356 ± 6.0525.063 ± 1.0200.372 ± 0.111
K562Ledidi58.009 ± 10.49013.035 ± 3.3781.563 ± 0.775
K562ToDNA47.609 ± 1.57110.504 ± 0.2870.971 ± 0.278
SK-N-SHLedidi41.289 ± 20.95111.312 ± 4.6901.732 ± 2.065
SK-N-SHToDNA23.045 ± 8.0366.147 ± 0.8300.831 ± 0.364
AuthorsOpenReview ↗

Response to Reviewer njXC

Q7: The evaluation covers only three enhancer targets and one promoter target, using a single MPRA-derived activity prediction setup. Additional cell types and complementary assay modalities, such as chromatin accessibility, would provide stronger evidence of generality.

A7: Thank you for this suggestion. In addition to the four displayed tasks, we have evaluated the JURKAT and THP-1 promoter targets. As shown in Table 4 on A5, ToDNA achieves a higher mean specificity score on the JURKAT and THP-1 promoter targets. This demonstrates that our model also performs well on tasks involving other cell types.

Q8: The mean-difference objective achieves the best MinGap for some targets, whereas directly optimizing MinGap does not consistently yield the highest score. Could the authors explain this pattern and clarify whether optimization budgets and hyperparameter tuning were comparable? The positive MinGap scores obtained by the Target-only objective also require clarification to address concerns about the accuracy of the reported results.

A8: Thank you for raising this point. Our objective design follows the Ctrl-DNA (Chen et al., 2025) formulation, which measures specificity as normalized target activity minus mean normalized off-target activity. We included Target-only and smooth-MinGap variants to examine alternative reward formulations.

Q9: Could the authors report evaluation metrics for the starting sequences sampled from the adapted prior, before test-time optimization? This would provide a reference for motif and composition metrics and help distinguish the contribution of optimization from that of the adapted prior.

A9: Thank you for your suggestions. We evaluated both the full candidate pool sampled from the adapted prior and the selected starting sequences before test-time optimization. As shown in Table 6, we observe that test-time optimization can generate more natural sequences with the desired properties.

Table 6. Characteristics of HepG2 candidate sequences before test-time optimization. Bold indicates the better mean according to the stated direction.

MetricBefore TTOAfter TTO
Design mean specificity margin ↑0.001 ± 0.0010.030 ± 0.003
Motif Pearson ↑0.800 ± 0.0260.818 ± 0.037
3-mer Pearson ↑0.947 ± 0.0120.921 ± 0.029
Mean GC content (%)46.619 ± 0.26954.068 ± 0.491
Sequences with GC content between 30% and 70% (%) ↑91.667 ± 1.50295.573 ± 1.966
Sequences with maximum homopolymer length ≤ 6 (%) ↑83.377 ± 2.21590.104 ± 1.626
Mean maximum homopolymer length (bp) ↓5.621 ± 0.1225.193 ± 0.255

Q10: How is the number of edits determined? Does the method use early stopping or a fixed number of iterations? Could the authors vary KL regularization strength and iteration count to characterize the trade-off between sequence fidelity and predicted specificity?

A10: Thank you for highlighting this concern. We assume edits are measured as the Hamming distance between each final discrete sequence and its own starting sequence. The edit count is a computational metric. We implement each trajectory run for a fixed number of iterations without early stopping, while KL regularization softly constrains deviation from the reference distribution. In the matched HepG2 comparison below Table 7, increasing the KL weight from 0 to 1 reduced edits while maintaining a slightly higher mean Design-predicted specificity.

Table 7. Matched KL comparison on HepG2. Both settings use the same Design predictor, paired pools and starts. Bold indicates the best displayed mean.

KL weightIterationsEdits ↓Design ΔR ↑
0.0001077.520 ± 1.3920.383 ± 0.015
1.0001064.494 ± 1.5390.391 ± 0.012

Q11: Could the authors compare KL regularization with a minimum-edit loss, as used in Ledidi (Schreiber et al., 2025)?

A11: Thank you for raising this point. We performed a regularizer ablation by replacing KL with a differentiable expected-edit penalty relative to the starting sequence, using the same starting pool and optimization budget. The edit penalty discourages nucleotide substitutions, while KL penalizes deviations from the full reference nucleotide distribution. As shown in Table 8, KL achieved a higher mean Design reward gain on all three enhancer targets at the tested regularization strengths.

Table 8. KL regularization versus an expected-edit penalty within ToDNA. The metric is Design reward gain relative to each retained candidate’s own starting sequence (↑). Bold marks the higher mean.

Enhancer targetEdit penaltyKL regularization
HepG20.009 ± 0.0010.024 ± 0.001
K5620.106 ± 0.0050.120 ± 0.005
SK-N-SH0.010 ± 0.0010.021 ± 0.001
Reviewer_6gM8OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper challenges those DNA design methods that require fine-tuning a model for each cell line. It enables gradients on sequence logits and backpropagates through them to optimize DNA sequences. One notable benefit is that the generator’s parameters can remain frozen.

Critical strengths
  1. Avoiding cell-specific fine-tuning has value.
Critical weaknesses
  1. The technical contribution is limited. Continually pretraining Evo 2 with LoRA does not need so much text and figure. A brief description should be enough. The method indeed freezes the models and performs gradient descent on the sequence logits. I think the title “preference-programmable test-time optimization” is unnecessarily complex. “Predictor-guided gradient-based sequence optimization” already describes the method quite well, and similar approaches have been used in prior work cited by the authors [1].
  2. The paper is hard to reproduce. I could not find the source code. Some implementation details are also unclear, including but not limited to the hardware used and how the sequences for LoRA continual pretraining were selected. The paper states that “aggregate results and scripts are provided,” but where can these materials be accessed? If these materials are available, please point me to them.
  3. I did not find the theoretical analysis particularly informative. I would suggest moving this section to the appendix.

[1] Reddy, Aniketh J., et al. “Designing Cell-Type-Specific Promoter Sequences Using Conservative Model-Based Optimization.” Advances in Neural Information Processing Systems 37 (2024): 93033–93059.

Questions for authors
  1. Would a simple edit-distance penalty work just as well as the KL term?
  2. Would randomly choosing starts from the generated pool give similar results?
  3. How much does Evo 2 itself help?
1 comment
AuthorsOpenReview ↗

Response to Reviewer 6gM8

Thank you for your detailed and insightful comments. We will address each of your concerns point by point in the following response.

Q1: How does ToDNA differ from prior predictor-guided sequence optimization, and could the title and LoRA description be simpler?

A1: Thank you for pointing this out. Our contribution is to separate adaptation to an element class from the choice of target cell. We adapt Evo 2 once for enhancers or promoters. Changing the target changes the reward; the adapter and predictor remain fixed. We propose the title ToDNA: Reward-Guided Test-Time Optimization for Regulatory DNA Design. We will condense the LoRA description to one paragraph.

Q2: The paper is hard to reproduce.

A2: We apologize for stating that scripts were provided without giving a reviewer-accessible link. We have prepared a source code archive at https://anonymous.4open.science/r/ToDNA-0642/.

Q3: Could the theoretical analysis be moved to the appendix?

A3: Thank you for this suggestion. We will move the propositions and proofs to the appendix.

Q4: Would a simple edit-distance penalty work as well as KL?

A4: Thank you for pointing this out. We implemented the experiments to compare KL and a simple edit-distance penalty. As shown in Table 1, the edit penalty made fewer substitutions than KL on all three tasks, but also produced smaller gains in the design predictor’s reward.

Table 1. KL versus an expected-edit penalty on three enhancer tasks. Bold marks the best mean in each row.

Target cell typeEdit penalty: reward gain ↑KL: reward gain ↑
HepG20.009 ± 0.0010.024 ± 0.001
K5620.106 ± 0.0050.120 ± 0.005
SK-N-SH0.010 ± 0.0010.021 ± 0.001

Q5: Would choosing starts at random from the generated pool give similar results?

A5: Thank you for pointing this out. As shown in Table 2, we observe that ranked starts gave higher initial and final design rewards. That demonstates adapt ranked starts gain more advantage start sequences for DNA design.

Table 2. Random versus generated on HepG2 tasks. Bold marks the best mean in each row.

Metric (HepG2)Random startsRanked starts
Initial design reward ↑0.002 ± 0.0050.037 ± 0.011
Final design reward ↑0.370 ± 0.0100.400 ± 0.007

Q6: How much does Evo 2 itself help?

A6: Thank you for bringing up this important point. We keep that candidate generation and logit initialization fixed to Base Evo 2. The comparative experimental results are shown in Table 3, we can observe that the adapted reference improved 3-mer agreement, specificity, and edit counts. This indicates that the Evo2 prior information can provide a better starting point for the sequences generated for our model.

Table 3. Analysis of the contribution of the adapted KL reference. Candidate generation and logit initialization were fixed to Base Evo 2. Bold marks the best displayed mean.

KL referenceEvaluation MinGap ↑FullMPRA MinGap ↑3-mer Pearson ↑Edits ↓
Base5.349 ± 0.2506.273 ± 0.4330.697 ± 0.01364.228 ± 2.679
LoRA5.357 ± 0.3126.281 ± 0.4960.759 ± 0.01664.177 ± 2.778
Reviewer_ToHgOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces ToDNA, a framework for designing regulatory DNA with cell-type preferences. It uses LoRA to adapt Evo 2 to enhancer or promoter sequences, then optimizes individual sequence logits through a frozen activity predictor. A KL penalty keeps the nucleotide distributions close to a fixed reference derived from the adapted model.

Critical strengths

Separating the sequence prior from the cell-specific objective is a practical design choice. The same adapted generator can support different target-cell preferences by changing the reward, which is useful when exploring several design objectives.

The motif-preservation results are encouraging. ToDNA achieves the highest mean motif Pearson correlation on all three enhancer tasks.

Critical weaknesses

My main concern is the methodological novelty. Optimizing sequence logits through a frozen predictor is not new: similar ideas appear in protein hallucination methods such as BindCraft and, more directly, in DNA activation maximization. ToDNA seems to combine an Evo 2 prior with existing optimization tools, but it is unclear what the main methodological advance is over these approaches.

It is difficult to tell which components account for the gains. The base-logit ablation keeps the same candidate pool and changes both the initialization and the local reference. It therefore does not fully separate the benefit of better LoRA-generated starting sequences from that of the adapted local reference.

Questions for authors
  1. How does ToDNA compare with PPO and GRPO in total GPU-hours, runtime, and peak memory, including candidate generation, scoring, and optimization? Matching predictor evaluations alone does not make the computational costs comparable.

  2. Could the authors test candidate generation, logit initialization, and the KL reference separately? This would help clarify how much each use of the Evo 2 prior contributes beyond sequence optimization itself.

3 comments
AuthorsOpenReview ↗

Response to Reviewer ToHg

Thanks for your detailed and insightful comments. We will address each of your concerns point by point in the following response.
Q1: This suggests that we should explain methodological novelty.

A1: Thank you for pointing this out. Our contribution is separating element-class adaptation from target-specific search. We adapt Evo2 once per regulatory-element class. At test time, the prior supplies candidates, initial logits, and a reference that stays fixed for each starting sequence. A change in target-cell preference changes the reward without requiring a new generator.

Q2: It is difficult to tell which components account for the gains. The base-logit ablation keeps the same candidate pool and changes both the initialization and the local reference. It therefore does not fully separate the benefit of better LoRA-generated starting sequences from that of the adapted local reference.

A2: Thank you for identifying this limitation. To isolate the adapted KL reference contribution, we conducted a reference-only comparison on HepG2. Both conditions used the same base-generated candidate pool, starting sequences, and base-model initial logits. We changed only the KL reference from pretrained Evo 2 to its LoRA-adapted version. As shown in Table 1, the adapted reference increased MinGap and 3-mer Pearson correlation metrics. These results demonstrate stronger support for the adapted KL reference contribution.

Table 1. Reference-only ablation on HepG2. Bold indicates the higher mean.

MetricBase referenceAdapted reference
Evaluation raw MinGap ↑5.349 ± 0.2505.357 ± 0.312
FullMPRA raw MinGap ↑6.273 ± 0.4336.281 ± 0.496
3-mer Pearson ↑0.697 ± 0.0130.759 ± 0.016

Q3: How does ToDNA compare with PPO and GRPO in total GPU-hours, runtime, and peak memory, including candidate generation, scoring, and optimization? Matching predictor evaluations alone does not make the computational costs comparable.

A3: Thank you for pointing this out. We audited all three methods (ToDNA, PPO, GRPO) on the HepG2 task; As shown in Table 2, ToDNA required 29.11 allocated GPU-hours and 8.12 cumulative runtime hours, with a peak allocated memory of 14.31 GiB per GPU. These correspond to 75.6% fewer allocated GPU-hours than PPO and 12.9% fewer than GRPO. This indicates that our model is reasonable and advantageous in terms of computational cost.

Table 2. Computational costs. Bold marks the lowest observed value in each column.

MethodTotal allocated GPU-hours ↓Cumulative runtime (h) ↓Peak allocated memory per GPU (GiB) ↓
ToDNA29.118.1214.31
PPO119.4537.8119.57
GRPO33.4333.4319.57

Q4: Could the authors test candidate generation, logit initialization, and the KL reference separately? This would help clarify how much each use of the Evo2 prior contributes beyond sequence optimization itself.

A4: Thank you for bringing up this important point. We keep that candidate generation and logit initialization fixed to Base Evo2. The compared experimental results are shown in Table 3, we can observe that the adapted reference improved 3-mer agreement, specificity, and edit counts. This indicates that the Evo2 prior information can provide a better starting point for the sequences generated for our model.

Table 3. Analysis of the contribution of the KL reference. Candidate generation and logit initialization were fixed to Base Evo 2. Bold marks the best displayed mean.

KL referenceEvaluation MinGap ↑FullMPRA MinGap ↑3-mer Pearson ↑Edits ↓
Base5.349 ± 0.2506.273 ± 0.4330.697 ± 0.01364.228 ± 2.679
LoRA5.357 ± 0.3126.281 ± 0.4960.759 ± 0.01664.177 ± 2.778

Thank you authors for the rebuttal. The compute comparison in Table 2 addresses my question, and the ablation shows that the adapted KL reference improves significantly in 3-mer agreement, but the gain in cell-type specificity is small and the methodological novelty advance over existing DNA activation maximization approaches remains still unclear to me, so I’m maintaining my score.

Thank you for your valuable feedback. We apologize that the methodological novelty of our work was not sufficiently clear in the initial manuscript. Compared to existing DNA activation maximization approaches, our core methodological contribution lies in decoupling reusable regulatory-prior learning from objective-specific sequence optimization, coupled via an explicit regularization mechanism:

(1) Objective-Agnostic Prior Adaptation: ToDNA adapts Evo 2 to a broad regulatory class (enhancers or promoters) without target-cell rewards, producing a single adapter shared across all cell targets within that class. At design time, the backbone, adapter, and activity predictor remain frozen; only candidate nucleotide logits are optimized. Switching target cells or off-target preferences requires only updating the optimization objective—avoiding the repeated generative-model retraining needed by reward-driven RL methods such as TACO (ICLR 2025) and Ctrl-DNA (NeurIPS 2025). While standard activation maximization also supports inference-time objective changes, ToDNA couples this flexibility directly with an adapted genomic foundation-model prior.

(2) Efficient Local Prior Regularization: The adapted prior provides both initial candidate proposals and a fixed, start-specific reference distribution. Unlike Ledidi (Schreiber et al., 2025)’s uniform edit-count penalty, our KL term penalizes deviations across the full per-position nucleotide distribution, preserving the prior's base-level preferences. Furthermore, unlike VAE-based regularization in FastSeqProp (Linder & Seelig, 2021), our local categorical reference is derived directly from the adapted foundation model and computed once before optimization. Because the large generative model is excluded from the iterative gradient loop, optimization remains lightweight.

Empirical Validation: Extensive experimental results further demonstrate consistent gains in sequence realism alongside competitive functional specificity:

  • Sequence Composition: Across all four main benchmark tasks (8 task–metric pairs), ToDNA achieves the highest mean Motif Pearson and 3-mer Pearson correlations.
  • Functional Specificity: In additional comparisons against FastSeqProp and Ledidi across six tasks, ToDNA achieves the highest mean specificity on 3/6 tasks and outperforms Ledidi on 4/6 tasks. We greatly appreciate your valuable feedback. If you have any further concerns, please raise them in the forum, and we will actively address them.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.