acceptodds
Withdrawn from ICLR 2027

Controlled DNA Design via Rollout Reasoning with Discrete Flow Matching

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

We introduce DREAM-DNA (controlled Design via REasoning And Matched-flows for DNA), a reinforcement learning–based generative framework for DNA sequence design. Traditional models use diffusion architecture and rely on stochastic, single-pass generation. These limit their ability to correct early structural errors. DREAM-DNA overcomes these limitations by replacing stochasticity with Discrete Flow Matching. This establishes a deterministic ”straight-line” mapping from noise to data for superior trajectory control. Our framework further introduces an Iterative Rollout mechanism. Specifically, we treat generation as a multi-round refinement process. By iteratively masking and resampling subsets of nucleotides in the DNA sequence, the model ”critiques” and revises its intermediate outputs based on the reward feedback. Experiments on human enhancer design demonstrate that DREAM-DNA outperforms state-of-the-art baselines. It achieves a 13% boost in enhancer activity and an 8% increase in open chromatin match levels.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
DNA Sequence Generation, Discrete Flow-matching, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_CFt6OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces DREAM-DNA, a reinforcement learning framework that iteratively refines a discrete flow matching generator for DNA design. The framework performs rollouts with progressive random remasking and reinforces high-reward sequences with the existing fine-tuning algorithm DRAKES. There are no clear theoretical or algorithmic contributions.

Critical strengths

None

Critical weaknesses
  • This is a simple application of DRAKES for discrete flow matching with random remasked rollouts and there are no clear theoretical or algorithmic contributions of this framework and relevant prior works are not referenced or benchmarked (e.g. SEPO [1], TR2-D2 [2], GLIDE [3])
  • Unlike DRAKES and prior masked diffusion finetuning methods, DREAM-DNA was validated only on DNA (which most prior works do as one of their experiments) and nothing more. The experimental validation is very limited and not convincing.

[1] Zekri, O., & Boullé, N. (2025). Fine-tuning discrete diffusion models with policy gradient methods.

[2] Tang, S., Zhu, Y., Tao, M., & Chatterjee, P. (2025). Tr2-d2: Tree search guided trajectory-aware fine-tuning for discrete diffusion.

[3] Cao, H., Shi, H., Wang, C., Pan, S., & Heng, P. A. (2025). GLIDE: A gradient-free lightweight fine-tune approach for discrete biological sequence design.

Questions for authors

No questions whose answers could change my assessment.

Reviewer_FjidOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents DREAM-DNA, a framework for reward-guided regulatory DNA generation that combines discrete flow matching (DFM) with iterative remasking and regeneration. The method builds on DRAKES, using Gumbel–Softmax relaxation to backpropagate activity rewards through sampled trajectories while regularizing the generator toward a frozen reference model. During fine-tuning, successive cycles generate a sequence, remask a subset of positions, and regenerate them using the retained sequence as context. Experiments primarily evaluate HepG2 enhancer design. DREAM-DNA improves predicted activity and accessibility relative to the main DRAKES baseline but has lower motif and 3-mer correlations.

Overall, I find iterative refinement interesting and well-motivated. However, the practical advantage over existing methods and several central methodological claims remain insufficiently established.

Critical strengths
  1. Adapting discrete flow matching to DNA sequence generation is an interesting direction.
  2. Iterative remasking extends DRAKES-style reward optimization by allowing generated sequences to be revised during training. The ablations help characterize the benefits and limitations of this procedure.
Critical weaknesses
  1. The practical benefit of DFM over masked diffusion (MDLM) remains unclear. Low activity in an unguided model (Table 1) does not itself establish a limitation of its generative framework. A comparison using matched training data and computational budgets, with consistent held-out references for sequence composition and motif agreement, would better establish the benefit of DFM as a contribution.
  2. DREAM-DNA improves predicted activity over DRAKES but reduces motif and 3-mer correlations in the main comparison. This raises concerns about low sequence realism which limits broad performance claims.
  3. The main benchmark omits relevant methods, including GPA (Sarkar et al., 2026), SVDD (Li et al., 2024), DNA-CRAFT (Awasthi et al., 2026), and especially RERD (Uehara et al., 2025), which also performs reward-guided iterative refinement. Although Ctrl-DNA (Chen et al., 2025) appears in the appendix, a matched comparison in the main evaluation would be useful. Comparisons using common reward objectives and evaluation predictors, with computational costs reported, would clarify the practical benefit of DREAM-DNA.
  4. Evaluation remains limited to one enhancer dataset. Although Appendix C includes specificity evaluations across three cell types, an additional regulatory design benchmark, such as promoter design from Ctrl-DNA, would strengthen the generality claims.
  5. The DRAKES results differ substantially between Tables 1 and 2, making the contribution of rollout difficult to assess. Please reconcile these results and specify the KL settings and training budgets used in the ablations.
Questions for authors
  1. What is deterministic in the implemented sampler, and are transition rates at already unmasked positions explicitly set to zero? Please also clarify which diffusion processes the irreversibility argument applies to.
  2. Is iterative rollout used during training, inference, or both in the reported experiments? Can you compare DREAM-DNA and DRAKES with matched rollout depths and computational budgets, including training and generation times?
  3. Can you compare DREAM-DNA and DRAKES across KL strengths, including settings with similar activity or motif agreement, to establish whether the activity–sequence-quality trade-off improves?
  4. What explains the different DRAKES results across the tables and the stated percentage improvements? Please specify the number of independent training runs and what the error bars represent.
  5. Can you clarify the overlap between data used for generator training, reward-model training, evaluation-oracle training, and model selection? Which results are affected by the overlapping subsets in Appendix H, and how well does the external CNN oracle perform on held-out real sequences?
Reviewer_ndf5OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces DREAM-DNA, a reward-guided generative framework for regulatory DNA sequence design that combines discrete flow matching with iterative rollout refinement. The framework uses a pretrained discrete flow-matching model to generate DNA sequences from fully masked inputs and extends the DRAKES reward-optimization approach through multiple rounds of partial remasking and resampling. Specifically, the proposed rollout mechanism progressively remasks 100%, 80%, and 50% of sequence positions, allowing the model to revise intermediate generations while optimizing predicted enhancer activity under KL regularization. The authors evaluate DREAM-DNA primarily on human HepG2 enhancer design, reporting improvements in predicted enhancer activity and chromatin accessibility over DRAKES and other guidance-based baselines. Additional experiments investigate rollout depth, remasking schedules, sequence naturalness, diversity, and transfer to a separately trained CNN activity predictor. The paper also evaluates cell-type-specific regulatory sequence design across HepG2, K562, and SK-N-SH, where DREAM-DNA improves over some baselines but does not outperform the strongest Ctrl-DNA configurations.

Critical strengths
  • The first critical strength is the integration of discrete flow matching with iterative refinement for reward-guided DNA sequence generation. The proposed framework addresses a meaningful challenge in regulatory sequence design: improving functional objectives while retaining the ability to revise previously generated nucleotide configurations.

  • The second critical strength is the relatively systematic empirical investigation of iterative rollout and its effects on functional performance. The paper evaluates multiple rollout depths, remasking schedules, and remasking strategies, and includes a Best-of-3 baseline to examine whether iterative refinement offers benefits beyond generating additional independent candidates.

Critical weaknesses
  • The most important weakness is that the methodological novelty and the claimed reasoning mechanism are not sufficiently established. The proposed iterative rollout consists primarily of randomly remasking previously generated sequences and resampling the masked positions, which resembles existing iterative refinement and masked-generation procedures rather than a distinct reasoning mechanism. Although the authors describe the approach as allowing the model to critique and correct its intermediate outputs, the method does not explicitly identify unfavorable regulatory motifs, evaluate intermediate biological errors, or use structured feedback to determine which positions should be revised. Moreover, reward-guided iterative refinement for biological sequence design has already been explored in related work, including the cited study by Uehara et al. (2025). The paper does not sufficiently distinguish its contribution from these existing approaches or demonstrate that the proposed rollout learns a qualitatively different revision strategy. The characterization of discrete flow matching as deterministic is also potentially misleading, since Appendix F describes categorical sampling and optional stochastic remasking during inference. Consequently, the current evidence supports iterative remasking as a useful optimization technique, but provides weaker justification for the stronger claims of reasoning-based sequence design and deterministic trajectory control.

  • The second major weakness is that the experimental improvements are accompanied by substantial limitations in biological validation, sequence naturalness, and generalization. Although DREAM-DNA achieves the highest predicted activity and chromatin accessibility in the main HepG2 benchmark, its naturalness metrics remain substantially below several competing methods. In Table 1, DREAM-DNA achieves a 3-mer correlation of 0.500 and JASPAR correlation of 0.398, compared with 0.804 and 0.709 for DRAKES, respectively. This raises concerns that reward optimization may favor sequences with high predicted activity while substantially altering regulatory motif composition. The external CNN evaluation provides useful supporting evidence, but both activity predictors are trained on related MPRA data, and no experimental validation is provided. Furthermore, the additional cell-type-specific benchmark shows that DREAM-DNA consistently underperforms the strongest Ctrl-DNA configurations in specificity margins across all three target cell types. Appendix H also states that the evaluation subsets contain sequences observed during training, limiting conclusions about generalization to unseen biological conditions. These limitations are consequential because the paper's central motivation is controllable and biologically meaningful regulatory DNA design, whereas the reported results primarily establish improvements under computational predictors in a relatively restricted evaluation setting.

Questions for authors
  • Could the authors clarify what specifically constitutes reasoning in the proposed iterative rollout mechanism, and how it differs from existing reward-guided iterative refinement methods? The current procedure randomly remasks a fixed fraction of positions and regenerates them, without an explicit mechanism for identifying incorrect motifs or selecting revisions based on biological feedback. The manuscript also cites Uehara et al. (2025), which studies reward-guided iterative refinement for DNA and protein design. A more direct comparison with this approach, or an ablation demonstrating that DREAM-DNA learns meaningful correction behavior beyond repeated masked sampling, would materially affect my assessment of the methodological novelty.

  • Could the authors provide a more controlled comparison between discrete flow matching and discrete diffusion under matched model capacity, training budget, sampling steps, and reward-optimization settings? The main results suggest that the flow-matching backbone improves predicted activity, but it is unclear whether the gains arise from the underlying probability path, differences in pretraining quality, or optimization dynamics. Additionally, the manuscript repeatedly emphasizes deterministic trajectories, whereas the implementation describes categorical sampling and stochastic remasking. Clarifying which components are actually deterministic, and isolating the contribution of flow matching from other implementation differences, would strengthen the central methodological claim.

  • Could the authors investigate why the improvements in predicted activity are accompanied by substantial reductions in sequence naturalness, particularly JASPAR motif correlation? In Table 1, DREAM-DNA improves predicted activity from 5.88 to 6.74 relative to DRAKES, while JASPAR correlation decreases from 0.709 to 0.398. Although the rollout mechanism partially recovers naturalness relative to the no-rollout variant, the resulting sequences remain substantially less similar to natural enhancers under the reported motif statistics. It would be valuable to determine whether these differences reflect enrichment of specific functional motifs, systematic changes in sequence composition, or potential exploitation of the learned reward model. An analysis of motif enrichment and activity-naturalness trade-offs across different KL regularization strengths would materially affect the interpretation of the reported improvements.

  • Could the authors clarify the evaluation protocol and provide stronger evidence that the improvements generalize beyond the training distribution and optimization predictors? Appendix H indicates that the validation and test subsets contain sequences observed during training, while the external CNN oracle is trained on the same underlying MPRA dataset as the primary activity predictor. Although the use of different predictor architectures is helpful, it does not fully establish independence of the biological evidence. In particular, it would be useful to report performance on genuinely held-out sequences, characterize the predictive accuracy of both oracles on independent measured data, and clarify whether the same generated candidate sets are evaluated across predictors. Additionally, given that DREAM-DNA underperforms the strongest Ctrl-DNA configurations on cell-type specificity, the authors should clarify whether the proposed framework is primarily intended for maximizing target-cell activity or for achieving cell-type-specific regulatory control. These distinctions would materially affect the scope of the paper's conclusions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.