acceptodds
Under review as a conference paper at ICLR 2027

Ariadne: Regulatory DNA Design with Adaptive Reward Shaping and Reinforcement Learning

Abstract

Cis-regulatory elements (CREs), including enhancers, regulate gene expression through the composition and organization of transcription factor binding sites (TFBSs) within their sequence context. The vast sequence space makes exhaustive experimental screening infeasible, motivating computational methods to design enhancers with high activity in a target cell type. A central challenge is to improve predicted activity while determining whether optimization narrows the generated sequence distribution and whether gains extend beyond the predictor used for training. We present Ariadne, a reinforcement-learning framework that combines an enhancer sequence prior with target-cell-directed optimization to address this challenge. Ariadne first adapts an autoregressive DNA language model to massively parallel reporter assay (MPRA)-derived enhancer sequences through a single shared supervised fine-tuning (SFT) stage and then trains separate target-cell policies with proximal policy optimization (PPO), using the corresponding Enformer prediction as the reward signal. To account for changing reward distributions during optimization, the evaluated reward combines exponential moving average (EMA) normalization, within-batch ranking, and a clipped bonus above an EMA-tracked upper-quantile threshold. Normalization controls favor an evolving reward reference over fixed SFT statistics, but do not establish a universal advantage of EMA over batch normalization. Component ablations support modest incremental Enformer gains from rank, whereas tail has cell-dependent returns that are not uniformly reproduced by Malinois. PPO improves the reported Enformer and Malinois summaries relative to the shared SFT policy, but Hamming diversity decreases from SFT and Enformer-embedding concentration increases. These results support Ariadne as a computational pipeline for improving predicted enhancer activity and characterizing the accompanying distributional trade-offs. They do not establish universal superiority of the complete EMA–rank–tail reward, experimental activity, or cell-type selectivity.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Cis-regulatory element design, enhancer design, DNA language models, reinforcement learning, reward shaping, proximal policy optimization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_hFo6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper presents Ariadne, a reinforcement-learning framework for regulatory DNA design. It first fine-tunes an autoregressive DNA language model on MPRA-derived sequences to establish a shared, unconditional sequence prior, then trains separate target-cell policies using Enformer-guided PPO. Its main contribution is an adaptive reward combining EMA-based activity normalization, within-batch ranking, and a bounded bonus for exceeding a historical upper-tail threshold.

Validation comprises computational experiments in HepG2, K562, and SK-N-SH, evaluated using both the training reward modelan external predictor. Reward ablations and analyses of training dynamics assess the contributions of individual reward components and their effects on predicted activity and sequence diversity.

Critical strengths
  • The paper clearly motivates the challenges in regulatory DNA design and presents the proposed method in a well-organized manner. Its explanation of the two-stage training pipeline and the roles of individual reward components makes the approach easy to understand and assess.

  • The appendices provide detailed hyperparameters, training algorithms, dataset splits, and evaluation procedures, including the reward-state updates and stagewise PPO settings. These implementation details improve transparency and facilitate reproduction of the proposed method.

Critical weaknesses
  • The paper has insufficient demonstration that the reward design addresses the motivating problems. Although the intended roles of EMA, rank, and tail are explained, the main paper does not directly characterize the changing reward distributions or demonstrate how the proposed components address the resulting optimization difficulties.
  • The paper has insufficient controlled comparison with existing RL methods. The paper lacks protocol-matched comparisons with alternative algorithms such as GRPO, while the TACO results come from a different training protocol. Comparisons using the same initialization, oracle, and query budget are needed to establish whether Ariadne offers a meaningful improvement over existing RL-based design approaches.
Questions for authors
  1. Is the proposed reward intended primarily to accommodate changing reward scales, or also to mitigate concentration of generated sequences? Table 1 reports Hamming diversity, while the appendix reports novelty and representation concentration. What do these results establish about the trade-off between predicted activity and sequence quality?
  2. Ariadne already incorporates adaptive KL regularization, a mechanism used in prior RL fine-tuning work to limit reward-model overoptimization [1]. What evidence separates the benefits of the proposed shaping from changes in the effective balance between activity reward and KL regularization? Prior work highlights the importance of controlling implementation details when attributing RL performance gains [2].
  3. How does the proposed method differ from established reward scaling and clipping [2], relative-ranking reward designs [3], and upper-quantile, risk-seeking policy optimization [4]? Is the methodological contribution primarily the particular combination of EMA normalization, ranking, and a historical tail threshold, or a distinct mechanism?

[1] Ouyang et al. Training Language Models to Follow Instructions with Human Feedback. NeurIPS, 2022.

[2] Engstrom et al. Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO. ICLR, 2020.

[3] Niu et al. From Absolute to Relative: Rethinking Reward Shaping in Group-Based Reinforcement Learning. ICML, 2026.

[4] Petersen et al. Deep Symbolic Regression: Recovering Mathematical Expressions from Data via Risk-Seeking Policy Gradients. ICLR, 2021.

5 comments
AuthorsOpenReview ↗

Reward attribution and training protocol

We thank the reviewer for asking us to distinguish reward shaping from scale, KL regularization, and prior methods. We audited the implementation and clarify the following.

Training protocol. All PPO configurations start from the same SFT checkpoint and use ten stages of 100 scheduled steps. Stages 2–10 start from the preceding stage's Enformer-selected best policy. We report the stage-10 final policy, without early stopping or selection across seeds. Thus, training has a fixed schedule but includes stagewise Enformer-based checkpoint selection.

Reward and KL. All configurations use reward whitening and a fixed KL coefficient of 0.12. Terminal PPO approximate KL values are broadly similar, but KL trajectories and applied update counts are unavailable. We cannot fully separate reward shaping from effective optimization strength.

EMA is a historical z-score clipped at ±5 and reset at each stage. Without clipping, per-rollout whitening cancels this affine transformation in exact arithmetic. Since clipping frequency was not logged, these runs do not establish a benefit from historical EMA. The original adaptive-KL protocol did not use this whitening and must be interpreted separately.

Interpretation. The reward acts on oracle scores; it does not explicitly prevent representation concentration. Full ERT does not consistently outperform simpler rewards across cell types and predictors. We will narrow the contribution to an empirical study of reward configurations and their limitations.

AuthorsOpenReview ↗

Predicted activity and cross-predictor agreement

Fixed-candidate results. The table reports Enformer top-16 (E) and Malinois on those same sequences (M), as mean ± standard deviation over five training seeds. Top-16 is computed within each 256-sequence draw and averaged over 59 generation seeds per policy.

RewardHepG2 E / MK562 E / MSK-N-SH E / M
EMA0.912±0.023 / 0.819±0.0200.990±0.006 / 0.765±0.0590.923±0.033 / 0.897±0.041
EMA+rank0.928±0.018 / 0.834±0.0310.995±0.007 / 0.781±0.0660.958±0.019 / 0.895±0.021
EMA+tail0.888±0.098 / 0.792±0.0750.995±0.005 / 0.785±0.0500.909±0.086 / 0.841±0.140
ERT0.933±0.021 / 0.794±0.0540.993±0.006 / 0.774±0.0560.940±0.034 / 0.910±0.014

Sequence quality. Exact uniqueness is 1.0 in each merged pool of 15,104 sequences. For the pooled Enformer top-128, median pairwise Hamming distance is 129.8–137.0 bp for EMA and 128.0–137.6 bp for ERT across cell types. Their mean pairwise Enformer-embedding cosines are 0.974–0.984 and 0.980–0.987, respectively. Sequence dispersion therefore coexists with embedding similarity; this alone does not prove oracle exploitation.

Prior work and GRPO. Our method combines historical normalization/clipping, within-batch ranking, and a bounded bonus above a historical threshold. These relate to established scaling, relative-reward, and upper-tail methods; we do not claim a new policy-gradient estimator.

PPO-ERT and GRPO-ERT share initialization, oracle, stagewise reference schedule, and scheduled update count. They differ in KL coefficient (0.12 vs. 0.04), whitening, and advantage estimation. GRPO can skip minibatches, and actual update counts are unavailable; therefore, this is an exploratory comparison.

MethodHepG2 E / MK562 E / MSK-N-SH E / M
PPO-ERT0.933±0.021 / 0.794±0.0540.993±0.006 / 0.774±0.0560.940±0.034 / 0.910±0.014
GRPO-ERT0.876±0.027 / 0.844±0.0760.967±0.008 / 0.785±0.0590.975±0.017 / 0.865±0.046

Neither method is better on both predictors across all cell types. GRPO-ERT has higher Enformer–Malinois Spearman correlation on all five HepG2 and K562 training seeds: 0.641±0.045 vs. 0.098±0.083 for HepG2, and 0.580±0.147 vs. 0.170±0.075 for K562 (GRPO-ERT vs. PPO-ERT). Top-128 overlap is also higher for GRPO-ERT in these two cell types. These results indicate stronger cross-predictor ranking agreement in these settings.

AuthorsOpenReview ↗

Sequence diversity and quality

Sequence diversity and quality. We computed Hamming diversity and embedding similarity on the Enformer-selected top-128 sequences from each merged pool, and computed sequence violations on the full merged pool of 15,104 sequences per training seed. The table reports median pairwise Hamming distance (bp), summarized across five training seeds. Paired differences are PPO-ERT minus GRPO-ERT.

Cell typePPO-ERTGRPO-ERTPaired differencePPO higher
HepG2137.6±3.4134.6±2.2+3.0±3.93/5 seeds
K562128.0±4.4118.8±2.8+9.2±6.45/5 seeds
SK-N-SH131.8±8.8111.8±32.5+20.0±33.14/5 seeds

The SK-N-SH mean difference is strongly influenced by one GRPO-ERT seed with Hamming distance 54. On HepG2, the merged-pool difference is positive in only three seeds. When diversity is instead computed within each 256-sequence generation draw and averaged, PPO-ERT is higher in 5/5, 5/5, and 4/5 seeds for HepG2, K562, and SK-N-SH, respectively.

Other metrics do not show a uniform PPO advantage. For the Enformer-selected top-128 from each merged pool, mean pairwise Enformer-embedding cosine is 0.981±0.004 vs. 0.979±0.003 in HepG2, 0.987 for both methods in K562, and 0.980±0.007 vs. 0.994±0.005 in SK-N-SH (PPO-ERT vs. GRPO-ERT). Novelty is similar in HepG2 and approximately 2 bp higher for PPO-ERT in K562 and SK-N-SH. Merged-pool uniqueness is 1.0 for both methods except GRPO-ERT SK-N-SH seed 2 (0.942).

In the full merged pool of 15,104 sequences per training seed, the sequence-violation fraction is lower for PPO-ERT on all five K562 and SK-N-SH seeds: 0.045±0.049 vs. 0.738±0.096, and 0.144±0.107 vs. 0.896±0.142, respectively. A sequence is counted as violating if its GC fraction is outside 0.25–0.75, it contains an invalid base, or its homopolymer length exceeds 6. This is a reporting metric, not a training objective. All comparisons are descriptive and do not isolate the effect of the RL algorithm.

Reviewer_di7oOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Ariadne, a reinforcement-learning framework for designing regulatory DNA sequences with high predicted activity in a target cell type. The framework first performs a single shared, unconditional supervised fine-tuning stage on a 1.2B-parameter GENERator-v2 model using MPRA-derived enhancer sequences. The resulting sequence prior is then used to initialize separate PPO policies for HepG2, K562, and SK-N-SH, with cell-specific Enformer predictors providing the optimization signal. The main methodological component is an adaptive reward that combines EMA-based normalization of predicted activity, within-batch rank information, and a bounded bonus for sequences exceeding a historically tracked upper-tail threshold.

Critical strengths
  • The first critical strength is the careful empirical analysis of reward shaping for predictor-guided regulatory DNA design. Rather than simply proposing the complete EMA+rank+tail reward and comparing it against an unshaped baseline, the paper studies the individual reward components, their combinations, raw Enformer reward, and alternative normalization strategies.

  • The second critical strength is that the evaluation explicitly examines failure modes and distributional trade-offs rather than reporting only the optimized oracle score. The authors evaluate Malinois as a reward-independent secondary predictor and additionally analyze Hamming diversity, embedding concentration, uniqueness, novelty, naturalness, motif profiles, and predicted target-versus-off-target activity.

Critical weaknesses
  • The most important weakness is that the evidence for biological functionality and generalization beyond the optimization oracle remains limited. Enformer is simultaneously used as the PPO reward, the checkpoint-selection criterion, and the primary evaluation model. Malinois provides a useful secondary evaluation, but it is also trained using related MPRA measurements and, importantly, independently selects its own top candidates rather than evaluating exactly the Enformer-selected set. The authors themselves acknowledge that this setup cannot rule out oracle exploitation without fixed-set cross-predictor evaluation, rank correlations, or top-set overlap. This concern is particularly important given the strong increase in Enformer embedding concentration after PPO. Without prospective MPRA experiments or stronger independent fixed-candidate validation, the paper demonstrates optimization of computationally predicted activity rather than experimentally validated regulatory activity.

  • The second major weakness is that the evidence does not establish the complete EMA+rank+tail reward as clearly preferable to substantially simpler alternatives. EMA-only already captures much of the improvement, adding rank produces relatively small but reasonably consistent Enformer gains, and the tail component is inconsistent across cell types. For example, EMA+rank achieves higher Enformer scores than ERT for SK-N-SH, while EMA-only gives the highest Malinois score; for K562, ERT and EMA+rank are nearly identical on Enformer metrics, while other configurations perform better under Malinois. Furthermore, reward magnitude and realized KL regularization are not matched across configurations, so some differences may reflect changes in effective optimization strength rather than the intended structure of the reward itself. Consequently, the experiments support adaptive normalization and, to a lesser extent, rank shaping, but provide weaker justification for the full ERT formulation that is presented as Ariadne's main reward design.

Questions for authors
  • Could the authors provide a controlled comparison of EMA-only, EMA+rank, and ERT in which the effective reward scale and realized KL divergence are approximately matched? The current experiments show that rank provides modest incremental gains but that tail is not consistently beneficial, while the manuscript notes that the different reward formulations can interact differently with adaptive KL regularization and the value loss. Such a comparison would materially affect my assessment of whether ERT itself constitutes a meaningful methodological contribution beyond adaptive reward normalization.

  • Could the authors evaluate exactly the same candidate sequences across Enformer and Malinois, for example, by taking the Enformer-selected top candidates and reporting their Malinois scores, rank correlations, and top-set overlap? Currently, each predictor independently ranks the candidate pool, meaning that improvements in the two reported Top metrics need not correspond to agreement on the same optimized sequences. A fixed-set analysis would substantially strengthen or weaken the evidence that PPO improvements transfer beyond the optimization oracle.

  • Could the authors provide the main ERT results using only prespecified, fixed-budget runs with a consistent checkpoint-selection protocol across all reward configurations? Appendix I.2 states that the archived endpoint summary can involve selection between available ERT evaluations using Enformer Fitness and that some archived runs enabled early stopping, which can make the reported Enformer endpoint optimistic. Because these ERT numbers are central to the main comparison, understanding how much of the reported advantage remains under a strictly matched protocol would materially affect the evaluation.

  • How should the very large increase in Enformer embedding similarity after PPO be interpreted with respect to possible oracle exploitation? The embedding similarity rises to approximately 0.97-0.98 across all three cell types, despite the generated sequences retaining relatively high Hamming diversity. It would be useful to know whether the authors have examined whether the high-scoring candidates occupy a narrow predictor-specific representation regime, and whether the sequences that transfer best to Malinois show the same degree of concentration. This would help determine whether the activity improvements reflect robust regulatory sequence design or increasingly concentrated optimization of features favored by Enformer.

3 comments
AuthorsOpenReview ↗

Reward attribution and fixed-budget protocol

We thank the reviewer for highlighting the limitations of reward attribution and checkpoint selection. We agree that the evidence does not establish full EMA+rank+tail (ERT) as consistently preferable to simpler rewards.

Reward scale and KL. We ran a supplementary five-seed comparison of EMA, EMA+rank, EMA+tail, and ERT. Rewards are shaped and then whitened within each rollout; the KL coefficient is fixed at 0.12. The table reports Enformer top-16 (E) and Malinois on those same sequences (M). Metrics are computed within each 256-sequence draw, averaged over 59 generation seeds, and summarized across five training seeds as mean ± sample standard deviation.

RewardHepG2 E / MK562 E / MSK-N-SH E / M
EMA0.912±0.023 / 0.819±0.0200.990±0.006 / 0.765±0.0590.923±0.033 / 0.897±0.041
EMA+rank0.928±0.018 / 0.834±0.0310.995±0.007 / 0.781±0.0660.958±0.019 / 0.895±0.021
EMA+tail0.888±0.098 / 0.792±0.0750.995±0.005 / 0.785±0.0500.909±0.086 / 0.841±0.140
ERT0.933±0.021 / 0.794±0.0540.993±0.006 / 0.774±0.0560.940±0.034 / 0.910±0.014

Full ERT does not consistently outperform EMA+rank across cell types and predictors. Terminal PPO approximate KL values are:

RewardHepG2K562SK-N-SH
EMA0.074±0.0470.059±0.0230.078±0.028
EMA+rank0.080±0.0380.069±0.0230.082±0.037
EMA+tail0.084±0.0440.064±0.0220.087±0.021
ERT0.077±0.0530.062±0.0180.077±0.031

These are terminal-step values; KL trajectories and applied update counts are unavailable. The comparison therefore only partially controls scale and KL. Reward shape may still affect advantages, value loss, and optimization dynamics.

Without clipping, per-rollout whitening cancels an affine EMA transformation shared across the rollout in exact arithmetic. Clipping frequency was not logged, so these runs do not independently establish a benefit from historical EMA. The original adaptive-KL protocol did not use this whitening.

Checkpoint selection. The supplementary runs use ten stages of 100 scheduled steps, without early stopping or selection across training seeds. We report the stage-10 final policy. However, stages 2–10 start from the previous stage's Enformer-selected best_policy. The runs therefore share a scheduled budget and stagewise rule but do not eliminate Enformer-based checkpoint selection. We have not rerun the comparison by continuing from the final policy at every stage.

AuthorsOpenReview ↗

Fixed-candidate evaluation

Fixed-candidate evaluation. We evaluated Malinois on the sequences selected by Enformer, using the stage-10 final policies from the fixed-whitening runs above. Metrics are computed within each 256-sequence generation draw, averaged over 59 generation seeds, and summarized across five training seeds as mean ± sample standard deviation.

EMA+rank: Malinois score onHepG2K562SK-N-SH
Enformer top-160.834±0.0310.781±0.0660.895±0.021
Enformer top-1280.829±0.0340.784±0.0620.888±0.025

Post-SFT Malinois scores on Enformer-selected sequences are absent from the imported summary, so these values do not quantify improvement over SFT.

Enformer–Malinois Spearman correlation and top-set overlap on the same draws are:

RewardCell typeSpearmanTop-16 overlapTop-128 overlap
EMA+rankHepG20.182±0.0390.083±0.0140.558±0.015
EMA+rankK5620.138±0.0890.061±0.0130.542±0.031
EMA+rankSK-N-SH0.387±0.1210.082±0.0240.625±0.047
ERTHepG20.098±0.0830.066±0.0260.525±0.029
ERTK5620.170±0.0750.064±0.0200.553±0.026
ERTSK-N-SH0.389±0.2190.093±0.0260.640±0.091

Overlap is the intersection size divided by k. For two independently and uniformly selected sets of size k from 256 candidates, the expected overlap fraction is k/256:

Random setsExpected overlap
Top-16 from 2560.0625
Top-128 from 2560.5000

The observed top-16 overlap is close to this reference, especially in K562. This reference is descriptive and is not a significance test.

Additional predictor. We evaluated the Enformer-selected top-16 with the 90-model MPRA-LegNet ensemble trained on Agarwal human lentiMPRA labels. Both rows report the mean raw LegNet score of those same 16 sequences within each 256-sequence draw.

ModelHepG2 LegNetK562 LegNet
Post-SFT0.364±0.1600.700±0.119
EMA+rank1.810±0.2141.198±0.053
ERT1.657±0.1621.206±0.054

ERT achieves high Malinois scores on the same Enformer-selected candidates, with results varying across cell types. Compared with EMA+rank, ERT has a higher mean fixed-set Malinois score in SK-N-SH but a lower score in HepG2. Together with the limited top-set overlap, these results support reporting ERT as a computational design method while tempering claims of consistent cross-predictor superiority.

AuthorsOpenReview ↗

Representation concentration

Representation concentration. For each training seed, we merged 59 generation draws into a pool of 15,104 sequences and selected the Enformer top-128. Mean pairwise Enformer-embedding cosine is:

RewardHepG2K562SK-N-SH
EMA0.978±0.0060.984±0.0030.974±0.007
EMA+rank0.983±0.0020.988±0.0010.982±0.003
ERT0.981±0.0040.987±0.0030.980±0.007

For the same pooled top-128, diversity is the median pairwise Hamming distance in base pairs:

RewardHepG2K562SK-N-SH
EMA136.0±4.8129.8±1.9137.0±5.6
EMA+rank134.0±4.9129.2±4.0133.4±2.1
ERT137.6±3.4128.0±4.4131.8±8.8

Exact uniqueness in each full merged pool is 1.0. Adding rank or tail did not consistently reduce Enformer-embedding similarity relative to EMA.

We also measured Enformer-embedding cosine within each 256-sequence generation draw. Each entry is the mean pairwise cosine of that draw's top-128, averaged over 59 generation draws and summarized across five training seeds as mean ± sample standard deviation.

RewardCell typeEnformer top-128Malinois top-128
EMAHepG20.971±0.0120.964±0.013
EMAK5620.981±0.0030.976±0.004
EMASK-N-SH0.965±0.0040.957±0.003
EMA+rankHepG20.974±0.0040.967±0.006
EMA+rankK5620.984±0.0020.980±0.003
EMA+rankSK-N-SH0.970±0.0090.961±0.012
ERTHepG20.973±0.0060.966±0.007
ERTK5620.982±0.0030.977±0.004
ERTSK-N-SH0.969±0.0110.960±0.016

High sequence-level diversity thus coexists with high Enformer-embedding similarity, including among candidates ranked highly by Malinois. This is consistent with representation concentration but does not establish its cause or prove oracle exploitation. Without prospective MPRA, we cannot claim experimentally validated regulatory activity. We will report these metrics together and narrow the corresponding claims.

Reviewer_M9jBOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The authors propose Ariadne, a reinforcement learning framework for designing CREs (i.e. 200 bp enhancers). They first fine-tune the GENERator-v2 DNA LM on MPRA sequences using SFT. They then train separate policies for HepG2, K562, and SK-N-SH cell lines with PPO, using rewards derived from a regression head on Enformer used to predict MPRA activity. Their custom reward (“ERT”) combines EMA-normalized activity, within-rollout percentile rank (“rank”), and a clipped term for scores above an upper-quantile threshold (“tail”). Enformer is used for both checkpoint selection and to score activity. Evaluation by Malinois and all sequence/representation-level metrics are reported post hoc and do not affect the reward or checkpoint selection. The authors report improved predicted activity over the SFT-only checkpoint under evaluation by both Enformer and Malinois. Ablations show that EMA is the strongest component of the custom reward, the rank component leads to small gains as evaluated by Enformer, and the tail improves activity in HepG2 but is comparable in K562 and slightly lower in SK-N-SH.

Critical strengths
  1. In Table 8 (Appendix), the authors show that Enformer embedding similarity increases substantially after PPO (0.401 to 0.976 in HepG2, 0.372 to 0.979 in K562, 0.423 to 0.971 in SK-N-SH), even though Hamming diversity remains high (Table 1) and there is 100% exact-sequence uniqueness (Appendix Table 9). This is a useful result on evaluating sequence design models. Exact uniqueness and pairwise Hamming distance can look high even when generated sequences become concentrated in the oracle’s representation space, which motivates using more robust metrics for evaluating model-designed CREs, e.g. representation quality and uniqueness.

  2. Ablation experiments are well designed to identify which components of the reward give reported gains. The three terms in ERT are tested individually, in pairs, and jointly. Each trial correctly starts from the SFT checkpoint with matched seeds and matched budgets (ten stages of 100 PPO updates).

Critical weaknesses
  1. There is circularity and leakage in the evaluation procedure, which I believe to be a critical flaw in the paper’s evaluation. Enformer is used to produce the PPO reward, to select checkpoints, and to score the primary activity metric. Malinois is used as another activity predictor, but it selects its highest-scoring candidate sequences (i.e. instead of evaluating the candidates selected by Enformer). Malinois was also trained on the same MPRA collection Gosai et al. 2024 used during SFT (Appendix A.1). The reported results thus do not establish cross-predictor agreement on the sequences that the oracle would prioritize. Another serious issue is that the main ERT results were obtained by selecting among available evaluations using Enformer Fitness, which is the same metric used for downstream eval. Some of the candidate runs also used early stopping (Appendix I.2; in contrast to the fixed ten-stage protocol reported in Section 4.1), which the authors themselves note could make the results look optimistic. This limits confidence that the reported gains reflect the reward's design rather than differences in checkpoint selection/training protocol.

  2. Novelty over closely related work is weak and there is no controlled comparison against these related methods. TACO (Yang et al. 2025) already combines RL on top of a pretrained autoregressive DNA LM for CRE design, including the same three cell lines and 200-bp enhancer setting used by the authors. Ctrl-DNA (Chen et al. 2025) also applies RL to autoregressive gLMs, while explicitly constraining for off-target activity. Ariadne's most significant distinction is in the particular combination of EMA normalization, within-batch ranking, and the upper-tail bonus term in the ERT reward. However, the authors show that EMA does not consistently outperform batch normalization, rank provides only incremental gains, and tail is not beneficial across all cell types or predictors. In addition, the baseline performance of TACO and other methods is copied directly from their source papers under different protocols, rather than rerun to match the author's experimental setup (Appendix E.3). Ctrl-DNA is not compared against at all. The authors do not establish that the reward design provides a substantial improvement over these approaches.

Questions for authors

Clarifications:

  1. The authors state that the reported ERT results involve selection among available evaluations using Enformer Fitness and that some runs used early stopping (Appendix I.2). Which evaluations were included during selection, and how did their training/checkpoint selection procedure differ from the controls? Can the authors report five-seed ERT results under the same fixed budget protocol used for the ablation experiments (Table 2)?

  2. Do the authors have any comparisons against TACO and Ctrl-DNA with matched initialization, budget, and eval procedures? Ctrl-DNA is particularly important here because it uses the same enhancer cell lines and Ariadne reuses its normalization intervals (Appendix F.2), allowing for easy comparison.

  3. Can the authors report Malinois scores for the Enformer top-16 and top-128 sequences from Table 1? The current evaluation independently ranks the same generated pool with each predictor, so improvements in the top-set metrics do not establish cross-predictor agreement on candidates that Enformer would prioritize. When candidates are selected exclusively by Enformer, does Malinois score these sequences higher over SFT and the simpler controls?

Future work suggestion:

  1. Evaluate generated candidates using a predictor trained on independent experimental data, instead of the Gosai et al. 2024 MPRA collection. This would help assess generalizability of the method.
5 comments
AuthorsOpenReview ↗

Fixed-schedule ERT results

We thank the reviewer for identifying the limitations of using Enformer for reward, checkpoint selection, and primary evaluation. We reran the reward configurations under a common fixed schedule and report all five training seeds. The results support ERT as a computational design configuration, but do not show that full ERT is consistently preferable to simpler rewards.

Correction to Appendix I.2. We audited the original runs and found an error in the manuscript: early stopping was not enabled, and all runs completed ten stages. We apologize for this misstatement. The original Appendix I.2 summary nevertheless selected among available evaluations using Enformer fitness. We will correct the description and distinguish that summary from the fixed-schedule results below.

Protocol. Each supplementary run uses ten stages of 100 scheduled PPO steps, with early stopping disabled. We report the stage-10 final policy without selecting across training seeds or among the archived Appendix I.2 endpoints. However, stages 2–10 start from the previous stage's best_policy, selected by an Enformer-based score at step 1, every five steps, and the final step. Thus, the runs share a scheduled budget and stagewise rule but still include Enformer-based checkpoint selection. We have not rerun the comparison by continuing from the final policy at every stage.

The 256,000 scheduled target-cell training sequences do not include all additional evaluation calls and are not a verified total oracle query count.

Results. E is the Enformer mean of the top-16 sequences within each 256-sequence generation draw; M is Malinois on those same sequences. Metrics are averaged over 59 generation seeds and then summarized across five training seeds as mean ± sample standard deviation.

RewardHepG2 E / MK562 E / MSK-N-SH E / M
EMA0.912±0.023 / 0.819±0.0200.990±0.006 / 0.765±0.0590.923±0.033 / 0.897±0.041
EMA+rank0.928±0.018 / 0.834±0.0310.995±0.007 / 0.781±0.0660.958±0.019 / 0.895±0.021
EMA+tail0.888±0.098 / 0.792±0.0750.995±0.005 / 0.785±0.0500.909±0.086 / 0.841±0.140
ERT0.933±0.021 / 0.794±0.0540.993±0.006 / 0.774±0.0560.940±0.034 / 0.910±0.014

ERT has the highest mean Enformer score in HepG2 and the highest mean fixed-set Malinois score in SK-N-SH among these configurations. The pattern varies by cell type and predictor; we will not claim consistent superiority of the complete reward.

AuthorsOpenReview ↗

Fixed-candidate evaluation

Fixed-candidate evaluation. We evaluated Malinois on exactly the sequences selected by Enformer. The table reports Malinois scores on each draw's Enformer top-16 and top-128.

RewardSelectionHepG2K562SK-N-SH
EMAEnformer top-160.819±0.0200.765±0.0590.897±0.041
EMAEnformer top-1280.819±0.0170.768±0.0570.886±0.035
EMA+rankEnformer top-160.834±0.0310.781±0.0660.895±0.021
EMA+rankEnformer top-1280.829±0.0340.784±0.0620.888±0.025
ERTEnformer top-160.794±0.0540.774±0.0560.910±0.014
ERTEnformer top-1280.796±0.0570.773±0.0560.902±0.006

These results evaluate the same candidates rather than allowing Malinois to select its own top sequences. ERT's mean fixed-set Malinois score is higher than EMA+rank in SK-N-SH but lower in HepG2. Post-SFT Malinois scores for Enformer-selected sequences are unavailable in the imported files, so this analysis does not establish improvement over SFT.

Enformer–Malinois top-16 overlap is close to the expected overlap fraction of 0.0625 for two independently and uniformly selected 16-element sets from 256 candidates. The observed values are 0.061–0.064 in K562 and 0.066–0.093 in the other cell types. Top-128 overlap is 0.525–0.640, compared with a random-set expectation of 0.500. These references are descriptive, not significance tests.

Malinois uses related Gosai MPRA data, and we have not performed prospective MPRA validation. We will restrict our claims to computationally predicted activity.

AuthorsOpenReview ↗

TACO and Ctrl-DNA comparison

TACO and Ctrl-DNA. We rerun both methods and generated held-out candidates using 59 generation seeds, with 256 sequences per draw. The comparison is informative but not fully matched: TACO uses Hyena, Ctrl-DNA includes off-target objectives, and the methods differ in optimization settings. Held-out Enformer scores below use the Ctrl-DNA min-max stored in these generation files. We do not mix them with training-round true_score.

The table reports Malinois on each method's own Enformer-selected top-16, averaged over generation draws and summarized across five training seeds.

MethodHepG2K562SK-N-SH
EMA+rank0.834±0.0310.781±0.0660.895±0.021
ERT0.794±0.0540.774±0.0560.910±0.014
TACO0.717±0.0330.697±0.0980.900±0.026
Ctrl-DNA0.657±0.2590.748±0.3350.670±0.269

EMA+rank has the highest mean for HepG2 and K562. ERT has the highest mean for SK-N-SH. These are descriptive comparisons under different protocols, not evidence that either reward is superior under matched conditions. Ctrl-DNA also shows substantial variation: HepG2 seed 2 has merged-pool uniqueness 0.213 and within-draw diversity 1.2. LegNet is a 90-model ensemble trained on the Agarwal human lentiMPRA for HepG2 and K562. Its training data are separate from the Gosai MPRA used to train the Enformer heads and Malinois. No LegNet model is available for SK-N-SH, so those entries are left blank. The reported values are raw computational predictions for the Enformer-selected top-16, not prospective MPRA measurements.

Off-target Ariadne and Ctrl-DNA. The Ariadne off-target runs use the ERT weights, fixed KL coefficient 0.12, no second whitening, and a penalty from the non-target Enformer models. We report the stage-10 final policy for training seeds 0, 1, 2, 42, and 100. On the same 59-draw held-out, E is the within-draw Enformer top-16 and M is Malinois on those sequences. LegNet is the raw 90-model score of that same top-16. SK-N-SH has no LegNet model.

MethodCellEM on Enformer top-16LegNet on Enformer top-16
Ariadne off-targetHepG20.722±0.0230.572±0.0490.776±0.118
Ariadne off-targetK5620.846±0.0550.635±0.0541.080±0.310
Ariadne off-targetSK-N-SH0.860±0.0410.902±0.107—
Ctrl-DNAHepG20.696±0.0360.657±0.2590.971±1.300
Ctrl-DNAK5620.896±0.0330.748±0.3351.255±0.613
Ctrl-DNASK-N-SH0.701±0.0690.670±0.269—

For the Ariadne off-target runs, the same Enformer top-16 has these scores in the other two cell lines. The margin is the target mean minus the larger off-target mean, calculated inside each draw and then summarized across training seeds.

TargetTarget E / MFirst off-target E / MSecond off-target E / ME marginM margin
HepG20.722±0.023 / 0.572±0.049K562 0.658±0.060 / 0.429±0.069SK-N-SH 0.596±0.082 / 0.466±0.0910.064±0.0720.103±0.046
K5620.846±0.055 / 0.635±0.054HepG2 0.583±0.056 / 0.533±0.041SK-N-SH 0.560±0.032 / 0.467±0.0330.256±0.0870.102±0.069
SK-N-SH0.860±0.041 / 0.902±0.107HepG2 0.699±0.009 / 0.798±0.068K562 0.837±0.041 / 0.750±0.1030.023±0.0110.105±0.047

The 59-seed Ctrl-DNA files do not contain non-target cell scores. Its available specificity numbers are the saved round-1000 batch of 256 sequences, on the 16 sequences with the highest true_score. That selection and pool are separate from the 59-draw table above.

TargetTarget E / MFirst off-target E / MSecond off-target E / ME marginM margin
HepG20.638±0.083 / 0.548±0.309K562 0.442±0.099 / 0.261±0.132SK-N-SH 0.379±0.105 / 0.272±0.0970.197±0.1650.249±0.371
K5620.876±0.045 / 0.614±0.280HepG2 0.371±0.060 / 0.389±0.029SK-N-SH 0.294±0.072 / 0.256±0.0370.505±0.0300.225±0.271
SK-N-SH0.611±0.119 / 0.566±0.165HepG2 0.321±0.091 / 0.318±0.089K562 0.366±0.071 / 0.242±0.1170.226±0.1330.222±0.115

On the shared 59-draw definition, Ctrl-DNA has the higher Malinois and LegNet means for HepG2 and K562, with large seed variation, while the Ariadne off-target run is higher on SK-N-SH. For SK-N-SH, the Ariadne off-target sequences selected by the target Enformer remain high on K562 Enformer, 0.837±0.041 against a target value of 0.860±0.041. We will disclose the differences in initialization, objectives, optimization, and query budgets rather than treating these runs as a fully matched benchmark.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.