- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies distributional collapse when using RL to optimize DNA discrete diffusion models. The main observation is that this collapse depends on generation order. Based on this, the paper proposes DP-DNA, which jointly learns where to generate and what to generate, together with a real-sequence training stream adapted from LoMDM. The results are generally strong across three enhancer-design tasks. My main concern is novelty and significance: most of the key ingredients already appear in DCoLT and LoMDM, so the new part is largely their combination in DNA RL. I am not yet convinced the current experiments isolate a large enough benefit from learned order itself to make the contribution sufficiently significant for ICLR.
Critical strengths
-
The paper identifies a meaningful failure mode beyond reward alone. Across both D3LM and SLM, RL improves predicted reward while substantially distorting GC content and local sequence diversity, and the reveal-order experiments show that these effects depend strongly on which positions are resolved first. This is a useful empirical observation for reward-based biological sequence design.
-
The empirical results are fairly strong. DP-DNA performs well across three cell types and multiple functional and distributional metrics, and the paper includes several component ablations rather than relying only on aggregate benchmark performance.
Critical weaknesses
-
Novelty: The methodological novelty appears limited relative to closely related work. DCoLT already jointly trains an unmasking policy and token generator with outcome-based RL, while LoMDM already learns context-dependent generation order using learned corruption and forward-reverse rate matching. DP-DNA combines these ideas in the DNA RL setting, but I am not yet convinced that this combination constitutes a sufficiently substantial methodological advance for ICLR.
-
Significance: The experiments do not cleanly isolate whether the gains come from adaptive generation order. The main D3LM+GSPO comparison changes several things at once, including the scheduler, advantage construction, and the additional real-sequence objective. The planner ablation is useful, but given how close the method is to prior work, a matched comparison varying only order learning is important for establishing the significance of the contribution.
Minor point: I also find the name DP-DNA confusing, since DP has a well-established meaning as differential privacy in ML. I would strongly suggest using a different acronym.
Questions for authors
-
How does DP-DNA compare with a DCoLT-style joint order-content RL baseline using the same D3LM initialization, reward, regularization, and optimization budget?
-
Can the authors provide a matched experiment where initialization, advantage construction, real-sequence supervision, and training budget are fixed, and only the learned planner is replaced by a fixed scheduler?
-
The motivating order experiment changes which ground-truth tokens are revealed before completion. Does the same reward-collapse effect remain when starting from a fully masked sequence and changing only the generation-order policy while keeping the content model fixed?
4 comments
thanks for the revision and clarifications. i had also read through other reviews and author responses. i am not satisfied. the current draft does not pass the bar. I believe this paper would benefit from polishing more. so i keep my score.
also DP is prevalently known as Differential Privacy in the field of machine learning
Thank you for taking the time to review our revisions and responses. We appreciate your feedback and respect your assessment. If you have any further specific concerns or suggestions, we would be happy to address them.
Regarding the model name, as mentioned in our previous response, we agree that the acronym DP may cause confusion with Differential Privacy and plan to rename the model. We have kept the original name during the review process solely to maintain consistency and avoid confusion for reviewers when referring to the manuscript and our responses.
Thank you again for your time and consideration.
Dear Reviewer,
As the discussion period is approaching its end, we would like to kindly follow up on our rebuttal.
We have addressed all your concerns about novelty and the contribution of learned generation order, including detailed methodological clarifications, matched planner ablations, DCoLT-style comparisons, and fully masked generation experiments.
We would also be happy to provide any further clarifications before the discussion closes.
Thank you for your time and consideration.