- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper develops a method for reinforcement learning with a reward model that depends on combinatorial inputs, and applies it to design libraries of building blocks and reactions for synthesizing small molecules, in the context of DNA encoded libraries
Critical strengths
The paper addresses an important practical problem, that is critical for designing many actual wetlab experiments: how to select a subset of building blocks and reactions that it is tractable to implement in the wetlab. The approach they develop is principled and scalable, and will likely have interest beyond the specific problem of designing DELs, helping chemists choose which reactions to focus on and which building blocks to choose.
Critical weaknesses
The theory, in particular proposition 2, did not seem well motivated. The empirical results seemed to be missing important definitions. See questions below.
Questions for authors
- The objective is not fully defined: How is it approximated from samples? What is Z_\theta, and how is it parameterized? As written, it seems to suggest we are minimizing over trajectories, but I believe you are actually minimizing over theta? To understand these points I had to look up the original trajectory balance paper - I think they should be clarified for the reader here.
- What is the point of Proposition 2? Why does it matter that this map is injective? It appears related to the proof of DeepSets’ flexibility, but this is not explained.
- Most important: What is the definition of diversity you are using and reporting in the experiments? How does the model promote structural diversity, if at all?
- What is the subsampling rate? It seems to never be defined or explained.
- What is the accuracy and performance of your surrogate model on its own? (Separate from the quality of the DEL designs.)
- What efforts, if any, were made to ensure the reactions and building blocks were compatible with DEL chemistry? Many reactions are not tolerated in the context of DNA encoded small molecule libraries.
Minor suggestion: I think the problem setting might be explained more clearly to a wide audience by starting with the concrete DEL design problem, rather than beginning with the abstract definition of a c-structure, or at least specifying what is the c-structure of interest before the definition.
Overall, I think the paper is a valuable contribution, and will raise my score if the points above are addressed.
4 comments
Q4 (What is the subsampling rate? It seems to never be defined or explained):
The meaning of the subsampling rate has been clarified in the Section 5.1, stating:
"DeepSets-GFN parameterizes flows over (state, action) pairs, which allows the flow to be evaluated on a random subset of the available actions at each step rather than on the full action space (Seo et al., 2025). We refer to the fraction of actions retained as the subsampling rate such that corresponds to evaluating every candidate building block and , for example, corresponds to evaluating of them. This allows the per-step cost of training to be decoupled from the size of the building block pool."
Additionally, we have included detailed theoretical justification in Appendix C.4 D.
Q5 (Accuracy and performance of surrogate model on its own):
We report surrogate accuracy independently of DEL quality in Appendix D. The element-level docking proxy attains validation Pearson of -- across the four targets (Table 4), and the library-level DeepSets model attains Kendall on held-out libraries (Table 5). Since serves as a ranking signal, we have added top- overlap against the enumerated on K held-out libraries (Table 6), showing agreement of -- at and strongest agreement in the upper tail, where it matters for training. Figure~2 further shows 's error decreasing over online learning iterations on an external stratified validation set.
Q6 (Efforts made to ensure the reactions and building blocks were compatible with DEL chemistry):
Our environments use building blocks from the Enamine catalog, with each having both a unique amine group and a unique acid group, assembled by amide coupling at both steps. Amide coupling is the most commonly used reaction in DEL synthesis [2], making the reaction chemistry in our environment representative of real DEL construction. Plainly, we made no effort to reject building blocks. We note that no general rule exists for predicting whether a given building block will be DEL-compatible; compatibility is established empirically, and even well-optimized on-DNA amidation consumes only 78% (423/543) of catalogue carboxylic acid substrates [3]. Restricting our pools to a validated subset would therefore have substantially narrowed the chemical space considered.
Ultimately, this work is primarily concerned with the combinatorial aspect of the DEL generation process, however it is possible to enforce any on-DNA compatibility criterion upstream of our pipeline by filtering building blocks prior to selection.
Additional suggestion (problem setting might be explained more clearly to a wide audience by starting with the concrete DEL design problem, rather than beginning with the abstract definition of a c-structure, or at least specifying what is the c-structure of interest before the definition)
Thank you for this suggestion! We have taken this into account and have updated the opening paragraph of Section 3 to specify what the -structure of interest is before the definition:
"Throughout, the -structure of interest is a tuple containing one building block from each pool, which corresponds to a single molecule of the resulting DEL"
We thank Reviewer XiRY for their attentive reading, and in particular for the suggestion to open the problem setting with the concrete DEL design problem rather than the abstract -structure definition --- a reframing that we think makes the paper substantially more accessible.
References
[1] Whitammer, Esmeralda S., et al. "Trajectory balance: Improved credit assignment in gflownets." Advances in neural information processing systems 35 (2022): 5955-5967.
[2] Hunter, James H., et al. “Highly efficient on-DNA amide couplings promoted by micelle forming surfactants for the synthesis of DNA encoded libraries.” Chemical Science 12.27 (2021): 9475-9484.
[3] Shi, Ying, et al. "DNA-encoded libraries (DELs): a review of on-DNA chemistries and their output." RSC advances 11.4 (2021): 2359-2376.
Thank you for the hard work and clear response; my concerns are addressed.
We thank reviewer XiRY and are grateful for their most helpful feedback.
We thank the reviewer for their detailed and constructive feedback. We address each question below. These changes appear in red in the updated manuscript.
Q1 (Objective is not fully defined):
We appreciate the reviewer for pointing out the lack of clarity in Section 3.1. We have since adjusted the main text to address each point.
"What is Zθ, and how is it parameterized?"
We now define Zψ (note a small non-substantive change to ψ from θ; this is a notational change only) where it first appears (Section 3.1), as the partition function of the flow network, that is the total flow leaving the initial state, Zψ=F(s0), which equals ∑x∈XR(x) at the TB optimum. We note that it is learned directly as a single scalar parameter logZψ trained jointly with ψ (again, previously denoted θ, a notational change). The threshold-conditioned setting of Section 5.2 replaces this scalar with a small network taking a Fourier encoding of λ, as described there.
"As written, it seems to suggest we are minimizing over trajectories, but I believe you are actually minimizing over theta?"
The reviewer is correct that we minimize over ψ (previously θ), and our previous phrasing was ambiguous with regards to this point. We have separated the two statements that had been previously conflated. The guarantee is now stated as a condition; i.e., if Equation 1 is satisfied for every trajectory τ, then the terminating distribution of PF satisfies p(x)∝R(x). The optimization is stated separately, and explicitly identifies ψ and logZψ as the variables being optimized rather than the trajectories themselves.
"How is it approximated from samples?"
We have added a description of the training procedure, which was previously excluded, in the paragraph following Equation 1 in Section 3.1. Since the constraint cannot be enforced over all τ exactly, Equation 1 is used as a training objective, with trajectories sampled from the current PF and stochastic gradient steps taken on the resulting loss with respect to ψ and logZψ.
We have additionally clarified in the same subsection that Zψ is the total flow out of s0, learned as a single scalar parameter logZψ jointly with ψ, and that in the combinatorial subset selection environments that we consider, all complete trajectories have the same length and PB is uniform so the log PB term is constant and absorbed into logZψ (with the argument given in Appendix C.3.4). We hope that the subsection is now readable without reference to [1].
Q2 (Additional clarity on the purpose of proposition 2):
The idea of viewing the soft count as a function of λ is to capture the empirical distribution of docking scores using a continuous function of λ, which is convenient to work with, and, most importantly, learnable by a neural network. The faithfulness of soft-count proposition is analogous to the uniqueness property of the Moment Generating Function (MGF): if X and Y are random variables with equal moment generating functions, then X and Y are equal in distribution. In this sense, the MGF is a functional avatar of a distributional object. Proposition 3 shows that the soft count enjoys a similar property, which justifies learning it as a function of λ, as we do in Section 5.2 (reward threshold conditioning) .
We thank the reviewer for pointing out that the justification for Proposition 2 (now Proposition 3 in restructuring) in the text was lacking. Since none of our results hinge on Propositions 2 and 3, which are purely intended as motivation, we have decided to move Propositions 2 and 3 to the Appendix, in Appendix A. The main text now instead clearly focuses on the ideas Proposition 2 and 3 were intended to convey, and refers the reader to the Appendix for technical details.
Q3 (Definition of diversity and does the model promote structural diversity?):
To directly answer the first query, "what is the definition of diversity you are using and reporting in the experiments?", we have adjusted the main text in Section 5.1, where we now state:
"We report diversity as the mean pairwise distance between the 100 highest-reward libraries, where each library is represented by concatenated mean Morgan fingerprints of its three building block pools and distances are computed with the Tanimoto coefficient (see Appendix C.3.3)."
A detailed description of precise calculation of diversity is also now provided in Appendix D.3.3.
(continued below)