- Rating
- 3
- Confidence
- 3(+1)
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Continuous Variational Synthesis (cVS), which addresses the challenge of training variational biological synthesis models under discrete chemical constraints through continuous spatial gradient optimization and post-training quantization. The method combines forward KL pre-training, reverse KL fine-tuning, reduction of gradient variance, and parameter quantization, and is validated on four tasks: enzymes, HLA-presenting peptides, antibody CDRH3, and regulatory DNA. Experiments show that cVS outperforms existing methods in multiple quality-diversity trade-off metrics, and the feasibility of the method is verified through real DNA synthesis and sequencing.
Critical strengths
-
Continuous optimization followed by quantization is intuitive, practical, and compatible with gradient-based training.
-
Includes in vitro DNA synthesis and sequencing, providing evidence beyond purely computational evaluation.
-
Demonstrates improved quality–diversity trade-offs against EM VS and PGLD.
Critical weaknesses
-
Ablations suggest that continuous relaxation and quantization contribute more than the proposed variance-reduction techniques. Their individual contributions need clearer quantification.
-
Some diversity metrics, particularly theoretical diversity, may not fully reflect experimentally realizable unique sequences or functional diversity.
Questions for authors
-
How much improvement comes from continuous relaxation, quantization, pretraining, and variance reduction individually?
-
How does cVS compare with alternative continuous optimization or discrete optimization methods under identical constraints?
-
How robust is the method to different synthesis catalogs, numbers of reaction wells, and quantization errors?
We are grateful to the reviewer for their time, their careful reading and their constructive suggestions. A point-by-point response is below (since some comments and questions cover similar points, we address them jointly).
"Ablations suggest that continuous relaxation and quantization contribute more than the proposed variance-reduction techniques. Their individual contributions need clearer quantification."... "How much improvement comes from continuous relaxation, quantization, pretraining, and variance reduction individually?"
Thank you for the valuable suggestion, we have run additional experiments quantifying the contribution of each component of our training algorithm. First, we ran additional experiments evaluating each component of our variance reduction strategy. Each separate technique offers a small but reliable improvement, which add together to a more substantial gap over the naive approach (Figure 21). Next, we evaluated our quantization strategy in greater depth. Our new experiments show our reverse KL quantization objective offers large improvements over a naive Euclidean quantization objective. We also considered quantizing during training rather than only post-training, by adding a penalty for violations of the quantization objective. We find this only offers modest improvements, at the cost of reduced wetware inter-operability (Figure 22). To evaluate the continuous relaxation, we can compare to PGLD, which does not use a continuous relaxation; here we find substantial gains (Figures 2, 3, 4). Finally, we found that pre-training offers substantial gains (Figure 11).
"Some diversity metrics, particularly theoretical diversity, may not fully reflect experimentally realizable unique sequences or functional diversity."
Thank you for raising this important point. Indeed, we only include “theoretical diversity” in the peptide design evaluation because it was used in a previous benchmark proposed in the PGLD paper (Sussex et al. 2026); we do not believe it is a useful estimate of diversity in practice, precisely because it does not fully reflect experimentally realizable unique sequences. We clarify these points in Section 2.2. We also agree that evaluating functional diversity is both important and challenging. In the absence of functional measurements, we evaluated diversity according to a range of different strategies: besides KL divergence to the prior, we looked at Shannon entropy (Figure 13), the kernel 2-Renyi entropy across many different kernel bandwidths (Figure 14), and the kernelized Stein discrepancy (KSD-B) across different kernels and bandwidths (Figure 18). The KSD-B is of particular note, since it is an integral probability metric which works by integrating over an infinite number of possible sequence-function relationships, effectively averaging over many possible measures of functional diversity (Amin et al. 2023).
"How does cVS compare with alternative continuous optimization or discrete optimization methods under identical constraints?"
Thank you for raising this important point. We do not know of any directly comparable method besides PGLD, which focuses just on the constraint of equal nucleotide mixtures. We have performed additional experiments evaluating different optimizers for the training phase of cVS, including REINFORCE leave-one-out, a widely used method (Figure 21). We also performed additional experiments evaluating different quantization methods used for discrete optimization, including quantization with the standard Euclidean distance, which performs much worse (Figure 22).
"How robust is the method to different synthesis catalogs, numbers of reaction wells, and quantization errors?"
Thank you for raising this important point. We have evaluated the method with different synthesis catalogs, reaction wells and quantization errors, observing substantial robustness. As the number of reaction wells increases, the performance steadily increases, approaching an asymptotic limit; small changes in wells have little effect (Figure 10). As the synthesis catalog increases in size, the quantization error decreases, and performance improves steadily (Figure 3b). Some amount of quantization error is unavoidable when synthesis uses mixtures of nucleotides, rather than trinucleotides (codons), even with a large catalog. Despite this irreducible quantization error, we do not observe a large gap between the performance of the unconstrained free cVS model and the quantized model in practice.