- Rating
- 3
- Confidence
- 1
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper identifies sequence-wide base-composition shifts as one failure mode of strong classifier-free guidance in regulatory-DNA generation. It proposes composition-neutral guidance (CNG), which subtracts the per-base positional mean from the conditional-minus-unconditional logit update, and applies it to a compact convolutional discrete diffusion model, BaseDiff. Models are compared through accessibility-versus-motif-quality trade-off curves rather than at one guidance scale. Experiments evaluate composition, motif frequencies/co-occurrence, predicted accessibility, and several robustness checks.
Critical strengths
-
The main finding is informative beyond a model leaderboard: increasing predicted accessibility can coincide with distorted base composition and motif statistics. CNG is a simple, training-free intervention targeting this mechanism. The same-network comparison in Fig. 3 and the composition/error analysis in Appendix J help isolate the effect from the backbone change; composition-matched forward-noise experiments provide an additional check.
-
The empirical validation is substantially broader than the main table alone. It includes a second accessibility predictor, a second genomic insertion locus, a second dataset, three independent BaseDiff training runs, and comparisons with APG/CFG-Zero* and alternate composition corrections (Appendices C, E–G, K–M). On K562, Table 1 shows co-occurrence trade-off area improving from 0.364 to 0.423 with CNG. These checks strengthen the case for a useful computational finding.
Critical weaknesses
-
The claimed portability of the correction remains under-tested. Other headline generators use ordinary CFG, while CNG and the additional correction comparisons are applied to BaseDiff. The results therefore establish a BaseDiff-plus-CNG system and a within-BaseDiff improvement, but do not show whether the intervention transfers to other discrete generators or how rankings change when competitors receive it. This matters to the broader contribution of CNG as a reusable sampler correction.
-
The biological conclusion remains limited by the evaluation proxies. Both motif-quality measures originate from the same JASPAR scan, and accessibility comes from predictors; Appendix D itself shows that short-range statistical generators can achieve high realism. The second predictor/locus/dataset checks are valuable but do not establish improved cellular function. I regard the evidence as supporting reduced compositional distortion and better computational trade-offs, with functional claims requiring more direct validation.
Questions for authors
-
Can CNG be applied to an independently trained competing discrete model, such as the D3 baseline, and evaluated at comparable accessibility? Does it improve quality and preserve accessibility, and how does the corrected competitor compare with BaseDiff-CNG?
-
Appendix K acknowledges that zeroing the mean logit update preserves composition only approximately after softmax. Can the main-text statement that the correction changes only composition be qualified, and can effects on motif probabilities be separated from the exact algebraic property of zero-mean logits?
-
How sensitive are the CNG gains to the motif library and co-occurrence pair-selection rules? If functional validation is not available, please keep conclusions explicitly at the computational level and distinguish matching motif statistics from preserving regulatory function.
2 comments
Author Response (2/2)
Motif library and pair selection. We rescanned the sequences with HOCOMOCO v14 CORE, which contains 1,595 motif matrices against JASPAR's 1,019, and recomputed the quality metrics. Below are area differences between BaseDiff-CNG and the same model with ordinary CFG.
| Cell type | JASPAR realism | HOCOMOCO realism | JASPAR co-occurrence | HOCOMOCO co-occurrence |
|---|---|---|---|---|
| GM12878 | ||||
| HepG2 | ||||
| K562 |
We also varied the number of selected pairs, the motif-prevalence window, and the Jaccard redundancy threshold. Each variant reselects pairs from the real training data and recomputes the co-occurrence metric. The table gives CNG's co-occurrence-area improvement over ordinary CFG.
| Pair-selection rule | GM12878 | HepG2 | K562 |
|---|---|---|---|
| Submitted rule | |||
| Top 250 pairs | |||
| Top 1,000 pairs | |||
| Prevalence 2–40% | |||
| Prevalence 0.5–60% | |||
| Jaccard threshold 0.3 | |||
| Jaccard threshold 0.7 |
The K562 improvement remains positive across all pair-selection variants. The effects on GM12878 and HepG2 are smaller, consistent with the original comparison.
Computational interpretation. We agree with your characterization of the evidence as reduced compositional distortion and improved computational trade-offs. Appendix D illustrates why motif agreement must be considered alongside predicted accessibility. On GM12878, the order-5 Markov model has realism of 0.897. Its predicted accessibility is 0.172, compared with 0.611 for real sequences.
Our study concerns building and evaluating DNA generators in silico. The cost of experimental characterization motivates careful computational comparison before selecting sequences for synthesis. Motif agreement and predicted accessibility describe specific properties of the generated sequences, while cellular enhancer function requires experimental validation. We will clarify this scope in the Introduction and Discussion and qualify wording that could imply preservation of regulatory function.
Author Response (1/2)
Thank you for recognizing the computational finding and the controlled comparisons. We tested CNG on D3 on both datasets and examined the sensitivity to the motif library and pair-selection rules.
Transfer to D3. We sampled the same trained D3 baseline with and without CNG, keeping its weights, conditioning and guidance scales fixed. DHS areas follow Table 1, and lentiMPRA areas use the shared accessibility range of Table 7.
CNG increases D3's area in all twelve comparisons, including WTC11 realism before rounding. The lentiMPRA comparison holds the accessibility range fixed across ordinary D3, corrected D3, and BaseDiff-CNG. BaseDiff-CNG retains larger areas in most comparisons with 21.6M parameters, compared with D3's 92.4M. The corrected models are similar on DHS GM12878 realism and HepG2 co-occurrence.
Exact property of the correction. Let Δp,b=ℓc,p,b−ℓ∅,p,b and let Δˉb be its mean over positions. CNG uses Δp,b=Δp,b−Δˉb. Each base's mean update is exactly zero, and variation across positions is unchanged in logit space. Softmax couples the four bases at each position, and the effect on sampled composition is approximate. Appendix K tests a probability-space variant of this correction, and it does not improve on the mean correction (CNG). We will qualify Section 3.3 to state this property precisely and distinguish it from effects on motif probabilities.