- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies watermark verification for DNA generated by genomic language models when the verifier does not know the strand orientation, 6-mer token phase, or location of the generated region. It instantiates the existing SynthID tournament watermark and proposes a verifier that scans both strands, every nucleotide start, and four window lengths. It computes a binomial-tail statistic for each candidate window and applies a Bonferroni correction to obtain one read-level decision.
Critical strengths
-
The paper identifies a concrete and practically relevant issue for genomic watermarking: unlike tokenized model outputs, a nucleotide read does not expose the original token phase, strand orientation, or generation boundary.
-
The detector is clearly specified. In particular, the paper correctly treats the final decision as a multiple-testing problem over the entire search space, and Eq. 3 uses a conservative correction that does not require overlapping windows to be independent.
Critical weaknesses
-
In Sections 3.4 and 4.3, every evaluated read has the same structure: a 384 bp prompt followed by a 3,072 bp generated suffix. The generated region is therefore always at the end of the read, rather than at an unknown internal position with unrelated DNA on both sides. Table 1 and Figure 4 establish detection for this fixed suffix geometry with one edit, but not for a generated segment embedded within arbitrary surrounding DNA. How does the detector behave when the generated segment occurs in the middle of a read, when both flanks are non-generated, or when the observed read is randomly cropped? This distinction matters because additional flanking sequence increases the number of searched windows and hence the Bonferroni penalty, while shorter generated segments contain less cumulative watermark evidence. Section 5 acknowledges this tradeoff qualitatively, but the paper does not characterize detection strength as a function of generated-region length, flank length, or segment placement.
-
Table 1 evaluates exactly one randomly selected, nonadaptive substitution, insertion, or deletion per read. Figure 4 provides a plausible qualitative explanation for why the detector can retain evidence after one indel, but it does not characterize the synchronization behavior beyond that setting. How does recall or window strength change with the number of edits, edit location, clustered indels, longer insertions/deletions, recombination, cropping, or score-aware modification? The current results leave unclear whether the proposed search resolves a general synchronization problem or only recovers sufficient aligned evidence after one mild perturbation.
-
Equation 2 treats reconstructed null mark bits as independent fair binary variables. Bonferroni controls the family-wise error rate without requiring different windows to be independent, but it still requires each window-level to be valid under the stated null. The only empirical diagnostic is the aligned GENERator ordinary-output analysis in Section 4.3, which observes positives across four lengths, versus roughly five expected at each length. This is not a calibration test for the deployed procedure, which searches both strands, every start position, and multiple window lengths. Does the assumed binomial null remain valid after HMAC construction, context reconstruction, repeated-context handling, and the full scan over candidate alignments? The paper does not provide full-search read-level -value calibration across independent keys, ordinary corpora, and wrong-key controls.
-
At the prompt level, the 95% upper bound on the false-positive rate is 1.90% when no control prompt is positive and 2.87% when one is positive. Thus, Figure 2 and Table 1 establish only that the observed controls are compatible with the nominal 1% target, not that the operational false-positive rate is below 1%. Given that the claimed verifier is defined around a declared 1% read-level threshold, the current control sample has limited precision in the relevant tail regime.
-
Section 4.3 states that 64 prompts were reserved for selecting between two window-score summaries, but it does not define the candidate summaries, the selection criterion, the selected option, or whether this process affected any other detector settings. What exactly was chosen using these prompts, and was any part of the detector design modified after observing their outcomes?
Questions for authors
See questions in weaknesses.