acceptodds
Under review as a conference paper at ICLR 2027

Tracing model generated DNA with position independent watermark verification

Abstract

Generation time watermarks support provenance for generative models, but verifi- cation becomes a synchronization problem when token alignment is unknown. In genomic sequences, a verifier may lack strand orientation, token phase, generation boundaries, and the location of the generated region; insertions and deletions can further disrupt alignment. We introduce a position independent verification frame- work that searches both strands, every nucleotide start, and four window lengths. Under an explicit null model for keyed bits, a Bonferroni correction converts the strongest window statistic into one read level significance decision, accounting for the full search without assuming independence between overlapping windows. Verification requires only the observed DNA, a key, and public settings. We instan- tiate the framework with the existing SynthID tournament watermark in Carbon- 500M and GENERator-v2 1.2B under canonical six mer sampling. Across 192 held out prompts with two draws each, the verifier detected 384/384 correct key marked reads per model, both unedited and after each tested single nonadaptive substitution, insertion, or deletion. Each ordinary and wrong key control family yielded at most one positive among 384 reads per model and condition at a de- clared 1% threshold. Neither model showed a statistically detectable difference across 14 prespecified model likelihood and sequence measures after correction. These results support verification without model access under unknown synchro- nization in the evaluated six mer and single edit regime.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Watermarking, Generative Models, Genomic Language Models, Sequence Provenance, Statistical Hypothesis Testing, Biological Foundation Models, Trustworthy AI.

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_zdRGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies watermark verification for DNA generated by genomic language models when the verifier does not know the strand orientation, 6-mer token phase, or location of the generated region. It instantiates the existing SynthID tournament watermark and proposes a verifier that scans both strands, every nucleotide start, and four window lengths. It computes a binomial-tail statistic for each candidate window and applies a Bonferroni correction to obtain one read-level decision.

Critical strengths
  • The paper identifies a concrete and practically relevant issue for genomic watermarking: unlike tokenized model outputs, a nucleotide read does not expose the original token phase, strand orientation, or generation boundary.

  • The detector is clearly specified. In particular, the paper correctly treats the final decision as a multiple-testing problem over the entire search space, and Eq. 3 uses a conservative correction that does not require overlapping windows to be independent.

Critical weaknesses
  • In Sections 3.4 and 4.3, every evaluated read has the same structure: a 384 bp prompt followed by a 3,072 bp generated suffix. The generated region is therefore always at the end of the read, rather than at an unknown internal position with unrelated DNA on both sides. Table 1 and Figure 4 establish detection for this fixed suffix geometry with one edit, but not for a generated segment embedded within arbitrary surrounding DNA. How does the detector behave when the generated segment occurs in the middle of a read, when both flanks are non-generated, or when the observed read is randomly cropped? This distinction matters because additional flanking sequence increases the number of searched windows MM and hence the Bonferroni penalty, while shorter generated segments contain less cumulative watermark evidence. Section 5 acknowledges this tradeoff qualitatively, but the paper does not characterize detection strength as a function of generated-region length, flank length, or segment placement.

  • Table 1 evaluates exactly one randomly selected, nonadaptive substitution, insertion, or deletion per read. Figure 4 provides a plausible qualitative explanation for why the detector can retain evidence after one indel, but it does not characterize the synchronization behavior beyond that setting. How does recall or window strength change with the number of edits, edit location, clustered indels, longer insertions/deletions, recombination, cropping, or score-aware modification? The current results leave unclear whether the proposed search resolves a general synchronization problem or only recovers sufficient aligned evidence after one mild perturbation.

  • Equation 2 treats reconstructed null mark bits as independent fair binary variables. Bonferroni controls the family-wise error rate without requiring different windows to be independent, but it still requires each window-level PwinP_{\mathrm{win}} to be valid under the stated null. The only empirical diagnostic is the aligned GENERator ordinary-output analysis in Section 4.3, which observes 2,6,6,42,6,6,4 positives across four lengths, versus roughly five expected at each length. This is not a calibration test for the deployed procedure, which searches both strands, every start position, and multiple window lengths. Does the assumed binomial null remain valid after HMAC construction, context reconstruction, repeated-context handling, and the full scan over candidate alignments? The paper does not provide full-search read-level pp-value calibration across independent keys, ordinary corpora, and wrong-key controls.

  • At the prompt level, the 95% upper bound on the false-positive rate is 1.90% when no control prompt is positive and 2.87% when one is positive. Thus, Figure 2 and Table 1 establish only that the observed controls are compatible with the nominal 1% target, not that the operational false-positive rate is below 1%. Given that the claimed verifier is defined around a declared 1% read-level threshold, the current control sample has limited precision in the relevant tail regime.

  • Section 4.3 states that 64 prompts were reserved for selecting between two window-score summaries, but it does not define the candidate summaries, the selection criterion, the selected option, or whether this process affected any other detector settings. What exactly was chosen using these prompts, and was any part of the detector design modified after observing their outcomes?

Questions for authors

See questions in weaknesses.

Reviewer_ot4KOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper asks whether a generation time watermark can be recovered from DNA produced by a genomic language model when the verifier does not know how the sequence lines up. Both models studied, Carbon-500M and GENERator-v2 1.2B, emit nonoverlapping six base tokens, so token boundaries leave no trace in the sequence and six phases are possible. Reverse complementation changes token identity and order, and the generated region sits somewhere inside a longer read with no marked boundary. The authors take the SynthID-Text tournament watermark and build a verifier around this uncertainty. It scores four prespecified window lengths at every starting base on both strands, 16,136 windows for a 3,456 base read, and Bonferroni corrects over the whole search so that one read level decision can be reported at a declared 1% threshold. The experiments cover sampler fidelity, fourteen likelihood and sequence quality measures, detection on 192 held out prompts per model, and the effect of a single substitution, insertion or deletion on both detection and on which window length carries the strongest signal.

Critical strengths

Treating genomic provenance as a synchronisation problem. Hidden token boundaries and strand ambiguity have no counterpart in text watermarking, and the paper is the first I know of to state the problem in this form and attach a calibrated decision procedure to it.

Critical weaknesses

W1. Contribution is instantiation rather than method. The watermark is SynthID-Text used unchanged, and the verifier applies a standard Bonferroni correction over an exhaustive alignment search. No new estimator, null model, or search strategy is introduced from its analysis insights. Evaluation is also confined to one scheme. Section 2.2 separates biased schemes such as KGW from distribution-preserving ones, but only the latter is instantiated and the two families are not compared.

W2. Cross-model claim rests on two points without accounting for their spread The weakest marked read scores 124.8 in Carbon and 19.6 in GENERator against a common threshold of 6.21, and the paper concludes portability without addressing the gap. A six fold difference in log units suggests the available watermark signal is model dependent, which matters because it predicts divergent behavior precisely in the short continuation and long read regimes that were not evaluated. Relating per read strength to per token entropy would test this.

Questions for authors
  1. Can the authors report a broad range of baselines, such as biased schemes and additional distribution-preserving methods?
  2. The continuation length of 3,072 bases coincides exactly with the longest window in the grid. How does detection behave when the generated region length falls between grid points?
  3. All experiments use a single continuation length. Since shorter regions give fewer scored bits while the threshold stays nearly the same, can the authors report detection across a range of continuation lengths?
  4. What accounts for the gap between 124.8 in Carbon and 19.6 in GENERator, and does the portability claim survive it?
  5. Given the observed insights, what kind of method would be good for this task?
Reviewer_GmoAOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a position-independent verifier for watermarked DNA generated by genomic language models. It uses the existing SynthID tournament watermark and searches both strand orientations, every nucleotide starting position, and four window lengths to accommodate unknown token alignment and generation boundaries. A Bonferroni correction converts the strongest window-level evidence into a read-level significance decision. Experiments on Carbon-500M and GENERator-v2 1.2B detect all 384 held-out marked reads per model, both unedited and after each tested single-base substitution, insertion, or deletion. Ordinary-output and wrong-key controls produce at most one positive per 384 reads in each condition. No statistically significant differences are observed across 14 prespecified sequence-quality proxies after correction.

Critical strengths

Explicit treatment of synchronization uncertainty and statistical significance. The verifier incorporates strand orientation, token phase, starting position, and window length into a single corrected read-level decision. This prevents reporting the strongest searched window without accounting for the selection process.

Consistent empirical detection with appropriate controls and qualified conclusions. The evaluation includes two genomic models, ordinary-output and wrong-key controls, sampler-validation checks, and prompt-level uncertainty estimates. Complete detection under the tested single-edit conditions supports feasibility within the specified regime. The authors appropriately avoid interpreting nonsignificant quality differences as biological equivalence.

Critical weaknesses

The incremental benefit of the proposed verifier is insufficiently established. SynthID already supports detection without access to the generating model, while window-based watermark detection and Bonferroni correction are established techniques. The paper does not compare the combined search against aligned detection, phase-only search, or an adapted window-search baseline at a common false-positive target. Consequently, the results do not establish which components provide consequential improvements or whether their computational cost is justified.

The evaluation insufficiently characterizes the synchronization problem and operating limits. All continuations contain 3,072 bases and follow a 384-base prompt, which preserves the original six-mer phase relative to the read start. Boundary position, initial phase, and strand orientation are not independently varied. Robustness is restricted to one nonadaptive edit per continuation. Performance on shorter marked fragments, longer surrounding sequences, and greater edit burdens remains unresolved, limiting the demonstrated generality of position-independent verification.

Questions for authors

At the same read-level false-positive target, how does the proposed verifier compare with aligned, phase-only, and adapted window-search detectors in detection power and runtime?

How does detection change when marked-fragment length, surrounding-sequence length, boundary position, token phase, and strand orientation are varied independently? These measurements would clarify when the complete search is necessary and effective.

What caused the Carbon protocol-hash discrepancy acknowledged in Section 6? Which configuration produced the reported results, and can that execution be reproduced with a consistent protocol record?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.