acceptodds
Under review as a conference paper at ICLR 2027

GenoTrace: Inheritable Watermarks for Genome Foundation Model Distillation

Abstract

Can a genome model retain a detectable record of the synthetic sequences used to train it? We study watermark inheritance through distillation with GenoTrace, a codon-aware extension of green-list watermarking. Two token-level factors modulate the teacher's generation bias using codon position and organism-specific codon usage. The resulting sequences train a smaller student, whose outputs are audited without an active watermark processor. In a three-seed GenomeOcean-500M-to-100M experiment, the joint configuration achieves a mean audit score of 17.88 and 94.5% detection at a fixed threshold. It retains 49.0% detection after key-aware token substitution, compared with 0% for the available single-seed plain-watermark comparator, and 47.0% after combined mechanism-targeted nucleotide edits. Additional experiments establish inherited signal across five organism-conditioned datasets and teacher–student size ratios up to 40. Component ablations and computational sequence-quality assays reveal distinct operating points for detection strength and coding coverage. GenoTrace provides a practical token-level construction and an empirical account of how genomic structure shapes inherited watermark signals. The findings concern shared-tokenizer distillation and the tested editing procedures, with calibration and biological utility treated as separate evaluation requirements.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
genome foundation models, watermark inheritance, knowledge distillation, model provenance, codon-aware watermarking

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_YBj2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes GenoTrace, a codon-aware watermarking method for tracing synthetic genomic data through model distillation. By incorporating codon-position and organism-specific codon usage information into traditional green-list watermarking, the method enables student models to inherit detectable watermark signals without modifying their inference process. Experiments demonstrate strong watermark detectability and improved robustness against certain editing attacks compared to conventional watermarking.

Critical strengths
  1. The proposed codon-aware design provides a meaningful domain-specific adaptation of conventional watermarking methods.
  2. The paper conducts extensive experiments across different watermarking configurations, adversarial editing strategies, model scales, and genomic datasets.
Critical weaknesses
  1. Lack of a preliminary section. The paper lacks a dedicated preliminary section that rigorously defines the problem setting and introduces essential biological concepts. This makes it difficult for readers without a biology background to understand the technical challenges and follow the proposed methodology.
  2. Uninformative illustration. Figure 1 does not effectively illustrate the key technical contributions or the logical flow of the proposed method. In particular, the core codon-aware watermarking mechanism is largely omitted, while considerable space is devoted to evaluation components. A more focused illustration of the proposed methodology would improve the paper's clarity.
  3. Insufficient baseline comparisons. The experimental comparisons do not convincingly isolate the benefits of the proposed codon-aware design. The conventional KGW baseline uses a different watermark strength and only one seed in the adversarial evaluation, making the robustness improvements difficult to attribute to the proposed mechanism.
  4. Unclear technical exposition and experimental justification. The paper is difficult to follow, particularly regarding how the proposed method addresses the identified technical challenges and how the experimental results establish its effectiveness. The intuition behind the codon-aware design is insufficiently explained, and the experimental comparisons do not clearly demonstrate its advantages over conventional watermarking methods. A more coherent presentation connecting the technical challenges, methodological choices, and empirical findings would significantly improve the paper.
Questions for authors
  1. Can you show Joint-F and Joint-B at δ=2\delta=2 (same as KGW baseline) in Table 1 to have a fair comparison?
  2. Are those in Figure 2 the scatter plots? If so, what is the x-axis for each watermark group? Or are they actually bar plots?
  3. Where will be KGW baseline if it is shown in the trade-off analysis by Figure 4?
  4. Does codon-aware weighting offer a meaningful advantage over simply increasing the strength of conventional KGW, particularly when controlling for watermark detectability and biological sequence quality?
2 comments
AuthorsOpenReview ↗

Thank you for the careful and encouraging review. Part 1 of 2: a matched KGW baseline and Joint at δ = 2

Dear Reviewer YBj2,

Thank you very much for your time and for reading our paper so carefully. We are glad you found the codon-aware design a meaningful domain-specific adaptation, and that you noted the breadth of our experiments. Your review points to two clear improvements: a fair comparison with plain KGW, and a presentation that readers outside biology can follow. We are grateful for both, and we have acted on both.

Since receiving your review, we have:

  • trained a plain KGW baseline with three seeds at δ = 2, 4, 6, and 8 (W3, Q4);
  • trained Joint-F and Joint-B at δ = 2 (Q1);
  • placed KGW on the trade-off plot of Figure 4 (Q3);
  • prepared a preliminaries section, a new Figure 1, a clearer Figure 2, and a plain account of the design intuition (W1, W2, W4, Q2).

All new students use the same teacher, student, prompts, real-genome mix, LoRA recipe, sampling settings, and edit code as the codon-aware rows of Table 1. Rates use the paper's scoring and threshold, as in Table 1. Each row averages seeds 44, 142, and 256.

W3, Q1, Q4: You are right that the single-seed KGW* row could not isolate the effect of the codon weights. KGW* also came from an earlier run whose student decoding used presence and frequency penalties. The table below replaces it with three-seed students decoded exactly like every codon-aware row, and we will use this baseline in Table 1, Figure 2, and the abstract. Mean push is δ times the average codon weight over the vocabulary, that is, the average bonus a green token receives. Teacher green is the share of tokens in the 2,000 teacher training responses that fall in the green list. CAI is the codon adaptation index of student outputs against E. coli.

SchemeδMean pushTeacher greenCleanKTKNCAI
KGW22.0084.795.56.25.50.618
KGW44.0099.394.810.39.00.616
KGW66.0099.896.811.88.00.620
KGW88.0099.895.210.27.30.612
Joint-F20.3938.620.83.20.30.649
Joint-B21.1574.691.816.85.30.715
Joint-B42.3098.691.842.019.30.727
Usage42.3198.689.044.820.00.728
Joint-F81.5488.594.549.035.80.719
Joint-F, shuffled81.5486.590.710.75.80.617

The shuffled row keeps the Joint-F weights at δ = 8 but assigns them to random tokens. Another reviewer suggested this control.

The table shows three things.

  1. Stronger KGW saturates. At δ = 4 the KGW teacher already marks 99.3% of tokens green. Beyond that, KT detection stays between 10% and 12%, and KN stays below 10%.
  2. Codon-aware weighting reaches a region KGW does not. Joint-F at δ = 8 keeps 49.0% after KT and 35.8% after KN. KGW keeps at most 11.8% and 9.0% at any strength. Clean detection is matched: every row except Joint-F at δ = 2 lies between 89.0% and 96.8%.
  3. The difference comes from codon structure, not bias strength. Joint-F at δ = 8, KGW at δ = 2, and the shuffled control mark their training corpora at similar rates (84.7% to 88.5%). Only the aligned codon weights keep high KT detection, and only they shift codon usage toward preferred E. coli codons (CAI 0.719, against 0.618 and 0.617).

Q1: Joint-B at δ = 2 is close to KGW at δ = 2 on clean detection (91.8% against 95.5%). It keeps 16.8% after KT, more than KGW at any strength, and it shifts codon usage (CAI 0.715). Joint-F at δ = 2 is weak, as expected. Its average codon weight is 0.19, so its mean push is only 0.39, one fifth of KGW at δ = 2. Equal nominal δ is therefore not a matched comparison, and mean push or teacher green is a better guide. We will add both rows to Table 1.

Q4: Thank you for this key question. Increasing KGW strength does not give the same benefit. Beyond δ = 4, a stronger KGW improves neither KT nor KN retention, and it never shifts codon usage. Codon-aware weighting provides both at similar clean detection. The cost is coding coverage, which we show under Q3. We will state this trade-off plainly and explain how to choose an operating point from the required quality level.

AuthorsOpenReview ↗

Thank you for the careful and encouraging review. Part 2 of 2: Figure 4 with KGW, clearer figures, and preliminaries

Q3: Thank you for this question. Below are ORF coverage and 3-mer entropy of clean student outputs, with detection after KT. We also add an unwatermarked control trained and decoded with the same settings.

SchemeδORF coverage3-mer entropyKT
Unwatermarked control00.3294.851.7
KGW2 to 80.454 to 0.4885.11 to 5.206.2 to 11.8
Position-B40.3654.898.3
Joint-B20.2454.2716.8
Usage40.1924.0544.8
Joint-F80.1663.9949.0

KGW would sit in the lower right of Figure 4, with high ORF coverage and low KT retention. Raising δ barely moves it. The usage and joint settings sit in the upper left. No single scheme dominates. KGW suits uses where coding coverage matters most. Codon-aware weighting suits uses that call for control over codon usage, and it retains more detections in our key-aware stress tests. We will also correct the reference line. The dotted line (0.538) came from the earlier run with different decoding. The matched control has ORF coverage 0.329, and the position-weighted students stay above it. We will add the KGW curve to Figure 4 and state the trade-off plainly in Section 5.5.

Q2: Thank you for catching this ambiguity. Figure 2 is a dot plot. The x-axis is categorical, and each tick is one condition (clean outputs or one edit type). Within a tick, markers for different schemes are shifted slightly left and right only so they do not overlap. The offset has no meaning. The y-axis is the detection rate. In panel (b), error bars are seed standard deviations. We will switch to grouped bars with value labels, add the three-seed KGW series, and say all of this in the caption.

W1: We agree. We will add a short section before the method. It will explain five ideas in plain terms:

  1. Codons and reading frames.
  2. Synonymous codons, which usually differ at the third base.
  3. Codon usage bias across organisms.
  4. Why subword tokens rarely line up with codons.
  5. Green-list watermarking and its z-score.

The section will end with a precise problem statement and one worked token example. The statement will say who holds the key, what the student trainer receives, and what the auditor observes.

W2: We agree that the current figure favors evaluation over mechanism. The new Figure 1 will have three panels. Panel (a) follows one decoding step for the token ATCG. It shows the green-list check, the position weight, the usage weight, and the final bonus. At phase 0, the position weight is one quarter for the fractional variant and one for the binary variant. Panel (b) shows distillation into the student. Panel (c) shows the audit. Evaluation components move to a thin strip with pointers to Section 5.

W4: Plain KGW gives every green token the same bonus. In a genome model, tokens are DNA strings of variable length that do not line up with codons, so a uniform bonus spreads over all green continuations. GenoTrace keeps the key and the green list, and only sets the size of each green token's bonus with two weights between 0 and 1. The first is the share of the token's bases at third codon positions. The second is the share of its complete codons that the organism prefers. The intended effect is to tilt the marked corpus toward the organism's preferred codons, so that the mark also shapes codon usage. The new data support this. Every usage-weighted student in the table shifts toward preferred codons (CAI 0.649 to 0.728, against 0.627 for the control). The shift disappears when we shuffle the weights across tokens (0.617), while clean detection stays similar. We will add this paragraph to Section 3, along with a short table linking each challenge to its design choice and its evidence.

Thank you again for your thoughtful and constructive feedback. Your comments on the baseline led directly to the matched experiments above, and they helped us state precisely what the codon weights add. Your comments on presentation will make the paper much easier to follow. We will include all of these changes in the revision. If anything is still unclear, or if another comparison would help, we would be happy to provide it during the discussion period. We would be very grateful if you would consider these updates in your final assessment.

Reviewer_DxfLOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper asks whether a watermark injected into a genome language model's training data is inherited by a student trained on those outputs. A teacher (GenomeOcean-500M) samples DNA with a codon-aware KGW-style token bias — reweighted by reading-frame position (fractional/binary third-position weighting) and organism codon usage — and a student (GenomeOcean-100M) is LoRA-fine-tuned on a mix of these sequences and real E. coli fragments. The watermark processor is removed before evaluation, so any signal must come from the student's learned distribution. The auditor scores outputs with the teacher's key and tokenizer via the standard KGW z-statistic. Experiments report clean detection, detection after token-level and mechanism-targeted nucleotide edits, extensions across organisms, model scale, training controls, and tokenizers, plus computational quality assays (ORF coverage, entropy, ESMFold, BLAST). The headline joint fractional configuration (δ = 8) reaches mean z = 17.88 with 94.5% clean detection and retains 49.0% after key-aware token substitution and 47.0% after combined nucleotide edits, at a substantial cost to sequence quality.

Critical strengths
  1. The teacher/student separation and the removal of the inference-time processor make the inheritance claim methodologically clean.

  2. The organism and scale extensions are broad enough to show the phenomenon is not a single-configuration artifact.

Critical weaknesses
  1. The paper is very hard to follow, and the technical contribution is unclear. This is the most serious problem. The writing is dense, hedged, and repeatedly defers to appendices; key definitions (e.g., the exact scoring event, what "joint fractional" means operationally, how the codon factor interacts with subword tokens) are scattered across Sections 3.2–3.4 and Appendices A.3–A.5, with worked token examples appearing only in the appendix. The reader is told in the introduction that the contribution is a "codon-aware token-bias construction," but the paper never states crisply what is new relative to prior watermark constructions beyond applying a position/usage reweighting to KGW's green-list bias. The relation to Sander et al. (2024) on "radioactive" watermarks — which is the closest prior work and establishes that watermarks can be inherited through training — is mentioned in two sentences and never differentiated. As written, a reader cannot easily answer: what is the technical contribution, and why is it not simply "we ran the radioactive-watermark experiment on DNA models with a codon-aware bias"?
  2. The technical contribution appears to be an empirical observation of watermark inheritance, not a new method. Stripped of the framing, the pipeline is: (a) apply a KGW-style green-list bias with a position/usage multiplier to teacher sampling; (b) distill a student on the outputs; (c) audit with the standard KGW z-score. The codon-aware factor is a reweighting of the existing bias term, and the paper itself concedes it "favors certain token continuations rather than enforcing synonymous editing or preservation of an existing protein" (Section 3.3). The theoretical analysis (Appendix B) is a fixed-context exponential-tilt calculation that the authors explicitly say "does not prescribe the student inheritance rate." So the method reduces to a variant of KGW plus an empirical study. That may be a publishable empirical contribution, but the paper presents it as a methodological one, and the mismatch is a problem.
  3. Only KGW-style watermarking is tested, so the generality of the findings is unknown. All main experiments use the KGW green-list construction (with and without codon-aware reweighting). The related work discusses unbiased/distortion-free watermarks (Hu et al., 2024; Kuditipudi et al., 2024), cryptographic undetectability (Christ et al., 2024), semantic partitions (Hou et al., 2024), and deployed sampling watermarks (Dathathri et al., 2024), but none of these is evaluated for inheritance. The conclusion that "genomic structure can shape an inherited watermark" is therefore a claim about one watermark family, and the paper does not argue why the findings should transfer. This is especially important because KGW's green-list bias is a particularly blunt distortion; it is plausible that distortion-free watermarks inherit differently, or not at all.
  4. The detection-quality trade-off is severe, and the paper's own numbers undercut the practical value of the headline configuration. The joint fractional δ = 8 configuration that achieves 94.5% clean detection has ORF coverage 0.166 vs. 0.538 for the unwatermarked control, 3-mer entropy 3.99 vs. 5.60 bits, and 5/450 vs. 12/450 significant BLAST hits. ESMFold pTM is near the 0.177 floor for both conditions, which the authors correctly say cannot establish non-inferiority. So the best-detecting configuration produces sequences that are substantially degraded in coding potential and diversity, and the paper does not identify any operating point that is simultaneously strong on detection and credible on quality. The paper is commendably honest about this, but honesty about a limitation is not a substitute for addressing it; the practical deployment case for the method is therefore weak.
  5. The evaluation scope is narrower than the framing suggests, and several claims are hedged to the point of being unfalsifiable. The main matrix is one teacher–student pair (500M → 100M) on E. coli, with 200 responses per condition and three training seeds. The organism results use plain KGW at δ = 2, not the joint method, so they do not establish cross-organism robustness of the proposed construction (the paper says so, but the abstract and introduction do not). The tokenizer probes (HyenaDNA, GROVER) show no transfer, which the paper reports as a scope restriction rather than a failure. Throughout, claims are qualified with "in this setting," "measured operating point," "not a guarantee," and "does not establish" — to the point where it is difficult to state what the paper does establish beyond "we observed inheritance in one configuration, with quality degradation, and it did not transfer across tokenizers."
Questions for authors
  1. What is the technical contribution relative to Sander et al. (2024)? That paper established watermark inheritance through training in the text setting. What does the codon-aware construction add beyond a reweighting of the KGW bias, and what experiment would fail if the codon-aware factor were doing no work? The ablations compare position/usage/joint at different strengths, but the "plain KGW" comparator in the edit matrix has only one seed and a different starting detection rate.
  2. Why only KGW? Could the authors at least discuss, or ideally test, whether a distortion-free or unbiased watermark (Hu et al., 2024; Kuditipudi et al., 2024) is inherited, and whether the codon-aware reweighting is compatible with such constructions? Without this, the generality claim is unsupported.
  3. Is there any operating point that is both detectably watermarked and biologically plausible? The paper reports the trade-off but does not identify a configuration that a practitioner could use. What is the best achievable detection at, say, ORF coverage ≥ 0.3 or 3-mer entropy ≥ 5.0, and how does it compare to the unwatermarked control?
  4. How should the reader interpret the "radioactive" framing? The paper cites Sander et al. only in passing and does not compare detection rates, edit robustness, or quality trade-offs to that work. If the contribution is an empirical demonstration in a new domain, a direct comparison (even qualitative) would clarify what is new.
  5. What is the status of the cross-tokenizer result? The paper reports that HyenaDNA and GROVER audits do not recover the signal and treats this as a scope restriction. Is this a fundamental limitation of token-level watermarking for genome models, or an artifact of the specific probes? The GROVER result (mean score −5.47 with a 14.5% positive tail) is described as "a distributional shift, not a centered null," which makes it hard to interpret.
3 comments
AuthorsOpenReview ↗

Thank you for the thorough and candid review. Part 1 of 3: our contribution, its relation to Sander et al., and clarity

Dear Reviewer DxfL,

Thank you for reading our paper so closely and for the detailed, constructive review. We are grateful that you found the teacher and student separation methodologically clean, and that you saw the organism and scale extensions as evidence against a single-configuration artifact. Your points on clarity, framing, and scope are fair, and they led us to new experiments and a clearer story. In short, we add a distortion-free watermark family, three-seed plain KGW baselines, a calibrated audit with a wrong-key null, and a re-analysis of the tokenizer probes.

W1, W2, Q1, Q4: We agree that the paper did not state its contribution crisply. We also agree with your reading in W2: the work is best presented as an empirical study, with codon-aware weighting as one design axis within it. We will revise the abstract and introduction to say so directly. Our contributions are:

  1. The first study of watermark inheritance through distillation in genome language models. Subword tokens do not align with codons, and output quality has a biological meaning. We cover five organisms, teacher-to-student size ratios up to 40, a second fine-tuning stage, and now two watermark families.
  2. A calibrated audit for DNA outputs. DNA generations repeat token pairs often. Scoring each unique pair once and building the null from wrong keys on the same outputs gives valid false-positive rates (FPR).
  3. Codon-aware weighting as a controllable design axis, with a map of detection against sequence quality.

Sander et al. established inheritance in text and noted that deduplication is needed for valid p-values. We ask whether this carries over to DNA. It does, under a protocol close to their closed-model setting: we prompt the student and score its outputs. At 1% FPR, each codon-aware setting in our main table and each new three-seed KGW student (δ = 2, 4, 6, and 8) is detected in 99.5% to 100% of random 10-response subsets. The matched unwatermarked control is flagged in 2.2%. Per response, joint fractional weighting at δ = 8 is detected in 92.0% of outputs.

On terminology (Q4): Sander et al. call the phenomenon radioactivity. We use the neutral term inheritance for the same phenomenon. We will state this correspondence explicitly and add a direct comparison paragraph covering protocol, scoring unit, and controls.

For the control experiment in Q1, the clearest test is codon usage, because the factor changes where the bias lands. If the usage factor did nothing, students would match plain KGW. They do not. Settings in our main table that use the usage factor reach a codon adaptation index (CAI) of 0.69 to 0.73. Plain KGW students stay at 0.61 to 0.62, position-only students at 0.62 to 0.63, and the unwatermarked control at 0.63. A shuffled-weight control makes this concrete. It keeps the joint weights at δ = 8 but assigns them to random tokens. Its students fall to a CAI of 0.62, against 0.72 for the aligned weights, while detection is unchanged (91.0% against 92.0% of single responses).

Inheritance also holds for plain KGW and for a distortion-free watermark (Part 2), so the phenomenon is not specific to our weighting. We therefore present the codon factor as a way to control codon-level properties of the marked outputs. We also replaced the single-seed comparator with three-seed KGW students trained and decoded exactly like the codon-aware students.

W1 (continued): We will move the scoring event, the operational definition of joint fractional weighting, and one worked token example into Section 3. We will define every factor in one place, and we will cut repeated qualifiers.

AuthorsOpenReview ↗

Thank you again for the generality question. Part 2 of 3: a distortion-free watermark and operating points that meet your quality targets

W3, Q2: Thank you for this suggestion. We ran the experiment. We implemented the Gumbel-max watermark of Aaronson (2023), which Sander et al. also study. The previous token and a secret key seed a uniform random vector r over the 4,096 tokens. The teacher emits the token v that maximizes log r(v) / p(v), where p is its usual sampling distribution (top-k 50, top-p 0.95). Over random keys, each step is an exact sample from the teacher. A step whose previous token already occurred in the response is sampled normally, which prevents loops. The detector scores -log(1 - r) for each unique pair. Everything else matches our main pipeline: 2,000 teacher sequences, 25% real E. coli replacement, LoRA training of GenomeOcean-100M, and 200 evaluation responses per seed.

Gumbel key applied toPer response, 1% FPR10 responses, 1% FPR
Gumbel teacher outputs100%100%
Gumbel students (3 seeds)99.2% (97.5% to 100%)100%
Unwatermarked students1.0%0.5%
Plain KGW students, δ = 21.5%1.2%

The distortion-free watermark is inherited. Both negative controls stay near the 1% target, so the audit is also specific to the key family. This matches what Sander et al. report for text.

The teacher outputs keep the sequence statistics of unwatermarked teacher outputs: ORF coverage 0.516 against 0.499, and 3-mer entropy 5.37 against 5.36. The students also match the unwatermarked control decoded the same way: ORF coverage 0.368 against 0.329, and entropy 4.98 against 4.85.

On compatibility: the codon factor scales a logit bias, so it cannot enter a distortion-free sampler without breaking exactness. A compatible version would choose which steps carry the watermark from the codon phase, which is known before sampling, so each step would remain an exact sample. We will describe this design, and we will expand our discussion of Hu et al. (2024), Kuditipudi et al. (2024), and Christ et al. (2024) accordingly.

W4, Q3: Yes, such operating points exist. Thank you for giving concrete targets. While answering, we found an error that matters here. The unwatermarked control in our quality table (ORF coverage 0.538, entropy 5.60) was decoded with presence and frequency penalties of 0.5, unlike the codon-aware students. We now compare against a control decoded exactly like the other students. The table uses clean outputs and three seeds. "Detected" is the share of single responses flagged at 1% FPR.

SettingORF coverage3-mer entropy (bits)Detected
Unwatermarked control0.3294.851.8%
Gumbel, distortion-free (new)0.3684.9899.2%
KGW, δ = 20.4565.1195.5%
KGW, δ = 40.4545.1798.0%
Position-B, δ = 40.3654.8994.0%
Joint-F, δ = 80.1663.9992.0%

Four settings match or exceed the control on both metrics, and they are detected in 94.0% to 99.2% of single responses. Plain KGW at δ = 2 and 4 meets both of your targets. Joint fractional weighting at δ = 8 sits at the high-bias end of the trade-off. Still, its gap to the control is much smaller than the paper reported: ORF coverage 0.166 against 0.329 (not 0.538), and entropy 3.99 against 4.85 (not 5.60). We will correct the table and the trade-off figure, recompute the folding and homology comparisons against the matched control, and recommend the quality-preserving settings as defaults.

AuthorsOpenReview ↗

Thank you once more for pushing on scope. Part 3 of 3: what the paper establishes, the tokenizer probes, and our revision plan

W5: We agree, and we appreciate this point. We will state what the paper establishes in plain terms and gather the caveats in one limitations paragraph. Under the calibrated audit, inheritance through shared-tokenizer distillation holds in each of these settings:

  • All five organism-conditioned datasets: 86.5% to 96.5% of single responses flagged at 1% FPR, against 0.5% to 4.0% for their unwatermarked controls.
  • Both 4B-teacher students: 84.0% to 96.5% for the 100M student and 64.5% to 70.0% for the 500M student.
  • A second fine-tuning stage: 90.0% to 97.5%, with the control at 1.0% to 2.5%.
  • A distortion-free Gumbel-max watermark: 97.5% to 100%, with negative controls at 0.0% to 2.5%.

We will also say in the abstract and introduction that the organism experiments use plain KGW at δ = 2.

Q5: Thank you for this question. Our new analysis suggests the probes were weak, not that transfer is impossible.

  1. Output length. The HyenaDNA students generated 512 nucleotides, about 97 teacher-token transitions, while GenomeOcean students give 511. Length alone costs much per-response power. When we cut our plain KGW GenomeOcean students to the same 512 nucleotides, per-response detection falls from 84.5% to 97.5% down to 11.5% to 85.0%.
  2. A weak but real signal. We pooled each student's responses and compared the score with the wrong-key null. HyenaDNA-large carries a signal: z = 2.3, 2.9, and 2.0 for the three seeds (p = 0.02, 0.01, and 0.04). HyenaDNA-medium is not significant. Transfer across tokenizers is weak, but it is not zero.
  3. GROVER is a null check. This probe retokenized joint outputs into GROVER's 610-token vocabulary and drew the keyed partition in that vocabulary. That partition has no relation to the teacher's, so the probe is a null test by design. Its mean of −5.47 and its 14.5% tail come from repeated token pairs. After deduplication, the real key behaves like a wrong key. We will relabel it as a null check.

Why is transfer hard? The scoring event is a pair of tokens in the teacher's vocabulary. A student with another tokenizer passes the signal on only if its nucleotide strings let the teacher's tokenizer recover the same pairs. Codon-level scoring would avoid this, and we list it as future work.

Revision plan. We will:

  1. Frame the work as an empirical study with a crisp contribution list.
  2. Make the calibrated audit the primary analysis.
  3. Add the distortion-free experiment and the three-seed KGW baselines.
  4. Report the operating-point table against the matched control.
  5. Move key definitions and a worked example into Section 3.
  6. Relabel the tokenizer probes and state the scope plainly in the abstract.

Thank you again for your time and for pointing out these issues so clearly. They have already made the paper clearer and more rigorous, and we will keep improving it along the lines above. We would be glad to answer any further questions.

Reviewer_5nfFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

A provider that releases synthetic DNA from a genome language model may want to know later whether a smaller model learned from it. Watermarks are known to carry over from teacher to student for text (Sander et al., 2024). The authors argue that DNA is harder, since tokens do not line up with codons and a strong watermark can distort sequences, and ask how to place the watermark so students inherit it with limited distortion.

The method builds on green-list watermarking (Kirchenbauer et al., 2023), where the teacher favors a secret set of tokens while generating. GenoTrace scales that favoring by two weights, for third codon positions and for codons an organism prefers. The provider audits by sampling the student without any watermark and testing, with the key, whether secret tokens appear more often than chance. The authors report that students inherit the signal, that the weighted versions keep more of it than a plain baseline after edits, and that stronger signals change sequence quality measures.

Critical strengths
  1. I find the problem worth studying. As far as I can tell, this is the first study of watermark inheritance under distillation for genome language models, a setting that the paper's cited related work does not cover. Tracing whether a model learned from a provider's released sequences matters for their provenance.

  2. Assuming the evaluation prompts are held out, the main experiments show a clear signal in students trained on watermarked outputs, repeated across seeds, with unwatermarked students near the expected level. The plain baseline already shows much of it, so this supports inheritance for a student sharing the teacher's tokenizer and family, not the codon weights in particular. The appendix reports every seed, and the authors state their limits openly.

Critical weaknesses

1. My main concern is that the paper does not show the codon weights beat a plain bias of matched strength.

  • The headline result compares the main joint setting with a plain baseline of different nominal strength and a single seed. The weights rescale the bias, so nominal strength says little about real strength. The authors acknowledge this, yet it is the abstract's main comparison.
  • The closer comparisons point both ways. At the same nominal strength, the usage weight beats the baseline under every edit. But a joint setting with about the same clean signal as the baseline detects less on clean outputs and does better only under the two key-using edits (Table 5). I cannot separate strength from structure.
  • Since a DNA sequence is just text to the model, plain green-list watermarking already transfers to the student, as the paper's own results and earlier work on text show (Sander et al., 2024; Gu et al., 2024). So the contribution rests on the codon weights.

2. The evidence does not show that the student's generations stay useful.

  • The settings that keep much of the signal after the key-aware and combined nucleotide edits change the sequences a great deal. Coding-like structure and diversity drop sharply (Table 17, Figure 4). Some milder settings survive the keyless synonymous edits but lose most of the signal under those two.
  • The quality measures are proxies for composition and structure, with limited samples for folding and homology. The authors say they do not measure function, so I cannot tell what a given drop means.
  • Quality is scored for student outputs only. I could not find a score for the watermarked teacher outputs that the downstream party receives, or for the plain baseline.
  • If the abstract's word "practical" covers output quality, these results do not yet support it.

3. Detection is not reported as a calibrated test, and the stress tests are light.

  • The audit applies the usual z-test to each response and counts every transition, even when contexts repeat, which the authors note makes tail probabilities hard to interpret.
  • Unwatermarked students are flagged at very different rates across the paper's own controls, near the nominal rate in the main runs but far above it in several others (Tables 4, 10, 12). Detection is reported at a fixed score threshold, not at a fixed low false-positive rate, and there is no wrong-key control, so I cannot tell what the audit would reach.
  • The edits act on the student's outputs and none adapts to the detector. If adversarial removal matters little for the intended use, that may be fine, but filtering the corpus or training the student further on clean data goes untested.
  • The main runs train on a small corpus seen repeatedly, and the paper leaves open whether the evaluation prompts were also in training.
Questions for authors
  1. Could you run the plain baseline at several bias strengths and plot every run, weighted and plain, as detection after the key-aware edit against a sequence-quality measure, with a control that shuffles the codon weights across tokens? If the codon weights come out ahead at matched quality, my main concern is resolved.

  2. What detection rate does the audit reach at a fixed low false-positive rate when the null is built from wrong keys on the same student outputs?

  3. Are the evaluation prompts disjoint from the training prompts? If they overlap, please report the main result on held-out prompts.

3 comments
AuthorsOpenReview ↗

Thank you for a rigorous and generous review. Part 1 of 3: matched KGW baselines and the shuffled-weight control

Dear Reviewer 5nfF,

Thank you sincerely for the time and care you gave our paper. We are grateful that you found the problem worth studying, and that you valued our seed-level reporting and our open statement of limits. Your concerns were precise, and each one pointed to a concrete experiment. We have now run all of them. Every new student uses three training seeds and the same teacher sampler, data mixture, training recipe, and decoding as the codon-aware students in the paper.

W1 and Q1. Do the codon weights beat a plain bias of matched strength?

Thank you for proposing this test. We trained plain KGW students at δ = 2, 4, 6, and 8. We also trained the control you suggested. It keeps the joint weights at δ = 8 but assigns them to random tokens. The weight distribution and the mean weight in each codon phase stay the same, and only the link between a token and its codons is removed. Its teacher marks 86.5% of training tokens green, close to the joint teacher (88.5%) and to KGW at δ = 2 (84.7%). So these three are matched in how strongly the training corpus is marked.

StudentCleanKTORF coverage3-mer entropyCAI
KGW, δ = 295.56.20.4565.110.618
KGW, δ = 494.810.30.4545.170.616
KGW, δ = 696.811.80.4855.200.620
KGW, δ = 895.210.20.4885.190.612
Joint-F, δ = 8, shuffled90.710.70.3494.940.617
Position-B, δ = 492.88.30.3654.890.627
Usage, δ = 489.044.80.1924.050.728
Joint-B, δ = 890.847.20.1443.930.731
Joint-F, δ = 894.549.00.1663.990.719

Detection rates are percentages at the paper's threshold. CAI is the codon adaptation index against E. coli. Each row averages three seeds.

  1. Bias strength alone does not reproduce the effect. Stronger KGW saturates. At δ = 4 its teacher already marks 99.3% of tokens green, and KT detection stays between 6.2% and 11.8% up to δ = 8. The shuffled control has exactly the joint weights, yet it falls to 10.7%, inside the KGW range. It also loses the codon-usage shift (CAI 0.617 against 0.719). The difference therefore comes from codon structure, not from the size of the bias.
  2. Clean detection is matched. All rows lie between 89.0% and 96.8%, so the differences after KT do not come from a higher starting rate.
  3. Quality. Plain KGW and position weighting keep higher coding coverage. The usage and joint settings trade coverage for codon-usage control. The two families occupy different regions of the plot, so we will present them as operating points to be chosen by the required quality level.

We will add the plot you requested, with every run of both families and the shuffled control. We will also replace the single-seed comparator in the abstract and in Table 1.

We also agree that plain green-list marks are learnable, as Sander et al. (2024) and Gu et al. (2024) show for text. Our matched baselines confirm this for DNA. What the codon weights add is control over the codon usage of the marked sequences, which plain KGW does not provide at any strength. We will cite Gu et al. (2024) and discuss it next to Sander et al. in Section 2.

AuthorsOpenReview ↗

Thank you for pressing on calibration. Part 2 of 3: detection at a fixed false-positive rate, and held-out prompts

W3, Q2: Thank you for asking for this directly. We scored the same student outputs with 1,000 random wrong keys. As you anticipated, counting every transition gives the wrong-key scores a heavy tail, because DNA outputs repeat token pairs often. Following Sander et al. (2024), we therefore score each distinct (previous token, token) pair once per response. With this scoring, wrong keys flag 1.1% to 1.2% of responses at z > 2.326, close to the nominal 1%. We then set thresholds at the empirical 1% and 0.1% quantiles of the wrong-key scores on each student's own outputs.

StudentFlagged at 1% FPRFlagged at 0.1% FPR
Unwatermarked, matched1.8%0.0%
KGW, δ = 295.5%88.7%
Joint-B, δ = 894.7%90.8%
Joint-F, δ = 892.0%84.7%

Thank you also for spotting the uneven control rates. The main unwatermarked control in Table 4 was decoded with presence and frequency penalties of 0.5, unlike the other students. With matched decoding, its nominal rate is similar to the controls in Tables 10 and 12. Under the calibrated audit, all of these controls fall to the expected range: 1.8% for the matched main control, 0.5% to 4.0% for the organism controls, and 1.0% to 2.5% for the fine-tuned-teacher control. Their watermarked students reach 86.5% to 96.5% and 90.0% to 97.5%. We will make this calibrated audit the primary analysis, and we will report the edit results separately as stress tests at the paper's threshold.

Q3: Thank you for raising this. Training and evaluation prompts were sampled as distinct windows of the E. coli genome, and no evaluation prompt appears in training. Because all windows come from one genome, some evaluation prompts partly overlap training material. To be strict, we kept only the evaluation prompts that share no 32-mer, on either strand, with any training prompt or response, teacher-generated or real. This leaves 375 of 600 prompts. On this held-out subset, the joint student is flagged in 91.7% of responses at 1% FPR, against 92.0% on all prompts. The result therefore does not depend on overlap with training material. We will state the sampling procedure and report this held-out check.

AuthorsOpenReview ↗

Thank you for keeping the focus on usefulness. Part 3 of 3: output quality, stress tests, and our revision plan

W2: Thank you for noticing that the teacher outputs were missing. The downstream party receives these, so we now score them on the same evaluation prompts, together with the plain baseline you asked about.

Teacher outputsORF coverage3-mer entropy (bits)
Unwatermarked0.4995.36
KGW, δ = 2 to 80.543 to 0.5525.50 to 5.55
Joint-F, δ = 80.3604.71

Plain KGW leaves these statistics at or above the unwatermarked level. Joint-F at δ = 8 lowers them, and it sits at the high-bias end of the trade-off. For student outputs, the matched control has ORF coverage 0.329 and entropy 4.85. Plain KGW (0.454 to 0.488) and Position-B at δ = 4 (0.365) stay above it, and 94.0% to 98.0% of their single responses are flagged at 1% FPR. We agree that these metrics are proxies and do not measure function. We will remove the word "practical", report teacher and student quality side by side, and recommend quality-preserving settings as defaults.

W3 (continued): We agree that our edits act on student outputs and do not adapt to the detector. We will call them fixed stress tests. Thank you also for naming corpus filtering and further training on clean data. Both are realistic ways to weaken an inherited signal, and our audit does not guard against them. We will state that the audit targets a student trained on the marked corpus, and we will add both to the threat model as open problems.

Revision plan:

  1. Make the calibrated audit, with an empirical FPR from wrong keys, the primary analysis.
  2. Add the three-seed KGW baselines, the shuffled control, and the detection versus quality plot with every run.
  3. Report the held-out prompt check.
  4. Report teacher and student quality against the matched control.
  5. Cite Gu et al. (2024), and present the edits as fixed stress tests.

Thank you once more for such a careful and fair review. Your questions translated directly into experiments, and they made the paper more rigorous and more precise about what it shows. We will keep improving it in this direction, and we would be glad to run any further check you find useful during the discussion.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Thank You to the SAC, AC, and All Reviewers

We sincerely thank the Senior Area Chair, the Area Chair, and all the reviewers for your time and for your careful reading of our paper. We are grateful for your encouragement, and just as grateful for the problems you raised. Your reviews led us to new experiments, including matched baselines, a calibrated audit, and a distortion-free watermark. They have already made the paper clearer and more rigorous, and we will use all of your comments to improve it.

GenoTrace asks a simple question: can a genome model keep a detectable record of the synthetic DNA it was trained on? Our study shows that it can. A watermark placed in a teacher's outputs is inherited by a smaller student, and it can be audited from the student's outputs alone.

We believe this question will matter more each year. Genome models now generate large amounts of synthetic DNA, and these sequences are shared, reused, and used to train new models. Without provenance, no one can tell which data shaped a model. Reliable provenance can give data providers credit for their work, help developers document where their training data came from, and support the accountable and responsible use of genome models.

We will release our code and audit tools, and we hope they help others test provenance in their own models. We are also working on next steps that your reviews helped us see, such as codon-level audits that work across tokenizers, and codon-aware marks that leave the teacher's sampling distribution unchanged.

We hope this work offers something useful to researchers, to developers, and to the wider community, so that genome models can be shared and used with trust.

Thank you again for your time and your support.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.