- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes GenoTrace, a codon-aware watermarking method for tracing synthetic genomic data through model distillation. By incorporating codon-position and organism-specific codon usage information into traditional green-list watermarking, the method enables student models to inherit detectable watermark signals without modifying their inference process. Experiments demonstrate strong watermark detectability and improved robustness against certain editing attacks compared to conventional watermarking.
Critical strengths
- The proposed codon-aware design provides a meaningful domain-specific adaptation of conventional watermarking methods.
- The paper conducts extensive experiments across different watermarking configurations, adversarial editing strategies, model scales, and genomic datasets.
Critical weaknesses
- Lack of a preliminary section. The paper lacks a dedicated preliminary section that rigorously defines the problem setting and introduces essential biological concepts. This makes it difficult for readers without a biology background to understand the technical challenges and follow the proposed methodology.
- Uninformative illustration. Figure 1 does not effectively illustrate the key technical contributions or the logical flow of the proposed method. In particular, the core codon-aware watermarking mechanism is largely omitted, while considerable space is devoted to evaluation components. A more focused illustration of the proposed methodology would improve the paper's clarity.
- Insufficient baseline comparisons. The experimental comparisons do not convincingly isolate the benefits of the proposed codon-aware design. The conventional KGW baseline uses a different watermark strength and only one seed in the adversarial evaluation, making the robustness improvements difficult to attribute to the proposed mechanism.
- Unclear technical exposition and experimental justification. The paper is difficult to follow, particularly regarding how the proposed method addresses the identified technical challenges and how the experimental results establish its effectiveness. The intuition behind the codon-aware design is insufficiently explained, and the experimental comparisons do not clearly demonstrate its advantages over conventional watermarking methods. A more coherent presentation connecting the technical challenges, methodological choices, and empirical findings would significantly improve the paper.
Questions for authors
- Can you show Joint-F and Joint-B at (same as KGW baseline) in Table 1 to have a fair comparison?
- Are those in Figure 2 the scatter plots? If so, what is the x-axis for each watermark group? Or are they actually bar plots?
- Where will be KGW baseline if it is shown in the trade-off analysis by Figure 4?
- Does codon-aware weighting offer a meaningful advantage over simply increasing the strength of conventional KGW, particularly when controlling for watermark detectability and biological sequence quality?
2 comments
Thank you for the careful and encouraging review. Part 2 of 2: Figure 4 with KGW, clearer figures, and preliminaries
Q3: Thank you for this question. Below are ORF coverage and 3-mer entropy of clean student outputs, with detection after KT. We also add an unwatermarked control trained and decoded with the same settings.
| Scheme | δ | ORF coverage | 3-mer entropy | KT |
|---|---|---|---|---|
| Unwatermarked control | 0 | 0.329 | 4.85 | 1.7 |
| KGW | 2 to 8 | 0.454 to 0.488 | 5.11 to 5.20 | 6.2 to 11.8 |
| Position-B | 4 | 0.365 | 4.89 | 8.3 |
| Joint-B | 2 | 0.245 | 4.27 | 16.8 |
| Usage | 4 | 0.192 | 4.05 | 44.8 |
| Joint-F | 8 | 0.166 | 3.99 | 49.0 |
KGW would sit in the lower right of Figure 4, with high ORF coverage and low KT retention. Raising δ barely moves it. The usage and joint settings sit in the upper left. No single scheme dominates. KGW suits uses where coding coverage matters most. Codon-aware weighting suits uses that call for control over codon usage, and it retains more detections in our key-aware stress tests. We will also correct the reference line. The dotted line (0.538) came from the earlier run with different decoding. The matched control has ORF coverage 0.329, and the position-weighted students stay above it. We will add the KGW curve to Figure 4 and state the trade-off plainly in Section 5.5.
Q2: Thank you for catching this ambiguity. Figure 2 is a dot plot. The x-axis is categorical, and each tick is one condition (clean outputs or one edit type). Within a tick, markers for different schemes are shifted slightly left and right only so they do not overlap. The offset has no meaning. The y-axis is the detection rate. In panel (b), error bars are seed standard deviations. We will switch to grouped bars with value labels, add the three-seed KGW series, and say all of this in the caption.
W1: We agree. We will add a short section before the method. It will explain five ideas in plain terms:
- Codons and reading frames.
- Synonymous codons, which usually differ at the third base.
- Codon usage bias across organisms.
- Why subword tokens rarely line up with codons.
- Green-list watermarking and its z-score.
The section will end with a precise problem statement and one worked token example. The statement will say who holds the key, what the student trainer receives, and what the auditor observes.
W2: We agree that the current figure favors evaluation over mechanism. The new Figure 1 will have three panels. Panel (a) follows one decoding step for the token ATCG. It shows the green-list check, the position weight, the usage weight, and the final bonus. At phase 0, the position weight is one quarter for the fractional variant and one for the binary variant. Panel (b) shows distillation into the student. Panel (c) shows the audit. Evaluation components move to a thin strip with pointers to Section 5.
W4: Plain KGW gives every green token the same bonus. In a genome model, tokens are DNA strings of variable length that do not line up with codons, so a uniform bonus spreads over all green continuations. GenoTrace keeps the key and the green list, and only sets the size of each green token's bonus with two weights between 0 and 1. The first is the share of the token's bases at third codon positions. The second is the share of its complete codons that the organism prefers. The intended effect is to tilt the marked corpus toward the organism's preferred codons, so that the mark also shapes codon usage. The new data support this. Every usage-weighted student in the table shifts toward preferred codons (CAI 0.649 to 0.728, against 0.627 for the control). The shift disappears when we shuffle the weights across tokens (0.617), while clean detection stays similar. We will add this paragraph to Section 3, along with a short table linking each challenge to its design choice and its evidence.
Thank you again for your thoughtful and constructive feedback. Your comments on the baseline led directly to the matched experiments above, and they helped us state precisely what the codon weights add. Your comments on presentation will make the paper much easier to follow. We will include all of these changes in the revision. If anything is still unclear, or if another comparison would help, we would be happy to provide it during the discussion period. We would be very grateful if you would consider these updates in your final assessment.
Thank you for the careful and encouraging review. Part 1 of 2: a matched KGW baseline and Joint at δ = 2
Dear Reviewer YBj2,
Thank you very much for your time and for reading our paper so carefully. We are glad you found the codon-aware design a meaningful domain-specific adaptation, and that you noted the breadth of our experiments. Your review points to two clear improvements: a fair comparison with plain KGW, and a presentation that readers outside biology can follow. We are grateful for both, and we have acted on both.
Since receiving your review, we have:
All new students use the same teacher, student, prompts, real-genome mix, LoRA recipe, sampling settings, and edit code as the codon-aware rows of Table 1. Rates use the paper's scoring and threshold, as in Table 1. Each row averages seeds 44, 142, and 256.
W3, Q1, Q4: You are right that the single-seed KGW* row could not isolate the effect of the codon weights. KGW* also came from an earlier run whose student decoding used presence and frequency penalties. The table below replaces it with three-seed students decoded exactly like every codon-aware row, and we will use this baseline in Table 1, Figure 2, and the abstract. Mean push is δ times the average codon weight over the vocabulary, that is, the average bonus a green token receives. Teacher green is the share of tokens in the 2,000 teacher training responses that fall in the green list. CAI is the codon adaptation index of student outputs against E. coli.
The shuffled row keeps the Joint-F weights at δ = 8 but assigns them to random tokens. Another reviewer suggested this control.
The table shows three things.
Q1: Joint-B at δ = 2 is close to KGW at δ = 2 on clean detection (91.8% against 95.5%). It keeps 16.8% after KT, more than KGW at any strength, and it shifts codon usage (CAI 0.715). Joint-F at δ = 2 is weak, as expected. Its average codon weight is 0.19, so its mean push is only 0.39, one fifth of KGW at δ = 2. Equal nominal δ is therefore not a matched comparison, and mean push or teacher green is a better guide. We will add both rows to Table 1.
Q4: Thank you for this key question. Increasing KGW strength does not give the same benefit. Beyond δ = 4, a stronger KGW improves neither KT nor KN retention, and it never shifts codon usage. Codon-aware weighting provides both at similar clean detection. The cost is coding coverage, which we show under Q3. We will state this trade-off plainly and explain how to choose an operating point from the required quality level.