- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how much of the underlying DNA sequence is leaked by mean-pooled embeddings from genomic foundation models. The authors propose Vec2DNA, a black-box embedding inversion attack that substitutes 1 to 3 nucleotides in each training sequence to form anchor–variant pairs, queries the frozen target model for both embeddings, and trains a non-autoregressive encoder-only inverter. The novel component is a feature-level consistency loss that constrains the L2 distance between pre-head features of anchor and variant at slots where the ground-truth token IDs match, on the grounds that a single substitution can change BPE segmentation and break direct positional correspondence. The authors analyze allele-level and locus-level disclosure, and propose TAG-Q, a post-hoc defense that fits linear probes for the intended downstream tasks, builds a basis that puts those probe directions first, and then spends more quantization bits on directions that are both task-relevant and high-variance.
Critical strengths
-
The paper correctly diagnoses why existing post-hoc defenses fail, and this diagnosis outlives TAG-Q itself. Section A.2 shows the two standard defenses failing in opposite directions rather than simply failing: Gaussian noise couples the two objectives, so on DNABERT-2 the attacker reaches the random baseline only once probing accuracy has collapsed from 0.81 to 0.63, while scalar quantization preserves utility even at one bit (0.77 on DNABERT-2, 0.93 on GENA-LM) yet floors inversion at 0.50 and 0.80.
-
The cross-model spread changes what the field should be measuring. Encoder-Only reaches 0.8476 on GENA-LM and 0.8300 on GROVER but only 0.5771 on DNABERT-2, all under BPE, refuting the tokenization-based explanation of inversion resistance. The spread across encoders exceeds any gain this paper’s attack achieves, so no single-encoder evaluation can establish that a model is safe. Vec2DNA holds its advantage at both ends, separating attack strength from encoder resistance as independent axes.
Critical weaknesses
-
The main comparison seems unfair: Vec2DNA was allowed four times as many queries as the baseline, and its own new idea explains only about a tenth of the gain. Every time the attack asks the target model for an embedding, that counts as one query. The baseline (Encoder-Only) was trained on 70,000 of them. Vec2DNA was trained on 280,000, because A.9 says the 70,000 original sequences plus the 210,000 mutated copies each cost one query. So the 14.5% improvement advertised in the abstract, introduction, and conclusion compares a method with a big budget against one with a small budget, and none of those three places mentions it.
-
The defense is only tested against an attacker who is not allowed to adapt to it, and its usefulness is only checked on the one task it was tuned for. A.9 states that the attacker is trained on undefended embeddings and then frozen, so it never learns what the defense did. That matters, because TAG-Q does not add randomness; it rotates the embedding in a reversible way and keeps the directions that carry the useful information. An attacker who simply retrains on defended embeddings would have very little to get past.
Questions for authors
I am willing to increase my score if author answer my questions well.
-
I am curious about the peformance if the baselines are retrained with the same 280k queries on unrelated hg38 sequences. Specifically, do Decoder-Only, ResNet, and NNL also close most of the gap, and does Vec2DNA remain the best method on all four encoders under equal budget?
-
How does the gap behave as the budget grows? If Encoder-Only, A2, and Vec2DNA are each trained at 70k, 140k, 280k, 560k, and 1.12M queries, do the curves converge?
-
What does Vec2DNA achieve when retrained directly on TAG-Q-defended embeddings at each target rate?
3 comments
Thank you for your insightful reviews.
A3. Limitations of the frozen-attacker protocol
We agree entirely, and this was the single most important correction prompted by the reviews. We ran the adaptive attacker and report the full results in the closing section of this response. In summary: against TAG-Q an adaptive attacker reaches LS 0.4204 on DNABERT-2 and 0.4248 on GENA-LM, both short of the composition oracle at 0.4613, with nucleotide accuracy below the uniform-random level. The same experiment also showed that our quantization baselines were only safe under the frozen protocol, which we report in full.
Adaptive attacker: an experiment we ran in response to the reviews
Our submitted evaluation froze an inverter trained on clean embeddings and applied it to defended ones. On reflection this protocol conflates two different things: that the information has been removed, and that the inverter has never seen this input distribution. We are grateful to the reviewers for pressing on this, and we ran the experiment it calls for: we let the attacker retrain on defended embeddings under the same query budget (70k anchors + 210k variants, all released through the defense) and evaluated it on the defended test split.
| Model | Defense | Frozen (submitted) | Adaptive | Adaptive NA | Margin vs oracle |
|---|---|---|---|---|---|
| DNABERT-2 | TAG-Q R=32 | 0.2338 | 0.4204 | 0.1813 | -0.0409 |
| DNABERT-2 | 1-bit quantization | 0.5036 | 0.5702 | 0.3338 | +0.1089 |
| DNABERT-2 | 4-bit quantization | 0.6325 | 0.6317 | 0.4004 | +0.1703 |
| DNABERT-2 | Gaussian σ=0.1 | 0.3707 | 0.4534 | 0.2357 | -0.0080 |
| GENA-LM | TAG-Q R=32 | 0.4910 | 0.4248 | 0.1825 | -0.0366 |
| GENA-LM | 1-bit quantization | 0.8022 | 0.8527 | 0.7235 | +0.3914 |
| GENA-LM | 4-bit quantization | 0.9650 | 0.9642 | 0.9251 | +0.5029 |
| GENA-LM | Gaussian σ=0.1 | 0.3902 | 0.4047 | 0.2437 | -0.0567 |
Margin = adaptive LS - composition-oracle LS (0.4613). The composition oracle is a random permutation of the target's own nucleotide multiset: the ceiling reachable from base composition alone, which is exactly what TAG-Q is designed to release. Uniform-random NA is 0.25.
TAG-Q holds under adaptation. On both encoders the adaptive attacker stays below the composition oracle (negative margin), and nucleotide accuracy is 0.181-0.182, below the uniform-random level of 0.25. Retraining does not recover sequence content; it converges to the composition-level ceiling the defense is designed to permit.
The quantization baselines, by contrast, were only safe under the frozen protocol. 1-bit quantization collapses against an adaptive attacker. Our submitted comparison therefore overstated how protective quantization is, and we have added an adaptive column to Table 5. We would not have found this without the reviews. The corrected claim is also more precise than the original: among the mechanisms we compare, TAG-Q is the only one that survives adaptation.
All of my concerns have been addressed. I have increased my score.
Thank you for your insightful reviews.
Dear Reviewer kBVG,
Thank you for the close and constructive reading of our experimental setup. The query-budget mismatch you identified was a genuine flaw in our comparison, and correcting it also required us to revise a figure in the abstract; we are grateful that you caught it before publication. We have rerun every learned baseline and measured the full budget curve you asked for. Our responses are set out below, and we report the outcomes in full, including those that were not in our favour.
A1. Matched-budget retraining of every learned baseline
The reviewer is correct, and we thank them for the observation. We retrained every learned baseline at an identical budget of 280,000 queries. Values are means over three seeds; ± is the seed standard deviation.
The 14.5% figure in the abstract was computed from the mismatched comparison, and we have replaced it with the matched-budget value. We are grateful to the reviewer for identifying this.
A2. Query-budget scaling curve from 35k to 280k
This was a valuable question, and the curve turned out to be informative in a way we had not anticipated. We reran the full pipeline at 35k, 70k, 140k and 280k victim queries. Each anchor consumes one query for itself and three for its SNP variants, so the query count is four times the anchor count. At every budget we also trained the same encoder with the consistency loss removed, which separates the budget effect from the contribution of the loss.
Three things are visible. First, the attack has not saturated: DNABERT-2 is still climbing at 280k (0.5538 → 0.5831 → 0.6170 → 0.6507), so the numbers we report are a lower bound on this threat rather than a ceiling. Second, the consistency loss helps at every budget (+0.0097 to +0.0151 on DNABERT-2, +0.0166 to +0.0329 on GENA-LM). Third, on GENA-LM the gain is largest at the smallest budgets, which indicates that the loss compensates for scarce data rather than merely accelerating what more data would eventually provide.
We state one limitation plainly. Our embedding pool is capped at 280k pairs, so we could not measure the 1.12M point the reviewer asks about. Given the slope of the curve we expect attack performance to be higher there, but we record this as unmeasured rather than estimated.