- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes HashMark, a watermarking framework for tabular data containing numerical values and free-form text. It embeds watermarks through keyed numerical modifications and semantic paraphrase selection, enabling blind detection without the original data or generative model. Experiments demonstrate robustness against various data modification attacks while preserving data utility.
Critical strengths
1.Practical applicability: HashMark supports both numerical and free-form text data, enabling blind watermark detection without access to the original data or generative model.
2.Theoretical analysis: The paper provides comprehensive mathematical analyses of watermark capacity, distortion bounds, and statistical detection.
3.Presentation: The presentation is clear.
Critical weaknesses
-
Limited methodological novelty: The core watermarking mechanisms of HashMark are closely related to existing approaches. Its numerical component relies on keyed hashing and small value perturbations share similar principles with prior tabular watermarking methods[1,2,3]. Its text watermarking component largely follows the semantic-space partitioning and rejection-sampling paradigm introduced by SemStamp[4]. A comprehensive comparsion and discussion should be added to clarify its novelty and improvements beyond integrating existing techniques.
-
Robustness: The numerical watermark relies on hashing values at a predefined decimal precision, making it inherently sensitive to small numerical perturbations. In practical data processing pipelines, rounding or numerical formatting, precision conversion is common. This limits the practical applicability of the proposed method. Appendix L.1 shows the strong fidelity and robustness results of HashMark are achieved under different operating conditions rather than simultaneously. A more systematic comparison of the fidelity–robustness trade-off would strengthen the paper.
-
Semantic Fidelity Concerns:The proposed text watermarking method selects paraphrases according to their positions relative to a secret hyperplane rather than explicitly optimizing semantic equivalence with the original text. These findings raise concerns about the the method for applications requiring precise preservation of factual information, such as medical records. The paper already include evaluates semantic fidelity using automated language models, but additional human evaluation or stronger semantic consistency constraints would be valuable.
[1]He, Hengzhi, et al. "Watermarking generative tabular data." arXiv preprint arXiv:2405.14018 (2024).
[2]Zheng, Yihao, et al. "Tabularmark: Watermarking tabular datasets for machine learning." Proceedings of the 2024 on ACM SIGSAC conference on computer and communications security. 2024.
[3]Ngo, Dung Daniel, et al. "Adaptive and robust watermark for generative tabular data." arXiv preprint arXiv:2409.14700 (2024).
[4]Hou, Abe, et al. "Semstamp: A semantic watermark with paraphrastic robustness for text generation." Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.
Questions for authors
See weakness. I am willing to revise my score based on the authors' response.
3 comments
W2 (rounding, and fidelity with robustness). The strongest fidelity result for Edit Mode and the strongest noise-robustness result for HashMark-Q describe different operating points. The revision adds a more systematic comparison by bin count (Table 44, Appendix L.1).
For Edit Mode, the owner declares a precision c and the attacker knows it. Rounding to c or more decimal places preserves the released grid values and hence the watermark. Rounding one decimal place more coarsely removes the exact-value evidence in Figure 2. For latitude at c = 6, that coarser scale is approximately 1.1 metres; the declared grid spacing itself is approximately 11 cm. Robustness to formatting at the declared precision does not extend to arbitrary numerical perturbations.
HashMark-Q trades larger changes for robustness by marking bins rather than exact values. There are operating points with both useful fidelity and rounding robustness: with 512–1024 bins, it retains z = 20–22 after rounding to two decimals, while a classifier distinguishes marked from unmarked rows with accuracy 0.51–0.55, close to chance.
Substantial multiplicative noise is more costly. At noise strength α = 0.2, 32 or fewer bins are needed to detect at least 99% of Shoppers tables; exact Edit Mode detects none (Table 31), and 32 bins detect 90% on Magic (Appendix J.2). The reported classifier accuracy at the 32-bin operating point is 0.898, so the perturbation is distinguishable. Table 44 shows classifier accuracy falling from 0.94 at 8 bins to 0.64 at 256 bins. The owner therefore chooses the operating point for the expected attack.
W3 (semantic fidelity). The hyperplane criterion does not ensure factual equivalence. A candidate can carry the mark and still be a poor rewrite. We have added human evaluation and experiments with factual-preservation instructions to examine this concern more directly (Appendix K.1).
First, we use DeBERTa-large-MNLI, independently of the watermark encoder, to test entailment in both directions between the original and released cells. Across 500 cells, HashMark leaves 335 unchanged and achieves a bidirectional-entailment rate of 0.746, compared with 0.282 when every cell is paraphrased. That aggregate result benefits substantially from leaving cells unchanged. Among the 165 rewritten cells alone, the keyed choice has a lower entailment rate than a random paraphrase, 0.230 versus 0.297, and a higher contradiction rate, 18.2% versus 7.9%. These results support your concern that selection can worsen an already imperfect candidate pool.
Second, a blinded annotator who was not involved in developing the method rated meaning preservation from 1 to 5 on 156 cells rewritten by HashMark. The mean was 2.92 for the keyed choice and 2.99 for a random paraphrase, a difference of −0.07 with 95% confidence interval [−0.19, 0.05]. The annotator found contradictions in 3.8% of cells in both conditions. This evaluation does not show a clear additional loss from keyed selection, but the absolute scores leave room for improvement, and one annotator provides limited evidence about precise factual preservation.
Finally, we tested 500 Drugs.com patient reviews. An LLM judge found that only 3% of the T5 rewrites HashMark released (and 1% of T5 paraphrases of the cells the local model rewrote) preserved every fact, and DeBERTa filtering did not adequately identify the failures. We therefore generated candidates with an LLM explicitly instructed to retain every fact, stopping at the first candidate on the valid side of the hyperplane. The rewritten reviews retained 96% of drug names and doses, compared with 72% for T5 rewrites of the same reviews; this particular check counts the names and doses without a model. Detection remained strong at z = 10.9. With a locally run 7B model, an independent judge rated 51% of rewrites as preserving every fact, compared with 1% for T5, while the watermark remained detectable at z = 8.2.
Better candidates therefore improve factual preservation and keep the watermark. Text whose exact facts matter should be rewritten only by a generator instructed to keep them, and its rewrites validated independently.
Thank you for your reply
Thank you for the detailed response and the additional experiments. The revision addresses some of my concerns.
-
The revision clarifies the contribution of adapting existing watermarking mechanisms to unordered mixed tables. I still see the novelty as modest, and the composed baseline leaves open how much HashMark improves over a stronger combination with separately calibrated numerical and text detectors.
-
Table 44 addresses my request for a systematic comparison. The revision now makes clear what fidelity cost accompanies each robustness result.
-
Semantic fidelity remains my main concern. Subtle numerical changes are hard for human raters to judge, so quantitative distortion and downstream utility are more informative for numerical columns. For rewritten text, the blinded evaluation is relevant, but the mean meaning-preservation score of 2.92 out of 5 leaves my concern about factual fidelity unresolved.
I really appreciate the time and effort the authors have put into the rebuttal and additional experiments. After careful consideration, I've decided to keep my original score. Thanks again to the authors for their efforts and thoughtful responses!
We thank the reviewer for their comments. W1 (novelty against [1–4]). HashMark keeps SemStamp's idea that a secret hyperplane in embedding space can carry a watermark. SemStamp's construction, however, relies on a sequence: each sentence's valid region is seeded by the previous sentence, and the language model is resampled until the next sentence lands in it. A table has no such sequence. Its cells are independent records, rows can be sorted or deleted, and a cell is often a single sentence. HashMark therefore paraphrases each cell on its own and keys every cell in a column to one hyperplane, so a cell's validity depends only on its own text and the column descriptor. This is simpler and order-free, but raises two problems SemStamp does not face.
Dependence between cells. With one shared hyperplane, similar cells land on the same side together, and sentence embeddings point in a common direction. An unmarked column can then show a large excess of valid cells: in our uncentered adaptation, clean scores have standard deviation 10.55 where an independent-bit test assumes 1 (Table 8), and a margin rule leaves it at 10.86 (Table 24). HashMark centers the embeddings on the column's published mean, derives the null variance from the pairwise angular correlations (Proposition 1), and decides ownership against fresh keys. Centered bits are still dependent (null standard deviation 0.71), so a raw z score is not a standard-normal p-value.
Capacity under a finite paraphrase budget. Meaning-preserving paraphrases of a cell cluster in embedding space, so extra candidates are not independent chances to cross the hyperplane, and some cells stay unmarkable. Proposition 2 predicts this failure probability from the candidates' geometry, and hence the expected valid fraction and score, with no fitted parameters. Appendix E.2 shows that a pilot pool of four paraphrases per cell already predicts the capacity of a pool of 32.
Numerical columns. WGTD [1], TabularMark [2] and Ngo et al. [3] move values into keyed intervals or sub-ranges. HashMark's Edit Mode instead searches the declared precision grid for the first valid point and leaves repeated inputs unchanged. Theorem 1 bounds both the largest change and the shift in each column's distribution. Detection counts each distinct value once, since repeats carry no new hash bit; without this, clean scores have standard deviation 22.6 instead of 1 (Appendix D.1). Coarser rounding than declared and noise need a different operating point (W2).
One decision for the table. HashMark declares the table marked if any column passes its corrected threshold, needing only the key and the published descriptor; prior tabular schemes need the original data, a generator at marking time, or a sampling protocol over candidate rows. The family-wise decision needs no independence between columns: on Airbnb, rounding away the numerical evidence leaves the text to decide, rewriting away the text leaves the numbers to decide, and only both attacks together remove the verdict (Table 17).
Why the combination needs these changes. The composed baseline combines WGTD-style binning and SemStamp-style selection under our table protocol, without centering or distinct-value counting. On an unwatermarked Airbnb table it scores 35.5, against 1.3 for HashMark. It separates marked from clean tables by 2.8 null standard deviations, against 67.8 for HashMark, and loses its evidence under rounding (Appendix F.1.2, Table 18). The revised related work states the inherited ingredients and these contributions.
Settings supported by the closest methods (revised Table 1):
Comparison with MUSE and Ngo et al. HashMark marks 10,000 numerical rows in about 0.2 seconds, where MUSE samples several candidate rows for each released row. On Shoppers, after 80% row or cell replacement, HashMark detects every tested table, against 26% and 34% for MUSE (Table 33). MUSE is more robust under value alteration and on Beijing (Tables 33 and 37). On Adult, HashMark's normalized C2ST is 0.996, equal to the unwatermarked release, against 0.883 for MUSE (Table 3). Our reimplementation of Ngo et al. (Appendix J.2), on Magic where its test is calibrated, survives shuffling and moderate row replacement, but detects at most 10% of tables at 80% value alteration and 1% after rounding to two decimals. On the same tables, Edit Mode detects 100% under value alteration, and HashMark-Q with 32 bins detects 100% after rounding, with a threshold calibrated in-sample.