acceptodds
Under review as a conference paper at ICLR 2027

CohortMark: Watermarking Text Cohorts While Preserving Every Proper Subset

Abstract

Text watermarks seek detectable signals with limited distributional change. Pre-serving individual distributions need not preserve subgroup laws; preserving thecomplete group's clean law conditional on key and metadata leaves no evidence.Our insight is to separate observation scopes: preserve every proper subset whileplacing evidence in the original group's higher-order relation. Token discretiza-tion can erase this dependence. CohortMark uses an analysis of this loss to designa finite-harmonic coupling whose shape controls evidence without altering sub-set laws. Source-model replay with required metadata gives the original group'sexact token-interval likelihood ratio. With ideal exact sampling, token probabili-ties based only on each member's own prompt and history, and fresh joint drawsper step, all predesignated nonempty proper subsets follow independent clean se-quence laws at each fixed key and nonce. Across two model families and twotasks, the three-member instance achieves 94.8-99.0% recall at separately cal-ibrated nominal 1% false-positive rates. All four settings meet predeclared re-call noninferiority and false-positive criteria against SynthID; both Llama settingssupport higher recall. Exchanging one member at the same three-text, 768-tokenbudget reduces positive rates to 1.0–1.2%, linking evidence to original joint gen-eration. In automatic ELI5 evaluation, intervals for changes from clean in thefraction of groups with at least one usable answer lie within ±3 percentage points.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
text watermarking, large language models, higher-order dependence, distribution preservation, provenance verification

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_VHmBOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CohortMark, a watermark for a cohort of texts generated jointly by a language model. The method proposed ensures every proper subset of the documents follows the original distribution and contains no detectable watermark, while watemark signal can be detected when the full cohort is presen. The paper tested the proposed method on real dataset C4,ELI5 and models Qwen,Llama. Results shows that the TPR is close to 1 under cohort level detection and falls to ~1% if member text is changed, indicating that the signal only exists when the cohort is examined as a whole.

Critical strengths

The paper proposed a different setting of watermark where detection is only available when a full cohort of text is present, individual and subset of text does not show detectable signal.

Critical weaknesses

One issue is the practical applicability of this scheme. Detection requires "original token IDs in member order, source-model replay with the same vocabulary" and other source data (Sections 3.3), verification can be difficult for anyone other than the text provider, since model replay might not be feasible, baseline methods KGW and Synthid compared in experiments does not require model replay or any model parameters.

The paper didn't evaluate roubustness to editing or rewriting. This might be more significant to this work since replacing a member voids the detetection.

Questions for authors

Could the author elaborate on the motivation of such design, in what situation would it be undesirable to detecte watermark in individual part of the text?

In experiment, KGW and SynthID does not rely on model replay, but still achieves comparable TPR, can we design other experiments to demonstrate the advantage of the proposed method?

Reviewer_VPCQOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

When generating multiple text outputs at once (a "cohort" of answers, such as 3 candidate responses), existing watermarks face a trade-off, which is, even if a watermark preserves the probability distribution of each candidate, it may distort the statistical relationships and similarities, as well as joint probabilities across subsets of those outputs (e.g., pairs of texts). Conversely, if you preserve the exact clean joint distribution of the entire group for a fixed key, there is no statistical difference left to detect the watermark. The authors' main goal is to separate these observation scope i.e., any incomplete subset of the generated group to be statistically indistinguishable from clean, unwatermarked, independent generation while hiding the watermark signal entirely inside the higher-order mathematical relationship of the complete group. To achieve this, they (i) couple the continuous sampling variables across the cohort members at each generation step using a one-periodic density while keeping each member's local token probability mapping intact (ii) analyze how discretizing continuous draws into token probability intervals attenuates this relational dependence, and design a finite-harmonic Fejér coupling kernel that boosts higher-frequency components so the signal survives interval averaging; and (iii) formulate an exact token-interval likelihood-ratio detector that verifies the complete original cohort via white-box source-model replay using the exact generation metadata, key, and nonce.

Critical strengths

I would admit that the authors gave an insightful analysis of how the watermarking signal is lost when continuous random numbers are converted into discrete text tokens, and designed a clever solution to overcome this. In language models, choosing a word from a probability distribution acts like a coarse filter that averages out subtle statistical patterns, which normally erases higher-order relationships across a group of texts. The authors clearly diagnose this bottleneck and introduce a specialized multi-frequency coupling method that concentrates the watermark signal into sharper patterns capable of surviving the word-selection process.

BUT (see weaknesses)...

Critical weaknesses

WHY?

  • This seems to me like an artificial problem formulation when simpler solutions exist. Forgive how that may have sounded, but while exposing how subset relationships behave under fixed watermarking keys is a neat theoretical observation, it appears to be a solution in search of a problem. Look at it this way... in practice, if a practitioner wants to generate multiple candidate responses without introducing unwanted correlations between them, they can simply use an existing unbiased or non-distortionary watermark with a fresh, independent random seed (nonce) for each candidate derived from a master key. Averaged over that randomness, the candidates are already statistically independent and distortion-free, while retaining the massive practical advantage that every single text can be detected on its own. This paper seem to be forcing a problem by insisting that subsets must remain completely independent even when the key and nonce are held fixed. It is unclear why anyone would ever need this strict fixed-key property, especially when achieving it requires completely sacrificing the ability to detect the watermark in individual texts.

  • Continuing my point above, perhaps the motivation could have been stronger if there is a realistic use case or threat model. The authors never describe a concrete real-world deployment scenario where this method would actually be useful. When a language model generates three candidate answers to the same prompt, those answers are semantically redundant drafts. In standard workflows, for example ChatGPT/Gemini offering multiple drafts or automated reranking (best-of-N), most users simply picksthe single best response and throw the rest away. Under CohortMark, the moment a user selects one (or even two) responses to keep or publish, the watermark vanishes completely by design. Furthermore, verifying the watermark requires the detector to possess all three unedited texts in their exact original order. If an auditor already has the unedited texts, exact ordering, prompts, and generation metadata, they are essentially looking at their own internal server logs. Statistical watermarking is unnecessary in this setting.

  • The authors highlight that swapping out just one of the three texts causes watermark detection to plummet from roughly 99% down to 1% (random chance), framing this as proof that the watermark successfully relies on the whole group's relationship. From a practical security and provenance standpoint, this represents extreme brittleness. Unlike standard watermarks (SynthID and KGW), which maintain over 90% detection in the exact same experiment because the remaining texts still carry their own watermark signals, CohortMark fails completely if even a single candidate is dropped, replaced, reordered, or lost. Combined with the authors' acknowledgment that text editing, retokenization, and missing-member recovery are unsupported, the watermark can be trivially defeated simply by not publishing all three redundant outputs together.

Questions for authors

(1) Why not use standard unbiased watermarks with independent per-candidate seeds? I mean, that would be the go to solution in practice. So, why is preserving subset distributions under a fixed, known key and nonce important enough in practice to justify losing the ability to detect individual texts? (2) What is a concrete real-world application for this? (3) How can the method survive basic selection or subsetting?

Reviewer_iqWzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CohortMark proposes a watermarking framework that embeds provenance exclusively across a full cohort of generated texts, ensuring that any individual output or proper subset follows the exact unwatermarked model distribution. The approach couples sampling across cohort members using a regularized Fejer kernel to mitigate token discretization loss, detecting the watermark via synchronous white-box model replay. On standard language models, full cohorts achieve high detection recall at a one percent false positive rate, whereas missing or swapped members drop detection to random chance.

Critical strengths

Moving watermarking from single-sequence carriers to higher-order cohort interactions is an innovative conceptual contribution. Guaranteeing that every proper subset remains completely clean and unbiased under a fixed key reframes how multi-turn and multi-candidate model outputs can be watermarked.

The circular translation property of the one-periodic coupling function cleanly guarantees that marginalizing out any coordinate yields a uniform product measure. The authors derive exact closed-form expressions for discrete rectangle integration and likelihood ratios over token intervals.

The paper adheres to pre-registered evaluation protocols, reports simultaneous confidence intervals across multi-hypothesis test families, and openly documents prior negative pilot findings and failure modes.

Critical weaknesses

The central technical guarantee that every proper subset exhibits zero watermark evidence is also the method's greatest operational vulnerability. In real-world deployment, users rarely publish or utilize all generated candidate texts together. If an adversary or benign user extracts only one or two outputs from a three-member cohort, all watermark evidence disappears completely, and detection collapses to random guessing. A watermarking scheme that fails when an end-user discards a single completion is difficult to justify for practical model provenance.

Detection relies on step-by-step synchronous alignment of token probability intervals across all members of the cohort during model replay. If any completion undergoes basic user modifications, such as deleting a sentence, fixing a typo, paraphrasing, inserting punctuation, or changing formatting, the coordinate alignment breaks down. While traditional single-text watermarks degrade gracefully under mild editing, CohortMark lacks robustness against localized text edits and length variations.

The detector cannot evaluate texts using simple n-gram hashing or black-box observation. It requires full source-model replay: executing simultaneous forward passes for all cohort texts under identical model checkpoints, vocabulary ordering, floating-point implementations, and decoding configurations. This restricts auditing exclusively to internal model owners and prevents third-party or lightweight detection.

The discrete survival factor for each Fourier harmonic decays exponentially as a product of per-member sinc terms. As cohort size grows beyond three members, the discrete coupling signal diminishes rapidly. This makes scaling CohortMark to larger cohorts or interactive multi-turn dialogue agents computationally impractical, as the required generation horizon would grow prohibitively large.

While automated evaluator metrics show small differences, the blind human audit across 100 questions yields wide reference confidence intervals spanning approximately negative ten to positive twelve percentage points. The sample size is insufficient to statistically confirm that cohort coupling preserves generation quality to the claimed precision.

Questions for authors

see weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.