acceptodds
Under review as a conference paper at ICLR 2027

Marking the Wrong Symptoms: Evaluating LLM Watermarks in Medical Texts

Abstract

Large language models (LLMs) are increasingly integrated into clinical workflows, stressing the need for reliable traceability of model-generated output with watermarking. Yet, most watermarks are evaluated on general-purpose benchmarks, leaving domains like medicine, where small token-level perturbations can result in significant semantic changes, under-explored. In this work, we present the first rigorous study of how LLM watermarks affect medical performance, benchmarking 5 watermarking schemes across 11 LLMs and 7 VLMs on various tasks spanning unimodal and multimodal clinical reasoning. Importantly, we complement existing evaluations by introducing a human-expert-validated pipeline for systematically auditing medical reasoning quality, terminological precision, and induced hallucinations. Our results reveal that watermarking can induce substantial degradation across multiple failure modes, including lexical corruption, hallucinated terminology, and amplified misattribution or omission of image findings. Notably, we find that the absence of domain-specific analyses, combined with aggregate metrics that miss failures inherent to clinical text, can systematically obscure practical watermark-induced degradations. Our findings establish domain-specific evaluation as a prerequisite for the safe deployment of watermarked models in medicine, where current benchmarks can otherwise mask clinically consequential failures.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM, watermarks, medicine

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_mAtbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper addresses the safety issue of watermark usage in medical images, conducting a comprehensive evaluation of five watermarking schemes across 11 LLMs and 7 VLMs on multiple tasks covering both unimodal and multimodal clinical reasoning. The paper's findings show that watermarks can cause significant degradation under multiple failure modes, including lexical confusion, terminological hallucination, and amplified misattribution or omission of image findings.

Critical strengths
  1. The safety of medical VLMs is a research problem that cannot be ignored;

  2. The evaluated models are relatively comprehensive, including 11 large language models and 7 vision-language models;

  3. The experiments and analysis are sufficient, the content coverage is comprehensive, and the evaluation metrics are reasonable.

Critical weaknesses
  1. Using watermarks for medical image provenance is a development trend worth considering, is it necessary to use watermarks for medical image provenance? If medical institutions do not use watermarks as a basis for provenance, then the premise of this research topic does not hold, since embedding watermarks into medical images is itself unsafe;

  2. The paper cites too many arXiv preprints;

  3. Most of the experimental content is placed in the supplementary material, making it difficult for readers to follow;

  4. Since there are already papers studying watermark attacks against medical VLMs, what is the significance of the evaluation experiments in this paper?

Questions for authors

If the authors can properly address the above shortcomings, I am inclined to raise the score.

4 comments

We thank Reviewer mAtb for their valuable feedback, and address their individual points on weaknesses below.

Q1: Is watermarking for medical image provenance necessary, and is the premise valid? (W1)

We thank the reviewer for this question. First, we would like to make clear that we never watermark images (they are inputs only). The object of interest in this study is text-decoding watermarks. What we find is that watermarking the generated text tokens degrades how VLMs interpret the image, which is fed unchanged into the VLM encoder. The underlying question remains whether watermarking medical text is a realistic premise. We believe it is:

  1. LLMs are already heavily used in the medical context, whether by professionals for report generation, decision support, patient communication or education (81% of U.S. physicians use AI professionally; AMA, 2026), or by the general public (Arora et al., 2025).
  2. Independently, major providers have deployed watermarking. For example, SynthID-Text (Dathathri et al., 2024) is used for Gemini. Anthropic watermarks text generated by its Claude models using a version of SynthID-Text, applied worldwide to comply with the EU AI Act (Anthropic, 2026). OpenAI uses a watermark for ChatGPT and Codex in the EU.

Text watermarking is now required by regulation in the EU. Article 50 of the EU AI Act has applied since 2 August 2026. Under the accompanying Code of Practice on Transparency of AI-Generated Content, providers must mark generated content in a machine-readable way. This implies that LLM/VLM-generated output is increasingly likely to be watermarked, and we study its medical-specific failures, which so far have only been evaluated on general-domain benchmarks, and are non-trivial to detect, as our work suggests.

Q2: Why does the paper cite so many arXiv preprints? (W2)

We thank the reviewer for this remark. Of our 40 citations, 24 are peer-reviewed publications. Of the remaining 16, six are the technical reports of the models and the benchmark we use, which are only released as preprints, and seven are non-academic sources (model cards, regulatory documents, etc.). The last three are recent research preprints, which we cite because the field moves fast and relevant work often appears on arXiv before peer review. We have replaced arXiv entries with their peer-reviewed versions where available.

Q3: Can more experimental content move from the appendix into the main paper? (W3)

We thank the reviewer for this feedback and agree. Using the additional page, we have moved the key results of Sec. 4.4 and 4.5 into the main paper: Table 3 compares answer-only and full watermarking of reasoning models (accuracy, detectability and output length), and Table 4 shows the detectability at which our audit vs. the HealthBench rubric score first detects degradation.

Q4: How does this work differ from existing papers on watermark attacks against medical VLMs? (W4)

We thank the reviewer for this question. As clarified in W1, we do not alter model inputs (images or text), nor do we study an adversarial setting. Instead, we study the effect of watermarking on output quality. To our knowledge, the only prior work on the effect of watermarks on generated medical text is Hastuti et al. (2025), which we discuss in Sec. 2.

Thank you for the reviewer's careful response, which has resolved some of my concerns. However, I believe that although watermark attacks targeting medical VLMs are relatively scarce, the method proposed by the authors should still be further compared with other watermark attacks (outside the medical VLM domain), since the approaches are similar and only the domain differs. In addition, the current version of the paper still cites a considerable number of arXiv papers that have not undergone peer review. One last point: I would like to understand why, in the AI usage statement, the authors say they used AI to implement the method—why did the authors not write the code themselves to carry out the adversarial attack, given that injecting watermarks into medical images and optimizing them is not a particularly difficult task?

We thank the reviewer for engaging with our response. However, the reviewer’s comments indicate a fundamental misunderstanding of the subject of our work, which we would like to clarify.

On comparison with other watermark attacks. Our paper does not study watermark attacks, and no images are watermarked at any point. As stated in the abstract, Sec. 2 and Sec. 3.1, we evaluate generation-time text watermarks, such as KGW (Kirchenbauer et al., 2023), which embed a detectable signal into the text a model generates. Medical images are passed unchanged to the VLM; we measure how watermarking the generated text degrades its clinical quality. Attack methods are therefore not a meaningful baseline.

On arXiv citations. As stated earlier, only 9 of our 40 references are arXiv preprints. Six of them are the official technical reports of the models and benchmark we evaluate, which are released only on arXiv and are the standard citations for these artifacts, as common practice in ML papers. We have re-checked the remaining three as of today; none has a peer-reviewed version yet. We will update them if they appear before the camera-ready deadline.

On the LLM usage statement. As above, the paper contains no adversarial attack and no watermark injection into images. While a substantial portion of the code was written by hand, we disclose that parts of the implementation were AI-assisted, as required by the ICLR policy. We used it deliberately, as we believe that, used properly, it makes code better and less error-prone.

Thank you to the authors for their patient response to my comments; I will give them careful consideration.

Reviewer_rxQMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents a systematic evaluation of watermark on the medical domain. Five watermarking schemes were evaluated across 11 large language models (LLMs) and 7 vision-language models (VLMs) on a variety of tasks, including unimodal and multimodal clinical reasoning, as well as open-ended, multi-turn clinical dialogue. It shows that the application of watermarks leads to significant performance degradation in several forms, and an increase in the omission of findings from imaging examinations. In addition, it is observed that the actual performance decline caused by watermarks may be systematically obscured because of a lack of domain-specific analysis.

Critical strengths
  1. The first large-scale study on the impact of watermarking in the medical domain, covering both large language models (LLMs) and vision-language models (VLMs), with the lowest strength achieving ≥ 99% TPR at 1% FPR.

  2. Broad coverage. five watermark schemes, 11 LLMs and 7 VLLMs.

  3. Clinically significant findings: In reasoning tasks involving clinical text and images, traditional metrics (such as accuracy or perplexity) may systematically underestimate performance degradation related to safety.

Critical weaknesses
  1. It shows heavy reliance on LLM-as-a-Judge frameworks while claiming to be "judge-agnostic,"

  2. It use standard datasets like MedQA. But in the real application, the clinical notes are messy, We don't know if watermarks behave differently on highly unstructured real-world medical data from the paper.

  3. This paper only shows the problem of watermark on the medical text. But not show effective solution to it.

  4. The methods used in the paper are mainly published in 2024. KGW (Kirchenbauer et al., 2023) (soft logit bias), DipMark (Wu et al., 2024) (CDF reweighting), AAR (Aaronson, 2023) (Gumbel-max sampling), PPL (Gloaguen et al., 2026) (perplexity-budgeted argmax), and SynthID (Dathathri et al., 2024) (tournament sampling). There has been no empirical comparison of recent advancements in distortion-free and robust watermarking techniques (such as MirrorMark, TextSeal, and TTMARK) or perceptually sensitive watermarking techniques (such as VISA-Mark).

Questions for authors
  1. Why is watermark important in the medical texts? In the e-system, it is very easy to track the source of text. In addition, we heavily rely on the specialized expert in the medical domain, not LLM generated results.

  2. What is the sequence length considered in this paper.

  3. For reasoning models, is “final-answer-only” watermarking the type that reaches ≥99% TPR @ 1% FPR at practical response lengths?

1 comment

We thank Reviewer rxQM for their valuable feedback and address each point below.

Q1: Why is watermarking important for medical text? Do the findings transfer to messy, unstructured real-world clinical data? (Q1)

We agree that EHRs can log who entered a text. However, they do not record whether it was LLM-generated, and this information is lost once text is copied in from external tools or leaves the system. Medical text is also not confined to EHRs. 81% of U.S. physicians report using AI professionally (AMA, 2026), hence VLM/LLM output reaches clinicians directly or indirectly, e.g., via drafted notes or literature summaries. Independently, patients and the general public ask chatbots health questions outside any clinical system (Arora et al., 2025). This is arguably the more dangerous setting, since no professional checks the output. As AI output is increasingly watermarked, including through regulation (EU AI Act, Art. 50), text will be watermarked in all of these settings.

On unstructured real-world data: beyond exam-style vignettes, we evaluate open-ended, multi-turn HealthBench conversations (Sec. 4.5), where our audit transfers unchanged. We did not use real patient records by design (Ethics Statement) and name in-hospital workflows as a limitation (Sec. 5).

Q2: Is the evaluation really judge-agnostic, given the reliance on LLM judges? (W1)

By "judge-agnostic" we mean that the conclusions do not depend on the choice of judge model. This is supported by:

  1. All five judge models show consistent results. We re-scored the 650 physician-annotated samples with five frontier judges (four closed-source, one open-weight). On every axis carrying our claims, the closed-source judges agree with the physicians as well as the two physicians agree with each other, and differ from one another by at most 0.05 κ (Sec. 4.3, Tab. 12).
  2. Evidence independent of our judges. On HealthBench, the official physician-written rubric score, graded with OpenAI's own protocol, confirms the degradation (Sec. 4.5, Tab. 4). Our clinician panel (App. D.2), rated by a physician without any LLM judge, shows the same behavior.

We clarified this in the paper.

Q3: Does the paper offer any solution? (W3)

Our paper offers one concrete mitigation and one tested negative result. Mitigation: for reasoning models, watermarking only the user-visible answer and leaving the hidden reasoning unwatermarked keeps reasoning quality intact while still giving reliable detection (Sec. 4.4, Tab. 3). Negative result: prior work reduces quality loss by watermarking only high-entropy tokens (SWEET; Lee et al., 2024) or only lexically redundant ones (WatME; Chen et al., 2024a). This doesn't carry over to clinical text. Fabricated entities appear at high-entropy positions, while misspellings and misapplied terms appear at low-entropy ones, so no single entropy threshold avoids both.

More broadly, regulation such as the EU AI Act increasingly requires watermarking. In practice, clinicians and patients use general-purpose models, which often match or beat specialised medical ones (e.g., Llama-3.3-70B vs. the medical 70Bs we evaluate). So medical text will be watermarked whether or not anyone designed the watermark for it. That's why we think it matters to measure how watermarks affect a safety-critical domain like medicine, rather than build domain-specific watermarks that deployed systems won't use. Our physician-validated protocol gives a way to do that.

Q4: Why are more recent watermarking methods not evaluated? (W4)

We selected schemes that are either deployed in practice, such as SynthID in Gemini (Dathathri et al., 2024), or serve as standard baselines in the watermarking literature (Liu et al., 2024; Tu et al., 2024). Our five schemes span both main families, distortionary and distortion-free watermarks, and include PPL, a state-of-the-art 2026 scheme. Our pipeline, which we will open-source, can include any new watermark or benchmark with little adaptation.

Q5: What sequence length is considered? (Q2)

All models are prompted with chain-of-thought to first give an explanation and then the answer choice, so that the watermarked text typically spans 250–450 words for non-reasoning models and its statistical signal is meaningful. Non-reasoning models are capped at 2,500 tokens; reasoning models have a 10,000-token budget, of which the post-</think> answer (what we call the "final answer") is capped at 2,500 tokens (App. A.2).

Q6: Does final-answer-only watermarking reach ≥99% TPR@1%FPR at practical lengths? (Q3)

For reasoning models, TPR is computed on the user-visible answer after </think>, which contains a summary of the reasoning followed by the final answer. Depending on the setting, the watermark may or may not also be applied to the hidden reasoning trace inside <think>…</think>. Answer-only watermarking reaches 92–99% TPR@1%FPR (Tab. 3), close to but not always above 99%.

Reviewer_eyWsOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether text watermarking can hurt the quality of medical LLM outputs even when the final benchmark accuracy looks stable. The authors test five watermarking methods on 11 language models and 7 vision-language models using MedQA, MedXpertQA-MM, reasoning tasks, and HealthBench dialogues.

The paper uses several LLM judges to look for problems that ordinary accuracy misses, including fabricated medical terms, wrong terminology, spelling errors, contradictory reasoning, invented clinical details, and incorrect image descriptions. The judges are checked against two physicians on 650 blinded samples.

The main result is that watermarking can leave the final answer unchanged while making the reasoning less reliable. For example, the faulty-but-correct rate increases from 11.4% to 26.3% for PHI-4-14B with SynthID, and fabricated entities increase by up to 39.2 per 100 responses for LINGSHU-7B. The paper also finds that watermarking full reasoning traces is harmful, while watermarking only the final answer is much safer.

Overall, the paper makes a useful case that medical watermarking should be evaluated with domain-specific quality checks, not just answer accuracy. The main remaining concerns are the fairness of cross-scheme comparisons, the small branching-point experiment, and the broad interpretation of the clinical implications.

Critical strengths
  1. The paper makes an important point: a model can choose the correct answer while its medical reasoning contains serious errors. The examples of fabricated terms, wrong terminology, and unsupported image claims make this clear.

  2. The experiments cover a wide range of models, watermarking methods, and tasks. The inclusion of reasoning models, vision-language models, and HealthBench makes the study more useful than a single benchmark experiment.

  3. The physician validation is a real strength. Two physicians independently reviewed blinded samples, and the authors compared them with several LLM judges. The validation is reasonably convincing for the main error types, especially fabricated entities and image-grounding errors.

  4. The full-trace versus answer-only comparison is practically useful. The results suggest that watermarking only the final answer can preserve reasoning quality much better than watermarking the entire reasoning trace.

  5. The authors are fairly open about the limitations, including the small human validation set, the use of open-weight models, and the limited HealthBench setup.

Critical weaknesses
  1. The cross-scheme comparisons are not fully fair. Each watermark is evaluated at the lowest strength that reaches 99% TPR, but this can mean very different amounts of distribution shift, perplexity increase, or output-length change. A method may look more harmful simply because its chosen operating point is more aggressive. The paper should also compare methods at matched TPR, perplexity, KL divergence, or output length.

  2. The branching-point experiment is interesting but very small. It uses only three hand-selected questions and two main watermarking methods. Also, the trials are paired by seed, but the paper uses two-proportion z-tests instead of a paired analysis. A paired bootstrap or permutation test would be more appropriate.

  3. The paper runs many tests across models, schemes, strengths, and audit categories. It is unclear which results were planned in advance and whether multiple-testing correction was applied. This makes some of the highlighted p-values harder to interpret.

  4. The judge validation is uneven across categories. Agreement is much weaker for clue misread and some fine-grained errors than for fabricated entities or image claims. The paper should make clearer which audit dimensions are reliable enough for the main conclusions.

  5. The clinical claims are a little broader than the evidence. The experiments show harmful errors in benchmark and dialogue outputs, but they do not directly measure effects on real clinical decisions or patients. The abstract and conclusion should make this distinction clearer.

  6. The results may depend on the chosen prompts, temperature, seeds, and hashing setup. More seed variation and a small sensitivity analysis would make the main claims more robust.

Questions for authors

A. Clarification requests.

  1. Could you compare the watermarking methods at matched TPR values, such as 50%, 80%, and 99%? It would also help to include matched perplexity increase, KL divergence, or output-length change.

  2. Since Table 11 uses trials paired by seed, could you reanalyse it with a paired bootstrap, paired permutation test, or McNemar-style test? Please report confidence intervals for the effects as well.

  3. Which results were chosen as primary outcomes before the experiments? Was any multiple-testing correction used for the many reported p-values?

  4. Which audit dimensions do you consider sufficiently validated for the main claims? Table 10 suggests that agreement is much weaker for clue misread than for fabricated entities and image claims.

  5. Appendix D.2 uses only five questions and one watermarked output per setting. How were these questions selected, and do the conclusions hold on a larger, independently sampled set?

  6. The study uses exam questions and benchmark conversations rather than real clinical workflows. Could you narrow the wording about clinical risk, or add experiments involving report writing, summarisation, or tool use?

  7. How much seed variation was used for the main results? Could you report seed-level variation for the headline audit metrics?

  8. Does the answer-only recommendation still hold when the final answer includes a longer clinical explanation or is followed by additional dialogue turns?

1 comment

We thank Reviewer eyWs for their valuable feedback and address each point below.

Q1: Are cross-scheme comparisons fair at the chosen operating point? (W1, Q1)

We compare schemes at matched TPR at a fixed FPR, the default metric for detectability. The distribution shift, perplexity increase or length change a scheme needs to reach it is part of what we measure. For the costly judge audits we use ≥99% TPR@1%FPR, a common operating point (e.g., Dathathri et al., 2024; Tu et al., 2024). For all other operating points, we report every strength of every scheme (Fig. 2, Tabs. 5–6, Figs. 4–6, Tabs. 7 and 14), so any operating point can be read off directly. Following the reviewer's suggestion, we added App. B.2 (Figs. 6–7), comparing all schemes directly at matched 50%, 80% and ≥99% TPR for accuracy and perplexity on both benchmarks.

Q2: Is the branching-point experiment large enough, and is the test appropriate? (W2, Q2)

On re-examination, the unwatermarked prefix baseline was sampled separately and shared across rows (no seed-paired trials); we corrected this description in App. E. We now use Fisher's exact test with bootstrap 95% CIs (Tab. 13), and all eight effects remain significant. Scale is limited because prefixes must be clinically plausible and are thus physician-authored. The experiment tests whether effects stay consistent when varying the hash key; generalisation across questions rests on our audits of 2,000 questions per benchmark (Sec. 4.2).

Q3: Which outcomes were primary, and was multiple-testing correction applied? (W3, Q3)

Unlike common practice in medical research, we did not pre-register the analysis; our hypotheses were fixed by the research question (whether watermarking increases medicine-specific errors, how they present, and whether standard benchmark metrics detect them). The significance markers in our tables are uncorrected per-cell tests (p<0.01p<0.01, paired bootstrap). Our claims rest not on individual cells but on effects that recur across models and schemes, which isolated false positives would not explain.

Q4: Which audit dimensions are validated well enough to support the main claims? (W4, Q4)

Our LLM judges agree with physicians about as well as physicians agree with each other on every axis except clue misread (Sec. 4.3, Tab. 12). None of our claims rely on that axis; we report it only for completeness. Tab. 12 now marks which axes support our claims.

Q5: How were the App. D.2 questions selected, and do results hold on a larger sample? (Q5)

The five items were randomly selected across specialties (App. D.2). Each was sampled with 50 seeds without watermark, giving a tight noise band, and rated with watermark at 12 strength levels. Given limited physician time, we traded breadth for many seeds per item. Generalisation rests on our judge audits of all 2,000 questions per benchmark, validated against two physicians on 650 blinded samples (Sec. 4.3).

Q6: Are the clinical claims broader than the evidence? (W5, Q6)

We agree. Measuring effects on real clinical decisions would require prospective studies in which clinicians act on watermarked vs. unwatermarked output, where isolating the watermark is very difficult. Our study instead measures errors in VLM/LLM outputs, including open-ended multi-turn dialogue graded against physician-written rubrics (Sec. 4.5). We have narrowed the wording: the abstract now speaks of failures with potential clinical consequences, and the limitations state that we do not directly measure effects on clinical decisions or patients (Sec. 5), where we also name report writing and tool-augmented workflows as open directions.

Q7: How sensitive are the results to seeds, prompts, temperature and hashing? (W6, Q7)

Prompt content is fixed by the benchmarks; we wrap it in a single standard chain-of-thought instruction (App. G.1), identical across models and between watermarked and unwatermarked runs. Each configuration is sampled once per question at temperature 0.7 with a fixed seed shared between the watermarked and unwatermarked completion (App. A.2). Uncertainty is estimated by bootstrapping over the 2,000 questions per benchmark. Robustness to hashing is shown in App. E, where the effect persists when averaging over 200 private keys, and our clinician panel uses 50 seeds per item for the unwatermarked baseline (App. D.2).

Q8: Does answer-only watermarking work for longer explanations or follow-up turns? (Q8)

Answer-only watermarking is a deliberate choice that applies only to reasoning models, which separate a hidden reasoning trace from the visible answer. A longer watermarked explanation is therefore subject to the same failures we measure for non-reasoning models, both in multiple-choice reasoning (Sec. 4.2) and in multi-turn dialogue (Sec. 4.5).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.