PRO: Enabling Precise and Robust Text Watermark for Open-Source LLMs

Text watermarking for large language models (LLMs) enables model owners to verify the origin and protect the intellectual property of AI-generated text. While text watermarking for closed-source LLMs is relatively mature, open-sourcing the LLM introduces unique challenges: existing methods inject the watermark during the model owner's decoding process, but for open-source LLMs, end users control decoding and can arbitrarily remove it at inference time. The watermark must therefore be embedded into the released model weights, so that standard decoding natively produces watermarked text. Recent methods attempt this by distilling from a decoding-based watermarked LLM, but two challenges remain: (i) learnability: the LLM may not perfectly internalize the predefined watermark pattern, producing a generation-detection inconsistency between what the LLM generates and what the detector expects; and (ii) robustness: open-sourcing the model exposes the embedded watermark to weight-level user modifications such as fine-tuning and model merging. A practical open-source LLM watermark must therefore align the LLM's learned watermark with the detection pattern, and remain robust against weight-level user modifications. Guided by these requirements, we propose PRO, a precise and robust open-source LLM watermarking method. PRO jointly trains a watermark policy model with the LLM so that the LLM's learned watermark aligns with the detection pattern, and further incorporates a perturbation-aware regularizer that improves robustness against weight-level modifications. Evaluated on several mainstream open-source LLMs, PRO improves watermark detectability to $0.92$ TPR at $1\%$ FPR while reducing perplexity by $20.5\%$ over the strongest prior method, and preserves detectability under aggressive weight-level modifications, e.g., AUC $0.87$ vs. $0.79$ under $0.5$-ratio model merging. The code is publicly available at https://anonymous.4open.science/r/PRO-44B7.