acceptodds
Under review as a conference paper at ICLR 2027

How Robust Are Code Watermarks, Really? Rethinking Robustness Beyond Rule-Based Rewriting

Abstract

As code LLMs are increasingly used to generate, transform, and learn from software, code watermarks are becoming important for establishing output provenance and protecting training-data ownership. Their utility, however, depends on robustness to semantics-preserving edits. Existing evaluations primarily rely on predefined transformation suites, which characterize robustness only to hand-specified edits and leave open whether the same conclusions hold under learned, input-adaptive semantic rewriting. Existing alternatives only partially address this gap: prompted rewriters offer limited control over rewrite magnitude and functional preservation, while token-focused adaptive rewriters cover only a narrow subset of code transformations. We therefore introduce WARP-Washer, a key- and detector-agnostic learned rewriting stress test trained exclusively on non-watermarked code. WARP-Washer combines supervised rewrite learning with execution-grounded GRPO to learn broad, functionality-aware transformations. At inference time, it produces a single rewrite without access to the watermark key, detector, or execution feedback, allowing the same policy to stress-test heterogeneous watermarks without target-specific guidance. Across four public inference-time and dataset-level watermarks, four programming languages, and four model backbones, WARP-Washer substantially disrupts heterogeneous watermark carriers while maintaining high syntax validity or compilation success, as appropriate to each language. On SrcMarker, it reduces message recovery by 81–91% relative to predefined AST transformations while retaining over 98% test-pass rates. Under matched-fidelity evaluation, it also achieves higher valid-removal rates than prior adaptive rewriting approaches throughout the evaluated threshold range. These results show that robustness conclusions drawn from predefined transformations cannot be assumed to transfer to learned semantic rewriting, motivating learned rewriters as a complementary stress test for code-watermark robustness. Our code is available at https://anonymous.4open.science/r/WARP-Washer-626D.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Code Watermarking, Watermark Robustness, Learned Rewriting Policy, LLM safety

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_DL22OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper argues that current robustness evaluations of code watermarks, which rely on predefined rule-based transformations, overestimate robustness and fail to reflect performance against adaptive, input-aware semantic rewriting. It introduces WARP-Washer, a rewriter trained on non-watermarked code using supervised transformation pairs followed by execution-grounded GRPO. The training objective encourages functional, non-trivial rewrites without watermark-detector feedback. At inference, the policy produces a single rewrite without access to the watermark key, detector, or execution-based candidate selection. Experiments across 4 watermark schemes, 4 languages, and 4 model backbones show that learned rewriting substantially reduces watermark detectability while maintaining functional validity, and outperforms rule-based and token-level baselines under matched fidelity. The paper concludes that learned rewriters should serve as a complementary stress test for code watermark robustness.

Critical strengths

Well-specified threat model. The paper clearly defines the attacker's knowledge constraints (no key, no detector, no watermark-specific training data) and adheres to it consistently throughout experiments. This strict black-box setting makes the evaluation meaningful as a stress test.

Comprehensive empirical coverage. The evaluation spans both inference-time and dataset-level watermarks, four programming languages, four model backbones, and multiple baseline categories. Beyond raw removal rate, the paper introduces additional rigor through matched-fidelity analysis, downstream utility tests, and retrained verification models for dataset-level watermarks that extend beyond simple carrier survival measurement.

Critical weaknesses

Limited methodological novelty and incremental research contribution. The core idea of using learned adaptive rewriting to attack watermarks is already well established in natural language watermarking, with works like SiRA and RLCracker having thoroughly explored this direction; the conceptual extension to code is straightforward, and the paper does not identify or solve any unique fundamental challenge that distinguishes code from NLP in this context. Methodologically, WARP-Washer is a direct combination of existing techniques: supervised fine-tuning on semantically equivalent code transformations followed by execution-rewarded reinforcement learning via the GRPO algorithm. This two-stage pipeline follows a standard fine-tuning paradigm, introducing neither new algorithms, novel attack principles, nor new theoretical insights into code watermark robustness. Overall, the work reads more like a solid engineering application of known ideas to the code watermark setting than a research contribution with substantial novelty.

Insufficient mechanistic analysis and limited prescriptive insights for watermark design. The paper convincingly demonstrates that learned rewriting can disrupt existing watermarks, but provides little deep analysis of why the attacks succeed or what the results imply for principled watermark design. The authors acknowledge overlap between the SFT transformation families and the carriers used by evaluated watermark schemes, but do not adequately disentangle whether removal comes from direct coverage of known carrier families versus adaptive selection or composition of transformations. The SFT/GRPO ablation studies confirm that both training stages contribute to performance, but do not resolve this attribution question. For example, it remains unclear how much SrcMarker disruption stems from identifier renaming, structural AST changes, or their interaction; nor is it tested whether the advantage of learned rewriting persists when the corresponding transformation families are excluded from training. The transformation-composition statistics count how many families appear per rewrite, but do not establish which specific modifications actually cause watermark extraction failure. Similarly, the hypothesis that naturally integrated carriers are more robust against untargeted rewriting is plausible, but remains largely qualitative and unvalidated. A systematic analysis linking concrete carrier properties to attack success would constitute a far more substantial contribution than additional aggregate removal results. Without such mechanistic analysis, the paper offers limited actionable guidance on where exactly current robustness claims break down, or what kinds of carrier designs might remain stable under learned rewriting.

Questions for authors
  1. Compared with existing adaptive rewriting attacks on natural language watermarks (e.g., SiRA, RLCracker), what is the core novel methodological contribution of WARP-Washer beyond adapting the idea to code and adding execution-based rewards? Please clarify what specific technical innovation distinguishes this work from prior works.

  2. Does the advantage persist when the relevant carrier family is excluded from the SFT data? Such an experiment would help distinguish generalization beyond the learned transformation inventory from effective application of transformations already covering the tested carriers.

  3. Can the authors isolate carrier-specific effects? For SrcMarker, separate identifier-only, structure-only, and combined rewriting would help determine what actually causes extraction failure. A comparison with comparable functional preservation and rewrite magnitude would materially affect my assessment.

Reviewer_GnNtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper focuses on the robustness of code watermarking, which studies semantic rewriting as an evaluation framework, and develops a code rewritter WARP-Washer for robustness evaluation.

The topic is important, and the paper is easy to follow.

However, I consider the contribution of the paper is not enough for publication at ICLR. I give my detailed review as follows.

Critical strengths

(1) This work focuses on a critical issue in the domain of code watermarking, i.e., robustness.

(2) The authors conduct extensive experiments, and the paper is easy to follow.

Critical weaknesses

My main concern regarding the work is the study and method designs and the corresponding evaluations, involving some 'targeted designs', as follows.

(1) The authors claim that their rewriting is watermark-agnostic, however, the SFT training transformations overlap with the carrier mechanisms of the evaluated watermarks, e.g., the prompt 'Avoid preserving unusual capitalization, underscores, prefixes, or placeholder-style local names'. The effectiveness may result from prior knowledge and targeted designs.

(2) The designed method shows limited contributions. Based on the experimental results, WARP-Washer has not shown significant advantages compared to directly rewriting the watermarked code.

In general,I consider the work involves certain amount of work with implementations and experiments, but the contribution is not enough for ICLR.

Questions for authors

Please kindly answer the questions in the above weaknesses (1) and (2).

Reviewer_jdCCOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents WARP-Washer, a watermark- and detector-agnostic code rewriting framework that combines supervised fine-tuning (SFT) on code transformation pairs with execution-guided GRPO. The authors evaluate the framework against four code watermarking schemes across multiple programming languages and model backbones. The results show that learned rewriting can substantially reduce watermark detectability compared with predefined AST transformations, highlighting limitations in existing robustness evaluation protocols.

Critical strengths

The paper addresses a meaningful problem: whether robustness conclusions obtained from predefined code transformations generalize to learned, input-adaptive rewriting.

The evaluation covers multiple watermarking mechanisms, programming languages, and model backbones. The results on SrcMarker are particularly compelling.

The authors consider both watermark removal and preservation, including fidelity-controlled comparisons, functional tests, downstream utility, and component ablations.

Critical weaknesses

1. Limited incremental contribution over existing LLM-based rewriting.

The incremental contribution of WARP-Washer is not sufficiently established. The untrained Base Model already achieves considerable watermark removal in several settings (Tables 2–3), and the additional improvements brought by SFT and GRPO are not consistently substantial across watermarking schemes. Moreover, the original PuzzleMark and DuCodeMark papers have already investigated LLM-based rewriting attacks and reported their effectiveness against dataset-level watermarks. Therefore, the vulnerability of code dataset watermarks to LLM rewriting is not entirely a new observation.

The key question is what fundamentally new capability or insight WARP-Washer provides beyond existing LLM rewriting attacks. Although the proposed method improves the removal–preservation trade-off in some settings, the paper should explicitly compare against the LLM rewriting results reported in the original watermark papers and demonstrate that the improvements are sufficiently substantial to justify the additional SFT and GRPO training pipeline.

2. Incomplete evaluation of structure-preserving transformation (SPT) based watermarks.

The evaluation omits CodeMark, a representative dataset watermark based on semantics-preserving program transformations (SPT). This omission is particularly relevant because WARP-Washer claims to evaluate robustness beyond predefined rewriting rules, yet its dataset-level experiments focus on PuzzleMark and DuCodeMark without directly examining CodeMark's transformation-based watermark mechanism.

Although CodeMark has been shown to be vulnerable to detection and removal by DeCoMa, this does not establish that SPT-based watermarking is inherently vulnerable to learned semantic rewriting. DeCoMa exploits statistical properties of watermark associations, whereas WARP-Washer attacks watermark carriers through code rewriting. These are different attack mechanisms, and success against one does not imply success against the other.

More importantly, future SPT-based watermark designs may use more robust carrier selection or transformation strategies. Without evaluating CodeMark or comparable SPT-based watermarks, the paper cannot establish whether WARP-Washer generalizes to this important class of code watermarking mechanisms.

The authors should include CodeMark as an additional baseline watermark and investigate whether learned rewriting can effectively disrupt its transformation-based carriers while preserving program functionality.

Questions for authors

Please address the concerns raised in the Critical Weaknesses section.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.