- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper argues that current robustness evaluations of code watermarks, which rely on predefined rule-based transformations, overestimate robustness and fail to reflect performance against adaptive, input-aware semantic rewriting. It introduces WARP-Washer, a rewriter trained on non-watermarked code using supervised transformation pairs followed by execution-grounded GRPO. The training objective encourages functional, non-trivial rewrites without watermark-detector feedback. At inference, the policy produces a single rewrite without access to the watermark key, detector, or execution-based candidate selection. Experiments across 4 watermark schemes, 4 languages, and 4 model backbones show that learned rewriting substantially reduces watermark detectability while maintaining functional validity, and outperforms rule-based and token-level baselines under matched fidelity. The paper concludes that learned rewriters should serve as a complementary stress test for code watermark robustness.
Critical strengths
Well-specified threat model. The paper clearly defines the attacker's knowledge constraints (no key, no detector, no watermark-specific training data) and adheres to it consistently throughout experiments. This strict black-box setting makes the evaluation meaningful as a stress test.
Comprehensive empirical coverage. The evaluation spans both inference-time and dataset-level watermarks, four programming languages, four model backbones, and multiple baseline categories. Beyond raw removal rate, the paper introduces additional rigor through matched-fidelity analysis, downstream utility tests, and retrained verification models for dataset-level watermarks that extend beyond simple carrier survival measurement.
Critical weaknesses
Limited methodological novelty and incremental research contribution. The core idea of using learned adaptive rewriting to attack watermarks is already well established in natural language watermarking, with works like SiRA and RLCracker having thoroughly explored this direction; the conceptual extension to code is straightforward, and the paper does not identify or solve any unique fundamental challenge that distinguishes code from NLP in this context. Methodologically, WARP-Washer is a direct combination of existing techniques: supervised fine-tuning on semantically equivalent code transformations followed by execution-rewarded reinforcement learning via the GRPO algorithm. This two-stage pipeline follows a standard fine-tuning paradigm, introducing neither new algorithms, novel attack principles, nor new theoretical insights into code watermark robustness. Overall, the work reads more like a solid engineering application of known ideas to the code watermark setting than a research contribution with substantial novelty.
Insufficient mechanistic analysis and limited prescriptive insights for watermark design. The paper convincingly demonstrates that learned rewriting can disrupt existing watermarks, but provides little deep analysis of why the attacks succeed or what the results imply for principled watermark design. The authors acknowledge overlap between the SFT transformation families and the carriers used by evaluated watermark schemes, but do not adequately disentangle whether removal comes from direct coverage of known carrier families versus adaptive selection or composition of transformations. The SFT/GRPO ablation studies confirm that both training stages contribute to performance, but do not resolve this attribution question. For example, it remains unclear how much SrcMarker disruption stems from identifier renaming, structural AST changes, or their interaction; nor is it tested whether the advantage of learned rewriting persists when the corresponding transformation families are excluded from training. The transformation-composition statistics count how many families appear per rewrite, but do not establish which specific modifications actually cause watermark extraction failure. Similarly, the hypothesis that naturally integrated carriers are more robust against untargeted rewriting is plausible, but remains largely qualitative and unvalidated. A systematic analysis linking concrete carrier properties to attack success would constitute a far more substantial contribution than additional aggregate removal results. Without such mechanistic analysis, the paper offers limited actionable guidance on where exactly current robustness claims break down, or what kinds of carrier designs might remain stable under learned rewriting.
Questions for authors
-
Compared with existing adaptive rewriting attacks on natural language watermarks (e.g., SiRA, RLCracker), what is the core novel methodological contribution of WARP-Washer beyond adapting the idea to code and adding execution-based rewards? Please clarify what specific technical innovation distinguishes this work from prior works.
-
Does the advantage persist when the relevant carrier family is excluded from the SFT data? Such an experiment would help distinguish generalization beyond the learned transformation inventory from effective application of transformations already covering the tested carriers.
-
Can the authors isolate carrier-specific effects? For SrcMarker, separate identifier-only, structure-only, and combined rewriting would help determine what actually causes extraction failure. A comparison with comparable functional preservation and rewrite magnitude would materially affect my assessment.