acceptodds
Under review as a conference paper at ICLR 2027

CodeKAST: Keyed AST-Safe Structural Watermarking for LLM-Generated Code

Abstract

Watermarking code generated by large language models poses the challenge of embedding detectable evidence without compromising functional correctness. Existing methods rely on the assumption that biasing token distributions at generation time is sufficient to leave a persistent watermark signal in code. Our analysis reveals a fundamental limitation of this assumption: program semantics are carried by structure rather than surface tokens, and token-level biases can perturb syntax-critical elements, thereby compromising functional correctness. To address this problem, we propose CodeKAST, a unified framework that embeds key-controlled structural evidence in the abstract syntax tree space rather than the raw token space and uses constrained syntax transformations to generate functional watermarked programs. Specifically, an HMAC-controlled encoding scheme derives a structural state for each supported AST site from a secret key and pre-declared verification information; a candidate program is released only after passing syntax checking, round-trip reconstruction, and execution-based screening. During verification, CodeKAST compares the structural states observed in the released code with those expected under a nominated key. The resulting evidence supports both watermark-presence detection and candidate-key attribution, while the verifier abstains when the available sites are insufficient to distinguish candidate keys.On HumanEval+ and MBPP+, CodeKAST achieves the best detection performance among the evaluated methods under a unified model and evaluation protocol, while preserving the functional correctness of the original code, without an extra generative model training process.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
code watermarking, code generation, large language models, code provenance

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_fD2FOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes CodeKAST, a training-free, post-generation watermarking framework evaluated on Python code. Appendix B describes four supported transformation families: parameter aliasing, loop-variable renaming, local-variable renaming, and return wrapping. An HMAC-SHA256 rule maps a secret key, public verification context, and site identity to a target state at recoverable AST sites. Transformed code is released only after syntax, round-trip reconstruction, and execution-based checks; otherwise, the original code is retained. Verification reconstructs the site states without the original program and computes a standardized match score for watermark detection and candidate-key attribution, with abstention for absent sites or tied evidence.

Experiments use Qwen2.5-7B-Instruct on MBPP+ (378 tasks) and HumanEval+ (164 tasks), comparing against locally rerun KGW, Unigram, SWEET, and STONE. Reported AUROC is 0.9118 and 0.9904, respectively; Recall@1%FPR is 82.54% and 93.29%, using thresholds selected on the evaluated samples. Strict EvalPlus Pass@1+ is unchanged at 69.05% and 64.02%, with zero reported additional failing tasks. On MBPP+, attribution among the embedding key and 256 wrong keys yields 311 correct decisions, zero wrong decisions, and 67 abstentions (82.28% coverage, 100% conditional accuracy). Formatting preserves these counts; local-variable renaming reduces coverage to 77.51%.

Critical strengths

1.Practical postprocessing that is model-agnostic by design, although evaluated with one generator. CodeKAST does not require decoding-time access or auxiliary training. Recovering evidence from the released code without the original program is a useful deployment property. The reported injection and pipeline timings suggest modest postprocessing overhead for the evaluated functions (Section 4.1).

2.Encouraging detection-utility evidence with transparent qualifications. Tables 1, 2, and 5 report strong detection and no observed additional strict EvalPlus failures on both paired cohorts. The manuscript also reports abstention explicitly, identifies threshold selection as in-sample, and keeps the 974-task development analyses separate. These practices make the empirical results easier to interpret.

Critical weaknesses

1.Attribution evidence supports closed-set ranking, not a rejection-capable provenance decision. The paper explicitly considers a finite candidate-key set (Section 3.1). In Algorithm 2 (p. 6, lines 270-280), the threshold affects only D for the nominated key; attribution returns a unique top-scoring candidate without an absolute acceptance condition. As specified, it can return a key even when all candidate scores are below the detection threshold. This is a specification concern, not a verified implementation error. The reported audit (Section 4.2; Appendix A.3) covers the 378 released MBPP+ programs with the embedding key always among 257 candidates; some released programs may be unchanged fallbacks, whose number is unreported. The 311/0/67 result supports accuracy in that candidate-set setting, but does not establish behavior on an explicit unwatermarked cohort or when the true key is absent. This concerns the scope of the additional attribution contribution, not the primary single-key detection AUROC.

2.Comparative and component-level evidence does not sufficiently isolate the contribution. All four head-to-head baselines are generation-time token methods operating under a different access model. The large reported detection gap is useful but does not determine the relative benefit over post-generation structural watermarking. Related Work discusses RoSeMary but omits SrcMarker, an earlier AST-based structural and identifier watermarking system. Learned structural methods are not equivalent to CodeKAST's training-free keyed design; the question is which differences provide the incremental value and whether a direct comparison is applicable. Tables 4 and 9 remove only the two renaming families with almost no change, leaving the roles of parameter aliasing and return wrapping unexplained. These gaps limit the assessment of significance, rather than establishing that the method is ineffective or lacks novelty.

Questions for authors

1.Does implemented attribution apply any acceptance condition to the top candidate beyond uniqueness within the tie tolerance? Using existing artifacts, could you report unique-attribution and abstention rates on (a) the Base programs and (b) released watermarked programs with the embedding key removed from the candidate set? If attribution is intended only as closed-set ranking with the true key present, please clarify that assumption alongside the attribution claims.

2.How many programs on each main benchmark were transformed versus retained unchanged, and what were the reasons for fallback? What exactly are screening tests P and how do they relate to final EvalPlus tests? How are Base programs that already fail P handled, and how many fallback programs appear among the 67 MBPP+ abstentions? These details are needed to interpret the reported utility and coverage, not to imply leakage.

3.Please provide baseline hyperparameters, detector input/preprocessing, implementation versions, and scored-token counts, particularly for STONE (AUROC 0.5229/0.5110). This would help explain the near-chance results without presuming an implementation error.

4.Please explain the essential differences from prior transformation-based code watermarking such as SrcMarker, and whether a direct comparison is feasible or prevented by language, training, or applicability constraints. A precise account of those constraints is useful even where rerunning a trained method is impractical.

5.What recall is obtained when the threshold is fixed independently of the evaluated Base/watermarked score pairs? With only 378 or 164 negatives, empirical 1% FPR corresponds to at most 3 or 1 false positives, so uncertainty should remain explicit. A focused check using disjoint calibration or a justified random-key null would help; any analytical calibration should state its assumptions about site construction and key-induced dependence. The current in-sample result is openly disclosed and does not invalidate AUROC.

Reviewer_xbxEOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper focus on the problem of how to embed detectable and traceable digital watermarks into code generated by LLMs without compromising the program's functionality, which existing methods typically embed watermarks by adjusting the token distribution during the code generation process. However, the methods may suffer from the results may alter the code's syntax or execution outcomes and often requires access to the model's internal generation process. To tackle the problems, the paper introduces CodeKAST, a post-generation watermarking framework for code that requires no additional training, which leaverage AST to to identify code structures that can be safely modified and HMAC to select specific transformation states for these identified structures.

Critical strengths
  1. This paper addresses an interesting problem which embed watermarks directly into already-generated code without access to the model’s generation process or requiring additional model training.
  2. The experimental results shows a good performance as claimed on two datasets.
Critical weaknesses
  1. The major concern is the novelty of this paper lies in anchoring the key-controlled watermark state to the AST transformation structure, rather than in the development of a new method. The ablation study only tested AST Structural Watermarking with local reanme and loop rename, which is not enough for proving the novelty.
  2. The experiment is not good enough, including ablation study. For example, CodeKAST is a post-generation method, whereas KGW, SWEET, and STONE are primarily generation-time methods. These two categories of methods operate under different control conditions. Furthermore, the authors have not provided a sufficient comparison with more directly relevant post-generation code watermarking methods. Some of the typical works such as Efficient and Universal Watermarking for LLM-Generated Code Detection, SrcMarker: Dual-Channel Source Code Watermarking via Scalable Code Transformations are not compared.
Questions for authors
  1. The paper primarily compares its approach against generation-time watermarking methods such as KGW, Unigram, SWEET, and STONE. However, there are more relevant post-generation or AST-based code watermarking methods such as SrcMarker (IEEE S&P 2024), "Efficient and Universal Watermarking for LLM-Generated Code Detection" (IEEE TSE 2026).
  2. Only AST formatting and local-variable renaming were tested. More significant attacks such as code refactoring, the elimination of redundant structures, and automated semantic-preserving rewrites were not considered. Such attacks could directly undermine the structural evidence upon which CodeKAST relies.
Reviewer_byboOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper takes the potential harm to code functional correctness caused by token-level red-green word list type watermark algorithms as its starting point and designs the CodeKAST algorithm, which embeds watermark signals into the AST space. The algorithm first screens AST sites that can be safely rewritten and can be restored by a verifier, then uses HMAC to select the target structural states based on key and site information, and embeds watermark signals through corresponding code transformations. Candidate code is only released after passing syntax checks, RoundTrip, and execution tests; otherwise, the original code is returned. During the detection phase, the algorithm detects watermarks and attributes candidate keys by comparing the actually recovered structural states with the expected states corresponding to the key. In the experimental results presented in this article, CodeKAST achieved detection performance significantly better than the compared baselines on HumanEval+ and MBPP+.

Critical strengths

This paper reveals the shortcomings of traditional token-level red-green word list watermarks in the field of code generation, and proposes using HMAC encryption to encode watermark information by altering the writing at permissible modification points on the AST, resulting in significant improvements.

Critical weaknesses

The source of code functionality and its trade-off with effective watermarking hasn’t been fully explained. This paper embeds watermarks through constrained AST rewrites and HMAC-controlled state selection, but candidate code is only released after passing syntax checks, RoundTrip, and execution tests; otherwise, it returns to the original Base. So the question is: do candidate rewrites naturally rarely break functionality, or is part of the functionality preservation just because problematic candidates are rejected and the original program is kept?

Questions for authors

Is the watermark evidence for each program primarily concentrated at a few structural sites, and to what extent does this affect its robustness against local rewriting? The paper mentions that some programs reject attribution due to an insufficient number of valid sites. Could you provide, for each program in the main evaluation set, the number of valid sites n, the number of optional states for each site q_i, and the distribution of these sites across different transformation families? 、

Additionally, Table 3 shows that most of the attribution coverage is retained after local variable renaming; however, in Table 4, removing local_rename has almost no effect on detection and attribution results. This makes it difficult to determine which part of the algorithm contributes to the robustness of the watermarking method.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.