acceptodds
Withdrawn from ICLR 2027

From Tokens to Files: Multi-Granularity Detection and Localization of AI-Involved Code

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

The increasing use of large language models for code generation has created an urgent need for reliable methods to distinguish purely human-written code from *AI-involved* code — files in which any part was generated or substantially modified by a model, however small that part is. Our goal is not only to determine whether a program is purely human-written or has been generated or modified by AI, but also to localize the specific lines or blocks attributable to AI. Existing approaches typically compress a file into a single score, either from surface-level stylistic features or from a global code representation, which averages away the evidence when only a few blocks of a file are AI-involved. We propose a detection framework that models AI authorship at the token level and supports predictions at the line, block, and file levels. A frozen code language model is equipped with two low-rank adapters, trained separately on machine-written and human-written files. The per-token log-likelihood ratio between the two adapters provides local evidence of AI authorship. Lightweight prediction heads then aggregate this evidence within lines and blocks, using the surrounding code context to estimate the probability that each line or block was generated or substantially modified by AI. The resulting local predictions are further aggregated to produce a file-level decision. We train these predictions using mixed-authorship files constructed from human originals and their AI-generated or AI-modified counterparts, including continuation and fill-in-the-middle scenarios. On an internal test set of 11,773 AI-involved files across six generation regimes against a shared pool of 2,541 human files, our method reaches 72.4 macro F1 at a 1% FPR operating point, compared with 60.4 for a classifier that shares its backbone, training files, truncation, and optimisation schedule, 58.4 and 56.5 for fine-tuned UniXcoder and CodeBERT, and 48.5 for a style-preference baseline. The gain concentrates on the mixed-authorship regimes, where our method achieves 60.8 and 50.4, compared with 42.2 and 41.3 for the corresponding baseline. With no adaptation at all, the method reaches 62.5 line-level F1 on HybridCodeAuthorship, an external human–AI co-writing benchmark that ships no training split, above the 56 that the benchmark's own paper reports as its best line-level result. We further run an extra transfer evaluation on a human-written-only test set: across 19,445 files from repositories created before 2022, our false-positive rate stays between 0.55 and 0.81% when the false-positive rate threshold is transported at 1%. So the operating point remains valid after transferring from paired synthetic negatives to real human code.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI-generated code detection, mixed-authorship code, token-level likelihood ratio, fine-grained attribution and localization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_AFYEOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper describes experiments in detecting code that has some (even minimal) machine-involvement for generation. The authors do this at various granularities. They experiment with different datasets and have detailed results.

Critical strengths

Detailed experiments.

Critical weaknesses

Unfortunately, the authors do not explain well what is novel about their work (and this is not clear to me). They also do not motivate their method (again it is not clear which part is new). They also report too many numbers, but do not discuss what they mean and what we shoudl remember about this work. The list of contributions is quite complex, and I am lost what exactly is being claimed. The results are in the appendix as well as the related work, which makes it hard to follow the work (I also see this as an implicit page limit violation as critical results are in teh appendix, even though they are discussed in the main text, i.e., I see some main-text tables put beyond the page boundaries).

Detailed comments:

  • The list of contributions is somewhat unfocused -- the sentences are too long, which makes it hard to assess what exactly is being claimed. For example, here is how teh second contribution starts: "A mixed-authorship data construction protocol that synthesizes continuation and fill-in-the-middle edits with character-level ground truth and a controlled AI ratio ρ, which supplies within-file supervision for the line and block heads and test cells covering both tail and interior segment geometries." -- By the time I read line 2, I have forgotten what this was about. All three contributions are like that, and I am lost what exactly is being claimed. It is also unclear to me where novelty is, based on such unclear list list of contributions: is it a new task formulation, is it a new dataset, is it a new method? These over-specified descriptions make it extremely hard to tell.

  • Related Work is in the appendix rather than in the main text. I strongly advice against this. This is a critical part of the paper: without it, it is hard to position this work. Moreover, the related work for some research lines is outdated: very little work after 2024 (there are a couple of 2026 paprrs cited, by I find it insufficient).

  • 091-093: "The file-level label is binary and AI-involved: y = 1 if any part of the file was generated by a language model, and y = 0 if the file is entirely human-written." -- I have a problem with this terminology. First, "The file-level label is binary and AI-involved" at first made me think this "and" meant "both" rather than "vs"; consider rephrasing. Second, "AI-involved" is not the same as "generated by a language model". Can you make it clear what the task is: is it to detect "AI-involved" or "LLM-generated" code? This is not the same thing. You need to pick one term and stick to it, but which one you choose has implications.

  • There are quite a few hyper-parameters set for the experiments. The authors need to explain how their values were set and to motivate these values.

  • Section 3 is badly written: It tells me what is done, but it does not explain why it is done that way. There is no motivation why things are done in a certain way. There is also no explanation what is novel vs. what follows previous work.

  • There is too much use of passive voice, which makes it hard to tell sometimes what the authors did vs. what was done by somebody else.

  • 195: "Table 20 (Appendix A.8) summarizes the splits:" -- This is very important information. It cannot be in the appendix! (just like related work)

  • The text refers to table in the appendix, e.g., Tables 7, 18, 22, etc. I cannot read the paper's main text only to get the idea. I see this as an implicit violation of the page limit. I should be able to understand teh paper from the main text only, but critical information such as the results, and related work are in the appendix.

  • I am somewhat lost in the many experiments: there is so much detail on results, but not much discussion on what this means, why it matters, and what I should remember about this paper.

  • The conclusion is weak: it goes again into numbers etc. and doe snot tell us much about why this work matters and what we should remember about it. There are summary sentences like "The mechanisms are therefore complementary rather than competing.", but I find them too generic: having read the paper, I could not remembered anything about it. It reads like a list of experiments -- more like a documentation of what is done, without much effort to explain why this is done that way and what we learn from the results.

  • I would suggest adding some note on future work.

Minor comment:

  • The main contributions of this paper are: --> "The main contributions of this paper are: as follows:" (do not break a clause in the middle by ":")
Questions for authors

What is new about this work? Which part is new? Can you motivate your method? Can you motivate your hyper-parameters?

Reviewer_mXXwOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors focus on detecting AI-involved code at multiple granularities, which is better aligned with the real-world form of AI-involved development. Specifically, they propose a detection framework that uses a token-level likelihood ratio derived from trained LoRA adapters together with line- and block-level discriminative heads. Experiments on multiple frontier LLMs and programming languages show that the proposed method outperforms several existing zero-shot and fine-tuning-based methods.

Critical strengths
  1. The paper focuses on multi-granularity AI-involved code detection, which is more practical than the existing binary detection task definition.
  2. The authors build a dataset with the latest frontier LLMs, and the evaluation across granularities, models and languages demonstrates the effectiveness of the proposed method.
Critical weaknesses
  1. The human source of the training set is not detailed: whether the human code is clean and free of AI involvement, and whether the files were screened for private or sensitive data such as credentials, keys or personal information before training and release.
  2. It is unclear how the proposed method could be used in real-world scenarios; the authors should provide concrete examples in the paper.
  3. Current AI-empowered software engineering usually uses AI agents rather than one-shot API calls to generate code, which creates a gap between the proposed dataset and real-world AI coding-assistant usage.
  4. Lack of comparison with recent supervised and sentence-level detection methods, such as Profiler [1] and SeqXGPT [2].
  5. Lack of cross-language OOD experiments and adversarial attack experiments to show the robustness of the proposed method.

[1] Guo, Hanxi, et al. "Profiler: Black-box AI-generated Text Origin Detection via Context-aware Inference Pattern Analysis." Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

[2] Wang, Pengyu, et al. "SeqXGPT: Sentence-level AI-generated text detection." Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023.

Questions for authors
  1. Can code statistics such as AST depth or compilation rate separate the AI and human parts?
  2. Is there any quality guard applied during AI code generation, such as preventing direct repetition of the human code?
  3. Why are the LoRA adapters not trained with line- and block-level detection tasks, instead of relying solely on the discriminator heads for the fine-grained task?
Reviewer_FtAZOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces localizable code attribution to AI that predicts at the line, block, and file levels. This gives better visibility compared to file level detection. A frozen code language model is equipped with two low-rank adapters, trained separately on machine-written and human-written files. The per-token log-likelihood ratio between the two adapters provides local evidence of AI authorship. The best internal system reports 72.4% average macro F1 across six generation conditions at a test-calibrated 1% file-level FPR.

Critical strengths

1.The mixed-authorship setting is well motivated. Continuation, interior insertion, and local refactoring expose a limitation of whole-file classification. The paper usefully distinguishes the amount of AI code from the geometry and length of its segments.

2.Likelihood-ratio component has meaningful supporting evidence. On the internal benchmark, the token-ratio system reaches 70.7% average macro F1 against 60.4% for the same-backbone LoRA classification control and 63.8% for DroidDetect. Table 2 also shows a substantial loss when either side of the ratio is replaced by the base model. These results support the usefulness of the two-sided score on this distribution.

3.The experimental coverage is substantial. The paper already includes unseen generators, external transfer versus adaptation, feature-family ablations, alternative aggregators and heads, token-category controls, and other ablations.

Critical weaknesses

1.Headline external localization comparison may not be sound. The paper compares 62.5 "line-level" F1 with HybridCodeAuthorship's 56, but Appendix A.9 scores attribution runs rather than individual lines. The claimed improvement needs evaluation on identical lines and subsets.

2.Reliable localization of small edits remains unestablished. Refactoring line AUROC is 0.599, and its 48.0 F1 falls below an all-AI baseline of approximately 50.1 (Table 8). Human-only code also produces 17.33% line and 8.89-9.90% block false positives at the separate localization thresholds (Table 19). Low file-level false-positive rates therefore do not establish reliable highlights. Report localization recall at controlled local false-positive rates, especially for short, sparse edits.

3.Novelty needs stronger support. The method combines established components, while SeqXGPT already uses likelihood profiles and contextual sequence modeling for local AI-text detection. The same-backbone file classifier does not isolate the benefit over equally supervised local alternatives. A code-adapted SeqXGPT, a same-backbone token/line classifier trained on the same mixed files, and a base-profile-only head would better separate the contributions of the adapters, local supervision, and architecture.

Questions for authors
  1. Can you clarify the "line" term in the HCA results? Please provide the evaluation setup, counts of physical lines versus attribution runs, class prevalences, and the role of true authorship boundaries. Also, I'd like the authors to recompute physical-line results on identical examples for every method, with all-positive baselines and matched threshold selection. Please also replace the 62.5-versus-56 comparison with a comparison using the same generator/triviality subsets and metric aggregation.

  2. How does the method perform against equally supervised local alternatives? Please compare with a same-backbone token/line classifier using the same mixed files and context, and a code-adapted likelihood-profile sequence detector. A base-model-profile-only head would isolate the value of the two adapters.

  3. How useful is localization under a controlled false-alarm budget? Please report precision-recall curves or AUPRC, local recall at development-selected 1% and 5% local FPR, and per-file localization metrics. Include short/scattered segments and low AI fractions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.