acceptodds
Under review as a conference paper at ICLR 2027

AIDAR: AI-Generated Code Detection via AST Structures

Abstract

Large language models can generate convincing solutions to programming problems, making surface cues such as comments, naming, and style increasingly unreliable for identifying AI-generated code. Yet most existing approaches remain text-based and overlook the rich structural information in programs. We first introduce AST-TED, a non-neural baseline that compares complete abstract syntax trees (ASTs) using normalized tree edit distance. We then develop AIDAR by improving this framework in two ways: Discriminative Subtree Selection identifies informative local AST regions, while Prototype Distribution Matching replaces direct edit-distance comparison by matching distributions over shared structural prototypes for complete ASTs and selected subtrees. On CodeNet100 and a historical contest benchmark (C1), AIDAR achieves 95.05% and 99.00% accuracy, respectively, the highest among the evaluated methods. In a subsequent live contest (C2), we deployed AIDAR as part of a screening workflow that referred 17 of 1,176 full-score submissions for human review. The workflow's eight highest-ranked referrals matched the eight cases human experts flagged as suspected AI-generated code. These results demonstrate the value of AST structure for detection and for prioritizing human investigation. Code is available at https://anonymous.4open.science/r/AIDAR-3BCA for anonymous review.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
AI Code Detection, Abstract Syntax Tree, Graph Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_jb3JOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AIDAR, an AST-based method for detecting AI-generated code. The motivation is that existing methods mainly rely on textual features, such as comments and coding styles, which can be easily changed by LLMs. Instead, the paper focuses on the structure of code and uses subtree selection and prototype matching to distinguish human-written and AI-generated code.

In evaluation, the paper compares AIDAR with several existing detectors on code datasets and programming contest submissions. The proposed method generally outperforms the baselines. The paper also includes ablation studies to show the effectiveness of its components and a real-world case study to demonstrate its practical use in programming contests.

Critical strengths
  • The paper is well-written and easy to follow. The proposed method is clearly described.
  • The paper provides a comprehensive evaluation of the proposed method against several existing detectors, and the results show that the proposed method achieves state-of-the-art performance on the evaluated benchmarks.
  • The paper also provides a detailed analysis of the proposed method, including a series of ablation study.
Critical weaknesses
  • Design: Is AST alone sufficient for distinguishing human-written and AI-generated code?
  • Dataset: Are the benchmarks sufficiently challenging and representative?
  • Robustness: How does AIDAR perform under adversarial AST-level modifications?
  • Detection Granularity: At what code level is AIDAR expected to work?
Questions for authors

1. Design: Is AST alone sufficient for distinguishing human-written and AI-generated code?

The paper mainly relies on AST structures to distinguish human-written and AI-generated code. However, I am not fully convinced that AST alone can provide sufficiently strong and reliable signals for this task.

AST mainly captures the syntactic structure of a program, such as how functions, statements, and expressions are organized. There might be some structural differences between human-written and AI-generated code. For example, human programmers may write code in a more casual or compact way, while LLMs may tend to produce more structured code with additional functions, blocks, or intermediate steps. However, these differences are not necessarily inherent to AI-generated code. An LLM could easily be instructed to mimic a human's coding style, simplify the implementation, or follow the structure of an existing human-written solution.

On the other hand, many other signals could also be useful for detection, such as naming conventions, comments, indentation, type annotations, and inconsistencies in coding style. Although these features can also be manipulated, they may still provide complementary information that AST alone cannot capture. Completely discarding them may unnecessarily limit the detector.

I have two main questions:

  • Why is an AST-only design preferable to combining AST with textual or stylistic features? Is there any evidence that using AST alone is more robust, rather than simply discarding potentially useful signals?
  • Can the authors compare the current AST-only detector with a combined detector using both AST and source-code features? This would help justify whether relying solely on AST is a reasonable design choice.

2. Dataset: Are the benchmarks sufficiently challenging and representative?

The paper mainly evaluates the proposed method on CodeNet100, which is derived from the relatively old Project CodeNet dataset, and C1, which is constructed from a single programming contest problem. Although the authors introduce polished human code and human-like AI-generated code to increase diversity, I am not sure whether these benchmarks are sufficiently challenging or representative of current AI-generated code detection scenarios.

For example, as shown in Table 1, several existing methods already achieve very high accuracy, especially on C1. GPTSniffer achieves 97.67% accuracy on C1, while the proposed method achieves 99.00%. This raises the question of whether the dataset contains relatively obvious differences between human and AI code, and whether the improvement would remain on more challenging datasets.

There are also more recent benchmarkes[1][2], which aim to cover more diverse human-written and AI-generated code. Such benchmarks could provide a more convincing evaluation of the proposed method.

My main questions are:

  • How representative are CodeNet100 and C1 of real-world AI-generated code, especially given the rapid improvement of recent LLMs?
  • Can the authors evaluate AIDAR on some more challenging and diverse benchmark[1][2], and compare it against the existing detectors? It would be interesting to see whether AST-based detection still provides an advantage when the differences between human and AI code become less obvious.

3. Robustness: How does AIDAR perform under adversarial AST-level modifications?

The paper argues that AST structures are more reliable than surface-level textual features, since an LLM can easily change comments, variable names, or formatting. However, AST structures themselves can also be modified, especially when the attacker knows that the detector relies on AST similarity.

For example, an attacker may have access to some human-written solutions and ask an LLM to generate code following similar structures. The attacker could also compare the AST of the generated code with human-written references and iteratively ask the LLM to modify the implementation to reduce their structural distance. Such modifications may involve changing loop structures, reorganizing conditional statements, merging or splitting functions, or replacing expressions with structurally different but functionally equivalent implementations.

More importantly, this type of attack seems particularly relevant to AIDAR, since its detection is based on comparing a query against human and AI reference distributions. If an attacker can make the generated code structurally similar to the human reference pool, the detector may become less effective.

My main questions are:

  • How robust is AIDAR when AI-generated code is explicitly instructed to mimic the AST structures of human-written solutions, rather than merely mimic human coding style?
  • Can the authors evaluate an adaptive attack where the attacker iteratively modifies AI-generated code to reduce its AST distance to human references, while preserving the program's functionality? This would provide stronger evidence for the claimed robustness of structure-based detection.

4. Detection Granularity: At what code level is AIDAR expected to work?

The paper evaluates the detector mainly on complete solutions to programming problems. However, it is unclear what the intended granularity of AI-generated code detection is.

For example, in real-world software development, AI-generated code may appear as a single function, a class, a file, a module, or even a small modification within an existing codebase. The structural information available at these levels can be very different.

My intuition is that AST-based detection may depend heavily on the size and complexity of the input code. For a very short function, there may be only a few reasonable ways to implement the same logic, so human-written and AI-generated code could have almost identical ASTs. In contrast, larger programs may contain more structural patterns that could help distinguish their origins. However, larger codebases can also contain a mixture of human-written and AI-generated components, which introduces another challenge.

My main questions are:

  • What is the intended detection granularity of AIDAR? Is it designed for individual functions, classes, files, modules, or only complete programming solutions?
  • How does detection performance change with code size and structural complexity? For example, can the authors report results separately for short functions, medium-sized programs, and larger code samples?
  • Can AIDAR detect a small AI-generated function embedded within an otherwise human-written program, or does it require the entire input to be generated by a single source?

References

[1] Orel, Daniil, et al. "AICD Bench: A Challenging Benchmark for AI-Generated Code Detection." EACL 2026.

[2] Guo, Hanxi, et al. "Codemirage: A multi-lingual benchmark for detecting ai-generated and paraphrased source code from production-level llms." arXiv preprint 2025.

Reviewer_pcvTOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AIDAR, an AST-based framework for detecting AI-generated code. Unlike existing methods that primarily rely on textual features, AIDAR explicitly models program structures to distinguish human-written from AI-generated code. The framework consists of two main components: Discriminative Subtree Selection (DSS), which identifies informative local AST structures, and Prototype Distribution Matching (PDM), which compares programs through distributions over shared structural prototypes. Experiments on CodeNet100 and a historical programming contest dataset (C1) achieve 95.05% and 99.00% detection accuracy, respectively, outperforming the evaluated baselines.

Critical strengths
  • Detecting AI-generated code is increasingly important in programming education and competitions. The paper is particularly motivated by a real-world contest scenario, making the research problem meaningful and relevant.
  • Instead of relying on superficial textual patterns, the authors explicitly leverage AST structures for AI-generated code detection. The combination of discriminative subtree selection and prototype distribution matching provides an interesting approach to capturing structural differences between human-written and AI-generated programs.
  • AIDAR achieves strong detection accuracy on both CodeNet100 and C1, outperforming several existing methods. The ablation study supports the effectiveness of the proposed components, and the case studies provide useful insights into the structural patterns used for detection.
  • Applying the framework to a programming contest with 4,316 participants demonstrates its potential practical utility. The screening workflow successfully narrows 1,176 full-score submissions to 17 candidates for human review, with the eight highest-ranked cases matching those flagged by human experts as suspected AI-generated code.
Critical weaknesses
  • Although the authors construct datasets using multiple LLMs, the current experiments do not explicitly evaluate whether AIDAR can reliably detect code generated by previously unseen models. Since AIDAR relies on structural similarities between query programs and labeled reference programs, its high detection accuracy may partially result from learning generator-specific coding patterns rather than generalizable structural characteristics of AI-generated code. In particular, while the reference and query sets contain different code samples, this does not necessarily mean that the code generators are disjoint between the two sets. Consequently, the reported results may primarily reflect performance on known generators. I recommend adding a leave-one-generator-out evaluation, where all code generated by a particular LLM is excluded from both the training and reference sets and used exclusively for testing.
Questions for authors

See the "Weaknesses" part.

Reviewer_KwhiOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies AI-generated code detection using program structure rather than surface cues such as comments, naming and formatting. It introduces AST-TED, a non-neural baseline that classifies code using normalized tree edit distance to labeled references from the same programming problem. It then proposes AIDAR, which learns to select informative, node-disjoint AST subtrees and uses a GGNN to represent complete ASTs and selected subtrees as distributions over shared structural prototypes. Queries are classified by comparing these distributions with aggregated human and AI reference distributions. The authors evaluate the method on CodeNet100, containing 4,200 Python solutions across 100 problems, and C1, containing 900 solutions in multiple languages to one contest problem. Both benchmarks include polished human code and AI code generated to resemble human writing. AIDAR achieves 95.05% and 99.00% accuracy, respectively, outperforming the evaluated baselines. The paper also reports ablations, sensitivity analyses and illustrative case studies. In a live contest, an ensemble of AIDAR, AST-TED and DeTeCtive referred 17 of 1,176 full-score submissions for human review, with the eight highest-ranked referrals matching the eight cases experts flagged as suspected AI use.

Critical strengths

Strengths:

  1. Explicit AST structure is shown to be informative for detection. The parameter-free AST-TED baseline reaches 89.08% on CodeNet100, comparable to DroidDetect and above DeTeCtive, and AIDAR achieves the highest accuracy on both benchmarks. This matters to my assessment because it is the paper's most robust finding and does not depend on the specific DSS/PDM design, whose individual contributions are less clearly established.

  2. The work addresses a real operational need through a live contest deployment, where the screening workflow reduces 1,176 submissions to 17 referrals for human review. While this does not establish AIDAR's standalone accuracy, the base rate reasoning and the clear distinction between screening evidence and proof of misconduct show a sound understanding of how detectors are actually used.

  3. The benchmarks label samples by provenance rather than apparent style, including LLM-polished human code and human-like AI code. This makes the results more informative about realistic evasion than a clean human versus AI split, though robustness to deliberate structural rewriting remains untested.

Critical weaknesses
  1. The evaluation does not adequately distinguish transferable authorship detection from problem-specific structural similarity. Every query is evaluated using labeled references from the same programming problem, with no reported evaluation on held-out problems or generators. Although Appendix C states that CodeNet100 shares no source program between the reference and query sets, it does not clearly document grouping of originals with their polished derivatives or removal of structural near duplicates. Furthermore, human CodeNet solutions are accepted submissions, whereas generated solutions only pass parsing and visible tests, leaving correctness as a potential confound. These limitations matter because the reported accuracies do not establish whether the detector captures authorship signals that generalize beyond the evaluated reference setting.

  2. The experiments do not sufficiently isolate the proposed method's contributions, and some deployment claims exceed the evidence. The introduction describes eight C2 submissions as identified as AI-generated, whereas Section 5.5 describes them as suspected cases. These referrals come from a three detector ensemble, so the results do not isolate AIDAR's contribution, and selecting human references through agreement among the same detectors risks reinforcing their existing biases. Methodologically, the PDM ablation changes both representation learning and matching relative to non-neural TED, so the specific benefit of prototype distribution matching remains unresolved without a matched GGNN baseline. Adding DSS to PDM yields only 0.54 and 0.67 percentage point improvements (two predictions on C1), without repeated runs or uncertainty estimates. Together, these issues weaken the evidence for both AIDAR's standalone deployment effectiveness and the value of its distinctive components.

Questions for authors
  1. Are human originals and their polished derivatives always assigned to the same side of the split, and was any structural deduplication applied between reference and query sets?

  2. Were generated solutions verified beyond parsing and visible tests? If not, how do you rule out correctness as a confound, given that human CodeNet solutions are accepted submissions?

  3. How does AIDAR perform on held-out problems and held-out generator families, both with newly collected same-problem references and with no target-problem references?

  4. How were the 17 C2 referrals ranked, were experts blind to model scores, and were any unreferred submissions examined?

  5. The introduction says the eight C2 cases were identified as AI-generated, while Section 5.5 calls them suspected. Was any independent confirmation obtained?

  6. How do AIDAR, AST-TED and DeTeCtive each perform individually on C2, and how many referrals would the ensemble produce without AIDAR?

  7. Which AIDAR model selected the 430 human references in the preliminary C2 pass, and how do you address the risk that this selection reinforces the detectors' own biases?

  8. How does a matched GGNN with a standard classification head compare with PDM, so that the benefit of prototype distribution matching can be isolated?

  9. Are the DSS gains of 0.54 and 0.67 points stable across repeated runs?

  10. How robust is AIDAR to behavior-preserving structural rewrites such as loop conversion, ternary to if/else, or helper extraction?

  11. What is the true positive rate at low false positive rates, which matters more than balanced accuracy for screening?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.