acceptodds
Withdrawn from ICLR 2027

Launder-Bench: Can Repository-Level Agentic Code Laundering Be Detected?

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Coding agents can now rewrite repository-scale codebases while preserving tested behavior, creating a new challenge for plagiarism detection, license auditing, and software provenance. We ask whether such rewrites can still be traced to their source. Existing clone benchmarks emphasize fragments and bounded edits, while repository-transformation benchmarks evaluate functional correctness rather than provenance. Moreover, ranking the most similar source does not establish derivation when independent implementations can be similarly close. We define repository-level agentic code laundering and introduce Launder-Bench, to our knowledge the first repository-level benchmark for provenance under agentic rewriting, constructed from 282 real-world repositories. A source-hidden reconstruction protocol decomposes source behavior, reconstructs modules and repository structure, and retains rewrites only after execution-based verification. We evaluate the same provenance problem in two complementary settings: Launder-Bench-Pair contains 1,128 pairwise instances asking whether two repositories share provenance, while Launder-Bench-Attr contains 229 source-attribution instances asking which candidate is the source, with abstention when evidence is insufficient. Measured same-domain independent repositories establish how much similarity can arise without shared provenance and provide detector-specific reference bands. Existing methods remain far from saturating Launder-Bench: the strongest static method reaches only 0.66 AUROC on pairwise detection, while repository encoders rank the true source first on up to 22% of attribution queries but earn calibrated attribution credit on only 1–2%. These results reveal a systematic gap between relative code similarity and source-specific provenance evidence, providing a reproducible target for provenance detection under agentic rewriting.

Primary area:
datasets and benchmarks
Keywords:
Code Provenance, Coding Agents, Code Clone Detection, Repository-Level Code Analysis, Software Engineering Benchmarks

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_L41GOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduces Launder-Bench, a benchmark for detecting code provenance after AI-assisted repository rewriting. The authors construct behavior-verified rewrites from 282 real-world repositories and evaluate existing detectors on pairwise provenance detection and source attribution. The results show that existing methods struggle to recover the original sources.

Critical strengths
  • Addresses a potentially relevant problem concerning software provenance under AI-assisted rewriting.

  • Provides a repository-level benchmark with execution-based verification and same-domain negative examples.

  • Distinguishes similarity-based retrieval from evidence-based source attribution.

Critical weaknesses
  • Fundamental concern about the problem formulation and identifiability. My main concern is whether provenance can be reliably inferred under the proposed source-hidden reconstruction setting. The agent reconstructs implementations from behavioral specifications without access to the original source code. This process may eliminate much of the implementation-specific information necessary for identifying provenance. More fundamentally, functional equivalence does not imply shared provenance: independently developed programs can implement identical behavior. Even with extensive execution-based testing, identifying the historical source may therefore be statistically difficult or fundamentally underdetermined. The paper does not sufficiently establish what source-specific information should survive reconstruction or under what assumptions reliable attribution is possible.

  • Limited novelty and significance of the empirical findings. Given the reconstruction procedure, it is unsurprising that conventional code-similarity detectors perform poorly. These methods primarily detect shared implementation features rather than historical derivation from behavioral specifications. Without establishing that sufficient provenance evidence remains recoverable, the observed failures are difficult to interpret as a meaningful limitation of existing methods rather than an expected consequence of the benchmark construction.

  • Limited evaluation of alternative detection approaches. The evaluation focuses on static clone detectors and pretrained code encoders. Stronger repository-level, execution-based, or LLM-based approaches are not evaluated. Although these methods would not resolve fundamental identifiability limitations, they could help determine whether the observed failures reflect the benchmark's intrinsic difficulty or limitations of the selected baselines.

Questions for authors
  • What source-specific information is expected to survive source-hidden reconstruction, and under what assumptions is provenance attribution identifiable?

  • How does the proposed task differ from attributing an independently reconstructed implementation to one of several functionally similar sources?

  • Can stronger semantic or execution-based baselines help establish whether provenance remains practically recoverable?

Reviewer_Bz8iOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Launder-Bench, a benchmark for evaluating whether existing code similarity and provenance detection methods can identify the origins of repositories rewritten by coding agents. The authors construct a source-hidden reconstruction pipeline that extracts behavioral specifications from original repositories, regenerates their implementations using coding agents, and verifies the resulting artifacts through execution-based tests.

The benchmark is constructed from 282 real-world repositories and includes 1,128 pairwise provenance detection instances and 229 source attribution queries. Experiments involving traditional clone detectors and pretrained code encoders show that existing methods struggle with the proposed tasks. The authors further investigate the gap between relative code similarity and source-specific provenance evidence.

Critical strengths

The paper investigates an important challenge arising from the increasing capabilities of coding agents. Understanding whether code provenance remains detectable after large-scale agentic rewriting is relevant to software auditing, plagiarism detection, and license compliance.

The distinction between pairwise provenance detection and source attribution is useful. Incorporating same-domain negative examples and evaluating whether the available evidence is sufficient for source attribution provides a meaningful extension beyond conventional code similarity benchmarks.

Critical weaknesses

The central claim of the paper concerns detecting code provenance after repository-level agentic rewriting while preserving functionality. However, the benchmark does not sufficiently establish that the reconstructed repositories retain a representative portion of the original functionality. The acceptance protocol allows an overall behavioral conformance score as low as 0.5, with a minimum of only five successful test cases. Furthermore, the reported median source-to-artifact size ratio is approximately 0.011, indicating that reconstructed implementations are often substantially smaller than their original repositories. Although code size alone does not determine functional coverage, this substantial reduction raises an important concern: are existing detectors failing because coding agents successfully conceal provenance while preserving functionality, or because much of the original implementation is no longer present? The authors should distinguish these two possibilities through stronger behavioral validation and systematic analysis of functional coverage. Without such evidence, the benchmark's central conclusions remain insufficiently supported.

Questions for authors

Given the permissive behavioral acceptance criteria and the significant reduction in generated code size, can the authors provide stronger evidence that the reconstructed repositories preserve a representative portion of the original functionality?

How would more recent code representation models, repository-level retrieval methods, and LLM-based provenance analysis perform on the proposed benchmark?

Reviewer_uFepOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Launder-Bench, a repository-level benchmark for detecting agentic code laundering, reflecting the realistic challenges brought by the increasing capability of LLM agents. Compared with existing fragment-level provenance detection, Launder-Bench focuses on whether detectors can still reliably detect code laundering at the repository level when an LLM agent rewrites the code with full repository context. Evaluation results show that existing detectors perform poorly on the two tasks of Launder-Bench.

Critical strengths
  1. The paper focuses on an increasingly important setting: with the increasing capability of LLM agents, code laundering may move from the file level to the repository level.
  2. The benchmark is built from real code repositories with a prevailing agent harness, providing a realistic simulation, and it evaluates several categories of detectors as baselines.
Critical weaknesses
  1. The threat model should be clarified. The paper does not clearly state which target scenario the benchmark aims to simulate, e.g., code imitation and copying, or even black-box reverse engineering. The current workflow may cover both, but the scope is never explicitly stated.
  2. Lack of discussion of partial laundering. In real-world code laundering, a common case is that a developer launders one core module of another project into their own project without notice. The current benchmark does not discuss this case.
  3. The generation process lacks an adequate quality guard. A valid laundered repository should at least pass the unit test suites and official examples of the original repository that correspond to the laundered modules; otherwise it is hard to attribute such laundering. The current pipeline only requires passing half of a small set of captured oracle cases.
  4. Lack of generator diversity. The benchmark mainly uses GPT-5.6-Sol as the LLM agent, with GLM-5.3 and DeepSeek-V4.1-Flash only on a 40-source subset. This is not adequate to test whether laundering detection difficulty depends on generator size and capability, which would be an important finding. Moreover, many repositories in the benchmark predate the release of the models used, so they may appear in the models' pre-training data and the source-hidden reconstruction may not be valid. The same concern applies to the detectors built on pre-trained encoders.
  5. Lack of an agentic detector. Since all detector baselines are essentially zero-shot, a simple but essential baseline is LLM-agent-as-a-judge, e.g., whether a Codex agent can identify the laundering.
  6. Minor reference problem: the URL given for "Source code authorship analysis for supporting the cybercrime investigation process" points to a different version of the paper with a different author list.
Questions for authors
  1. Can the authors provide dataset generation details such as average token usage, monetary cost, and the generation traces of the agents? These are important for future research.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.