- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper introduces Launder-Bench, a benchmark for detecting code provenance after AI-assisted repository rewriting. The authors construct behavior-verified rewrites from 282 real-world repositories and evaluate existing detectors on pairwise provenance detection and source attribution. The results show that existing methods struggle to recover the original sources.
Critical strengths
-
Addresses a potentially relevant problem concerning software provenance under AI-assisted rewriting.
-
Provides a repository-level benchmark with execution-based verification and same-domain negative examples.
-
Distinguishes similarity-based retrieval from evidence-based source attribution.
Critical weaknesses
-
Fundamental concern about the problem formulation and identifiability. My main concern is whether provenance can be reliably inferred under the proposed source-hidden reconstruction setting. The agent reconstructs implementations from behavioral specifications without access to the original source code. This process may eliminate much of the implementation-specific information necessary for identifying provenance. More fundamentally, functional equivalence does not imply shared provenance: independently developed programs can implement identical behavior. Even with extensive execution-based testing, identifying the historical source may therefore be statistically difficult or fundamentally underdetermined. The paper does not sufficiently establish what source-specific information should survive reconstruction or under what assumptions reliable attribution is possible.
-
Limited novelty and significance of the empirical findings. Given the reconstruction procedure, it is unsurprising that conventional code-similarity detectors perform poorly. These methods primarily detect shared implementation features rather than historical derivation from behavioral specifications. Without establishing that sufficient provenance evidence remains recoverable, the observed failures are difficult to interpret as a meaningful limitation of existing methods rather than an expected consequence of the benchmark construction.
-
Limited evaluation of alternative detection approaches. The evaluation focuses on static clone detectors and pretrained code encoders. Stronger repository-level, execution-based, or LLM-based approaches are not evaluated. Although these methods would not resolve fundamental identifiability limitations, they could help determine whether the observed failures reflect the benchmark's intrinsic difficulty or limitations of the selected baselines.
Questions for authors
-
What source-specific information is expected to survive source-hidden reconstruction, and under what assumptions is provenance attribution identifiable?
-
How does the proposed task differ from attributing an independently reconstructed implementation to one of several functionally similar sources?
-
Can stronger semantic or execution-based baselines help establish whether provenance remains practically recoverable?