acceptodds
Under review as a conference paper at ICLR 2027

Origin Attribution for Unsafe Behavior In Self-Evolving Multi-Agent Coding Systems

Abstract

As language-model-based coding agents evolve their programs and reuse code from one another, safety failures can acquire a history that extends beyond the agent in which they are detected. An unsafe change may originate in one agent, propagate through intermediate recipients, and survive subsequent revisions before an audit reveals it. Understanding this history is important for identifying the introducing change and distinguishing inherited behavior from independently occurring failures. We study origin attribution in self-evolving multi-agent coding systems: recovering the agent and program version that introduced an observed unsafe behavior into the target's code lineage, or determining that it arose locally. This task is challenging because rewriting obscures textual similarity, independent occurrences confound temporal ordering, and intermediate recipients can resemble the source. We propose a method that combines behavioral fingerprint matching with historical localization and recursive tracing. It compares candidate implementations beyond the unsafe outcome, retains evidence from multiple historical versions, and follows matching candidates upstream. On 158 queries across 85 controlled scenarios, the method achieves 96.8% exact attribution accuracy, compared with 77.2% for exhaustive fingerprint matching without recursion. Component ablations and evaluation on 34 additional queries using the same method and hyperparameters demonstrate the complementary roles of multiple representatives and recursive tracing. Further checks extend the evaluation to approval-boundary and resource-owner errors. Together, these results show how behavioral and historical evidence can recover the origins of unsafe behavior through code revision and cross-agent propagation.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Agent Saftey, Self-evolving, Multi-Agent

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_kp6fOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies origin attribution in self-evolving multi-agent coding systems, aiming to identify the specific agent and program version that first introduced an unsafe behavior. The authors contribute a controlled benchmark with manually injected faults and propose an attribution framework that traces unsafe behaviors through historical localization, behavioral matching, and recursive tracing.

Critical strengths

Timely Problem and Practical Importance: Studying safety and accountability in self-evolving multi-agent systems is a novel and promising research direction. Tracing the origin of unsafe behaviors across complex, multi-agent code evolution histories is a critical problem for ensuring the reliability of autonomous coding systems.

Critical weaknesses

Limited Methodological Novelty and Weak Baselines: The proposed framework (combining historical localization, behavioral matching, and recursive tracing) is relatively straightforward and lacks significant technical innovation. The evaluation lacks strong baselines designed for agent fault attribution.

Unrealistic Fault Injection and Limited Agent Autonomy: Unsafe behaviors are introduced via manual, direct fault injection rather than emerging naturally from the agents' self-evolution. This intervention undermines "self-evolving" and fails to reflect true multi-agent autonomy.

Limited Diversity of Risks and Tasks: The benchmark only covers a narrow set of fault types: literal-string routing bypasses, boundary errors, and field-substitution faults. Because these faults have distinct syntactic or behavioral signatures, they may be easy to detect and trace. Evaluating against a broader and more realistic spectrum of vulnerabilities (e.g., more CWE categories) is necessary to demonstrate generalizability.

Questions for authors

Cross-Agent Code Adoption and Autonomy: Under what exact conditions do agents adopt code from one another? In realistic multi-agent systems, agents autonomously decide when and how to integrate peers' code. How does your benchmark and attribution method account for this autonomy?

Generalizability to Diverse Vulnerabilities: Have you considered evaluating your method on a broader set of security risks, such as CWE vulnerabilities?

Reviewer_LwLDOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes a method for tracing the origin of unsafe code in multi-agent software development systems, aiming to identify which agent and code version first introduced the unsafe behavior. Rather than relying solely on structural similarities or syntactic patterns in the code, the proposed approach incorporates semantic analysis to capture similarities in program behavior, even when the underlying code has been modified. Specifically, the method combines behavioral fingerprint matching, historical code-version analysis, and recursive tracing to systematically track the propagation and evolution of unsafe code across different agents and code revisions. By examining both the structural and behavioral characteristics of the code, the approach aims to distinguish the original source of unsafe behavior from subsequent agents that may have inherited, modified, or propagated it. The paper reports promising experimental results, demonstrating the effectiveness of the proposed method in accurately attributing unsafe behavior to its originating agent and code version.

Critical strengths

1. Important and Timely Problem: The paper addresses the important problem of tracing unsafe code across multiple agents, which is increasingly relevant for debugging, accountability, and code provenance in multi-agent systems.

2. Systematic Experimental Evaluation: The paper presents controlled experiments across multiple agents, code rewriting scenarios, and unsafe behaviors, supported by several baselines and ablation studies to validate the proposed approach.

Critical weaknesses

1. Ambiguity Between Code Reuse and Independent Implementation: My main concern is whether the method can reliably distinguish copied code from independently developed code with similar behavior. Behavioral matching on a fixed set of test cases, combined with timestamps and recursive tracing, does not necessarily establish code provenance. This is particularly problematic for small or simple functions, where independent implementations may exhibit identical behavior. While the paper includes experiments on independent implementations and code rewriting, it remains unclear whether these adequately address this ambiguity or generalize to larger, more complex codebases. The method may therefore identify an earlier behaviorally similar implementation rather than the actual source.

2. Limited Connection to Existing Software Engineering Research: The problem and approach overlap substantially with established research on code clone genealogy, cross-project code reuse, and software provenance. The paper does not clearly establish how tracing code across multiple agents differs fundamentally from tracing code clones across repositories. Moreover, the evaluation lacks comparisons with established clone detection, semantic clone analysis, and code genealogy techniques.

3. Limited Experimental Realism: The evaluation relies primarily on controlled code propagation scenarios, leaving open questions about the method's effectiveness in realistic multi-agent development workflows, particularly when independently developed implementations exhibit similar behavior.

4. Practical Assumptions About Provenance Availability: The approach appears to assume access to complete agent trajectories and code evolution histories, potentially including code adoption events. If such detailed records are available, it is unclear why provenance cannot be established more directly and reliably through explicit tracking rather than retrospective behavioral inference. The paper should clarify what information is actually available and why the proposed inference-based approach is necessary.

5. Unsafe Behavior. I am also not sure how the authors define unsafe behavior. In the experiments, it seems to be a specific bug or code change that is already known and can be detected using a test case. But in real-world agent systems, unsafe behavior may not be so easy to define or reproduce. The paper assumes that we already know what went wrong and have a test to detect it. I wonder how much the method depends on this assumption and whether it would still work for more complicated safety issues.

Questions for authors
  1. How can we tell whether one agent actually copied code from another, or whether they just happened to write code that behaves the same way? I wonder how the method would perform when two independently written functions produce exactly the same outputs on all the test cases. This is particularly true for small functionalities.

  2. There is a huge volume of literature on code clones, code genealogy, and code reuse across GitHub projects (see this as an example: https://ieeexplore.ieee.org/abstract/document/7962379/). I would like to understand what makes this problem different when the code is shared between agents. Also, why were existing techniques from this literature not included in the comparison?

Reviewer_V5TROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The author investigated the source attribution problem of unsafe behaviors in self-evolving multi-agent programming systems, determining which Agent's code version initially introduced the unsafe behavior. The author proposed a source attribution method that combines historical positioning, behavior fingerprint matching, and recursive tracking. First, the historical version of the Agent where the unsafe behavior first occurred was located, and the Top-K candidate Agents were selected based on code similarity. Subsequently, the most two representative historical versions were retained for each candidate Agent, and the candidate sources were further verified through code replacement and behavior fingerprint matching, and the possible sources were selected based on behavior consistency and time information. Finally, the original source after multiple code reuse and modification was identified through recursive tracking. In terms of experiments, the author built a controlled multi-Agent code evolution environment based on OpenEvolve, introduced unsafe modifications in a specified version, simulated the inheritance, rewriting, and propagation of unsafe behaviors among different Agents, and recorded the actual propagation relationships as attribution labels. The experiments included 85 scenarios and 158 attribution queries. The accuracy of precise attribution of this method reached 96.8%, which was superior to the 77.2% of the exhaustive behavior fingerprint matching baseline. Additionally, ablation experiments and 34 additional attribution queries further verified the complementary effect of multi-representative version selection and recursive tracking mechanism.

Critical strengths

The research topic is of practical significance and provides a new research perspective. It studied the source attribution problem of unsafe behaviors in the self-evolving multi-Agent programming system. Different from the traditional Agent error localization, this paper focuses on the original source of unsafe behaviors after code reuse, modification, and multi-hop propagation among different Agents. The author pointed out that the Agent that detects the unsafe behavior is not necessarily the one that initially introduced the behavior, and the code similarity and the time of the occurrence of the unsafe behavior are not sufficient to accurately determine the true source. This research topic has practical significance and provides a valuable research perspective for understanding and tracking the security issues in the multi-Agent code evolution process.

Critical weaknesses
  1. The experiment is mainly based on a controlled constructed multi-Agent environment, lacking sufficient validation from real complex scenarios. The core experiments of the paper are mainly based on a controlled constructed multi-Agent code evolution environment. By introducing unsafe modifications in the specified version, it simulates the inheritance, rewriting, and dissemination among different Agents. Although this approach can obtain clear real source labels, it differs from the natural unsafe behaviors occurring in real environments. There is a lack of sufficient validation of real complex scenarios.
  2. The method heavily relies on the completeness of historical code and candidate retrieval results for source attribution, and the robustness of source attribution remains limited. The author's method relies on historical code, behavior fingerprint matching, and time information for source attribution. The experiments in Section 7 of the paper show that when the historical records of the real source Agent are removed, 128 out of 135 queries are wrongly attributed to other Agents. Additionally, when the candidate number K is reduced from 3 to 2, the accuracy rate drops from 96.8% to 58.9%, indicating that the method is sensitive to the coverage of candidate retrieval. In actual multi-Agent systems, the absence of historical records or the failure to retrieve candidate sources may be difficult to avoid. Therefore, the current method is prone to incorrect attribution under incomplete historical information and still requires further evaluation of its robustness, and consideration of more reliable source uncertainty judgment or rejection attribution mechanisms.
  3. There is still room for improvement in the citation of references and the presentation of charts. The paper cites many pre-print studies that have not been officially published. It is recommended that the author further checks the literature basis for key arguments and supplement relevant published and peer-reviewed works when appropriate. Additionally, the layout and visual presentation of some charts are not standardized. For example, there is room for improvement in the alignment and format uniformity of some contents in Table 5. It is suggested that the author standardize the layout format of the charts to enhance the professionalism and readability of the paper.
Questions for authors

The paper constructed a dataset consisting of 85 scenarios and 158 attribution queries in a controlled multi-Agent code evolution environment, and used this as the basis for the main experimental verification. However, when checking the code repository provided by the paper, I did not find the corresponding complete experimental dataset. Could the authors please indicate whether they plan to make this dataset public, including the historical code versions of each Agent, the code dissemination records, and the actual source labels? If it has already been made public, could you provide the specific access path? Since this dataset is an important foundation for the experimental results of the paper, its accessibility will directly affect the independent replication and verification of the experiments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.