acceptodds
Under review as a conference paper at ICLR 2027

When a Bad Procedure Outlives the LLM Agents That Wrote It

Abstract

Shared notes let language-model agents build on one another’s work, but they can also carry an invalid practice to agents who never saw where it came from. Understanding collective safety therefore requires separating the emergence of a violation from its propagation and its correction. We study these steps in teams of four LLM analysts performing executable data-analysis tasks under an explicit validity rule. Every published result is independently re-executed, and each team is replaced twice, so successors inherit only agent-written notes. An invalid proce- dure is planted privately with one founder, unexposed teams serve as controls, and paired branches from identical notes either replace them with verified guidance or offer an advisory validity review. Across ten model configurations, violations without exposure were rare in completed runs, and eight configurations adopted the planted procedure at its source. What happened next differed sharply. Among adopting configurations that completed their entire allocation, invalid publications after two complete turnovers ranged from 14% (DeepSeek) to 99% (Qwen), while both GPT-6 configurations never adopted the procedure at all. Replacing the notes with verified guidance produced valid work in all 3,952 complete-block compari- son slots. An accurate review did not guarantee correction: after a failed review, Gemini published a valid result 23 of 27 times, Qwen 0 of 239 times. Initial susceptibility alone does not characterize downstream persistence or correction.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
large language models, multi-agent systems, AI safety, shared memory, procedural inheritance, error propagation, agent handoffs, feedback uptake

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_9mbfOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper evaluates the safety setup where the aim is to separate emergence violations from propagation. The paper shows that most violations appeared with exposure. It works with frontier models. It uses a few steps: a founder (G0), a successor (G1), and another successor (G2). The idea is to check the impact of earlier models on successor ones.

Critical strengths

This is an important question for evaluating the impact of earlier models on successor models.

Critical weaknesses
  1. This paper is hard to follow, and there are missing definitions and examples to follow the setup the paper uses. In addition, the paper can benefit from more intuition and more explanation of how to interpret the results.
  2. The paper introduces a specific setup to evaluate the impact of earlier models on successor models; to validate the results, ablation of the configuration this work uses is needed.
  3. The number of blocks used across the paper is low (in many results less then a 100) reducing our comprehending the results. In addition I have not found what the questions look like or which dataset they are from.
Questions for authors
  1. Line 130: What are a cohort and a successor branch? These definitions are important throughout the paper, and when they are first introduced, there is no explicit definition.
  2. What is invalid advice (Figure 1)?
  3. Line 158: What is “turnover”?
  4. Section 3 can benefit from a running example.
  5. Table 2 is referred to before Table 1; please fix the order of the tables.
  6. Please add examples to Table 2. This table is not intuitive, and it is hard to understand what each setup means.
  7. Line 188: “PRESSURE adds performance-review language favoring higher“ - What is this? Is this text something the model gets somehow in the prompt? What does this text look like?
  8. What datasets are used in Section 4?
  9. Please add intuition on how to read each figure and the results behind it.
  10. Figure 2,3 can benefit from improving their visuality, they are hard to read in their current format.
Reviewer_z4gfOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

This paper tests the impact of upstream errors on downstream performance in a chain of agents.

Critical strengths

The paper studies an interesting and important question from a somewhat novel perspective.

Critical weaknesses

I am going to keep this review brief and mostly say that I don't know who this paper is written for — as I think it provides little insight to those not already familiar with the work.

The experimental procedure, which is the entire contribution, is full of, or somewhat odd and completely unexplained choices. The tasks, the cohorts, the experiments, are all barely described and not at all explained of judtified.

The results are presented as a wall of numbers with little or no interpretation, except for the underlined conclusions. To the best of my understanding, the findings are basically that the level of error propagation depends on the model as well as on the instructions it receives -- which I would say is quite unsurprising.

Questions for authors

-- Are you able to explain and justify your process better? Especially why this specific task?

-- Can you give a more nuanced and meaningful analysis of your findings?

Reviewer_26mZOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies how an error propagates across different LLM agents. The authors utilized a controllable environment to test error generation, propagation, and repair. The comprehensive experimental results demonstrate the important role of shared notes in error propagation.

Critical strengths
  1. The paper's experimental framework is well-designed.
  2. The experimental results show an interesting difference in error persistence between agent generations.
Critical weaknesses
  1. The idea of this paper is not novel, as there are already several studies in this area. For example, Mind Viruses (2026) studied error propagation with 20 hops, yet this paper only shows two hops. MemSecBench (2026) goes further and checks memory poisoning and repair. Thus, the research question and methods are not novel or contributive to this area.

  2. The experimental results are not enough to support the main claim, as there are only two synthetic tasks, which involve the same kind of error. Such a simple setting limits generalization.

  3. The baseline is not sufficient. For example, there isn't a single-agent baseline to investigate the mechanism that drives error propagation.

Questions for authors

See Weaknesses

Reviewer_n4MvOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how invalid procedures, (violations of the repo policies), can propagate across multiple rounds of coding agents working with each other. They call this phenomenon a "ghost swarm". The key finding is that the downstream persistence can vary a lot depending on different models, even though their source adoption rates are identical. While the experiment is rigorous, the current setup is kind of synthetic and tiny, limiting the generalizability of the findings.

Critical strengths
  1. The core insight addresses a novel gap, (even though minor,) in the current language model coding agent systems in terms of how mistakes propagate across rounds.
  2. The experiment design adopts an intervention design, which is rigorous, and it can enable causal inference. The existing experiment design is clean.
  3. Provide findings about how the propagation rate can vary and can be independent from the initial adoption rates
Critical weaknesses
  1. The current violation type is limited to excluding flagged observations and is narrow and synthetic. Other types of violations can behave differently, so the generalizability of the findings is unclear.
  2. The scale of the repo is also small, limited to rather simpler tasks instead of modern, more sophisticated tasks
Questions for authors

What specific mechanism explains GPT-6's complete non-adoption of the invalid procedure? Are there implicit checks that GPT-6 adopts to totally avoid violation adoption? Could the implicit reasoning chain (unshown to users) be an unfair factor when compared to other models?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.