acceptodds
Under review as a conference paper at ICLR 2027

When Source Correction Is Not Enough: A Theory of Mistake Influence in Agents

13 views

Abstract

Correcting an agent’s past mistake may leave its influence on later decisions intact. We distinguish a mistake’s presence in context, its repetition probability, and its causal influence, then separate this influence into the original output’s direct effect and a trail effect carried by subsequent history. A three-state construction shows that removing direct influence gives no uniform bound on recovery time, even when influence eventually vanishes. We derive sufficient conditions for fading and finite-horizon persistence, and identify an explicit fading-to-persistence threshold in a specified nonlinear feedback model. Exact enumeration of four-step histories under recorded binary-action policies spans 300 arithmetic problems and three models: Qwen3-Coder, Llama 4 Maverick, and DeepSeek V4 Flash. At decision four, trail influence exceeds direct influence in magnitude in 90.3–98.4% of settings, although its magnitude and direction vary substantially. On six development problems, supplementing source correction with recomputation and a visible repair record reduces mean mistake reuse by 32.7–53.4 percentage points across two menu orders. A separate coding-agent study finds that reminders can reduce subsequent violations while leaving files created by earlier violations intact, with responses differing across models.

Primary area:
learning theory
Keywords:
large language models, mistake persistence, error propagation, error correction, stochastic processes, martingale concentration, casual influence

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_WigFOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary
  • The authors study the effect of correcting mistakes an agent has made in the past and to what effect it has on future actions.
  • They describe a simple state-space based examples and show that even if the source becomes inaccessible (i.e. falls out of 'context' window, for example), it can leave traces in the subsequent states the agent visits such that there is no uniform recovery time of the influence of the mistake.
  • They also give conditions for fading/persistence of the effect and show with help of a simple non-linear feedback model how a parameter can control the threshold of where the influence persists or fades away.
  • They run experiments with a small maths dataset and with a subset of the SWE-benchmark to benchmark the behaviour of LLM models and show reduced mistake reuse by doing repairs that go beyond just source correction.
Critical strengths
  • Excellent presentation of simple ideas.
  • Elegant explanatory examples and an interesting experimental setups for evaluating their claims.
  • Interesting theoretical results which establish no uniform recovery time, bound differences between continuations sharing a common history, establish conditions for fading, and population persistence.
Critical weaknesses
  • The experimental design is neat, it removes the Monte Carlo error and makes W, SWAP and C exact for the recorded policy. The binary choices keep the tree enumerable. However, it is unclear how agent-like this is. The model only makes choices rather than reasoning; and "agents reusing their own mistakes" is a bit of a stretch given that the first token scores may not align with what the agent would've actually done. Maverick's menu sensitivity (Appendix I.3) of 0/100 and 100/100 also suggests that suggests position bias may dominate in these cells. Similarly, depth 4 is a very short to be able to talk about agent's long term behaviour and 6 problems (the repair experiments) may not offer sufficient diversity.
  • The SWE setup, while elegant, also don't address the "agents reusing their own mistakes" either because the actions are planted. The planted effect may be in-context instruction following?
Questions for authors
  • The notion of 'accessibility' is never formally defined; only Appendix B makes it concrete: A one-line definition, e.g., “the source is accessible at t if the rendered input under Q^1_t and Q^0_t differs for some k”, would help.

  • To be able to actually study "agents reusing their own mistakes" in the wild; I can only imagine an observational study done by means of data-mining a large repository of multi-step agentic traces. Given such a repository; can the authors describe how one may design the experiments to test their claims? This may be a future work.

  • A visual illustration of the Slacks mentioned in the results near line 394--397 would help; as would a tree schematic of W, SWAP and C, ideally annotated with the path probabilities with a real math problem overlayed on it. That would help in understanding D vs. M.

  • The details of the 'full repair' arm of the experiment are missing. It would help to have an example prompt showing what the repaired ledger and repair record look like. Having the full protocol as an dedicated appendix section would help.

  • The coding study doesn’t map onto D vs. M because is lacks the C and SWAP arms. Is it possible to design experiments which map directly on to the decomposition to make the experiments more coherent?

Minor Nit: the use of H to indicate a state was mildly confusing in Theorem 2.2

2 comments

We appreciate your interest in our theory. We've conducted additional experiments and clarifications for your concerns.

1. Accessibility.

We agree that we should have defined source accessibility earlier. In the main paper, we will define a source as accessible at time t if changing only the source changes the agent's rendered input for some fixed state k. We will also state the assumption that the source cannot affect the agent's decisions through anything outside the rendered input and K_t. We will rename the states H/L/N to s+/s−/s0 to avoid using H for both a state and the horizon.

2. Studying mistakes in existing traces.

We agree with the idea that a large study of naturally occurring mistakes would be useful. Such a study would be outside the time scope of this rebuttal session, so it would indeed be a future work. What could be done is for a given repository, we would identify mistakes using a fixed rule before examining later outcomes, restoring the checkpoints before these mistakes, and run W and C. For future checkpoints, we would construct SWAP by correcting only the source while keeping later transcript and workspace. Recorded traces alone wouldn't establish these comparisons without executable replay.

3. Slack plots and the W/SWAP/C tree.

We made the requested figures and will add them to the revised paper, since we cannot attach them here. For the tree, we used a real root-sum problem to show the probabilities of each path and mistake, alongside how we calculate D and M. For the slack plots, we show which assumptions held, which failed, and whether the bounds gave useful information.

4. Full-repair prompt and protocol.

We checked the exported prompts against the original requests and made sure that they match exactly. The full repair adds a record to the source-correction prompt with the corrected workspace, old and new values, and labels showcasing which results were corrected. Take for example the root-sum problem, its record changes the stored sum from -1 to 0. It then marks the quantity as invalid and then recomputes, while keeping the earlier result for reference and stating that it has been replaced. The executor computes these corrections, without adding any additional model decisions or executing future operations. We will include the exact prompt and full protocol within the appendix for the revised paper.

5. Coding comparisons for D and M.

For this experiment, we implemented W, C, and SWAP on five SWE-bench tasks with Qwen3-Coder, with two W/C pairs for each task and 480 next-turn probes. W retains the injected command, C corrects its destination to /tmp before execution, and SWAP edits only the source command while keeping W’s later transcript and workspace.

TurnD = W − SWAPM = SWAP − C
5−1.2 pp+8.7 pp
15−5.0 pp+11.3 pp

C had no observed violations. Effects were concentrated in two tasks, and task-bootstrap ranges included zero. This pilot used a different provider route. Further details are in our response to reviewer m6qA.

6. Menu choices and repair diversity.

Here we compare the original menu, reversed options, and free-form responses at 12 different Maverick checkpoints. At 5 checkpoints, where the original menu retained its mistaken convention, 38/38 parseable free-form responses also retained it. However, 1/12 of the checkpoints showcased a position effect. This supports agreement within the tested cases, though does not establish reasoning over many steps without being given fixed answer options.

We also expanded the repair study to 30 new DeepSeek problems. The original package reduced target error by 9.1 pp as opposed to the source-only correction (95% CI: 4.72–13.66). Workspace correction alone increased errors, while combining it with a value-free repair record gave no clear improvement. The full comparison is in our response to reviewer m6qA.

7. Testing a mistake made by the agent itself.

We also ran Qwen3-Coder on SWE-bench tasks without injecting a mistake. While working on astropy-13977, the agent created a backup inside the repository instead of /tmp. We saved the checkpoint and applied W/C/SWAP to the command the agent had generated. After changing the source command to /tmp, SWAP still used the backup inside the repository in 4/16 probes on turn 15, yet C didn't use them at all. So, by fixing the command, the agent did not stop using the file created by the earlier mistake.

Using this backup was allowed. We also checked whether the agent broke the rule again: at turn 5, the violation rates were 18.8% for W, 6.2% for SWAP, and 0% for C. We found no violations at turns 15 or 30. So, the agent was still using the file from its own mistake at turn 15, even when we no longer observed new violations.

We hope these additions address your concern.

Thank you for the detailed rebuttal and the additional experiments. I’ll keep my assessment, but a few points would benefit from clarification:

  1. Repair effect: I might be wrong, but the original paper doesn't explicitly call out which model was used; so I am at a loss as to whether a model change caused the drastic drop from 32-57pp (values in abstract) to 9pp? Is it because DeepSeek had any particular property, like high trail effects? The model choices across experiments could use some justification.

  2. Self-generated mistake: At turn 5, the violation rates imply D = 12.6pp and M = 6.2pp, meaning D > M? In the injected coding pilot, D is -ve (SWAP > W). Is that expected or can that be explained? Further:

The direct and trail effects went in opposite directions … in 0/720 Maverick settings with complete scores, 3/1,200 Qwen settings, and 236/1,200 DeepSeek settings. In these cases, the trail effect often outweighed the direct effect and reversed the overall direction.

Does “reversed the overall direction” mean that sign(Δ) ≠ sign(D)?

  1. Model behaviour: Do the authors have a hypothesis for why DeepSeek shows this pattern so much more often than the others? What might be causing the differences between models?
Reviewer_oUMeOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

The paper studies the following problem. Assume that an LLM is working on some task and at some point hallucinates. A natural idea is to try to "correct" this mistake. The problem however is that even if the mistake is corrected, the effect of the mistake on the LLM's context remains. The goal is to provide a theory for this. In doing so, the paper distinguishes three quantities. First one is what happens when we do not intervene (call it WW) Second is what happens if we correct later (call it SWAPSWAP). Third is if we correct at the time of mistake (call it CC)

Critical strengths
  • Topic seems nice. Like, the fact that even if we correct mistake its effects will "linger" is important, though not sure if it's particularly novel. Similar concepts appear in, e.g., unlearning where undoing the effect of false data is difficult.
Critical weaknesses
  • I don't think there's high value in a simple "this problem can occur" kind of result in ML unless it's surprising somehow. Like, "neural net training can fail to converge effectively" may be some paper with a complicated proof but zero value in it without teaching us something else. Not sure the paper presents interesting enough theory for me to be leaning towards acceptance. This may be due to presentation (see weakness 2) but I think the onus is on the authors to make the paper sufficiently understandable for a broad technical audience.
  • I cannot understand the paper. The "theory" section needs to be rewritten and explained much better. As is, I cannot understand a theorem given the explanations provided before I see the theorem. E.g., what is "k" in 2.2? What do you mean by "execution history"? Are you modeling things as an MDP? If not, what is "state" in theorem 2.2? If so, why not mention that it is an MDP? Overall, I think the paper needs HEAVY edits so it is clear what the results even are. Before that, i don't think it is ready for publication. Abstract should be re-written for clarity as well.
Questions for authors

(see weakness 2 above)

4 comments

We appreciate your review; however, we disagree with the assessment of the theory’s value.

This matters for agents (i.e. coding agents) that reuse earlier work after correcting the original mistake. Previous work has studied how errors spread and how to correct them, alongside highlighting their importance:

Jiang et al. [9] show benefits from correcting the source of errors; Feng et al. [5] show that incorrect information can spread through later reasoning even when the model is instructed to correct it; and Cheng et al. (2026) show that recognizing errors may not remove their influence.

Building on these works, we study what source correction can guarantee once a mistake has spread through the agent’s work. We separate what correction removes from what remains through later history, and show that even removing the source and knowing recovery eventually occurs gives no uniform bound on recovery time. We then give conditions for when fading or persistence occurs.

This work matters since correcting the source and letting the agent continue doesn't establish if decisions will stop being affected.

1. What the theory establishes.

Theorem 2.2 shows that, for any given number of steps, a mistake's influence can stay close to 100 pp throughout its steps, even if its source becomes inaccessible or if the influence eventually disappears. So, knowing that the influence eventually fades does not give a general bound on how long recovery takes after removing the source.

Theorem 2.3 bounds the remaining influence using differences in the agent’s later work when the original mistake is kept and when it is corrected immediately, accounting for how these differences change given time. Proposition 2.4 gives conditions for when the agent is likely to repeat the mistake and Theorem 2.5 gives conditions for when the mistake's influence persists.

Equation 2.13 also shows why checking each part of the agent’s state separately is insufficient for the fading guarantee. Even when differences shrink within each part, feedback between them can prevent the overall bound from shrinking.

This also means that recovery does not imply correctness. The agent can have the same error rate whether the original mistake was kept or corrected, but still make mistakes in both cases. So, we study whether the original mistake still affects later decisions, not whether the agent stops making errors entirely.

2. What the notation means.

In equation 2.2, k is a particular value of K_t: the agent’s state excluding the original source slot. This includes the ordered history, summaries, workspace, memory-access information, and execution status. The probability p_t^a(k) is the chance of a target mistake at the next decision with that state and source value a.

By execution history, we mean the full record up to that point, including the agent’s actions and tool results. We keep the agent’s policy fixed and study the resulting stochastic process. We do not assume that a shorter summary of this history is Markov. Theorem 2.2 uses a Markov process with three states for its counterexample.

We compare three cases: keeping the mistake, correcting it immediately, and correcting only the source while keeping the later work produced after the mistake. Comparing the last case with immediate correction shows how much influence remains after fixing the source.

3. Predictions on new problems.

We tested the fading bound on 30 new Maverick problems across 120 settings. We fixed predictions from the first three stages before scoring the remaining two. Of 74 settings with sufficient scores, 43 had final error rates within 5 pp between correcting only the source later and correcting it immediately. The bound identified 36 of these 43 settings, with no incorrect predictions.

The assumptions held in 71/74 settings, including 35 of the 36 settings predicted to have small influence. We checked these assumptions after scoring the remaining stages, so the theorem’s guarantees apply where they held. This result is limited to Maverick on new problems from the same families. The initial DeepSeek test did not produce any predictions of small influence.

We also tested whether going beyond source correction changes the agent’s error rate. On 30 new DeepSeek problems, the original repair package reduced target error by 9.10 pp compared with source-only correction (95% CI: 4.72–13.66). However, correcting only the workspace increased errors by 1.96 pp. So, changing more of the agent’s state does not necessarily improve its performance, which further motivates studying what each correction changes.

So, these experiments show that the theory can give useful predictions on new problems within the tested setting, beyond showing that influence can remain after correction. We hope this addresses your concerns and would appreciate your reconsideration of your scores.

We ask that the reviewer re-consider our work in light of our response, which clarifies the raised concerns, alongside clarifying what our contributions are, alongside its significance.

Thank you for your rebuttal. Regarding Theorem 2.2, this should be better clarified in the paper. More generally though, my assessment remains unchanged. I agree with Reviewer m6qA: the theory is fairly restrictive given the strong assumptions.
While I appreciate the paper's experiments, given that the paper's title is "A Theory of Mistake Influence in Agents", I think the theory does not meet the bar for ICLR. I also remain concerned about the writing. In particular, it was hard to read the paper because concepts and notation were used before they were introduced (see also comment of Reviewer WigF regarding accessibility). Again, if the goal is to provide an understanding of why an empirical phenomenon occurs, then this is very important.

We appreciate your reply. Once again, we respectfully disagree with your current assessment. While the theory may be restrictive, as many theoretical contributions within machine learning are (seen later within this rebuttal), we showcase its value through predictions on new problems and checks of how often the fading assumptions are held.

We tested 30 new Maverick problems and fixed predictions before scoring the later outcomes. Of 120 settings, 74 had sufficient scores for evaluation. The bound identified 36 of the 43 settings with remaining influence within 5 percentage points of zero, with no incorrect predictions. The fading assumptions held in 71/74 evaluable settings, including 35/36 predictions of small influence. We checked these assumptions after scoring, so the guarantees apply where they held. These results show that the conditions can hold and produce informative predictions on new problems within the tested families.

To test whether source correction also removes an earlier mistake’s effect on a coding agent, we ran controlled comparisons on five SWE-bench tasks. Correcting the original command while keeping the agent’s later work left it 8.7 pp and 11.3 pp more likely to create files or edit tests against its instructions than correcting the command before execution, after 5 and 15 turns. This tests the theory’s distinction between fixing the source and removing its later influence. The full set up and results can be in our response to reviewer m6qA.

These coding comparisons address reviewer m6qA’s concern by testing the theory’s distinction between source and history effects in agents working on code.

We agree that notation should be introduced before its use. Several definitions appear in the appendix, but we will introduce them earlier in the revised paper and add a concrete W/C/SWAP example before the theoretical results. These changes address the presentation concerns without changing the results.

ICLR has previously accepted theoretical contributions that establish useful guarantees under restrictive assumptions, on a similar level to ours. The following oral papers provide specific comparisons for both the restrictions in their setups and the types of theoretical results they establish.

(Transformers Provably Solve Parity Efficiently with Chain of Thought, Kim and Suzuki, 2025) studies a one-layer, single-head transformer on parity problems, with position-only attention and a fixed parity-computing component. This ICLR oral paper gives useful guarantees despite their restrictions on the task, model, and training setups. Our theory similarly gives guarantees under similarly restrictive assumptions, while allowing different architectures and tasks.

(When is Task Vector Provably Effective for Model Editing? A Generalization Analysis of Nonlinear Transformers, Li et al., 2025) also studies a single-layer, single-head transformer, with specific orthogonality requirements on its input patterns. They establish guarantees within a restricted setup and use experiments to test their relevance. Likewise, we state the conditions our guarantees require and test whether they hold, as shown earlier in this comment and in our other rebuttals.

The following papers were also ICLR orals and made similar types of theoretical contributions:

(Limits to Scalable Evaluation at the Frontier: LLM as Judge Won’t Beat Twice the Data, Dorner, Nastl, and Hardt, 2025) uses an existing statistical result (Cramér–Rao bound) to establish limits on what correcting judge bias can achieve. This is close to our contribution: we use established mathematical tools to show what source correction cannot guarantee and identify conditions that give stronger guarantees.

(Exploring the Loss Landscape of Regularized Neural Networks via Convex Duality, Kim, Mishkin, and Pilanci, 2025) builds on existing convex formulations to establish connectivity thresholds and counterexamples to uniqueness. The similarity is in giving conditions for different behavior, identifying thresholds between them, and constructing examples showing why stronger claims fail. Our paper does this for fading, persistence, and recovery after correction.

Therefore, we call upon the reviewer to once again re-consider the results of our paper, as we explain empirical phenomena which are insightful within the current agentic sphere.

Reviewer_m6qAOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper asks whether fixing an agent’s original mistake is enough to stop that mistake from affecting later decisions.

The authors distinguish between two effects. The first is the direct effect of the original mistake. The second is the effect of the later history that was created because of that mistake. They argue that fixing the original output may remove the first effect while leaving the second one behind.

The paper gives several theoretical examples and conditions under which this remaining influence disappears or persists. It also runs controlled experiments using binary arithmetic decisions from three language models, a small repair experiment on six math problems, and a coding-agent experiment on ten SWE-bench tasks. The results suggest that later history can sometimes matter more than the original mistake. However, the experiments are still far from showing that this causes persistent harm in realistic long-running agents.

Critical strengths

The paper makes a useful distinction between fixing the original mistake and removing everything that happened because of it. This is a clear way to explain why simply editing an old answer may not fully repair an agent’s state.

The authors are careful about the limits of their theory. They clearly state that the β = 2 threshold comes from a specially designed toy process and should not be interpreted as a real phase transition in language models. The small arithmetic model also makes it possible to inspect the proposed effects exactly.

Critical weaknesses

The theory is interesting, but it is mostly based on toy examples and fairly strong assumptions. The three-state example is constructed to show that recovery can be slow. The general fading result assumes that differences between two agent histories shrink over time, but the paper does not show that real LLM agents satisfy this assumption. The β = 2 result applies only to one specially designed feedback model. Thus, the theory explains what could happen under certain conditions, but it does not yet tell us when this happens in real agents.

The experiments do not provide strong evidence for the main claim. The largest study uses short binary decision trees rather than realistic long-horizon agent behavior, and one model requires replacing missing action scores in many settings. The repair study uses only six math problems and changes several things at once, so it is unclear which part of the repair actually helps. The coding study uses ten selected tasks and does not include all of the counterfactual conditions needed to measure the proposed direct and history effects. Finally, a larger history effect does not automatically mean harmful persistence: it can reduce the mistake rate, or cancel out the direct effect. Therefore, the result that the history effect is often larger than the direct effect should not be treated as proof that source correction is insufficient in practice.

Questions for authors
  1. Which assumptions of the theoretical results can actually be checked on real agent traces? Can the authors use the assumptions to make predictions on new tasks, rather than only explain results after the experiments are complete?

  2. Can the coding experiment include all of the counterfactual conditions from the theory: keeping the original mistake, correcting it before later use, and correcting it while keeping the later history fixed? This is needed to show whether the remaining files or conversation history actually cause later violations.

  3. Can the arithmetic results be rerun without replacing missing action scores for Llama 4 Maverick? It would also help to report separately when the history increases the mistake rate, decreases it, or cancels the direct effect.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.