acceptodds
Under review as a conference paper at ICLR 2027

Recovery Under Drift: When Failed Context Helps or Hurts

Abstract

Failed context can contain both misleading content and correct task information, so removing it changes two things at once. We study this trade-off with controlled interventions that vary correct facts inside a partly wrong record, correct facts supplied separately, failed-code visibility, the source of failures used for training, and conversation retention at fixed files. In tool-use and ALFWorld recovery states, adding correct facts to the retained record raises its benefit for Qwen3-8B and Gemma-4-12B-it. A complete current reference lowers retention benefit in the tool setting, whereas the ALFWorld intervals do not establish the same change. A secondary Qwen tool comparison finds that partial external facts can first increase the record's benefit before a complete reference lowers it. Code experiments show that measured recovery also depends on restart context and failure source: for one 7B model, omitting failed code and feedback raises success within 64 attempts from 22.1% to 55.5%, and matched-source training gains exceed mismatched gains across five model settings. On 34 MettleBench failures at identical repository files, retaining the pre-checkpoint conversation raises mean recovery from 11.8% to 35.8%. Recovery evaluations should state what each condition retains, compare repair with fresh starts, and match inference budgets.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, self-correction, self-repair, recovery training, LLM agents, code generation, context effects, test-time inference

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_u5XFOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper systematically investigates when retaining context from failed attempts helps or hurts LLM recovery across tool-use and coding benchmarks. The authors find that effect is highly nuanced: while hiding failed code often improves retry success and recovery training tends to overfit specific failure distributions, retaining conversation history after repository rollbacks boosts recovery from 11.8% to 35.8% on MettleBench. Overall, the work provides a controlled empirical characterization of how LLM recovery depends on retained information, failure provenance, and restart boundaries.

Critical strengths

None

Critical weaknesses
  1. The writing logic and organizational structure of this paper are very poor, making it difficult to follow.
  2. The current paper even has incomplete experimental results (for the ALFWorld benchmark), and does not meet the acceptance level of ICLR.
Questions for authors

See weeknesses.

3 comments

Thank you for your review.

ALFWorld results. We would like to clarify the statement that our ALFWorld experiments are incomplete.

In the original submission, we completed all nine experimental settings for both Qwen3-8B and Gemma-4-12B-it across 60 ALFWorld states, totaling 8,640 recovery runs. Appendix H.3 (Table 33) reports the success rates for every setting and the effects of keeping versus removing the earlier memory.

Adding one correct object location to the old memory increased its benefit for recovery:

  • Qwen3-8B: +9.2 percentage points [4.2, 14.8].
  • Gemma-4-12B-it: +7.6 percentage points [2.8, 11.8].

Both findings are statistically supported.

We also tested whether providing correct locations separately reduces the usefulness of the old memory. The estimates suggested a reduction, but the results were statistically inconclusive. We explicitly reported this limitation.

To further strengthen the evidence, we repeated the experiment on 60 new ALFWorld states across 39 additional floor plans. The positive effect of adding correct information to the old memory was reproduced for both models:

  • Qwen3-8B: +6.2 percentage points [2.8, 9.7].
  • Gemma-4-12B-it: +5.6 percentage points [1.0, 10.4].

The effect of supplying correct information separately remained inconclusive in the new experiment.

The original ALFWorld experiments were completed and reported in full. Some results were inconclusive, but no experimental settings or results were missing.

Organization. In response to all reviewer feedback, we have improved the paper's readability by revising the introduction, related work, contribution statements, and experimental design. The revised manuscript now more explicitly explains the purpose of each experiment, how the four studies connect, and how our findings extend prior work.

We would appreciate more specific feedback on the organizational concern, as the review does not identify which sections or arguments were difficult to follow.

Could the reviewer clarify which ALFWorld experiments or results they consider incomplete, which sections were difficult to follow, and whether there are specific methodological concerns underlying the soundness rating?

We would be happy to address these points directly.

Thank you for your consideration.

Thank you for your explanation; I stand by my score.

Dear Reviewer,

We hope you are doing well. Could we check for the final time on which part of the paper was hard to follow and what part of the alfworld results are missing? This would greatly help us in addressing those. Thank you!

Authors

Reviewer_wjacOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

When an agent retries after a failure, the previous context (failed code, old tool map, earlier conversation) contains both wrong information and correct information. So removing that context changes two things at the same time. This paper tries to separate this trade-off with controlled interventions. It does not propose a new retention rule; it is closer to a methodological argument that “evaluation should state what is retained”.

Critical strengths
  • The intervention design is careful. The dose grid keeps the misleading content fixed and only moves correct facts inside or outside the record, which separates “where the information is” cleanly. The MettleBench experiment that fixes files and only changes conversation also decomposes the AgentRewind intervention nicely.

  • It shows well why a fresh-start baseline matters. Recovery training gain looks large under the repair prompt but disappears or reverses against a fresh-start baseline. This is a useful methodological lesson for recovery research in general.

  • Controls and statistics are careful. Training tasks, targets, response tokens, and optimizer steps are matched, and there are prespecified contrasts, cluster bootstrap, label/presentation controls, feedback sanitization, and extreme-value bounds for missing outcomes. Negative or unclear results are reported as they are.

Critical weaknesses
  • The unifying contribution is weak. The four experiments are loosely connected, and the authors themselves say they do not propose a new rule. The remaining message is a reporting recommendation: “report what is retained, compare with fresh start, and match budgets”. Each finding is also mostly an extension of prior work. Verma (2026) already showed failed code hinders retries, Wu et al. (2026) and SCoRe showed correction trained on injected errors transfers poorly to real errors, and Olausson et al. (2024) compared repair with resampling. So the increment feels small.

  • The predictions of the information experiments are almost obvious. It is intuitive that putting correct facts inside a record makes it more useful, and giving the same facts separately makes the record redundant. The theory in Appendix F is also a basic result (“information that can be ignored does not make the optimal policy worse”) and does not give new insight. The more interesting non-monotone pattern (partial external information actually increases the record’s benefit) is a secondary contrast only seen in Qwen, and it is not explained well.

  • The experimental regime and scale are weak. Qwen has 0–7% success without reference (floor), and Gemma has 96–97% with full reference (ceiling), so many cells are in ranges where effects are hard to detect. Number of states is also small (80 with 21 episodes, and 60), and the external-fact effect in ALFWorld is not established for either model.

  • Failure states are artificially constructed. The stale map is injected by the designer, and ALFWorld failures are made by rule-based scripts. Gemma and Llama experiments also reuse failure histories collected from Qwen. It is unclear whether the same results hold on the natural failure distribution of real agents.

  • The headline code result depends on a special model. The representative “failed code hinders” result (22.1% → 55.5%) comes from Direct Recovery, a separate legacy model that seems overfitted to copy the failed code strongly (token-copy .95). For the Clean model the effect is much smaller and its direction depends on failure source; on controlled mutations, showing the code actually helps.

  • Training scale is very small. With 43 tasks, LoRA 60 steps, and about 5.6 epochs of data, it is hard to extend “recovery training specializes to failure source” to recovery training in general. Also AST mutations are small changes of the correct code, so they are close to the original answer, and specializing to that distribution is expected. It is unknown if the same holds for RL-based correction training (like SCoRe) or bigger data.

  • The MettleBench experiment is narrow. One hosted model, 34 tasks, 3 continuations per condition, and 12 continuations have no outcome. The native prefix condition is not length-matched and includes reasoning items kept by Codex, so we cannot tell if the gain is from factual information or from reasoning state. The checkpoint is also chosen by the model itself, and there is no comparison with the full AgentRewind system.

  • There is no practical conclusion. The paper does not give a signal for deciding what to retain at test time; there is only the negative result that entropy cannot be used. Practitioners do not get a guide they can use directly.

Questions for authors
  • What is the core claim that unifies the four experiments? What signal can a practitioner use at test time to decide whether to retain context?

  • Can you run the tool and ALFWorld experiments on failure states naturally made by agents, instead of designer-injected failures?

  • Why do Gemma and Llama experiments use failure histories collected from Qwen? Do results change with each model’s own failures?

  • How do you plan to handle the issue that Qwen is near the floor and Gemma near the ceiling? Does the dose effect hold in a middle-difficulty regime (for example, weaker or stronger models, or a different budget)?

  • What mechanism explains the non-monotone pattern where partial external information actually increases the record’s benefit (Qwen, e=2)?

  • Besides Direct Recovery, which is trained to copy too much, does “failed code hinders” appear with similar size on natural failures for the base or Clean model? Can you show quantitatively the relation between copy rate and performance drop?

  • Does failure-source specialization still hold with much more training data and steps, or with RL-based correction training?

6 comments

Thanks for a careful and fair review. We go through your points in order and answer each question.

Q1: what ties the four experiments together?

Whether it's worth keeping a failed attempt around isn't fixed. Across the tested settings, it can go from helping to hurting depending on what else the model is given, what kind of failure it is, what you compare against, and where you restart. Each experiment keeps the failure the same and changes one of these:

What we changeWhat stays the sameWhat happens
What else the model is givenSame stale tool map, same failuresKeeping the map helps when no current info is given (+7.7 pp) and hurts when it is (−6.6 pp). The same flip shows up on Llama.
Type of failureSame model; failed programs fixed within each failure typeShowing the failed code hurts the plain model on real model-made failures (−16.0 pp) but helps on edited programs (+9.8 pp).
What you compare againstSame recovery-trained modelOn BigCodeBench it beats the plain model by +35.9 pp when both repair, but is 6.6 pp worse than the plain model just starting over.
Where you restartSame restored filesHere keeping context helps: keeping the earlier conversation takes recovery from 11.8% to 35.8%.

So one overall "keep or drop" number can point the wrong way for a different setting. You're right that the end of our introduction undersold this. We'll rewrite the introduction around this point and add a column to Table 2 saying what each experiment changes.

On prior work, yes, each part has earlier work behind it, and we'll be more precise about how we differ:

  • Verma (2026) showed failed code can hurt retries. We find it depends on the model and the failure: the same plain model is hurt on real model-made failures but helped on edited programs, and Controlled Recovery gains +32.5 pp from seeing edited ones.
  • Wu et al. (2026) and SCoRe looked at related problems. Wu showed that training on planted errors doesn't transfer well to errors the model makes itself. SCoRe showed that training on corrections can fail when the model's own errors differ from its training data. We test both directions in code with everything else matched. Training on model-made failures also doesn't transfer well to edited ones (−10.9 pp at 7B). This pattern holds across five model settings.
  • Olausson et al. (2024) compared repairing with just starting over, across several models. We compare a model trained specifically for recovery against a task-only fine-tuned model that's allowed to start fresh, at the same budget. On MBPP+ at 7B, the +18.3 pp gain drops to +4.8 [−4.0, +13.5], and on BigCodeBench it flips to −6.6 pp [−14.1, −0.5]. We think this matters when judging whether recovery training actually helps.

W 2: The information predictions are obvious..

Fair point, the two main predictions are intuitive. We wrote them as hypotheses in Section 3 and will make that clearer. Three things are less obvious, though:

  1. The model doesn't notice its map has gone stale. The exact same map goes from helping to hurting when only the separate information changes.
  2. With a stale map that has no correct facts, the model gets more confident and less correct at the same time.
  3. Giving more correct information separately doesn't lower the map's value in a straight line, and the pattern differs by model (your Q5, below).

On Appendix F: agreed, it's basic decision theory. It's only there to separate "the information is there" from "the model actually uses it well". We'll call it motivation and remove anything that makes it sound like a theory contribution.

Q5: why does a partial reference make Qwen's map more useful?

The original per-state results had been lost, so we reran with the same settings and seeds. The rerun matched the original exactly, so these are the original results.

Before looking at the results, we grouped the 80 failures by how much the reference repeats the map's correct facts. One catch: whenever the reference repeats nothing, it also happens to give the name of the tool the model just failed on, and it never does when it repeats everything. So we decided in advance that the real test is inside the middle group.

Reference vs. mapFailuresHow much keeping the map helps
Gives exactly the 2 facts the map is missing13+38.5 pp [17.7, 57.8]
1 fact repeated50+11.5 [5.4, 19.6]
Repeats both of the map's correct facts17+2.9 [−4.7, 9.1]

Inside the middle group, getting the failed tool's name doesn't explain it (−8.9 [−22.7, +1.8], if anything the other way). And with no reference, there's no clear difference in how much the map helps across the three groups.

So the map and the partial reference fill each other's gaps. In those 13 failures, together they cover the task and the model succeeds 47% of the time, vs. 9% with the reference alone. When the reference just repeats the map, the map adds nothing. We'll add this as an exploratory analysis (80 failures, one model), not a proven mechanism, and still describe the pattern as Qwen-specific, since it isn't clear for Gemma.

Q4: floor and ceiling. The memory result isn't just a floor effect. For Gemma with two facts given, success goes from 30.3% without the map to 46.6% with it (+16.2 pp, Table 25). The "full reference makes the map hurt" result isn't just a ceiling effect either: Llama shows −8.3 pp at 38–46% success (Table 20).

You're right that Gemma is near 100% with the full reference at 30 steps, so we looked again at fewer steps. At 5 steps, full-reference success is only about 80%. Adding correct facts to the retained map still increases its benefit (+4.8 [2.1, 7.9]), but the separate-facts effect isn't clear (−3.8 [−8.7, 0.9]). At 10 and 20 steps it's clearly negative (−4.0 and −9.5 pp), but success is already 94–96% at those budgets, so only the 5-step result really gets away from the ceiling.

Size, and ALFWorld. We built a second ALFWorld set: 60 new failures, 10 per task type, on 39 floor plans that don't overlap the original ones. We used the same code and setup, and fixed the analysis plan before running. Both models ran it with no errors.

EffectOriginal 60New 60
Correct facts in the memory, Qwen+9.2 [4.2, 14.8]+6.2 [2.8, 9.7]
Correct facts in the memory, Gemma+7.6 [2.8, 11.8]+5.6 [1.0, 10.4]
Correct facts given separately, Qwen−10.2 [−19.8, +0.1]−5.7 [−15.7, +4.3]
Correct facts given separately, Gemma−3.8 [−16.5, +7.6]+1.0 [−8.4, +10.3]

The memory effect holds up for both models. The separate-facts effect doesn't: Qwen points the same way, and Gemma shows nothing. Put together with the original set, Qwen comes out at −8.0 [−15.2, −0.8], but since we ran the new set after seeing an unclear result, we don't count the combined number as a real test. So we'll keep that claim to the tool setting. The new games are a bit easier, so we report both sets separately.

Q2: real failures. Building the failures by hand is what lets us keep the wrong information fixed while changing only the correct facts, and we'll explain that better. To check whether it carries over, we collected 50 failures Gemma made by itself, with no planted map. With the current tool names given, keeping the earlier history clearly lowers success from 99.75% to 94.00% (−5.75 pp). Success is close to 100%, so we treat this as supporting evidence. However, we cannot clearly show that giving current tool names changes how much keeping the history helps or hurts overall. The code failures and the MettleBench failures in the paper are also real model failures. We'll keep the controlled and real-failure evidence clearly apart.

Q3: why use Qwen's failures for Gemma and Llama? Using the same failures for every model meant any difference came from the model, not from different failures. That only applies to the tool study; the ALFWorld failures come from the simulator, not from any model. To answer your question directly, we collected each model's own failures. On Gemma's 120, giving the current tool names drops the map's benefit from +21.3 to +2.8 pp, a clear −18.4 pp change. On Llama's 114, it goes from +2.5 to −0.2 pp, which isn't clear (−2.7 [−9.3, 3.5]).

Reviewer_ePwJOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies recovery by controlling what information survives a failed attempt. In fixed tool-use and ALFWorld states, it varies correct facts inside a partly misleading record and in a separate reference while preserving the misleading content. Further experiments examine failed-code visibility, training-failure source, and conversation retention at identical repository files. More correct retained facts increase average retention benefit in both environments; complete external references reduce that benefit in the tool setting, while the ALFWorld external-reference results remain inconclusive. The code and repository studies show how fresh-start baselines and restart boundaries change measured recovery gains.

Critical strengths
  1. The information interventions isolate more than a generic keep-versus-delete effect. Correct task facts are varied inside and outside a partly wrong record while holding the failed state and misleading content fixed. Across Qwen3-8B and Gemma-4-12B-it, the average retained-fact endpoint contrast is positive in both environments, while the external-reference contrast is established only for tools. The paper carefully separates interactions, effects within individual conditions, and inconclusive near-ceiling results. This makes the evidence for conditional retention value convincing within the tested settings.

  2. The recovery comparisons expose consequential evaluation choices. At 64 attempts on MBPP+, the 7B Natural Recovery advantage over a task-only model shrinks from 18.3 points under a shared repair prompt to an unresolved 4.8 points against a fresh-start baseline (Table 4). At identical MettleBench files, retaining the earlier conversation instead raises task-mean recovery from 11.8% to 35.8% (Table 28). These contrasting results make the evaluation lesson concrete. Paired inference, failure-source controls, and sensitivity bounds for missing continuations support the comparisons without claiming a universal retention rule or internal mechanism.

Critical weaknesses

I found no critical flaw in the stated comparisons. The main limitation is their scope: the information experiments use 80 constructed tool states from 21 episodes and 18 task templates, and 60 selected ALFWorld states. The tool replications share Qwen-collected histories, and the repository experiment uses one model on 34 eligible failures. These designs leave open how frequently the effects occur in naturally collected failures and how widely they transfer. The paper acknowledges these boundaries; they limit generalization rather than undermine the controlled findings.

Questions for authors

None.

1 comment

Thanks a lot for the positive review, and for summarizing the paper so accurately. Your main point was about scope, which wjac also raised, so we tried to test it directly.

For hand-built vs. real failures, we did two things. First, we collected each model's own tool failures instead of reusing Qwen's, while still inserting an outdated map: 120 for Gemma and 114 for Llama. For Gemma, giving the current tool names clearly reduces how much the old map helps (−18.4 pp [−23.8, −12.9]). The map still helps a little (+2.8 pp), so it shrinks rather than flips. For Llama the effect is small and not clear.

Second, we collected 50 failures that Gemma made by itself, with no stale map planted. When the current tool names are given, keeping the earlier history lowers success by 5.75 pp [−11.25, −1.50]. Success is close to 100% there, so we treat this as supporting evidence only. The −5.75 pp effect is supported, but the overall reference-by-history interaction is inconclusive at 30 steps. At 10 and 20 steps, that interaction is negative (−22.5 and −15.3 pp).

For ALFWorld, we built a second set of 60 states on new floor plans and ran the same setup. Adding correct facts to the memory helps again for both models: Qwen +6.2 pp [2.8, 9.7] and Gemma +5.6 [1.0, 10.4]. Giving the facts separately still doesn't show a clear effect there, so, as your summary says, we keep that claim to the tool setting.

The MettleBench result is one model on 34 failures, at checkpoints the model picked itself, and we'll keep the claim at that size. The +24.0 pp gain stays positive even under the worst-case filling-in of the 12 missing outcomes. We'll make all of these limits explicit in the paper.

Comments on the paper

1 comment

Dear Reviewer,

Could you please take a look at our rebuttal and clarification regarding the experiments? We believe a misunderstanding led you to believe our results are incomplete on ALFWorld.

The other two reviewers appreciated the paper, and one of them increased their score. We hope you will be able to review our rebuttal and the paper when you get some time.

We are more than happy to address any further concerns you may have.

Thank you!

--

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.