acceptodds
Under review as a conference paper at ICLR 2027

VetEx: Beyond More Experience, Toward Filling Experience Gaps in LLM Agents

Abstract

Training-free experience reuse typically improves frozen LLM agents through two routes: extracting reusable corrective experience from failed and successful trajectories, or using successful trajectories, directly or after compaction, as few-shot references. However, both routes lack a principled criterion for deciding which experience should actually be admitted for reuse. Reorganizing the receiver’s own failure history alone cannot supply correct behaviors missing from that history; similarly, leaving a successful trajectory unprocessed can preserve misleading actions or unsupported state assumptions, while compaction can remove critical details needed by later decisions. We propose VetEx, which combines corrective experience construction targeting the receiver’s behavioral gaps with reference trajectory cleaning. For corrective experience, VetEx anchors same-task success–failure contrasts on receiver failures and admits donor success trajectories only for tasks the receiver has failed but never succeeded on, filling gaps in positive behavioral evidence. For task references, VetEx cleans retrieved successful trajectories, reducing redundant content while preserving task-relevant action–observation relations, and injects them together with abstract corrective experience into the frozen agent’s initial context. Across ALFWorld, HotpotQA, and WebShop with three frozen receiver models, VetEx outperforms ReAct in all nine settings and achieves a higher three-run mean than the strongest reproduced experience-learning baseline selected by single-run performance in each setting, by an average three-run mean margin of 5.12 points. Controlled analyses show that gains from experience extraction depend on whether newly added evidence fills behavioral gaps in the receiver’s history rather than on experience quantity alone, and that corrective experience and cleaned few-shot trajectories are complementary.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, LLM Agents, Agent Memory, Experience Learning, Trajectory Reuse

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_FS9zOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2(+1)
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes VetEx, a training-free experience-reuse framework for frozen LLM agents. VetEx constructs corrective experience by contrasting successful and failed trajectories on the same task, incorporating successful trajectories from donor models when the receiver has no observed success for that task. It also retrieves successful trajectories, removes redundant or unsupported actions, and validates the cleaned trajectories through environment replay. The resulting corrective experience and reference trajectories are injected into the receiver's initial context. Experiments demonstrate the effectiveness of VetEx.

Critical strengths

VetEx demonstrate consistent improvement over competitive experience-learning baselines. Across three benchmarks and three receiver models, VetEx achieves higher scores than the strongest selected external baseline in all nine settings. These results provide empirical support for the effectiveness of the overall framework across different models and task domains.

Critical weaknesses

W1. Overstated positioning and insufficient differentiation with prior work.

1-1. Oversimplified characterization of prior work.

The manuscript contrasts VetEx with existing methods that focus on "how available evidence is extracted and organized" to establish the gap in positive evidence coverage from experience organization. However, I think this framing oversimplifies some work. ExpeL already addresses this issue through Reflexion retries: it obtains successful trajectories for previously failed tasks and uses them to construct same-task success-failure contrasts. The main differences in corrective experience construction appear to be how missing successful trajectories are obtained (same-model retries versus donor successes) and how success-failure contrasts are organized into uniform construction units. Thus, the paper's positioning overstates its conceptual differentiation from prior work, making the paper's actual contribution difficult to identify.

1-2. Insufficient evidence for the claimed distinction.

The core question is: which approach is more effective at filling positive-evidence gaps on tasks without successful receiver trajectories, ExpeL's same-model reflection retries or VetEx's donor successes? Although ExpeL is included as a baseline, the comparison in Table 2 varies multiple components simultaneously, making it difficult to isolate the benefit of donor-based gap filling. I cannot find controlled comparison that answers this question.

W2. Limited and insufficiently justified benefits of the trajectory cleaning pipeline

2.1. Limited support for the claimed benefits of trajectory cleaning.

The paper motivates trajectory cleaning as an independent solution to the misleading or redundant behaviors contained in successful trajectories. In Table 6 (without corrective experience), however, 'Clean-only' performs slightly worse than 'Raw-only' on average (-0.31%p), improving only 4/9 settings. Notably, the largest performance change is -4.47%p on (ALFWorld, Qwen3.6-27B), and all three ALFWorld results show degradation. So, there is limited evidence for trajectory cleaning as an independent solution to the problem the authors argue it addresses ("gap in how it prepares task references"), with its benefits primarily with corrective experience added.

2.2. Limited gains even when combined with corrective experience.

In Table 6 (when combined with corrective experience), cleaning yields an average improvement of +2.73, but the gains are uneven. Some shows marginal improvements: +1 on (ALFWorld, Qwen3.6-27B) and +0.33 on (HotpotQA, Qwen3.6-27B). One shows regression: -0.49 on (ALFWorld, GLM-5.1). Notable gains are concentrated on WebShop: +4.71 on average. Thus, even in the joint setting where cleaning improves average performance, its contribution is often modest and varies considerably across benchmarks and receivers.

2.3. Unclear contribution of replay validation.

The paper presents replay validation as a key component of trajectory cleaning, which aims to remove misleading behaviors while preserving action-observation dependencies (lines 80-85). However, replay only verifies terminal success on the source task; it does not explicitly check whether the actions are supported by preceding observations or whether the cleaned trajectory provides a better reference for future tasks. Furthermore, the paper does not evaluate 'cleaning without replay' or report replay stats (e.g., reply pass rates and raw-fallback frequencies). Consequently, it remains unclear whether replay validation meaningfully filters harmful cleaning or contributes to the observed downstream gains.

W3. Limited practical applicability due to replay

VetEx validates cleaned trajectories by replaying the selected original actions in the source-task environment. This assumes that past task environments can be reproduced and their actions can be safely re-executed. This may be impractical in real-world agent deployments. Moreover, the cleaning prompt potentially requires irreversible actions to be repeated solely for validation. When replay is unavailable, the described raw fallback mechanism can reduce VetEx to VetEx-Exp+RT, eliminating the reported benefits of trajectory cleaning.

Notes

In Table 6, the "External baselines" header spans not only Ext. but also Raw-only and Clean-only. According to Section 4.4, Figure 6(b), and Table 16 that treat Raw/Clean Trace-only as the paper's own trace-preparation configurations, I interpret Raw-only and Clean-only as internal ablation conditions, and W2 is based on this interpretation. If my interpretation is correct, the header should be corrected to clearly distinguish external baselines from internal ablation conditions.

Questions for authors

Please see weaknesses.

10 comments

Thank you for your careful review and constructive questions. Your feedback has helped us identify and address several important issues, clarify our contributions, and improve the manuscript. We sincerely appreciate your time and thoughtful suggestions.We will address your concerns point by point below.

W1-1. Oversimplified characterization of prior work.

Thank you for pointing out the imprecise characterization of prior work. Your comment has helped us clarify both our positioning and our contribution.

We agree that ExpeL actively acquires successful trajectories through Reflexion retries and uses same-task success–failure comparisons to extract corrective experience. Our original description of existing methods as focusing on “only extracting and organizing available evidence” did not adequately acknowledge this active acquisition process. We have corrected this characterization in the Introduction and Related Work (Sections 1–2).

Our work investigates what kinds of experience are useful to a receiving agent, the mechanisms underlying their effectiveness, and the conditions under which they help. We address this question through methodological designs concerning experience sources, extraction, and use, together with controlled experiments examining their practical effects. Our contribution therefore encompasses both the design of VetEx and empirical findings about experience effectiveness, including the value of targeted evidence supplementation, the complementarity between corrective experience and concrete task references, and the dependence of their utility on task requirements and execution context. We build on prior work in success–failure comparison and do not claim to have originated this technique.

Regarding the specific issue of successful-evidence acquisition raised in your comment, VetEx and ExpeL share the use of same-task success–failure comparisons, but differ in how successful evidence is acquired and admitted. ExpeL uses the receiver’s own Reflexion retries to seek successful trajectories. VetEx supplements the receiver’s history with same-task donor successes through a gap-gated admission policy: when a task has an observed receiver success, only receiver successes are used to construct Self pairs; a donor success is admitted to construct a Peer pair only when the receiver has a failure but no observed success on that task in the available history. This makes targeted supplementation of missing successful evidence one concrete mechanism through which we investigate experience effectiveness.

We evaluate successful-evidence acquisition and coverage in Section 4.3 and Appendix C.5 (Tables 12–13), and examine targeted admission through the matched-count random-admission comparison in Section 4.4 and Appendix C.7 (Table 16). These experiments distinguish the value of obtaining external successful evidence from the value of selecting it according to the receiver’s observed evidence gaps. We discuss the acquisition and coverage results in detail in our response to W1-2 below.

W1-2. Insufficient evidence for the claimed distinction.

Thanks for your constructive comment.

We agree that overall comparisons that change multiple components simultaneously cannot adequately establish the specific benefits of filling evidence gaps with external sources. To address this concern, we conducted strictly controlled experiments across three benchmarks and three receiver models, together with an analysis of successful-evidence coverage, to examine both the downstream utility and the complementarity of external evidence. These experiments and analyses have been incorporated into Section 4.3, Figure 5, and Appendix C.5, Tables 12–13 of the revised manuscript.

First, we compare different sources of successful evidence while holding the experience-construction pipeline and downstream configuration fixed. We compare two ways of filling gaps in successful evidence:

  • Retry (ExpeL’s native mechanism): Uses ExpeL’s native Reflexion retry mechanism to fill gaps in successful evidence, then generates corrective experience using the same experience-construction pipeline as VetEx.
  • VetEx-E: Uses successful trajectories from a prespecified donor to fill gaps in successful evidence and construct corrective experience.

Both arms use exactly the same experience-construction pipeline, experience-extraction model, and downstream evaluation configuration. Both adopt the experience-only setting: only the constructed experience is injected, without task-reference trajectories, downstream Reflexion retries, or online recovery.

Each benchmark–receiver combination was evaluated over three runs. The mean scores are reported below, with differences calculated as VetEx-E minus Retry. Table 12 in Appendix C.5 additionally reports the sample standard deviations.

Benchmark / ReceiverRetryVetEx-EDifference
ALFWorld / DeepSeek-V3.272.3975.87+3.48
ALFWorld / Qwen87.0691.30+4.24
ALFWorld / GLM91.5490.80−0.74
WebShop / DeepSeek-V3.252.6053.66+1.06
WebShop / Qwen58.2859.54+1.26
WebShop / GLM56.5460.45+3.91
HotpotQA / DeepSeek-V3.244.0047.67+3.67
HotpotQA / Qwen43.3345.00+1.67
HotpotQA / GLM49.3352.33+3.00

VetEx-E achieves higher mean downstream scores in eight of the nine combinations. This indicates that, under the same experience-construction pipeline and an experience-only evaluation setting, corrective experience constructed from external successful evidence provides greater downstream utility in most settings.

To further investigate whether external sources supply successful evidence that the receiver has not obtained through its own attempts, we measured each method’s coverage of the receiver’s initial gap-task set, deduplicated by task ID. The following results are also reported in Appendix C.5, Table 13:

Benchmark / ReceiverInitial gap tasksDonor coverageExpeL coverageDonor-only coverageExpeL-only coverage
ALFWorld / DeepSeek-V3.24129 (70.7%)22 (53.7%)92
ALFWorld / Qwen1411 (78.6%)9 (64.3%)42
ALFWorld / GLM117 (63.6%)5 (45.5%)31
WebShop / DeepSeek-V3.25914 (23.7%)10 (16.9%)62
WebShop / Qwen578 (14.0%)6 (10.5%)42
WebShop / GLM5910 (16.9%)10 (16.9%)33
HotpotQA / DeepSeek-V3.24917 (34.7%)10 (20.4%)92
HotpotQA / Qwen5214 (26.9%)12 (23.1%)86
HotpotQA / GLM4410 (22.7%)6 (13.6%)62

External sources provide additional positive evidence in all nine combinations. The donor covers more gap tasks than ExpeL in eight combinations and the same number in one. Together, these two supplementary analyses clarify the specific value of external evidence: with the subsequent experience-extraction pipeline held fixed, supplementing trajectories through an external donor yields broader successful-evidence coverage overall and greater downstream utility.

We also observe complementarity between the two approaches. This suggests a valuable direction for further improving our algorithm: combining autonomous reflection with external assistance to achieve more comprehensive experience supplementation at a larger scale. The donor-only and ExpeL-only coverage columns in Table 13 document this complementarity; combining the two acquisition routes remains a direction for future work.

Reviewer_YizGOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes VetEx, an experience reuse framework for LLM agents that requires no model parameter updates. The core idea is that the value of experience lies not in its quantity, but in its ability to fill the gaps in the current agent's evidence for successful behaviors. The method distills error-correction rules by contrasting successful and failed trajectories on the same task, and incorporates successful experiences from other models only when the agent lacks its own success records. Meanwhile, it cleanses the retrieved successful trajectories by preserving key dependencies and validates them through environment replay, before injecting both the rules and the refined demonstrations into the context. Across nine configurations spanning three benchmarks and three receiver models, VetEx consistently outperforms the compared external baselines. Ablation studies indicate that simply increasing the amount of experience or cleansing trajectories in isolation is not necessarily effective; rather, combining error-correction rules with cleansed demonstrations yields more stable gains. Overall, the paper provides a clear and implementable approach to experience selection and reuse for agents. However, its experience gap identification remains relatively coarse, and further validation is needed regarding resource fairness, long-term online updates, and generalization in real-world scenarios.

Critical strengths

S1. The problem is clearly defined and practically meaningful. The proposed VetEX provides a clear direction for how agents should select experience. It shifts the focus from "gathering more experience" to "filling in the missing success evidence for the receiving model", and points out that simply reflecting on failures or blindly adding successful trajectories is not enough.

S2. The proposed method is well-designed and practical.It first uses the agent's own success-failure pairs. Experience from other models is only used when no success records exist. After cleaning, trajectories are checked by replaying them in the environment. If a replay fails, the trajectory is discarded. The whole process has clear rules and checking steps.

S3. The experiments cover multiple tasks and receiving models, and the ablation studies are comprehensive.The method is tested on three benchmarks and three models. The authors separately checked the effects of the experience source, including the gating mechanism, how the experience is organized, trajectory cleaning, and the compilation model. This shows exactly where the improvements come from.

Critical weaknesses

W1. The novelty of the proposed method mainly comes from combining and improving existing techniques. There are previous research works covering success-failure comparison, experience distillation, demonstration retrieval, and trajectory compression. The most notable new contribution of the paper is the gap-gating mechanism and its integration with the verification process. However, the algorithmic breakthrough itself is relatively limited.

W2. The definition of "experience gap" is coarse. Using a binary criterion based on whether success occurred within a limited number of attempts makes it difficult to distinguish between genuine capability deficits, accidental failures, and lucky successes. Furthermore, task-level gating cannot precisely identify which specific behavioral capabilities are missing.

W3. The evidence for generalization and long-term effectiveness is limited. The evaluation involves a relatively small scale of tasks and only examines offline reuse with a fixed experience library. Since successful replay on source tasks does not guarantee that demonstrations will be beneficial for new tasks, it remains unclear how the method would perform in complex real-world environments or continual learning scenarios.

Questions for authors

How does the success–failure comparison within the same task prevent the extraction of incorrect rules? The two trajectories may differ across multiple decision points simultaneously, and behaviors present in the successful trajectory are not necessarily the true causes of success.

6 comments

We sincerely thank you for your thoughtful review and constructive feedback. We particularly appreciate your recognition of the practical importance of selecting experience to fill the receiver’s gaps in successful evidence, as well as your positive assessment of our method’s design and comprehensive ablation studies. We are encouraged that you found the paper helpful in deepening your understanding of experience selection and reuse. Your concerns about novelty, the precision of gap identification, and generalization help us clarify both our contributions and their boundaries. Your question about distinguishing useful corrective rules from incidental differences between successful and failed trajectories also highlights an important issue that deserves careful discussion. We will address each point in turn and look forward to further discussion with you. If our responses satisfactorily address your concerns, we would be grateful if you would consider raising your score. Thank you sincerely for your time and consideration.

W1: The novelty of the proposed method mainly comes from combining and improving existing techniques. There are previous research works covering success-failure comparison, experience distillation, demonstration retrieval, and trajectory compression. The most notable new contribution of the paper is the gap-gating mechanism and its integration with the verification process. However, the algorithmic breakthrough itself is relatively limited.

Thanks for your constructive comment. As you point out, success–failure comparison, experience distillation, demonstration retrieval, and trajectory compression all have foundations in prior research. Our methodological contribution centers on a fundamental question: What kinds of experience are effective for an agent?

Each component of our method addresses this shared question. Same-task success–failure comparison starts from the agent’s own failures and extracts corrective experience from successful behavior on the same task, helping consolidate capabilities that the agent already possesses but cannot reliably exercise. For tasks the agent has never successfully completed in its historical attempts, coverage gating selectively introduces external successful trajectories, supplying positive behavioral evidence missing from its own history. At the experience-use stage, trajectory retrieval provides concrete behavioral demonstrations relevant to the current task. Trajectory cleaning removes redundant or observation-unsupported behavior, while replay in the native environment checks whether cleaning has disrupted execution dependencies required to complete the source task. Together, these components translate the question of experience effectiveness into experimentally testable designs concerning experience sources, extraction, and use.

Building on this problem formulation and methodological exploration, we identify several important findings about experience effectiveness. First, abstract corrective experience and cleaned concrete trajectories are complementary: the former provides guidance and constraints that generalize across tasks, while the latter supplies behavioral demonstrations that help the agent put this guidance into practice. In our experiments, trajectory cleaning alone does not yield consistent gains, whereas its benefits are more consistent when combined with corrective experience. Second, expanding the amount of experience does not necessarily improve its utility. Adding experience or expanding the successful-trajectory bank may yield no gains and can even degrade performance. In controlled WebShop experiments that match the number of admitted trajectories, targeted supplementation of the receiver’s positive-evidence gaps outperforms random admission of external successful trajectories across all three receivers. These results indicate that experience quantity alone cannot explain the gains; we must also examine what the additional evidence contributes.

Finally, the specific utility of experience also depends on task requirements and the agent’s current execution state. In ALFWorld and WebShop, the action preconditions and environment feedback recorded in trajectories can provide reusable operational guidance, and native replay can identify execution dependencies disrupted by cleaning. In HotpotQA, however, whether to continue searching, verify evidence, or submit an answer depends more strongly on the current question and available evidence. A strategy conveyed by the same successful trajectory can therefore have opposite effects in different contexts. These findings clarify the complementarity between abstract experience and concrete trajectories, the distinction between experience quantity and practical utility, and the conditions under which experience is useful across different task requirements. We hope the reviewer will assess our contribution through this complete line of inquiry: formulating concrete questions about experience effectiveness, investigating them through corresponding methodological designs, and deriving empirical insights that can inform future experience-learning methods.

We therefore hope to characterize our contribution more precisely as formulating concrete questions about experience effectiveness, developing corresponding evidence-selection and trajectory-validation mechanisms, and systematically examining how experience quantity, representation, and task requirements affect its practical utility. Together, these methods and findings constitute our contribution beyond the combination of existing components.

W2. The definition of "experience gap" is coarse. Using a binary criterion based on whether success occurred within a limited number of attempts makes it difficult to distinguish between genuine capability deficits, accidental failures, and lucky successes. Furthermore, task-level gating cannot precisely identify which specific behavioral capabilities are missing.

Thanks for your constructive comment. We have incorporated both the historical-success validation and the matched-count random-admission experiment into Section 4.4 and Appendix C.7 of the revised manuscript (Tables 17 and 16, respectively).

We acknowledge that the original manuscript did not directly validate the use of “no observed success” as an indicator of a capability gap. To assess whether “no observed success” is a reliable indicator of an observed capability gap, we compared historical success frequency with success rates on independent attempts. For each benchmark–receiver setting, we used three ReAct runs as historical observations and another three as independent validation. Tasks were grouped by their number of historical successes (0, 1, 2, or 3), and we measured each group’s success rate over the three validation runs on the same tasks.

The table below reports the validation success rate for each historical-success group across all nine benchmark–receiver settings.

Benchmark–receiver setting0/3 historical successes1/3 historical successes2/3 historical successes3/3 historical successes
DeepSeek–ALFWorld8.1%43.9%65.3%91.8%
Qwen–ALFWorld4.8%29.6%87.0%98.9%
GLM–ALFWorld9.1%29.6%50.0%98.8%
DeepSeek–HotpotQA6.8%24.2%46.2%86.4%
Qwen–HotpotQA3.8%6.7%60.0%86.9%
GLM–HotpotQA2.3%30.8%93.3%95.6%
DeepSeek–WebShop0.6%30.6%77.1%100.0%
Qwen–WebShop1.8%11.1%80.0%97.2%
GLM–WebShop2.8%9.5%66.7%92.2%

Of the tasks with no historical success, 91.5% failed all three independent validation attempts. Their aggregate validation success rate was 3.7%, compared with 27.66% for tasks with one historical success. These results support using the absence of success across three historical attempts to identify positive-evidence gaps.

To assess the role of gated admission in filling gaps in successful evidence, we conducted a controlled experiment on WebShop, a representative benchmark for our method. We used its 100 tasks under four-fold evaluation: for each fold, experience was constructed from the other 75 tasks and evaluated on the 25 held-out tasks. The baseline adopted a random-admission strategy. Donor trajectories were eligible if they achieved full success, regardless of whether the receiver had succeeded on the corresponding task or whether the task was identified as a capability gap. We performed three sampling trials without replacement using fixed random seeds. Within each fold, the sample size K matched the number of distinct successful trajectories from the donor admitted by the gating mechanism. Each sampling seed was evaluated three times. Each seed score averages its three evaluations; the overall mean and sample SD are computed across the three seed means. The results are shown below.

ReceiverSeed 1 meanSeed 2 meanSeed 3 meanRandom mean ± SDVetEx-E meanMean difference
DeepSeek-V3.244.7651.1851.4149.12 ± 3.7753.66+4.54
Qwen3.6-27B56.4252.9955.9555.12 ± 1.8659.54+4.42
GLM-5.151.8452.4653.1552.48 ± 0.6660.45+7.97

These results provide strong evidence that the gating mechanism effectively supplements missing successful evidence. By targeting the receiver’s positive-evidence gaps and introducing corrective experience derived from same-task success–failure comparisons, it provides more targeted guidance for correcting errors and achieves greater gains than randomly admitting external successful trajectories.

The matched-count comparison yields an average gain of 5.64 points across the three WebShop receivers. These results support the gate as an operational criterion for missing successful evidence in the available history; they do not make it a fine-grained diagnosis of every latent behavioral capability.

Reviewer_ozQzOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary
  • The paper proposes VetEx, a training-free way to reuse past experience with a frozen (not fine-tuned) LLM agent. It combines two pipelines built offline:
    • Corrective experience: pairs of failed and successful trajectories on the same task are turned into short do/avoid rules by an LLM compiler, then deduplicated and conflicts resolved. Successful trajectories from a second "donor" model are admitted only for tasks the agent has never solved across three rollout temperatures (the "gap gate").
    • Cleaned task references: for each new task, the two most similar past successes are retrieved. An LLM removes redundant or unsupported steps, and the shortened trajectory is kept only if replaying it in the original environment still succeeds; otherwise the raw trajectory is used.
  • Both are added to the agent's initial context before its first action.
  • Validation:
    • three benchmarks (ALFWorld, HotpotQA, WebShop) crossed with three frozen agent models (DeepSeek-V3.2, Qwen3.6-27B, GLM-5.1), using fold-rotated evaluation;
    • comparison with ReAct, ExpeL, ReasoningBank, Synapse, AdaMEM and SkillGen;
    • three repeated runs against the strongest baseline in each setting;
    • ablations on experience source, how experience is organized, gating, trajectory cleaning, compiler model and donor model.
  • VetEx beats ReAct in all nine settings and beats the strongest baseline's three-run mean by 5.12 points on average.
Critical strengths
  • Consistent gains under a careful protocol. VetEx improves over ReAct in all nine settings (+15.08 points on average) and over the strongest reproduced baseline in all nine three-run comparisons, usually with lower variance (Table 6). Fold-rotated evaluation, faithful reimplementation of five recent baselines, and complete prompts in the appendix make the main result credible.

  • Candid, informative analysis. The paper reports negative and null findings rather than only favorable ones:

    • adding donor trajectories alone does not help (−0.85 points on average);
    • cleaning alone does not help (−0.31);
    • stronger compiler models can reduce downstream performance (Table 15).

    It also gives paired repair/regression audits (Tables 7, 13). These results support the paper's broader message that more experience is not necessarily better, which is useful to the field.

Critical weaknesses
  • The comparison with baselines is confounded, so the gains cannot be cleanly attributed to the proposed ideas.
    • VetEx is the only method given donor trajectories (Table 1). The donor, compiler and cleaner is GLM-5.1, which is also the strongest agent model; for DeepSeek and Qwen this means extra information from a stronger model.
    • VetEx also injects the most context (3.07k tokens versus 0.40–2.05k for baselines).
    • Margins over the strongest baseline are modest relative to evaluation size (100–134 tasks): an average of 3.81 points in single runs. Several settings are within one standard deviation:
      • GLM–HotpotQA: 51.67 ± 2.08 vs. 51.33 ± 3.21
      • GLM–ALFWorld: 93.78 ± 1.87 vs. 91.53 ± 2.14
      • Qwen–ALFWorld: 92.04 vs. 90.55
    • Without baselines given the same donor pool, compiler and context budget, it is unclear whether the improvement comes from gap-aware design or simply from more and better-sourced evidence.
  • Limited novelty; the central claim is weakly supported.
    • The components are largely known:
      • same-task success–failure contrasts (ExpeL, AutoGuide);
      • retrieving similar successful trajectories as examples (Synapse);
      • LLM-based compression of trajectories.
    • The paper's main conceptual contribution is the "gap-filling" principle, but the evidence for it is weak:
      • gated admission beats ungated in only 6 of 9 settings, with losses of up to 4.24 points (Table 12);
      • adding donor experience hurts Qwen–HotpotQA by 4.0 points;
      • the gap is estimated crudely: a task counts as unsolved if none of three rollouts succeeded.
    • There is no control that admits the same number of donor pairs at random, so the benefit of the gate cannot be separated from simply adding more pairs.
    • The paper itself notes (Sec. 4.6) that the mechanism behind the complementarity between experience and trajectories rests mainly on selected case studies.
    • Evaluation is limited to subsets of older benchmarks (100 tasks for HotpotQA and WebShop), where ALFWorld is near ceiling for two of the three models.
Questions for authors

Clarification

  1. How do the strongest baselines (e.g., ExpeL, Synapse) perform when given the same donor trajectories, the GLM-5.1 compiler, and a matched context budget? This would determine whether the gains come from VetEx's design.
  2. Does the gate outperform admitting the same number of donor pairs chosen at random? Please report this to isolate the gap-filling effect.
  3. Can you report paired significance tests over tasks for the VetEx versus strongest-baseline comparisons in Fig. 4 and Table 6?
  4. ALFWorld and WebShop tasks are highly templated. How similar are the retrieved reference trajectories to the held-out tasks (e.g., same task type and objects)? Are donor rollouts restricted to the construction folds?
  5. How often does replay validation reject the cleaned trajectory and fall back to the raw one, per benchmark?
  6. What is the total construction and inference cost (LLM calls and tokens), including cleaning for each test task, compared with the baselines?

Suggestions

  1. Evaluating on larger or harder benchmarks (e.g., the full WebShop test set, AppWorld) and in a continual-update setting would strengthen the generality claims.
  2. A quantitative measure of complementarity (e.g., per-task attribution between experience and trajectories) would be stronger than the current case studies.
11 comments

We sincerely thank you for your thoughtful and detailed review and your positive overall assessment of our work. We particularly appreciate your recognition of our evaluation protocol and our effort to report negative and null results alongside performance gains. It is encouraging that you found the broader finding—that more experience is not necessarily better—useful to the field, as understanding when experience helps and when it does not is a central motivation of this work. Your concerns about fair comparisons and the evidence for gap filling, together with your suggestions for stronger quantitative analyses, give us concrete directions for strengthening the paper. We have taken these comments seriously and will address each point below. Thank you again for the time and care you devoted to evaluating our work.

Q1/W1-1, W1-2: How do the strongest baselines (e.g., ExpeL, Synapse) perform when given the same donor trajectories, the GLM-5.1 compiler, and a matched context budget? This would determine whether the gains come from VetEx's design.

Thanks for your comment. We have clarified in Appendix A.2 of the revised manuscript that GLM-5.1 is used for all LLM-based operations involved in experience construction across methods; Appendix A.3, Table 5 lists the VetEx configuration. This configuration applies to both VetEx and the relevant baselines, so GLM-5.1 is not an additional resource exclusive to VetEx. We also separately evaluate how the capabilities of the experience-extraction model and the donor affect downstream performance; these experiments are reported in Appendices D.3–D.4 and Tables 26–27. The results show that a stronger experience-extraction model does not consistently produce experience with greater downstream utility, and a stronger donor does not improve performance in every reuse configuration. These findings indicate that auxiliary-model capability alone does not determine downstream performance; how the evidence is organized and used also matters.

First, without donor-source expansion, we compare experience-extraction methods under experience-only inference. Detailed comparisons are provided in Section 4.3 and Figure 6(a) of the revised manuscript. The following table summarizes Self Experience’s score gains over each control, averaged equally across the nine benchmark–receiver settings.

ControlMean score gain of SelfSelf wins / ties / losses
ExpeL insight-only+2.977 / 1 / 1
ReasoningBank+5.498 / 0 / 1
SkillGen+7.538 / 0 / 1
ReAct+7.878 / 0 / 1

Furthermore, holding source material and downstream deduplication and conflict resolution fixed, we compare different experience-extraction methods. Same-task success–failure contrasts achieve the highest score in six of nine settings, including one tie, against ReasoningBank-style, ExpeL-style, and SkillGen-style organization. These results are shown in Figure 6(b), with absolute scores reported in Appendix C.6, Table 14 of the revised manuscript.

Second, we added a controlled comparison between donor supplementation and ExpeL’s native Reflexion retries for acquiring successful evidence. Both conditions use the same experience-construction pipeline, experience-extraction model, and experience-only evaluation configuration. VetEx-E achieves higher three-run mean scores than Retry in eight of nine settings, as shown in the newly added Figure 5. Detailed results are reported in Appendix C.5, Table 12 of the revised manuscript. Together, these ablations demonstrate the value of our algorithmic design.

Regarding donor trajectories, Figure 8(a) examines source expansion while keeping the retrieval method and Raw Trace injection protocol unchanged. Adding successful trajectories from the donor to the retrievable bank improves only two of nine settings when trajectories are used alone, with an average change of −0.85 points. When the same All Experience is supplied to both conditions, it improves four of nine settings, with an average change of −0.12 points. These results show that simply expanding the pool of retrievable references does not consistently improve performance. This is distinct from using donor successes to construct corrective experience. Section 4.3 shows that donor evidence can help when it fills the receiver’s positive-evidence gaps.

The revised manuscript also includes a matched-count random-admission experiment (Section 4.4; Appendix C.7, Table 16). Within each fold, random admission uses the same number of distinct donor successful trajectories as gap gating. Across three sampling seeds, each evaluated three times, VetEx-E outperforms random admission for all three WebShop receivers, with an average gain of 5.64 points. This comparison supports targeted admission of missing successful evidence beyond simply admitting more donor trajectories.

Regarding the context budget, we acknowledge that VetEx injects more experience context on average than some baselines, as reported in Figure 10(b) and Appendix E.6 of the revised manuscript. However, context length and final performance do not exhibit a monotonic relationship, a pattern also observed among the baselines. Comparing baseline results for which complete context-length records are available, we find that seven of the nine benchmark–receiver settings contain cases in which a longer experience context is associated with a lower score. For example, on HotpotQA–DeepSeek, Synapse uses approximately 3.8 times as much experience context as ReasoningBank but achieves a lower score (26.0 vs. 45.0). On WebShop–Qwen, ExpeL uses approximately 3.5 times as much experience context as Synapse, yet also scores lower (53.6 vs. 59.9). The source-expansion and coverage-gating ablations in Section 4.4 of the revised manuscript further support this view. These observations show that a larger context alone is insufficient to guarantee better performance; the selection and organization of evidence, as well as its relevance to the current task, are also critical.

Taken together, the results of the main experiments, the controlled ablations in Sections 4.3–4.5, and the task-paired statistical analyses in Appendix F indicate that VetEx’s additional context overhead is accompanied by consistent performance gains across multiple models and benchmarks. Furthermore, the extensive ablations in Sections 4.3–4.5 of the revised manuscript validate the contributions of receiver-conditioned corrective experience construction and task-reference trajectory cleaning with environment replay validation, while clarifying how these mechanisms work together and under what conditions they are effective. The revised manuscript includes these additional experiments and the corresponding analyses in Appendices C–D.

The main experimental results, together with the statistical significance tests reported in our response to Q3/W1-3 and Appendix F, Table 33, show that VetEx achieves higher three-run mean scores than the selected strongest external baselines in all nine settings, with lower run-to-run variability in most settings, while using a larger experience context. These results demonstrate the value of our approach alongside its additional context cost. Furthermore, the extensive ablations in Sections 4.3–4.5 of the revised manuscript validate the contributions of receiver-conditioned corrective experience construction and task-reference trajectory cleaning with environment replay validation, while clarifying how these mechanisms work together and under what conditions they are effective. The effectiveness of our approach is therefore supported from multiple perspectives. We respectfully ask the reviewer to consider the performance, stability, and ablation evidence above when assessing the value and design of our approach.

These controls isolate specific design choices within the evaluated configurations. The external-baseline comparisons still do not simultaneously match donor trajectories and experience-context budgets, so we do not attribute the entire overall performance difference to any single component.

Reviewer_6X8yOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how frozen LLM agents can reuse past experience more effectively. It proposes VetEx, which combines two components. First, it extracts corrective guidance by comparing successful and failed trajectories on the same task. When the receiver has failed but has no successful run, it uses a donor model’s successful trajectory to supply the missing positive evidence. Second, it cleans retrieved successful trajectories while preserving action–observation dependencies, checks them through replay in the original environment, and falls back to the raw trajectory if validation fails. Both the guidance and reference trajectories are added to the receiver’s context at task start.

The evaluation covers ALFWorld, HotpotQA, and WebShop with three receiver models, using complementary task folds for experience construction and evaluation. VetEx outperforms ReAct and the selected external baselines across all nine settings. The ablations suggest that adding more experience does not consistently help, and that trajectory cleaning is more effective when combined with corrective guidance than when used alone.

Critical strengths
  1. I like that the method selects experience based on what the receiver is missing. If an agent has never succeeded on a task, reflecting on its own failures may offer little guidance on what to do instead. A donor’s successful run on the same task provides a concrete reference, making this a simple and practical rule for deciding when to bring in external experience.

  2. The results support combining corrective guidance with concrete examples. With corrective experience held fixed, cleaned trajectories outperform raw ones in eight of the nine settings. This suggests that the two components offer complementary support: guidance helps the agent avoid known mistakes, while examples show how to carry out a solution.

  3. The cleaning procedure pays attention to the observations needed by later actions and checks the shortened trajectory by replaying it in the original environment. I find this more convincing than relying on an LLM-generated summary alone. The replay check provides a concrete safeguard against removing steps that are necessary to complete the source task.

  4. I appreciate that the experiments examine individual design choices rather than only reporting the full method’s performance. The findings that more donor trajectories and cleaning alone do not consistently help are particularly informative. They give a clearer picture of the method’s limits and make the analysis useful beyond the final benchmark scores.

Critical weaknesses
  1. I am not sure how well “no observed success” captures an experience gap. A receiver may fail several times because of sampling or step limits, and a single success does not mean it can solve the task reliably. The gate is a reasonable heuristic, but I would like stronger evidence that it identifies missing behavior rather than variation across attempts.

  2. The baseline comparisons leave some uncertainty about where the gains come from. VetEx has access to additional donor successes and uses more experience context than most baselines. Comparisons with the same source trajectories and similar context budgets would help establish how much the proposed design contributes.

  3. The evidence for trajectory cleaning is mixed. Cleaning alone slightly lowers average performance; it helps mainly when combined with corrective experience. Replay confirms that a shortened trajectory still solves its original task, but not that it provides a better example for a different task. The explanation for when cleaning helps—and when removing exploratory steps hurts—still relies largely on a few cases.

  4. Some of the reported gains are too small to assess confidently from three runs. On GLM–HotpotQA, for example, VetEx improves over the selected external baseline by about 0.34 points, less than either method’s run-to-run variation. The overall results are encouraging, but having a higher mean in all nine settings is not enough to establish a consistent advantage in each one.

  5. The overall cost is still unclear. VetEx requires trajectory collection, experience extraction, cleaning, environment replay, and additional context during execution. The reported 48.52% saving applies only to the input-evidence tokens used for Peer extraction. A fuller accounting of model calls, token usage, and latency would make it easier to judge whether the performance gains justify the added overhead.

Questions for authors
  1. Could you clarify whether the external baselines were evaluated with the same donor trajectories and similar experience-context budgets as VetEx? If not, how much can the existing controlled experiments separate the benefit of the proposed design from the benefit of additional information?

  2. For gap gating, is there evidence that “no observed success” distinguishes missing effective behavior from occasional failure? For example, are tasks with only one historical success solved more reliably later than tasks with no historical success? This would help clarify whether the gate identifies behavioral gaps or mainly reduces the amount of experience.

  3. In Table 6, cleaned trajectories alone do not improve average performance, but they become more useful when combined with corrective experience. Beyond the examples, do you have task-level statistics showing which failures are repaired, which tasks regress, and whether these changes relate to preserving information dependencies? I would like to understand whether the gains come from the cleaning mechanism emphasized in the paper.

  4. Across the three repeated evaluations, were the experience banks and retrieval results fixed, or were they reconstructed each time? Did you also compute task-paired significance tests or confidence intervals? Some improvements are smaller than the run-to-run variation, so I would like to understand which advantages have stronger statistical support.

6 comments

We sincerely thank you for your careful review and constructive feedback. We appreciate your positive assessment of our receiver-conditioned experience selection, the complementary roles of corrective guidance and concrete demonstrations, and the use of native replay to check execution dependencies. Your concerns help us identify where stronger evidence and clarification are needed. We will address each of your concerns and questions in turn and look forward to further discussion with you.

Q2/W1: For gap gating, is there evidence that “no observed success” distinguishes missing effective behavior from occasional failure? For example, are tasks with only one historical success solved more reliably later than tasks with no historical success? This would help clarify whether the gate identifies behavioral gaps or mainly reduces the amount of experience.

Thanks for your constructive comment. We acknowledge that the original manuscript did not directly validate the use of “no observed success” as an indicator of a capability gap. To address this concern, we conducted an additional analysis using existing execution records to quantify the relationship between historical success frequency and performance on independent attempts. This validation analysis has been added to Section 4.4 of the revised manuscript, with the detailed experimental design and results provided in Appendix C.7, Table 17.

To validate this definition, we use three ReAct runs as historical observations and another three as independent validation for each benchmark–receiver setting. Tasks are grouped by their number of historical successes, and we measure each group’s success rate over the three validation runs on the same tasks. The table below reports the validation success rate for each historical-success group across all nine benchmark–receiver settings.

Receiver–benchmark0/3 historical successes1/3 historical successes2/3 historical successes3/3 historical successes
DeepSeek–ALFWorld8.1%43.9%65.3%91.8%
Qwen–ALFWorld4.8%29.6%87.0%98.9%
GLM–ALFWorld9.1%29.6%50.0%98.8%
DeepSeek–HotpotQA6.8%24.2%46.2%86.4%
Qwen–HotpotQA3.8%6.7%60.0%86.9%
GLM–HotpotQA2.3%30.8%93.3%95.6%
DeepSeek–WebShop0.6%30.6%77.1%100.0%
Qwen–WebShop1.8%11.1%80.0%97.2%
GLM–WebShop2.8%9.5%66.7%92.2%

Of the tasks with no historical success, 91.5% failed all three independent validation attempts. Their aggregate validation success rate was 3.7%, compared with 27.66% for tasks with one historical success. These results support using the absence of success across three historical attempts to identify positive-evidence gaps.

Q3/W3: In Table 6, cleaned trajectories alone do not improve average performance, but they become more useful when combined with corrective experience. Beyond the examples, do you have task-level statistics showing which failures are repaired, which tasks regress, and whether these changes relate to preserving information dependencies? I would like to understand whether the gains come from the cleaning mechanism emphasized in the paper.

Thanks for your constructive comment. Building on the existing case studies, we provide quantitative comparisons and task-level paired statistics to investigate the complementarity between corrective experience and task-reference trajectories, and examine its underlying mechanisms and operating boundaries across benchmarks. These analyses are now included in Sections 4.3 and 4.5, Figure 9, and Appendix D.2.1 (Tables 19–22) of the revised manuscript.

Based on our analysis, we find that adding corrective experience improves performance with both raw and cleaned trajectory references in all nine settings, as shown in the table below. Conversely, adding raw references to experience-only inference lowers performance in five settings, whereas adding cleaned references improves performance in eight. These results indicate that raw task references do not consistently improve experience-only performance, whereas cleaned references yield more consistent gains when combined with corrective experience. These bidirectional gains provide quantitative evidence for the complementarity between corrective experience and cleaned task references.

Score changes from adding experience or traces. E denotes All Experience (VetEx-E), R denotes Raw Trace-only, and C denotes Clean Trace-only; E+R is VetEx-Exp+RT, and E+C is VetEx. The four columns report (E+R)−R, (E+C)−C, (E+R)−E, and (E+C)−E, respectively. Values are differences between the reported three-run mean scores in manuscript Table 7, computed from its rounded means.

BenchmarkReceiverAdd E to RAdd E to CAdd R to EAdd C to E
ALFWorldDeepSeek-V3.2+2.73+9.70+9.70+14.68
ALFWorldQwen3.6-27B+3.98+9.45-0.26+0.74
ALFWorldGLM-5.1+3.47+3.98+3.47+2.98
HotpotQADeepSeek-V3.2+5.67+8.34-1.00+1.00
HotpotQAQwen3.6-27B+5.67+4.33+2.00+2.33
HotpotQAGLM-5.1+0.67+3.00-3.33-0.66
WebShopDeepSeek-V3.2+12.62+18.58-0.28+6.81
WebShopQwen3.6-27B+5.98+10.77+1.15+5.18
WebShopGLM-5.1+3.98+4.05-2.38+0.62

Appendix D.2.1, Table 20 further reports corrective-experience repairs on both directions of Raw-only/Clean-only disagreement, aggregated over three runs. For example, on ALFWorld with Qwen3.6-27B, adding corrective experience repairs 32 of the 35 Clean-only failures on task–run pairs where Raw-only succeeds. This complements the component-score comparisons with evidence about which failures are repaired.

Comments on the paper

1 comment

Dear Reviewers,

We sincerely appreciate your careful reviews and constructive feedback, which have helped us improve our work.

During the rebuttal, we have addressed your concerns through additional experiments, analyses, and clarifications. We have updated our submission (PDF), with the main additions and revisions highlighted in BLUE. Below is a summary of the key updates:

  • Problem formulation, contribution positioning, and related work (Sections 1–2).
  • Successful-evidence acquisition ablation: VetEx-E versus ExpeL’s Reflexion retries (Section 4.3; Appendix C.5).
  • Validation and controlled ablations of coverage gating (Section 4.4; Appendix C.7).
  • Analyses of the complementarity between corrective experience and cleaned trajectories, including an additional weaker-receiver evaluation (Section 4.5; Appendix D.2.1).
  • Native replay validation ablations and mechanism analyses (Sections 4.3 and 4.5; Appendix D.2.3).
  • Statistical significance analyses (Section 4.2; Appendix F).
  • Comprehensive construction and inference cost analyses (Appendix E).

In addition, we have made two structural changes to improve the manuscript’s organization:

  • Separated ablation studies from mechanism analyses (Sections 4.3–4.5).
  • Moved Limitations and Operating Boundaries to Appendix H.

We sincerely hope that our responses and revisions have addressed the issues you’ve raised satisfactorily and strengthened VetEx as an effective framework for experience reuse in LLM agents. We would be happy to discuss with you if there are any further points you wish us to clarify or address for a better score.

Sincerely,
Paper 20767 Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.