acceptodds
Under review as a conference paper at ICLR 2027

AgentEAES: Effect-Aware Experience Selection for LLM Agents

Abstract

Recent advances have enabled large language models (LLMs) to act as agents capable of solving increasingly complex tasks in scientific research, office productivity, and software development. Prior work often improves agent performance at inference time by retrieving relevant experiences from past interactions. Although such experience-based augmentation offers a promising path toward continual adaptation, we identify a fundamental limitation of similarity-based retrieval in agentic settings. Highly similar experiences can degrade performance, even on tasks that the agent would otherwise solve successfully. This finding suggests that experience utilization is a decision problem rather than merely a retrieval problem. We propose effect-aware experience selection (AgentEAES), a framework that learns which experience to use from experience-augmented rollouts. AgentEAES trains a model to predict the sign of each task-experience pair's effect and selects the experience with the largest positive predicted effect. Experiments cover two agentic benchmarks with backbones of different strength, allowing us to evaluate both a strong backbone on a more challenging benchmark and a smaller model on an easier one. On AppWorld, with Qwen-3.7-Max, AgentEAES improves task success rates on the challenging domain by 13.3 percentage points. On WorkBench, even with the smaller Qwen3-30B-A3B backbone, AgentEAES still yields an improvement of up to 21.5 percentage points. The code is available at https://anonymous.4open.science/r/AgentEAES-BDBF.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
LLM agents, experience-augmented agents, experience selection, effect estimation, continual adaptation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jiGYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors point out that relying on experience retrieved via similarity-based methods is not always beneficial and can even lead to task failure. To address this, the paper proposes AgentEAES, which frames the utilization of experience as an effect-aware decision problem. This approach leverages lexical and cross-features derived from tasks, experiences, and trajectories to train two logistic regression classifiers—one to predict positive outcomes and the other to predict negative ones. The difference between these predictions serves as an effect score, and an experience is incorporated only if its corresponding score is positive.

Experiments were conducted on the AppWorld and WorkBench benchmarks, utilizing Qwen-3.7-Max and Qwen3-30B-A3B as the underlying task models. The results demonstrate that AgentEAES consistently outperforms random selection as well as retrieval methods based on text or vector similarity, yielding significant improvements across certain task domains.

Critical strengths
  1. This paper points out that correlation does not equal utility. Through experiments on AppWorld and WorkBench, it demonstrates that the correlation between similarity and actual performance is weak—or even negative—revealing a frequently overlooked issue of "negative transfer" in experience-augmented agents.

  2. Rather than reporting only accuracy, the authors provide a deeper analysis of how factors such as the number of interaction turns, token consumption, the cost of generating training labels, and the size of the training dataset affect the quality of performance predictions. These findings offer valuable insight into the practical costs associated with this approach.

Critical weaknesses
  1. Although the classifier is lightweight, each base model requires a large number of additional agent rollouts to generate performance labels; consequently, retraining may be necessary when transitioning to a new model. Performance relies on lexical overlap and predefined cross-patterns, which may struggle to capture synonymous paraphrasing, long-range semantic dependencies, abstract strategy mappings, or cross-lingual transfer.

  2. The full model achieves a GAUC of 0.634, which drops by only 0.005 to 0.629 when interaction features are removed; meanwhile, a model using only candidate-based features achieves 0.613. This suggests that the model may primarily be learning which experiences are generally effective, rather than relying heavily on specific task-experience pairings. The paper’s claim that interaction features are "crucial" is an overstatement.

Questions for authors
  1. How consistent are the G labels when the same task-experience pair is executed repeatedly? How much noise does a single rollout introduce relative to the true expected reward? Does using the average reward from multiple rollouts significantly improve the performance model?

  2. Please provide the proportions of samples with positive, zero, and negative rewards, as well as the class distributions for the two binary classification tasks. Specifically, in the absence of class weighting, does class imbalance affect the prediction results and the abstention thresholds?

  3. Generating performance labels for AppWorld requires approximately 1.6 * 10^9 tokens; furthermore, the construction of experiences involves using a GPT-5.4-class model for result extraction, segmentation, and hindsight relabeling. During inference, the total token consumption of AgentEAES on AppWorld is 62%–69% higher than that of the baseline without experiences. The paper suggests that these offline costs can be amortized over subsequent invocations; please specify the number of tasks required to reach a cost break-even point.

  4. Table 6 in the appendix reports 441 entries per trial, yet the full development sweep for GAUC utilizes 889 candidates. Could you explain why there is roughly a twofold difference between these figures?

2 comments

We thank the reviewer for the helpful comments. Our responses are below.


W1: Although the classifier is lightweight, each base model requires a large number of additional agent rollouts to generate performance labels; consequently, retraining may be necessary when transitioning to a new model.

A1: To test transferability within the Qwen family, we replace the backbone with Qwen3.6-Flash while keeping the same effect-aware model used for Qwen3.5-35B-A3B, without any retraining. Even in this zero-adaptation setting, the method continues to improve over the no-experience baseline, similarity-based retrieval, and CQR, showing that it possesses at least some transferability within the same model family. Due to budget and time constraints, we do not further evaluate transfer to backbones outside the same family in the current study, but we plan to report such cross-family results in the open-source release.

MethodEmail (45)Calendar (55)Analytics (60)CRM (40)PM (40)Multi (105)MacroMicro
No injection80.00 ± 3.8586.67 ± 2.7772.78 ± 2.5579.17 ± 2.8965.00 ± 0.0060.95 ± 0.9574.09 ± 0.6572.17 ± 0.29
Random Top-188.89 ± 0.0074.55 ± 5.4676.66 ± 2.8980.00 ± 2.5065.83 ± 8.0459.37 ± 3.8574.22 ± 0.9771.79 ± 0.89
Text-Sim (Dice)90.37 ± 6.4177.57 ± 9.3377.78 ± 4.2081.67 ± 1.4461.67 ± 5.2068.25 ± 0.5576.22 ± 2.7575.07 ± 2.53
Vector-Sim (BGE-M3)88.15 ± 2.5683.03 ± 5.2580.00 ± 1.6781.67 ± 5.7756.67 ± 1.4458.73 ± 4.8974.71 ± 0.9572.56 ± 1.75
CQR86.67 ± 6.6793.94 ± 4.5876.67 ± 5.7784.17 ± 1.4468.33 ± 1.4460.63 ± 4.8878.40 ± 1.5675.75 ± 2.41
AgentEAES89.63 ± 3.3987.88 ± 3.7880.00 ± 5.7791.67 ± 5.2073.33 ± 1.4461.90 ± 5.7280.74 ± 2.4577.58 ± 3.21

W2: The full model achieves a GAUC of 0.634, which drops by only 0.005 to 0.629 when interaction features are removed; meanwhile, a model using only candidate-based features achieves 0.613. This suggests that the model may primarily be learning which experiences are generally effective, rather than relying heavily on specific task-experience pairings. The paper’s claim that interaction features are "crucial" is an overstatement.

A2: Yes, we agree that the term crucial was not sufficiently precise. We have revised the wording accordingly and further added an ablation study examining the impact of removing cross features on the final performance. We have added a baseline that selects the top-ranked experience using a supervised Candidate-Quality Ranking(CQR) estimator in Table 1 and Table 2 to show the end-to-end performance

AppWorld benchmark

Methodtest-normal TGCtest-normal SGCtest-challenge TGCtest-challenge SGC
No experience64.3 ± 3.044.6 ± 1.861.1 ± 1.435.7 ± 1.5
CQR73.41 ± 2.0957.74 ± 5.4670.34 ± 1.1350.84 ± 2.91
AgentEAES (ours)74.4 ± 3.154.2 ± 3.774.3 ± 0.953.2 ± 2.5

WorkBench benchmark

MethodEmailCalendarAnalyticsCRMProj. Mgmt.MultiAvg.
No experience67.41 ± 1.2865.45 ± 1.8262.22 ± 1.9251.67 ± 8.7845.00 ± 4.3345.08 ± 4.7956.14
CQR75.56 ± 5.8867.88 ± 2.7866.11 ± 5.3666.67 ± 1.4465.83 ± 5.2042.22 ± 2.7564.04
AgentEAES (ours)65.19 ± 1.2866.06 ± 5.5576.67 ± 3.3367.50 ± 2.5065.83 ± 2.8949.52 ± 1.9065.13

Q1: How consistent are the G labels when the same task-experience pair is executed repeatedly? How much noise does a single rollout introduce relative to the true expected reward? Does using the average reward from multiple rollouts significantly improve the performance model?

A3: In Appendix C.1, we provide the detailed success rates over three rollouts. In addition, Appendix B.5 discusses how the number of rollouts affects the model’s estimated gAUC on the AppWorld development set. As shown there, increasing the number of rollout-generated samples consistently improves the estimated gAUC.

Sample sizeTraining pairsDev help-versus-harm GAUC
K=30K=302,7000.602
K=60K=605,4000.624
K=101K=1019,0900.634
K=200K=20018,0000.643
K=400K=40036,0000.652
Full sweep79,1200.660

Q2: Please provide the proportions of samples with positive, zero, and negative rewards, as well as the class distributions for the two binary classification tasks. Specifically, in the absence of class weighting, does class imbalance affect the prediction results and the abstention thresholds?

A4: In Table 31, we report the ratio between samples with positive effects and those with negative effects. Class imbalance does indeed affect model performance. However, it does not affect the choice of the abstention threshold in our method: we consistently use τ^f(t,m)>0\hat{\tau}_f(t,m)>0 as the criterion for deciding whether to use an experience.

DistributionClassCountProportion (%)
Gain signPositive (Δ>0\Delta>0)209323.29
Zero (Δ=0\Delta=0)496755.27
Negative (Δ<0\Delta<0)192721.44
Positive head (Δ>0\Delta>0)Positive target209323.29
Negative target689476.71
Negative head (Δ<0\Delta<0)Positive target192721.44
Negative target706078.56

Q3: Generating performance labels for AppWorld requires approximately 1.6 * 10^9 tokens; furthermore, the construction of experiences involves using a GPT-5.4-class model for result extraction, segmentation, and hindsight relabeling. During inference, the total token consumption of AgentEAES on AppWorld is 62%–69% higher than that of the baseline without experiences. The paper suggests that these offline costs can be amortized over subsequent invocations; please specify the number of tasks required to reach a cost break-even point.

A5: This method incurs two types of cost. The first is the cost of rollouts, and the second is the additional context cost introduced each time an experience is used. The amortization referred to in the paper concerns the rollout cost: the number of sampled rollouts used to construct the training data is controllable, and the LLM cost of generating these samples can be amortized over subsequent deployments. However, compared with not using experience at all, the overall cost is still always higher. The key point is that, as the number of downstream calls increases, the one-time cost of sample construction can be spread across later usage. We will also clarify this more explicitly in the cost analysis section of the paper. In Appendix D. we give detail cost analysis.


Q4: Table 10 in the appendix reports 441 entries per trial, yet the full development sweep for GAUC utilizes 889 candidates. Could you explain why there is roughly a twofold difference between these figures?

A6: 441 is the sum of the separately rounded pool-wise per-trial means (11+58+131+241)(11+58+131+241). 889 denotes the number of unique candidate IDs obtained after merging all three source trials and resolving repeated IDs. Candidate IDs do not encode the trial identifier, so records from different trials may share the same ID; under our parsing scheme, such collisions are resolved with a last-wins rule, which retains the later record. This procedure reflects ID resolution rather than deduplication based on identical text.


Additional clarification. Because the Qwen3-30B-A3B model had already been taken offline before the rebuttal period (see \url{https://www.alibabacloud.com/en/notice/model_studio_notice_of_retirement_for_selected_legacy_mainline_models_79d?_p_lc=1}), we reran all WorkBench experiments with Qwen-3.5-35B-A3B backbone. For the failure-mode analysis, however, we retained the previously sampled results obtained with Qwen3-30B-A3B to avoid incurring substantial additional computational cost.

Reviewer_6g4tOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+2)
Read review
Paper summary

This work claims addresses the shortcoming of experience-retrieval agents, and claims that retrieval based on the task similarity only might bring limited gains to no gains at all, and the choice of the trajectory should take into account the "potential effect of this trajectory" on the final task execution. Therefore, the authors propose to train a classifier that would predict for each trajectory m in the bank of experiences its usefulness for the task t. Each pair of (m, t) is represented with the bag of unigrams and bi-grams. Authors distinguish 1) task lexical features (n-grams covering task prompt) , 2) trajectory lexical features (n-grams covering task prompt and exectution, including thinking, code and actions) 3) interaction lexical features which is an overlap between task and trajectory lexical features after IDF-based filtering, to preserve only informative features.

Two binary logistic regression classifiers are trained on top of these lexical features prediction : prediction of positive gain or prediction of negative gain ; the overall effectiveness of the trajectory is computed as the difference between positive grain probability minus negative gain probability. The trajectory is prepended in the context only if expected effectiveness is positive.

First, in order to support the assumption that the task-similarity The classifiers are trained on the sampled trajectories from the experience bank, where the training tasks have been executed without experience, and with sampled experiences from different tasks.

The method is compared agains 4 simple baselines: no retrieved experience, random experience and top-1 experience retrieved either based on text dice similarity, or vector-sim with BGE-m3 retriever, based on the task prompt. The results of the proposed method seems to perform better than all the baselines. Authors also report cost of the different methods in terms of rounds, and tokens/round. They demonstrate that their method (as well as all the retrieval-based methods) lead to 20-30% less rounds/task on AppWorld, which means that the agent finds the solution faster. However since each round requires far more tokens, in terms of total amount of tokens it is not leading to the efficiency gains. And the authors are transparent about this matter.

Critical strengths

The motivation is meaningful, the authors provide some support to their initial hypothesis by measuring correlation between task similarity and the gain from the relevant trajectory, demonstrating that just task similarity is not a good indicator, since there is no clear correlation.

The results confirm the hypothesis, even though I would be more convinced if there were more models or benchmarks.

Authors provide a lot of experimental details in the appendix, perform multiple ablations studies, and honest about the limitations of their approach.

Critical weaknesses

While the motivation is appealing, that simple task-based similarity might not be enough to make it useful to the new task solving, it is not really clear to what extent the method proposed in the paper really does solve this problem.

  • According to results in the paper, the main contribution in the classifier performance comes from the trajectory features, while the motivation rather hints that it should be interaction between the trajectory and task features. What if we perform offline filtering of all the trajectories that do not bring any gain according the classifiers trained on trajectory features only? At inference you can perform either random or top-1 retrieval and see the performance. I would be curious to see whether it is comparable to AgentEAES performance?
  • it is not sure that overlap between the trajectory and task lexical features can really provide the hint about usefulness of the trajectory, I would like to see the examples of such trajectories.

Method is very basic, and there is no meaningful comparison to other experience retrieval methods in the paper. Some very simple comparisons:

  • eg. what if we use top-2 instead of top-1? how would it compare in terms of success rate and tokens/round?
  • what if you perform top-1 retrieval based on the similarity between the current task and the trajectory (from the experience bank), rather than task only?
  • what if you finetune the retriever on the data you've created for "useful trajectories" ?

It would be interesting to have more benchmarks to better assess the effectiveness of the method

Questions for authors

Few suggestions to improve this work:

  • provide better motivation for the way you define the classifier and its features. Giving some examples where the classifier effectively distinguished useful trajectory from harmful trajectory, and which features contributed to this, would be interesting. This is a strong advantage of the logistic regression classifier: it's decision is much more interpretable.

  • there is no explicit Related Work section? Is it intended? The domain of agents learning from experiences is very active, and it worth providing some positioning of your work with respect to the litterature

  • more baselines from the litterature or even very basic baselines (eg. top-2 retrieval instead of top-1)

4 comments

We sincerely thank the reviewer for the thoughtful and constructive feedback. We address each concern in detail below.


W1: What if we perform offline filtering of all the trajectories that do not bring any gain according the classifiers trained on trajectory features only? At inference, you can perform either random or top-1 retrieval and see the performance. I would be curious to see whether it is comparable to AgentEAES performance?

A1: In Appendix B.3.3, we additionally report a baseline that uses Vector-Sim retrieval while filtering out all trajectories that do not yield any gain. The results, shown below, indicate that this method performs better than standard Vector-Sim retrieval. However, on WorkBench, the number of trajectories filtered by quality is very limited, which suggests that the expected results would be nearly identical, and the experimental results are very close to those of standard Vector-Sim retrieval.

Methodtest-normal TGCtest-normal SGCtest-challenge TGCtest-challenge SGC
Vector-Sim71.4 ± 0.056.0 ± 3.768.4 ± 2.850.8 ± 6.3
Vector-Sim w/ quality filter70.83 ± 3.0954.17 ± 7.4373.06 ± 1.7053.96 ± 4.01

Q3: more baselines from the literature or even very basic baselines (eg. top-2 retrieval instead of top-1)

A2: In Appendix B.6.1 and B.6.2, we further compare against a baseline that uses top-2 retrieved experiences. The results show that, even when restricted to using the top-2 experiences for tasks that require experience, AgentEAES still achieves better final performance than the remaining baselines, as detailed below. appworld

MethodTGC countsTGC (%)SGC countsSGC (%)
Normal: 168 tasks / 56 scenarios per trial
Vector-Sim (BGE-M3)126/127/12875.60 ± 0.6039/36/3867.26 ± 2.73
Random115/122/11469.64 ± 2.5930/33/2954.76 ± 3.72
AgentEAES136/137/13580.95 ± 0.6034/38/3664.29 ± 3.57
Challenge: 417 tasks / 139 scenarios per trial
Vector-Sim (BGE-M3)304/302/30973.14 ± 0.8680/76/8056.59 ± 1.66
Random306/291/30071.70 ± 1.8177/69/7352.52 ± 2.88
AgentEAES310/313/31775.14 ± 0.8473/72/7853.48 ± 2.31

workbench

MethodEmailCalendarAnalyticsCRMPMMultiAvg.
Random62.96 ± 7.1461.82 ± 3.6471.67 ± 5.0061.67 ± 5.2049.17 ± 5.2046.35 ± 3.0658.94
Vector-Sim (BGE-M3)65.93 ± 10.9663.03 ± 2.1076.11 ± 3.4753.33 ± 3.8240.00 ± 2.5053.02 ± 4.5058.57
AgentEAES67.41 ± 4.6363.64 ± 8.3377.22 ± 5.0972.50 ± 4.3369.17 ± 3.8250.16 ± 3.6166.68

W2: it is not sure that overlap between the trajectory and task lexical features can really provide the hint about usefulness of the trajectory, I would like to see the examples of such trajectories. Q1:provide better motivation for the way you define the classifier and its features. Giving some examples where the classifier effectively distinguished useful trajectory from harmful trajectory, and which features contributed to this, would be interesting. This is a strong advantage of the logistic regression classifier: it's decision is much more interpretable.

A3: We have added an interpretability case study in Appendix E.5 to better motivate the classifier design and feature choices. The analysis compares two tasks with the same candidate but opposite observed outcomes, showing that the classifier can distinguish helpful from harmful trajectories through interpretable lexical and cross features. For example, the same Venmo refund experience receives a margin of +0.0605+0.0605 for a roommate return-and-reimbursement task, where it helps in all three trials, but a margin of −0.0168-0.0168 for an Amazon replacement-purchase task, where it harms performance in two of three trials. The candidate-only feature contributions are identical across the two cases, whereas 150 task ×\times trajectory interaction features are activated. Appendix E.3.1 further shows that cross features can demote a highly similar candidate whose injection changes a success into a failure.


W3: what if we use top-2 instead of top-1? how would it compare in terms of success rate and tokens/round? what if you perform top-1 retrieval based on the similarity between the current task and the trajectory (from the experience bank), rather than task only? what if you finetune the retriever on the data you've created for "useful trajectories" ?

A4: The utility comparison for using top-2 retrieval has already been discussed in A1. In addition, in Tables 1 and 2, we report results for a retriever fine-tuned with reward supervision. We also include in Appendix B.3.2 a comparison with trajectory-similarity-based retrieval, whose performance does not show a clear advantage over the similarity-based retriever, as shown in the table below.

MethodNormal TGCNormal SGCChallenge TGCChallenge SGC
Vector-Sim (prompt only)71.4 ± 0.056.0 ± 3.768.4 ± 2.850.8 ± 6.3
Vector-Sim (Task-to-trajectory)68.85 ± 3.2852.97 ± 8.2669.54 ± 1.5753.03 ± 1.67

Q2: there is no explicit Related Work section? Is it intended? The domain of agents learning from experiences is very active, and it worth providing some positioning of your work with respect to the litterature

A5: We have added a new section on related work. Broadly speaking, existing efforts on agent memory optimization mainly fall into two directions. The first direction focuses on experience construction and lifecycle management, with the goal of improving the quality of the experiences themselves and managing the memory lifecycle. This line of work is complementary to our method. The second direction focuses on experience selection, typically based on heuristic combinations of similarity retrieval and value-based scoring. To the best of our knowledge, this line of research has not yet directly optimized utility with an abstention rule in an end-to-end manner. Our method complements the literature by addressing this perspective.


Additional clarification. Because the Qwen3-30B-A3B model had already been taken offline before the rebuttal period (see \url{https://www.alibabacloud.com/en/notice/model_studio_notice_of_retirement_for_selected_legacy_mainline_models_79d?_p_lc=1}), we reran all WorkBench experiments with Qwen-3.5-35B-A3B backbone. For the failure-mode analysis, however, we retained the previously sampled results obtained with Qwen3-30B-A3B to avoid incurring substantial additional computational cost.

Reviewer_6g4tOpenReview ↗

comments on your results

The results show that, even when restricted to using the top-2 experiences for tasks that require experience, AgentEAES still achieves better final performance than the remaining baselines, as detailed below.

It doesnt' seem to be the case for SGC, but the results on workbench are convincing

Reviewer_6g4tOpenReview ↗

thank you for the clarifications

Dear authors,

thank you for your answers and additional clarifications. Based on your answers I've updated the soundness score, since I think you have addressed most of my concerns. I still think that having more than 1 model would make the results more convincing, especially taking into account your comment about Qwen3-30B-A3B model being taken offline.

And while the motivation is meaningful and results are convincing, I am still leaning towards weak reject since I feel like the contribution is maybe relatively light for a conference like ICLR. This is however an interesting submission for a specialized workshop.

Reviewer_SvrZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that similarity-based experience retrieval from past interaction can degrade LLM performance and treats experience utilization as a decision problem. Specifically, the proposed effect-aware experience selection (AgentEAES) trains a model to predict the effectiveness of past experience and only selects the experiences with the largest positive predicted effect. Reported results on two benchmarks showcase huge wins on task success rate

Critical strengths

The decision-oriented framing addresses a practical limitation of similarity retrieval: similarity is not necessarily a reliable proxy for an experience's usefulness, making selective experience reuse a well-motivated problem. The selector is concrete and lightweight at inference: two sparse logistic heads implement the positive-gain decision rule without changing the task-solving model

The main task-success gains are substantial. AppWorld TGC rises from 64.3 to 74.4 on test-normal and from 61.1 to 74.3 on test-challenge; the reported WorkBench macro-average is 48.3 versus 39.0 for the strongest retrieval baseline

Critical weaknesses

The experiments do not sufficiently isolate the contribution of task-conditioned effect prediction: AgentEAES is trained on 9,090 labeled task-experience rollouts on AppWorld and 21,585 on WorkBench, whereas the main retrieval baselines use no outcome supervision. In addition, those baselines always inject a candidate, while AgentEAES can abstain. These comparisons demonstrate the usefulness of the complete policy, but do not distinguish the proposed effect-learning formulation from supervised candidate-quality preferences and selective non-use. The most informative controls would include: (i) a retriever/reranker trained on the same gain labels; (ii) a task-independent candidate selected using training outcomes, within each domain on WorkBench; (iii) similarity retrieval with learned abstention; and (iv) end-to-end success and abstention rates for candidate-only/no-cross selectors, together with success rates for AgentEAES forced to inject its top-ranked candidate.

The motivating negative-transfer claim is insufficiently quantified for the actual similarity-retrieval policies: the abstract and introduction emphasize that highly similar experiences can harm tasks the agent would otherwise solve. However, the similarity-gain correlations and binned averages do not report how often the actual top-1 retrieval policies turn baseline successes into failures. Weak correlation is not itself evidence of harm, while positive aggregate improvements do not rule out harmful effects on individual tasks.

Questions for authors
  1. How does AgentEAES differ from supervised example retrieval/reranking, learned or value-based agent-memory selection, and uplift/CATE modeling? Please identify the closest prior approaches and clarify whether the main contribution is the gain supervision, lexical representation, abstention mechanism, or their integration for agent experience selection

  2. For the actual top-1 text- and vector-similarity policies, what is the observed harm rate, and how does it compare with random selection? Please report paired success-to-failure and failure-to-success counts across trials, including whether harm occurs among highly similar retrieved experiences.

  3. How is the help-vs-harm GAUC computed? Could you provide the baseline value or values and counts of negative, zero, and positive gains used in GAUC? Please specify whether baselines are rerun per pair or averaged, the actual binary AUC target, and how zero-gain pairs are handled. If the diagnostic compares helpful versus neutral or harmful versus neutral experiences rather than helpful versus harmful experiences, please clarify its definition and interpretation.

  4. What are the end-to-end success rates for candidate-only and no-cross selectors using the same abstention rule, and for full AgentEAES forced to inject its top-ranked candidate? Please report abstention rates for the non-forced variants. These controls would clarify the respective contributions of task-conditioned ranking, candidate-quality preferences, and selective non-use.

2 comments

We thank the reviewer for the careful reading of our paper and the insightful comments, which have helped us improve the clarity and rigor of this work. Please find our point-by-point responses below.


Q1: How does AgentEAES differ from supervised example retrieval/reranking, learned or value-based agent-memory selection, and uplift/CATE modeling? Q4: What are the end-to-end success rates for candidate-only and no-cross selectors using the same abstention rule, and for full AgentEAES forced to inject its top-ranked candidate? W1: These comparisons demonstrate the usefulness of the complete policy, but do not distinguish the proposed effect-learning formulation from supervised candidate-quality preferences and selective non-use

A1: We further introduce three additional experimental controls: 1) selecting the top-ranked experience using a supervised Candidate-Quality Ranking (CQR) estimator, 2) removing the abstention mechanism from AgentEAES, and 3) training a value-based variant by fine-tuning the BGE backbone with reward supervision. Here we would like to clarify that the condition raised in Q4, namely “no-cross selectors using the same abstention rule,” cannot be satisfied simultaneously. This is because the abstention rule is defined at the task level, whereas candidate-quality ranking operates at the candidate level and therefore cannot use the same mechanism directly. We report the final results in Tables 1 and 2 in the paper.
On the AppWorld benchmark, supervised candidate-quality and AgentEAES achieve comparable performance on the test-normal split, whereas AgentEAES shows a clear advantage on test-challenge. Because the experience utilization rate is 100%, the observed differences between AgentEAES and AgentEAES w/o abstention reflect execution variability rather than an abstention benefit. In contrast, on WorkBench, our method outperforms the other baselines on average. Because the experience utilization rate is not 100% on WorkBench, AgentEAES shows an advantage over AgentEAES w/o abstention.

AppWorld benchmark

Methodtest-normal TGCtest-normal SGCtest-challenge TGCtest-challenge SGC
No experience64.3 ± 3.044.6 ± 1.861.1 ± 1.435.7 ± 1.5
Random65.7 ± 2.747.6 ± 2.768.8 ± 1.647.5 ± 3.3
Text-sim (Dice)69.4 ± 1.252.4 ± 5.770.6 ± 4.151.1 ± 9.7
Vector-sim (BGE-M3)71.4 ± 0.056.0 ± 3.768.4 ± 2.850.8 ± 6.3
VBS67.86 ± 2.1551.19 ± 3.7269.78 ± 0.8351.32 ± 3.40
CQR73.41 ± 2.0957.74 ± 5.4670.34 ± 1.1350.84 ± 2.91
AgentEAES w/o abstention74.01 ± 0.6951.19 ± 4.1273.14 ± 1.4649.88 ± 6.49
AgentEAES (ours)74.4 ± 3.154.2 ± 3.774.3 ± 0.953.2 ± 2.5

WorkBench benchmark

MethodEmailCalendarAnalyticsCRMProj. Mgmt.MultiAvg.
No experience67.41 ± 1.2865.45 ± 1.8262.22 ± 1.9251.67 ± 8.7845.00 ± 4.3345.08 ± 4.7956.14
Random62.22 ± 3.8555.76 ± 3.7870.00 ± 5.7759.17 ± 1.4450.00 ± 6.6145.08 ± 2.4057.04
Text-sim (Dice)64.44 ± 5.8857.58 ± 2.7862.22 ± 1.9257.50 ± 0.0043.33 ± 3.8248.57 ± 2.8655.61
Vector-sim (BGE-M3)60.00 ± 2.2264.85 ± 4.5869.44 ± 0.9652.50 ± 2.5039.17 ± 1.4448.25 ± 3.8555.70
VBS57.04 ± 1.2859.39 ± 2.7866.11 ± 2.5550.00 ± 2.5035.00 ± 2.5043.17 ± 2.2051.79
CQR75.56 ± 5.8867.88 ± 2.7866.11 ± 5.3666.67 ± 1.4465.83 ± 5.2042.22 ± 2.7564.04
AgentEAES w/o abstention71.85 ± 2.5764.85 ± 3.7872.78 ± 5.3666.67 ± 2.8963.33 ± 7.6444.44 ± 2.2063.99
AgentEAES (ours)65.19 ± 1.2866.06 ± 5.5576.67 ± 3.3367.50 ± 2.5065.83 ± 2.8949.52 ± 1.9065.13

Q2: For the actual top-1 text- and vector-similarity policies, what is the observed harm rate, and how does it compare with random selection?

A2: In Section 6.4, we further report the transition matrix and harm rate of the experiences selected by different methods. As shown in the table below, although our method does not achieve the lowest harm rate, it attains a relatively high net gain, which is aligned with our optimization objective.


Q3: How is the help-vs-harm GAUC computed?

A3: We first define the gain as Δ\Delta = reward with experience−reward without experience. We then construct a binary AUC target, where samples with $ \Delta > 0 $ are treated as the positive class and samples with $ \Delta < 0 $ as the negative class, and score them using P(Δ>0)−P(Δ<0)P(\Delta>0)-P(\Delta<0). Samples with zero gain are excluded from the evaluation. The help-vs.-harm metric thus measures the model’s ability to distinguish helpful observations from harmful ones. For aggregation, we compute the AUC on a per-task basis, assign ties a value of 0.5, and report the unweighted average over all tasks that contain both positive and negative classes. We also add the statement in Appendix A.1.2.


Additional clarification. Because the Qwen3-30B-A3B model had already been taken offline before the rebuttal period (see \url{https://www.alibabacloud.com/en/notice/model_studio_notice_of_retirement_for_selected_legacy_mainline_models_79d?_p_lc=1}), we reran all WorkBench experiments with Qwen-3.5-35B-A3B backbone. For the failure-mode analysis, however, we retained the previously sampled results obtained with Qwen3-30B-A3B to avoid incurring substantial additional computational cost.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.