- Rating
- 3
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces MetaExp, a framework for training LLM agents to acquire, update, and reuse transferable experience through verifiable cross-task environments. It constructs Shared-Tool and Shared-Workflow task families to generate sequential execution and reflection trajectories for supervised fine-tuning. Experiments demonstrate improved experience-driven adaptation on three held-out agent benchmarks without further parameter updates.
Critical strengths
-
The paper addresses an important problem by treating experience learning as a transferable model capability rather than task-specific adaptation.
-
The complementary Shared-Tool and Shared-Workflow designs provide a scalable approach to generating structured cross-task supervision.
-
The evaluation covers heterogeneous benchmarks, domain-excluded training, and behavioral analyses, providing useful evidence of improved experience utilization.
Critical weaknesses
- The training objective is not directly aligned with the claimed cross-task experience learning capability. Section 4.3 filters trajectories based on current-task success, while Eq. (8) merely imitates teacher behavior without evaluating whether experience updates benefit subsequent tasks. More importantly, the absence of independent-task SFT and frozen-bank controls makes it difficult to distinguish genuine experience learning from improved general task-solving or skill-utilization abilities.
- The ablation studies do not convincingly establish the complementarity of the two task-family designs. In Table 4, the full model combines training data from both designs, whereas each single-design variant uses only one subset.
Questions for authors
see wekaness.
2 comments
Thank you to the authors for their detailed response, which has addressed all of my concerns. Taking into account the other reviewers' comments and the authors' rebuttal, I have decided to keep my original positive score.
Official Response from MetaExp Authors
Dear Reviewer 4am6,
Thank you for your constructive comments. We clarify the training design and its supporting evidence below.
W1(a). Alignment between the training objective and cross-task experience learning
Although Eq. (8) uses standard SFT, the supervised behaviors are shown within task chains: each execution is conditioned on experience accumulated from earlier family members, while each reflection uses execution feedback to create, revise, or retain skills for subsequent tasks. Cross-family interleaving further encourages selective reuse in the presence of irrelevant experience. Thus, training supervises experience-conditioned execution and feedback-driven updating rather than isolated task solving.
Importantly, filtering does not require task success. We retain trajectories with complete execution, evaluation, and reflection stages and valid skill artifacts, including those from failed tasks. These failures provide supervision for learning from negative feedback. We will clarify the ambiguous wording "verified task completion" in Section 4.3.
As discussed in Appendix D, explicitly optimizing downstream experience utility through utility-based filtering or cross-episode reinforcement learning is a promising extension of our current SFT approach.
W1(b). Distinguishing experience learning from general task solving and skill utilization
Our existing analyses provide complementary evidence for improvements in both experience utilization and updating.
Moreover, experience-driven gains are not coupled with improvements in initial task-solving ability. On SpreadsheetBench, MetaExp's S(0) decreases from 68.9 to 65.0 (62.7 for Strict-OOD), yet Δ40 improves from −2.2 for the backbone to +8.5 (+9.3 for Strict-OOD). Together, these findings support transferable experience-learning behavior beyond general task-solving improvements.
W2. Complementarity of the two task-family designs
The two designs exhibit distinct performance and efficiency patterns. Shared-Tool achieves stronger improvement on SpreadsheetBench (+9.7 versus +6.1 for Shared-Workflow), consistent with operational experience over stable tools. Shared-Workflow, meanwhile, reduces execution turns on APEX-Agents (−2.6) and AutomationBench (−2.4), consistent with learning reusable procedures.
Combining both designs yields broader benefits. The full model achieves positive gains across all three benchmarks, including +8.5 on SpreadsheetBench with only +0.5 additional execution turns, +1.8 on APEX-Agents (versus −0.5 and +0.4 for the single-design variants), and +14.8 on AutomationBench (versus +7.6 and +7.1), alongside a 7.0-turn reduction.
These differences are not simply monotonic with training-data volume: Shared-Tool alone (approximately 2.5K trajectories) achieves a larger SpreadsheetBench gain than the full model (approximately 5K), while Strict-OOD MetaExp reaches +9.3 with 3,634 trajectories. The observed patterns support the complementary roles of operational and procedural experience supervision.