acceptodds
Withdrawn from ICLR 2027

ReGE: Stronger Agents through Experience Synthesis

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Repeated attempts at interactive tasks uncover facts, useful operations, and failures that can inform subsequent execution. We introduce ReGE, a framework for turning this recorded experience into guidance for a fresh attempt. Its main implementation, ReGE-Syn, uses a single coach call to jointly synthesize bounded execution records and available artifacts into self-contained guidance, connecting task-specific findings with proposed actions and relevant pitfalls. A student starts from the task's initial environment state and adapts this guidance to current observations. Source generation, synthesis, and student execution use the same base model, without parameter updates or a stronger teacher. Across four benchmarks, reporting the higher observed score of joint-synthesis or selected-trajectory guidance for each model–benchmark pair yields mean task-success gains of 9.9 percentage points on GPT and 8.8 on Claude over unguided execution. On matched BrowseComp-Plus subsets, ReGE-Syn reduces guidance-construction tokens by 98.2% on GPT and 95.5% on Claude relative to adapted AggAgent, with similar observed success. Combined construction and execution tokens decrease by 15.7% and 35.4%, respectively, counting input plus output and excluding source generation. BrowseComp-Plus ablations further associate access to process information with fewer student steps and show that joint synthesis produces substantially shorter guidance than direct history or independent summaries, with similar observed success. These results demonstrate how agents can use their own recorded experience to improve subsequent task completion.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Agents, Test-Time Scaling, Experience Synthesis, Trajectory Aggregation, Inference Efficiency

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_eLGTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

ReGE turns records from multiple attempts at the same task into guidance for a fresh run. Its main variant, ReGE-Syn, summarizes the attempts together in one model call. The same base model generates the attempts, writes the guidance, and executes the task, without parameter updates. Results show improved task success and lower guidance-construction costs in some comparisons.

Critical strengths
  1. The method is simple and practical. It preserves useful details such as commands, paths, and mistakes to avoid, and is evaluated across four benchmarks.

  2. The compression results are convincing. On the matched BrowseComp-Plus subset, ReGE-Syn produces much shorter guidance than direct history or separate summaries. It also uses far fewer construction tokens than adapted AggAgent, with similar observed success.

Critical weaknesses
  1. The cost comparison needs care. The main efficiency analysis excludes ReGE’s initial attempts, while Reflexion’s cost includes all its trials and reflections. This comparison therefore does not show that ReGE costs less overall. Table 2 provides estimates that include the initial attempts, but success under equal total token budgets remains unclear.

  2. The benefit of joint synthesis over simpler reuse methods is less clear. ReGE-Sel is competitive, and direct history achieves the same success rate as ReGE-Syn on the matched BrowseComp-Plus subset. These results support compression more strongly than an accuracy advantage from combining trajectories.

  3. The evaluation focuses on repeating the same task from a reset environment. It does not test whether the guidance remains useful when the environment changes or cannot be reset, or whether experience transfers to other tasks. The practical conclusions should reflect this limited scope.

  4. The distinction from earlier experience-summary methods needs to be clearer. Summarizing trajectories together in one call is simple and inexpensive, but stronger evidence is needed to show that combining complementary information adds value beyond reusing an existing answer or a good trajectory.

Questions for authors
  1. What are the exact SQL grading rules, and how was the BrowseComp-Plus answer judge validated?

  2. Can the authors report actual token usage for the initial attempts, guidance construction, and final execution, and compare methods under equal total budgets?

  3. Can the main success-rate comparisons include confidence intervals or repeated runs?

  4. Does the guidance remain effective when the initial state changes or when it is applied to a related task?

Reviewer_rYa4OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

ReGE runs the same LLM agent several times on a task, then uses another call to that model to combine the attempts’ useful findings, commands, and mistakes into a single guide. A fresh agent starts from the reset environment and uses that guide to complete the task, checking and adapting it as needed. No model weights are updated. The main idea is to reuse experience from multiple attempts through synthesized guidance

Critical strengths

NA

Critical weaknesses

If I understand this correctly, the novelty in the method is extremely limited. The paper practically replaces reflexion's “try → write a reflection → retry” loop with “make several independent tries → read them together → write one guide → try again.”

What is perhaps more concerning to me is that the baselines are not at all fair. For example, the authors show some performance improvement (~4-5%) on top of reflexion, at 2x the cost. The baselines have been deliberately made weak. For eaxmple, reflexion sees only 3 retries, while rege sees 6. I sincerely request the authors to clarify this, and make the comparison clear.

Questions for authors

See weakness

Reviewer_V5EaOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ReGE, a framework that synthesizes multiple agent execution trajectories into actionable guidance for subsequent execution, without parameter updates. Experiments across four agent benchmarks demonstrate improvements in task success rates and guidance-construction efficiency.

Critical strengths
  1. Comprehensive evaluation. The method is evaluated across four diverse agent benchmarks and two model families, demonstrating the practical value of reusing execution experience.
  2. Efficient experience aggregation. ReGE-Syn significantly reduces guidance-construction costs compared to AggAgent while maintaining comparable task success rates.
Critical weaknesses

1. Limited methodological novelty.

The core approach of summarizing agent execution trajectories into guidance closely resembles existing work, including ExpeL (Zhao et al., 2024), AutoGuide (Fu et al., 2024), AutoManual (Chen et al., 2024), and AggAgent (Lee et al., 2026). The main distinction is performing joint synthesis through a single LLM call, without introducing a substantially new mechanism for experience selection or integration.

2. Insufficient evidence for the necessity of joint synthesis.

In Table 8, directly concatenating historical trajectories achieves the same success rate as ReGE-Syn (88.3%), while Table 9 shows that selecting a single trajectory slightly outperforms ReGE-Syn (89.3% vs. 88.3%). These results raise questions about whether joint synthesis provides meaningful performance benefits beyond simpler experience reuse strategies.

3. Limited end-to-end computational efficiency.

Although ReGE-Syn substantially reduces guidance-construction costs, generating multiple source trajectories introduces considerable additional computation. Table 2 shows that the source-inclusive token cost is approximately 4–5 times that of a single execution, raising concerns about its efficiency under matched end-to-end computational budgets.

4. Insufficient analysis of complementary information across trajectories.

The largest improvements occur when source pools contain both successful and failed trajectories, but this does not establish whether the method effectively combines complementary information. In particular, improvements are limited and inconsistent when all source trajectories fail, leaving the benefits of synthesizing multiple unsuccessful attempts unclear.

Questions for authors

see weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.