acceptodds
Under review as a conference paper at ICLR 2027

REx: Small Language Model Agents with Recursive Execution and Subtree Compression

Abstract

Small language models are attractive for assistants because they can reduce the cost and latency of individual model calls, while local inference can limit exposure of sensitive data. However, long workflows require these models to coordinate dependent steps and retain useful results as execution histories grow. To address these demands, we introduce REx, which combines flexible planning with recursive execution and compression guided by unfinished work. Each planning round selects a batch, an ordered list of one or more subtasks, and assigns each to direct execution or recursive decomposition. Batch length can vary, with updated results guiding replanning after batch completion or subtask failure. To limit context growth, REx selectively replaces completed execution subtrees, the shared records of recursively decomposed subtasks and their nested subtasks, with compact memories. Task goals and any next planned subtask guide which outcomes and evidence to retain for unfinished work. We evaluate REx on Claw-Eval, GAIA, and Agent-Diff with three Gemma-4 models. Under matched models and limits on tool calls, REx outperforms the strongest evaluated baseline in all nine settings, improving task success by up to 19.88 percentage points over standard ReAct. Ablations support flexible planning and subtree compression, while further analysis shows the benefit of supplying task goals during compression.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Agents, Hierarchical Execution, Context Compression

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_oiyWOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how small language models can complete long-horizon agent tasks without changing the model itself. The authors identify two problems in recursive agents, namely planning that repeats without advancing execution, and context growth that buries or deletes evidence needed later. They propose REx, which has two parts. First, progressive planning with recursive execution. Each planning round emits either END or a variable length batch of subtasks, each labeled for direct execution by a local ReAct loop or for recursive decomposition. Planning resumes only when the batch finishes or a subtask fails. Second, subtree compression. When a decomposed subtask completes, its execution subtree in the shared global context is replaced by a compact memory, conditioned on the current goal, the parent goal, and the next planned sibling. On Claw-Eval, GAIA, and Agent-Diff with three Gemma-4 backbones, REx beats the best of four baselines in all nine settings, with gains of up to 19.88 points over ReAct. Ablations support the flexible batch length and the goal-conditioned compression, and a depth sweep shows that recursion helps but the gain is not monotonic in depth.

Critical strengths
  1. A well motivated design that separates task-level control from action-level execution. The batch mechanism directly addresses the failure mode observed in ReCAP traces, where small models keep replanning without producing new actions.
  2. Consistent gains across three benchmarks and three model scales, supported by careful ablations. The compression ablation is especially informative, since compression without goal conditioning performs worse than no compression at all, showing the benefit comes from retaining what unfinished work needs rather than from summarization alone.
  3. Honest efficiency reporting. The paper measures input tokens, output tokens, and model calls, and notes that REx uses fewer input tokens than ReAct but more output tokens and more calls.
Critical weaknesses
  1. Each setting appears to be a single temperature-0 run with no variance reported. On benchmarks of 161 to 224 tasks, some margins are only a few points, so it is hard to tell how stable the wins are.
  2. The compression component is only compared against REx variants. A comparison with existing context management methods such as HiAgent or AgentFold under the same backbone would make this contribution clearer.
Questions for authors
  1. How were the compression thresholds chosen for each benchmark, and how sensitive is the performance to this value?
  2. All experiments use the Gemma-4 family. Have you tried small models from another family to check that the gains are not tied to Gemma?
  3. The planner freely chooses the batch length and the EXECUTE or DECOMPOSE labels. How often do these outputs fail to parse and trigger the correction prompt, and does performance depend on that retry?
6 comments
AuthorsOpenReview ↗

Official Comment by Authors

Weakness 1: Each setting appears to be a single temperature-0 run with no variance reported. On benchmarks of 161 to 224 tasks, some margins are only a few points, so it is hard to tell how stable the wins are.

Response: We thank the reviewer for raising this important concern. To assess the reliability of our results, we conducted additional task-level paired statistical analyses across all nine benchmark–backbone settings. REx achieves higher observed Pass@1 than ReAct in all nine settings, with six showing 95% confidence intervals entirely above zero. These results provide additional statistical evidence while distinguishing clear improvements from smaller, inconclusive differences.

(1) Paired Statistical Analysis. We computed 95% paired bootstrap confidence intervals for the success-rate differences between REx and ReAct, using 10,000 task-level resamples per setting. Each resample preserves the paired outcomes of both methods on the same tasks. The resulting intervals quantify task-resampling uncertainty conditional on the observed runs.

(2) New Results. The results across all nine settings are summarized below.

BenchmarkModelDifference (pp)95% Paired Bootstrap CI (pp)
Claw-EvalE4B+16.78[9.94, 23.60]
Claw-Eval12B+16.78[9.94, 23.62]
Claw-Eval26B+19.88[11.80, 27.95]
GAIAE4B+4.84[0.00, 9.70]
GAIA12B+4.84[0.61, 9.09]
GAIA26B+10.91[4.24, 17.58]
Agent-DiffE4B+6.25[2.24, 10.71]
Agent-Diff12B+2.24[−3.13, 8.04]
Agent-Diff26B+3.13[−2.24, 8.48]

Results. Six of the nine 95% confidence intervals lie entirely above zero, covering all three benchmarks and all three model scales. The strongest evidence comes from Claw-Eval, where REx improves Pass@1 by 16.78–19.88 percentage points. For smaller gains, such as Agent-Diff/26B (+3.13 pp), the confidence interval includes zero ([−2.24, 8.48] pp), indicating that the observed difference is not statistically conclusive. These analyses strengthen the evidence for REx’s effectiveness while providing a more precise characterization of result uncertainty.

AuthorsOpenReview ↗

Official Comment by Authors

Weakness 2: The compression component is only compared against REx variants. A comparison with existing context management methods such as HiAgent or AgentFold under the same backbone would make this contribution clearer.

Response: We thank the reviewer for this valuable suggestion. We agree that comparing REx with existing context-management methods would strengthen the evaluation. We therefore added HiAgent, a closely related method supporting subgoal-based summarization and historical retrieval, as a new baseline. REx outperforms HiAgent in all nine benchmark–backbone settings, with improvements ranging from 3.64 to 27.95 percentage points.

(1) New Experiment: Comparison with HiAgent. We implemented HiAgent following its original subgoal-memory mechanism, including memory updates and historical retrieval through update_subgoal and retrieve_subgoal_history. We evaluated both methods on Claw-Eval, GAIA, and Agent-Diff using the same three Gemma-4 backbones within each setting.

BackboneBenchmarkHiAgent Pass@1REx Pass@1
E4BClaw-Eval16.15%29.81%
E4BGAIA12.12%15.76%
E4BAgent-Diff9.38%16.07%
12BClaw-Eval21.74%49.69%
12BGAIA21.21%33.93%
12BAgent-Diff32.14%39.29%
26B-A4BClaw-Eval29.81%55.90%
26B-A4BGAIA30.30%40.00%
26B-A4BAgent-Diff47.32%60.27%

(2) Results and Discussion. REx achieves higher Pass@1 than HiAgent across all nine settings, with particularly large improvements on Claw-Eval (+27.95 pp with 12B and +26.09 pp with 26B-A4B). Unlike HiAgent’s subgoal-based summarization, REx compresses completed recursive subtrees using current, parent, and next-sibling goals to preserve information needed by unfinished tasks. These results demonstrate REx’s end-to-end advantage over a closely related context-management method. We will include HiAgent in the main comparison table in the revised paper.

AuthorsOpenReview ↗

Official Comment by Authors

Question 1: How were the compression thresholds chosen for each benchmark, and how sensitive is the performance to this value?

Response: We thank the reviewer for this important question. We selected benchmark-specific compression thresholds based on the characteristics of execution traces and conducted additional sensitivity experiments across all three benchmarks. The default thresholds achieve the highest observed Pass@1 in all three settings, while alternative thresholds also yield comparable performance in several cases.

(1) Compression Threshold Selection. We use 3,072 tokens for Claw-Eval and 6,144 tokens for GAIA and Agent-Diff. Claw-Eval primarily involves structured workflows with relatively compact execution records, whereas GAIA often produces lengthy web observations, such as DOM and accessibility trees, and Agent-Diff involves structured records from multi-service API interactions. We therefore use larger thresholds for GAIA and Agent-Diff to avoid compressing useful execution information too early. Importantly, the threshold determines when compression is triggered, rather than the target length of the compressed memory: a completed subtree is compressed at a recursive return only when its current size exceeds the threshold.

(2) New Experiment: Threshold Sensitivity. We evaluated four thresholds on all three benchmarks using Gemma-4-26B-A4B, keeping all other settings unchanged.

Compression ThresholdClaw-Eval Pass@1GAIA Pass@1Agent-Diff Pass@1
1,53651.55%39.39%59.82%
3,07255.90% (default)36.97%55.80%
6,14452.80%40.00% (default)60.27% (default)
8,19249.69%34.55%57.59%

Results. The results show that compression-threshold sensitivity is benchmark-dependent and non-monotonic. On Claw-Eval, the default 3,072-token threshold achieves the highest Pass@1 (55.90%). On GAIA and Agent-Diff, the 6,144-token default also performs best, while the smaller 1,536-token threshold achieves similar results (39.39% vs. 40.00% on GAIA and 59.82% vs. 60.27% on Agent-Diff). These results indicate that REx does not require a single precise threshold to achieve competitive performance, although overly frequent or delayed compression can reduce accuracy.

Reviewer_jyyrOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes REx, a training-free agent framework for small language models on long-horizon tasks. It builds on ReCAP and makes two changes. First, planning is progressive and batched. Each subtask is labeled EXECUTE (a local ReAct loop) or DECOMPOSE (a recursive call to the same procedure). Second, when a decomposed subtask returns, its whole execution subtree is replaced in a shared global context by a compact memory. The memory is written by the same model conditioned on the child goal, the parent goal, and the next planned sibling. The experimental results are based on Claw-Eval, GAIA and Agent-Diff, with Gemma-4 E4B, 12B and 26B-A4B. Results showed that REx has the best Pass@1 in all nine settings. Ablation studies covered planning horizon, compression on/off, removal of goal conditioning from the compressor, and recursion depth. REx also cuts average peak context and uses fewer input tokens than ReAct.

[After Rebuttal] Based on the update on "weaknesses" and "questions" part, I would raise my score from 2 -> ~2.5 but still keep the system score as 2, leaving the work to AC for further consideration.

Critical strengths
  • Overall the design of the proposed framework is natural, clear, and well-motivated.Existing solution such as ReCAP has issues (1) planning that repeats without acting, and (2) deletion of evidence that later work needs. In this work, conditioning compression on current, parent, and upcoming goals gives a clear rationale for retaining information useful to unfinished work.
  • The analysis on the cost perspective is comprehensive and is helpful for practitioners, including input tokens, output tokens, model calls, and peak context. For example, compression reduces average peak context by 24% on Claw-Eval/26B, and the depth sweep identifies D=2 as a lower-resource setting with accuracy close to D=4.
  • Also in general gains without training are valuable.
Critical weaknesses
  • The novelty of this work is limited in my view. The method is a combination of ReCAP's shared recursive context, Plan-Exec style planning, and HiAgent style subgoal summarization. I acknowledge that there is a recursive execution as an improvement. Each piece is reasonable, but I feel the contribution is more on engineering and is empirical. [I still think it is an integration contribution and narrow down the scope to small model. But I acknowledge authors' response and explanation. I would raise my score from 2 -> ~2.5 but still keep the system score as 2, leaving the work to AC for further consideration.]
  • Some baselines are still missing. For example, I still think HiAgent is still a highly relevant baseline we should compare with. I acknowledge that the main differences are discussed in related work section. For example, HiAgent summarizes completed subgoal histories using their corresponding goals and can retrieve original records. REx compresses entire recursive subtrees, including descendant summaries, using current, parent, and next-sibling goals to guide retention. But I still would like to see the numbers the in the main table to understand the gap among SOTAs. This can make this work stronger. [Missing baselines are provided]
  • More detailed ablation can be included. For example, in the compaction stage, three kinds of signals are used including current, parent, and next-sibling goals. However, in the ablation study, the three components are removed as baseline. However, a more reasonable baseline should be summarizing using current goal as the HiAgent (which is an existing solution). This can show the parent or next-sibling goals' influence. [Extra ablations are provided, although it does not distinguish benefits of parent and next-sibling goals.]
Questions for authors
  • Do you have a direct measure of memory utility, such as how often a later step needed information that was dropped? [DIRR evaluation is provided.]
  • Do the gains hold for a stronger model? [Provided in the author response; Also in the response to Reviewer oiyW, Qwen family results are provided. Those are also the results I would like to see. I do see REx did not improve the DeepSeek V4 this kind stronger model but improving stronger model is not part of the claim of original paper. I originally would like see if the technique is generalizable to larger model.]
6 comments
AuthorsOpenReview ↗

Official Comment by Authors

Weakness 1: The novelty of this work is limited in my view. The method is a combination of ReCAP’s shared recursive context, Plan-Exec style planning, and HiAgent style subgoal summarization.

Response: We thank the reviewer for this thoughtful comment. We would like to clarify that REx is not simply a combination of existing methods, but is designed around the limited long-horizon planning and context-handling capabilities of small language models. Our key observation is that small models struggle with long-horizon planning but can make progress through local execution and recursive decomposition. REx builds on this observation through two connected designs: jointly controlling horizontal planning and vertical recursion, and using the resulting execution hierarchy to guide context compression.

Why Existing Planning and Recursion Methods Struggle with Small Models. Horizontal planning (e.g., Plan-Exec) requires models to anticipate future steps upfront, which is difficult for small models without sufficient execution feedback. Vertical recursion (e.g., ReCAP) can repeatedly decompose or replan tasks without making execution progress. Our traces show that ReCAP can reach a recursion depth of 16 without corresponding progress. These limitations motivate a more flexible approach that avoids both unreliable upfront planning and excessive recursive decomposition.

(1) Joint Horizontal Planning and Vertical Recursion. REx jointly decides how many subtasks to plan and whether each should be executed directly or recursively decomposed. It generates variable-length batches with EXECUTE/DECOMPOSE decisions, allowing manageable subtasks to execute through local ReAct loops while recursively decomposing more complex ones. This balances planning breadth and decomposition depth while incorporating execution feedback. On Claw-Eval/26B, REx achieves 55.90% Pass@1, outperforming single-step planning (49.69%) and one-shot planning (52.80%).

(2) Execution-Guided Context Compression. REx uses recursive execution boundaries to determine when and what to compress, and surrounding task goals to determine what to retain. Unlike HiAgent’s current-goal-based summarization, REx selectively compresses completed subtrees using current, parent, and next-sibling goals to preserve information needed by unfinished tasks. On Claw-Eval/26B, compression reduces peak context by 24.0% while improving Pass@1 from 51.55% to 55.90%. Full goal conditioning further outperforms current-goal-only compression by 3.73 percentage points.

AuthorsOpenReview ↗

Official Comment by Authors

Weakness 2: Some baselines are still missing. For example, I still think HiAgent is still a highly relevant baseline we should compare with. I acknowledge that the main differences are discussed in related work section. But I still would like to see the numbers the in the main table to understand the gap among SOTAs. This can make this work stronger.

Response: We thank the reviewer for this valuable suggestion. We agree that HiAgent is an important baseline for evaluating REx, as both methods manage execution histories through task-based memory mechanisms. We therefore implemented HiAgent and evaluated it across all nine benchmark–backbone settings. REx outperforms HiAgent in all nine settings, with improvements ranging from 3.64 to 27.95 percentage points. We will include these results in the main comparison table.

(1) New Experiment: Comparison with HiAgent. We implemented HiAgent following its original subgoal-based memory management, including memory updates and historical retrieval through update_subgoal and retrieve_subgoal_history. Both methods were evaluated on Claw-Eval, GAIA, and Agent-Diff using the same three Gemma-4 backbones within each setting.

BackboneBenchmarkHiAgent Pass@1REx Pass@1
E4BClaw-Eval16.15%29.81%
E4BGAIA12.12%15.76%
E4BAgent-Diff9.38%16.07%
12BClaw-Eval21.74%49.69%
12BGAIA21.21%33.93%
12BAgent-Diff32.14%39.29%
26B-A4BClaw-Eval29.81%55.90%
26B-A4BGAIA30.30%40.00%
26B-A4BAgent-Diff47.32%60.27%

(2) Results: Consistent Improvements over HiAgent. REx achieves higher Pass@1 in all nine settings, with particularly large gains on Claw-Eval (+27.95 pp with 12B and +26.09 pp with 26B-A4B). These results demonstrate that REx’s improvements extend beyond the original baselines to HiAgent, which already supports subgoal summarization and historical retrieval.

We also inspected execution traces to better understand the differences. For example, in an SLA compliance analysis task, HiAgent successfully retrieved the relevant SLA configurations but terminated before completing the requested analysis. This illustrates how completing a local subgoal does not necessarily ensure progress toward the overall task, motivating REx’s explicit coordination of recursive execution and task completion.

AuthorsOpenReview ↗

Official Comment by Authors

Weakness 3: More detailed ablation can be included. For example, in the compaction stage, three kinds of signals are used including** current, parent, and next-sibling goals. However, in the ablation study, the three components are removed as baseline. However, a more reasonable baseline should be summarizing using current goal as the HiAgent (which is an existing solution). This can show the parent or next-sibling goals’ influence.

Response: We thank the reviewer for this constructive suggestion. We agree that current-goal-only compression provides a more informative baseline than removing all goal information. We therefore conducted an additional ablation that isolates the benefit of using parent and next-sibling goals. The results show that full goal conditioning improves Pass@1 by 3.73 percentage points over current-goal-only compression, supporting the value of considering unfinished work during compression.

(1) New Experiment: Current-Goal-Only Compression. Following the reviewer’s suggestion, we evaluated a current-goal-only variant on Claw-Eval using Gemma-4-26B-A4B. This variant retains REx’s recursive execution and subtree-compression mechanisms but uses only the completed subtask’s current goal to guide summarization, similar to HiAgent’s goal-based summarization strategy. We compare it with the original ablations and full REx.

ConfigurationPass@1
REx w/ Compression, No Goal Conditioning50.31%
REx w/o Compression51.55%
REx w/ Current Goal Only52.17%
REx w/ Full Goal Conditioning55.90%

(2) Results: Benefits of Additional Goal Information. Current-goal-only compression improves Pass@1 from 50.31% to 52.17% compared with compression without explicit goal conditioning. More importantly, full REx achieves 55.90%, outperforming current-goal-only compression by 3.73 percentage points. Since both variants use the same execution framework and subtree-compression procedure, this improvement supports the benefit of incorporating parent and next-sibling goals when deciding what information to retain. These additional goals help preserve evidence relevant to unfinished tasks, rather than only summarizing what the completed subtask accomplished.

Reviewer_6SxdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper targets long-horizon tool use with small language models. REx keeps one shared context and, at each planning round, emits a batch of subtasks, each marked for direct execution or recursive decomposition, replanning after the batch or a failure. When a recursive subtask completes, its whole subtree is replaced by a short memory conditioned on the current goal, the parent goal and the next planned sibling. On three benchmarks and three Gemma-4 models it has the highest pass rate in all nine settings, with ablations for batch length and compression.

Critical strengths
  1. The design is clean and training-free, and the two components attack real failure modes observed in traces, namely recursive replanning without progress and context truncation that removes evidence still needed later. The compression ablation supports the second point directly, with success rising in all six settings when compression is on.
  2. The evaluation spans three benchmarks of different kinds and three model sizes, and the trend is consistent rather than confined to one setting, which is more than most agent-framework papers show.
Critical weaknesses
  1. Compute is not matched. The paper equalizes tool calls, but Figure 4b shows REx using more model calls and about fifty percent more output tokens than ReAct, and more of everything than the other three baselines. Extra inference compute alone can raise pass rates, so the nine-for-nine result cannot be attributed to the framework without a comparison at equal total calls or equal total tokens, or a simple more-compute baseline such as repeated sampling with ReAct.
  2. The evidence has no uncertainty estimate and the main baseline is weak. Pass rates come from a single attempt on 161 to 224 tasks with no seeds and no intervals, so differences of two or three points are within noise, yet they are reported as consistent wins. ReCAP, the closest baseline, is also run with a depth limit under which the paper says small models cannot follow its control protocol and falls below ReAct in several cells, which suggests it was not given its best configuration.
Questions for authors
  1. What happens to the comparison when total model calls or total tokens are held constant instead of tool calls, and how does REx compare with best-of-several ReAct runs at the same compute?
  2. Can you report confidence intervals, or at least per-task paired comparisons, for the nine settings? Are the Agent-Diff differences between REx and ReAct on the 26B model significant?
  3. Was ReCAP tuned for the small backbones, for example with a different depth limit or its own replanning schedule, and what is its score with that configuration?
4 comments
AuthorsOpenReview ↗

Official Comment by Authors

Question 1: What happens to the comparison when total model calls or total tokens are held constant instead of tool calls, and how does REx compare with best-of-several ReAct runs at the same compute?

Response: We thank the reviewer for raising this important concern. We agree that matching tool-call budgets alone does not ensure equal inference compute. To address this concern, we further examine inference consumption and conduct an additional experiment with repeated ReAct sampling. The results show that REx Pass@1 outperforms ReAct oracle Pass@3, while consuming substantially fewer tokens and model calls.

(1) Inference Costs in Original Experiments. Our original experiments use the same backbone models and tool-call budgets, but do not strictly match inference compute. As shown in Figure 4b, REx achieves higher accuracy than ReAct with 22.4% fewer input tokens, despite using more output tokens and model calls. We therefore conduct an additional experiment to examine whether increasing ReAct’s inference compute can close this performance gap.

(2) New Experiment: ReAct with Repeated Sampling. Following the reviewer’s suggestion, we evaluated ReAct with three independent attempts per task (temperature = 1) on GAIA using Gemma-4-26B-A4B, a challenging multi-step benchmark and our strongest backbone, consistent with the setting in Figure 4b. We report oracle Pass@3, where a task succeeds if any of the three attempts succeeds, giving ReAct multiple execution opportunities and oracle outcome selection. In contrast, REx uses only one attempt (Pass@1). We count all input tokens, output tokens, and model calls across the three ReAct attempts, including all planning and compression overhead for REx.

MethodPass RateInput Tokens (M)Output Tokens (M)Model Calls
ReAct oracle Pass@3 (T=1)38.18%1,410.513.7129,135
REx Pass@140.00%496.352.0113,144

As shown above, REx Pass@1 achieves a higher success rate than ReAct oracle Pass@3 (40.00% vs. 38.18%), despite using only one attempt. Moreover, repeated ReAct consumes 2.84× more input tokens, 1.85× more output tokens, and 2.22× more model calls than REx. Although this experiment does not strictly match inference compute, it shows that even with substantially more inference resources and oracle selection, ReAct does not match REx’s single-attempt performance. These results provide additional evidence that REx’s gains cannot be explained simply by increased inference compute.

AuthorsOpenReview ↗

Official Comment by Authors

Question 2: Can you report confidence intervals, or at least per-task paired comparisons, for the nine settings? Are the Agent-Diff differences between REx and ReAct on the 26B model significant?

Response: We thank the reviewer for this valuable suggestion. We conducted additional task-level paired statistical analyses for all nine benchmark–backbone settings. The results show that REx achieves higher observed success rates than ReAct in all nine settings, with six showing 95% confidence intervals entirely above zero. For Agent-Diff/26B, the improvement is positive but not statistically significant, as explained below.

(1) Paired Statistical Analysis. We computed paired bootstrap confidence intervals for the success-rate differences between REx and ReAct. For each setting, we resampled tasks with replacement 10,000 times, preserving the paired outcomes, and calculated the 95% confidence interval for the difference in success rates (REx minus ReAct). This paired analysis accounts for the fact that both methods are evaluated on the same tasks.

(2) New Results and Findings. The results for all nine settings are summarized below.

BenchmarkModelDifference (pp)95% Paired Bootstrap CI (pp)
Claw-EvalE4B+16.78[9.94, 23.60]
Claw-Eval12B+16.78[9.94, 23.62]
Claw-Eval26B+19.88[11.80, 27.95]
GAIAE4B+4.84[0.00, 9.70]
GAIA12B+4.84[0.61, 9.09]
GAIA26B+10.91[4.24, 17.58]
Agent-DiffE4B+6.25[2.24, 10.71]
Agent-Diff12B+2.24[−3.13, 8.04]
Agent-Diff26B+3.13[−2.24, 8.48]

Finding 1: Consistent Improvements Across Benchmarks. REx achieves higher observed success rates across all nine settings, with six 95% confidence intervals entirely above zero, covering all three benchmarks and all three model scales. The improvements are especially clear on Claw-Eval, reaching 16.78–19.88 percentage points.

Finding 2: Positive but Inconclusive Improvement on Agent-Diff/26B. REx improves Pass@1 from 57.14% to 60.27% (+3.13 pp), but the difference is not statistically significant (95% CI: [−2.24, 8.48] pp; exact McNemar p = 0.324). This smaller gain may partly reflect ReAct’s already strong performance with the 26B backbone, leaving less room for improvement. With 224 evaluation tasks, the observed difference is insufficient to establish statistical significance. Nevertheless, REx maintains higher observed success rates across all nine settings, with statistically supported improvements in six of them.

AuthorsOpenReview ↗

Official Comment by Authors

Question 3: Was ReCAP tuned for the small backbones, for example with a different depth limit or its own replanning schedule, and what is its score with that configuration?

Response: We thank the reviewer for this important question. We clarify that ReCAP used unrestricted recursion (D = ∞) in our original experiments, rather than the depth limit of D = 4 used by REx. Our additional experiments confirm that unrestricted recursion achieves the highest Pass@1 (37.88%) among the tested configurations, indicating that ReCAP was not disadvantaged by an unfavorable depth limit.

(1) Original ReCAP Configuration. We followed ReCAP’s original planning prompts, replanning schedule, and execution protocol without modifying them for the smaller backbones. Specifically, REx used D = 4, while ReCAP used D = ∞. We apologize for the unclear descriptions in Section 4.2 and Appendix A and will revise them to explicitly distinguish these settings.

(2) New Experiment: ReCAP with Different Recursion Depths. To examine whether a different depth limit could improve ReCAP’s performance, we evaluated D ∈ {2, 4, 6, 8, 16, ∞} on Claw-Eval using Gemma-4-26B-A4B, keeping its original planning prompts, replanning schedule, and execution protocol unchanged. The results show that ReCAP’s performance varies non-monotonically with recursion depth, with unrestricted recursion achieving the highest Pass@1 (37.88%). This is the configuration already used in our main experiments. Under the same benchmark and backbone, REx achieves 55.90% Pass@1, outperforming ReCAP by 18.02 percentage points. These results confirm that REx’s advantage is not due to imposing an unfavorable recursion-depth limit on ReCAP.

Maximum Recursion DepthPass@1
D = 234.78%
D = 431.06%
D = 634.78%
D = 831.68%
D = 1629.81%
D = ∞ (Original)37.88%
AuthorsOpenReview ↗

Follow-up on Rebuttal – REx (Paper #727)

Dear Reviewer 6Sxd,

Thank you again for your valuable feedback. As the author–reviewer discussion period is approaching its end, we would greatly appreciate any further feedback on our rebuttal and the additional experiments presented in our revised manuscript. We would be happy to clarify any remaining questions or concerns.

Paper number: 727

Paper title: REx: Recursive Execution for Small Language Model Agents

To view the comment, click here: https://openreview.net/forum?id=6Zkm41r219&noteId=CtJBXe2tmd

Thank you for your time and consideration!

Best regards, The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to the AE and Reviewers

We sincerely thank the AE and reviewers for their constructive feedback. We conducted additional experiments to strengthen the evaluation, clarify REx’s contributions, and examine its generalization and robustness. The key updates are summarized below.

(1) Fair Comparisons and Statistical Reliability (Reviewers 6Sxd, jyyr, and oiyW). We added HiAgent comparisons across all nine settings, where REx improves Pass@1 by 3.64–27.95 percentage points. On GAIA/26B, REx Pass@1 (40.00%) exceeds ReAct oracle Pass@3 (38.18%), while consuming fewer tokens and model calls. We also conducted a ReCAP depth sweep, confirming that its original unrestricted configuration performs best among the tested settings. Furthermore, paired bootstrap analyses show that six of nine 95% confidence intervals exclude zero, providing stronger statistical evidence for the observed improvements.

(2) Method Contributions and Memory Utility (Reviewer jyyr; related to Reviewer oiyW). We clarify that REx jointly controls horizontal planning and vertical recursion for small models, while using the resulting execution hierarchy to guide context compression. A new ablation shows that full goal conditioning improves Pass@1 by 3.73 percentage points over current-goal-only compression. Our direct memory evaluation further shows that REx retains 77.73% of downstream-required facts, compared with 70.21% and 59.48% for the two compression variants.

(3) Generalization and Robustness (Reviewers jyyr and oiyW). We extended the evaluation to two Qwen models, where REx outperforms ReAct and ReCAP, and DeepSeek V4 Flash, where REx achieves slightly lower Pass@1 than ReAct (68.32% vs. 69.57%). We also evaluated compression-threshold sensitivity across three benchmarks and measured planner-output correction rates, confirming that REx maintains higher observed performance than ReAct without correction retries in the tested setting.

Detailed results and experimental protocols are provided in our responses to the individual reviewers. We thank the reviewers for helping us strengthen the evidence and clarify the scope of REx.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.