- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes RACE, a framework for adaptive reasoning in LLM-based agents. The core idea is to leverage cross-turn reasoning persistence to determine when additional reasoning is necessary. Specifically, the authors introduce LoGiC, which uses reference-action likelihood changes to identify removable reasoning steps. The resulting signals are incorporated into both SFT and RL training. Experiments on four agent benchmarks demonstrate that RACE substantially reduces reasoning tokens while maintaining competitive task performance.
Critical strengths
- The motivation is clear and well supported. The observation that reasoning from earlier turns can continue to support subsequent actions provides an intuitive motivation for adaptive reasoning in multi-turn agent tasks.
- The paper is clearly written and well organized. The overall framework and its components are presented in a structured manner, making the method relatively easy to follow.
- The experimental evaluation is comprehensive, including comparisons with multiple baselines, ablation studies, and scaling analysis. The results demonstrate a promising performance–efficiency trade-off across different agent benchmarks.
Critical weaknesses
- The overall method is somewhat complicated, involving multiple training stages and components, making it less elegant than its core intuition. In particular, the two-stage SFT-RL pipeline and additional hyperparameters (e.g., and ) introduce extra complexity.
- The additional computational overhead introduced by RACE is not sufficiently discussed. Although the method significantly reduces reasoning tokens during inference, LoGiC requires additional likelihood evaluations during both data synthesis and RL training. It would be helpful to quantify this overhead and report how much additional computation or wall-clock time RACE introduces relative to the overall training pipeline.
Questions for authors
- Given the results, what do the authors consider to be a desirable performance–token trade-off? Under what level of accuracy degradation (or improvement) do the authors believe token savings become practically meaningful?
- How reliable is reference-action likelihood decrease as a proxy for reasoning necessity? Although the paper demonstrates an empirical correlation between likelihood decrease and action recovery rate, a small likelihood decrease does not necessarily guarantee that the reference action remains recoverable. Could the authors provide further quantitative analysis of the reliability of this proxy, particularly in cases where the reference-action likelihood is already low?
3 comments
Response (2/3)
Q1: What constitutes a desirable performance–token trade-off.
Thank you for raising this practical question. We consider a desirable trade-off to be a substantial reduction in reasoning computation while keeping task performance close to the corresponding full-reasoning training baseline. The acceptable performance difference depends on the intended application and the consequences of failure.
Quantitative comparison. For the settings studied in this paper, Figure 4 provides the following comparison against standard SFT+GRPO:
| Benchmark | SFT+GRPO success | RACE success | Change (pp) | SFT+GRPO reasoning tokens | RACE reasoning tokens | Token reduction |
|---|---|---|---|---|---|---|
| ScienceWorld | 74.55% | 73.28% | −1.27 | 2,073 | 1,403 | 32.3% |
| WebShop | 52.00% | 53.40% | +1.40 | 2,702 | 535 | 80.2% |
| AppWorld | 65.30% | 65.13% | −0.17 | 3,545 | 1,900 | 46.4% |
| DeepSearch | 35.75% | 35.00% | −0.75 | 2,163 | 822 | 62.0% |
Interpretation of the trade-off. Across the four benchmarks, RACE reduces reasoning tokens by 32.3%–80.2%, with a maximum observed success decrease of 1.27 percentage points and a 1.40-point improvement on WebShop, compared with standard SFT+GRPO. On WebShop, RACE improves success by 1.40 percentage points while reducing reasoning tokens by 80.2%. On AppWorld and DeepSearch, it reduces reasoning tokens by 46.4% and 62.0%, with observed success decreases of 0.17 and 0.75 points, respectively. ScienceWorld presents a more noticeable trade-off: a 32.3% token reduction with a 1.27-point success decrease.
For these benchmark settings, we view reductions of roughly one-third to four-fifths in reasoning tokens, accompanied by these small observed performance differences or an improvement, as promising.
Q2: Reliability of reference-action likelihood decrease as a proxy for reasoning necessity — Part 1
Thank you for raising this important question. To more directly evaluate reference-action likelihood decrease as a proxy for reasoning necessity, we conduct paired reference-action recovery experiments and further examine reference-action recovery when the reference-action likelihood is already low. We also trace whether the corresponding candidate reasoning removals are rejected by the complete LoGiC check.
Paired reference-action recovery experiments. We use 100 trajectories per dataset, each generated by Qwen3.5-9B and successfully completing its task. Keep retains the candidate reasoning step, while Skip removes only that step. Both conditions use the same current compressed history, including previously accepted removals, with all other reference actions, observations, and reasoning held fixed. At each target action position, we sample four actions under each condition and measure reference-action recovery. The sampled actions are neither executed in the environment nor inserted into subsequent histories.
The following tables report the average reference-action recovery rates when the candidate reasoning step is retained (Keep) or removed (Skip). The first table shows results when the reference-action likelihood decrease after removal is between 0 and 0.001 (). The second shows results when the decrease is greater than 0.001 ().
Likelihood decrease: .
| Dataset | Keep recovery | Skip recovery | Recovery loss (pp) |
|---|---|---|---|
| ScienceWorld | 97.97% | 97.76% | 0.21 |
| WebShop | 99.33% | 99.32% | 0.01 |
| AppWorld | 91.19% | 89.24% | 1.95 |
| DeepSearch | 82.64% | 79.74% | 2.90 |
Likelihood decrease: .
| Dataset | Keep recovery | Skip recovery | Recovery loss (pp) |
|---|---|---|---|
| ScienceWorld | 86.72% | 50.62% | 36.10 |
| WebShop | 92.14% | 54.96% | 37.18 |
| AppWorld | 81.51% | 44.37% | 37.14 |
| DeepSearch | 60.99% | 49.09% | 11.90 |
Recovery loss denotes the difference between Keep and Skip recovery rates in percentage points. Across all four datasets, small reference-action likelihood decreases correspond to Skip recovery rates close to their Keep counterparts, with additional recovery losses of only 0.01–2.90 percentage points. Skip recovery reaches 97.76% on ScienceWorld, 99.32% on WebShop, 89.24% on AppWorld, and 79.74% on DeepSearch.
When the reference-action likelihood decrease is greater than 0.001, the recovery loss is much larger, ranging from 11.90 to 37.18 percentage points across the four datasets. These results show that larger likelihood decreases correspond to larger reductions in reference-action recovery, supporting the use of likelihood decrease to assess the effect of reasoning removal.
The analysis of recovery when the reference-action likelihood is already low continues in the next comment.
Response (3/3)
Q2: Reliability of reference-action likelihood decrease as a proxy for reasoning necessity — Part 2
Recovery when the reference-action likelihood is already low. We group the comparisons according to whether the reference-action likelihood under Keep before removal is at least or below . We report the average reference-action recovery rates for each likelihood range and include only comparisons with . Recovery rates are averaged within each trajectory and then equally across trajectories in each likelihood range. Both accepted and rejected candidates are included. A trajectory may contribute comparisons to both likelihood ranges.
| Dataset | Reference-action likelihood before removal | Trajectories / Comparisons | Keep recovery | Skip recovery | Recovery loss (pp) |
|---|---|---|---|---|---|
| ScienceWorld | 94 / 6,133 | 98.11% | 97.90% | 0.21 | |
| ScienceWorld | 3 / 8 | 8.33% | 8.33% | 0.00 | |
| WebShop | 96 / 731 | 99.36% | 99.34% | 0.02 | |
| WebShop | 1 / 1 | 0.00% | 0.00% | 0.00 | |
| AppWorld | 100 / 1,714 | 91.54% | 89.66% | 1.88 | |
| AppWorld | 2 / 3 | 12.50% | 12.50% | 0.00 | |
| DeepSearch | 48 / 138 | 87.40% | 84.40% | 3.00 | |
| DeepSearch | 15 / 45 | 20.00% | 20.00% | 0.00 |
When the reference-action likelihood before removal is at least , Skip recovery ranges from 84.40% to 99.34%, with recovery losses of 0.02–3.00 percentage points relative to Keep. When the reference-action likelihood is below , recovery is already low under Keep, with no additional recovery loss observed in these samples. This illustrates the distinction: a small likelihood decrease can correspond to a small additional recovery loss, while the reference action may still be difficult to recover because its likelihood is already low.
We further examine how these low-likelihood comparisons affect the actual reasoning-removal decisions. A small likelihood decrease for one reference action does not by itself establish that the candidate reasoning step can be removed. LoGiC requires the likelihood decreases of the current and all subsequent affected reference actions to remain within the threshold. Across the four datasets, there are 1,014 comparisons in which the reference-action likelihood before removal is below . Of these, 57 comparisons (5.62%) have a likelihood decrease between 0 and 0.001. Since one candidate removal may affect multiple reference actions, these 57 comparisons correspond to 50 distinct candidate removals.
Of these 50 candidates, 47 (94%) are rejected by the complete LoGiC check because another affected reference action has a likelihood decrease exceeding 0.001. These results show that a small likelihood decrease for one low-likelihood reference action does not automatically lead to reasoning removal.
Ablation of cross-turn estimation. Table 2 of the revised manuscript compares full RACE with “w/o Cross-Turn Estimation” (current-action-only), which evaluates the likelihood decrease only for the current reference action. We report this comparison below. Each cell reports task success / average reasoning tokens per trajectory.
| Method | ScienceWorld | WebShop | AppWorld | DeepSearch |
|---|---|---|---|---|
| Current-action-only | 72.40% / 809 | 51.20% / 501 | 43.93% / 1,808 | 32.50% / 897 |
| Cross-Turn Estimation (RACE) | 73.28% / 1,403 | 53.40% / 535 | 65.13% / 1,900 | 35.00% / 822 |
Removing cross-turn estimation lowers task success across all four datasets, with the largest decrease of 21.20 percentage points on AppWorld. On DeepSearch, current-action-only checking also increases reasoning tokens from 822 to 897 while reducing success from 35.00% to 32.50%. These results suggest that checking only the current action can miss reasoning that supports subsequent decisions. They further support checking all affected reference actions when assessing whether a reasoning step can be removed.
These results show that small reference-action likelihood decreases consistently correspond to small additional recovery losses, the complete LoGiC check rejects most of the examined candidates containing low-likelihood actions with small likelihood decreases, and retaining cross-turn estimation improves task success across all four datasets.
We have added the paired reference-action recovery experiments and the analysis of recovery when the reference-action likelihood is already low to Appendix A.5 "Analysis of Likelihood Decrease and Action Recovery".
Response (1/3)
We sincerely thank the reviewer for the constructive and encouraging feedback. We particularly appreciate your recognition of the motivation behind cross-turn reasoning persistence, the clarity and organization of the paper, and the breadth of our experimental evaluation. Your questions highlight practical aspects of RACE that deserve a clearer explanation. Below, we clarify the roles of the training components, quantify the computational cost, and provide further evidence on the performance–token trade-off and the reliability of the likelihood-based signal.
W1: Complexity of the training pipeline and additional hyperparameters
Thank you for this helpful observation. We agree that the relationship between the core intuition and the training components should be explained more directly. RACE uses one shared reasoning-removal criterion, LoGiC, throughout the SFT–RL pipeline. The components translate this criterion into learning signals at different stages.
SFT provides an initial policy that can either reason or act directly, using trajectories whose removable reasoning has been identified by LoGiC. During RL, the policy and its generated trajectories continue to change, so LoGiC is applied to successful on-policy trajectories to update the supervision accordingly. Within RL, reasoning masking prevents the policy-gradient objective from reinforcing reasoning identified as unnecessary, while closure supervision explicitly teaches the policy to terminate reasoning at those turns.
The component ablations in Table 2 of the revised manuscript support the distinct roles of these training components. Removing closure supervision lowers task success and substantially increases reasoning tokens across all four benchmarks. For example, on WebShop, success decreases from 53.40% to 49.20%, while reasoning tokens increase from 535 to 3,690. On DeepSearch, success decreases from 35.00% to 34.75%, while reasoning tokens increase from 822 to 3,985. These results support the role of closure supervision in learning to terminate unnecessary reasoning while preserving task performance. Removing reasoning masking also lowers task success across all four benchmarks, with success decreasing from 53.40% to 46.00% on WebShop and from 65.13% to 56.75% on AppWorld, supporting its contribution to effective policy optimization. Furthermore, retaining the RACE-SFT initialization but removing LoGiC during RL substantially increases reasoning tokens across all four benchmarks. For example, reasoning tokens increase from 1,900 to 8,574 on AppWorld and from 822 to 3,507 on DeepSearch. This highlights the value of maintaining LoGiC-derived supervision during RL to preserve the reasoning efficiency learned during SFT.
The two additional hyperparameters have direct interpretations: ϵ is the likelihood-decrease threshold used to determine whether a reasoning step can be removed, and λ controls the strength of closure supervision. The SFT–RL structure is shared by the standard training baseline in Figure 4. RACE adds the computation and use of LoGiC-derived supervision within that structure. At inference time, the resulting policy directly decides whether to generate reasoning; it requires no separate LoGiC verification procedure.
W2: Computational overhead during data synthesis and RL training
We appreciate the request for explicit cost measurements. LoGiC introduces additional scoring computation during training, and this cost should be considered alongside the inference-time token savings.
Scoring time breakdown. The table below reports the proportion of time spent on LoGiC scoring in each stage. The SFT and RL percentages are calculated relative to the total SFT data-synthesis time and total RL training time, respectively.
Scoring cost across datasets. LoGiC accounts for 1.58%–6.08% of SFT data-synthesis time on three datasets and 37.24% on AppWorld. AppWorld has substantially fewer training examples, so its overall SFT data-synthesis time is relatively short. LoGiC scoring consequently accounts for a larger fraction of this shorter synthesis stage. During RL, the proportion of total training time spent on LoGiC scoring ranges from 14.54% to 17.31% across the four datasets.
Inference-time execution. LoGiC scoring is performed during data synthesis and RL training. At inference time, the trained policy directly decides whether to generate reasoning, without additional LoGiC scoring.
We have added this LoGiC scoring-time breakdown to
Appendix A.4 "Analysis of LoGiC’s Proportion of Training Time".