acceptodds
Under review as a conference paper at ICLR 2027

When Should Agents Think? Adaptive Reasoning via Cross-Turn Estimation

Abstract

Large language model (LLM)-based agents have demonstrated strong capabilities on complex tasks. They typically perform reasoning before each action throughout an interaction trajectory. However, reasoning may not be necessary at every turn, as reasoning produced earlier can continue to support subsequent actions. A key challenge is therefore to determine when existing reasoning remains sufficient and when a new reasoning step is needed, without relying on costly generation-based verification. We find that decreases in the likelihood of subsequent reference actions after removing additional reasoning closely track whether those actions remain recoverable given earlier reasoning, providing an effective and lightweight signal for estimating cross-turn action support. Based on this observation, we propose Reasoning Adaptation through Cross-Turn Estimation (RACE), a training approach for adaptive agent reasoning. RACE introduces a Likelihood-Guided Progressive Reasoning Cover Detection (LoGiC) procedure that progressively identifies reasoning turns whose removal has limited impact on the current and subsequent reference actions. The resulting removal signals are incorporated into both supervised fine-tuning and agentic reinforcement learning, enabling the policy to learn when to reason and when to act directly. Extensive experiments on four representative agent benchmarks show that RACE substantially reduces reasoning cost while maintaining or improving task performance.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agent Reasoning, Reasoning Efficiency, LLM-based Agents

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_oeSoOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes RACE, a framework for adaptive reasoning in LLM-based agents. The core idea is to leverage cross-turn reasoning persistence to determine when additional reasoning is necessary. Specifically, the authors introduce LoGiC, which uses reference-action likelihood changes to identify removable reasoning steps. The resulting signals are incorporated into both SFT and RL training. Experiments on four agent benchmarks demonstrate that RACE substantially reduces reasoning tokens while maintaining competitive task performance.

Critical strengths
  1. The motivation is clear and well supported. The observation that reasoning from earlier turns can continue to support subsequent actions provides an intuitive motivation for adaptive reasoning in multi-turn agent tasks.
  2. The paper is clearly written and well organized. The overall framework and its components are presented in a structured manner, making the method relatively easy to follow.
  3. The experimental evaluation is comprehensive, including comparisons with multiple baselines, ablation studies, and scaling analysis. The results demonstrate a promising performance–efficiency trade-off across different agent benchmarks.
Critical weaknesses
  1. The overall method is somewhat complicated, involving multiple training stages and components, making it less elegant than its core intuition. In particular, the two-stage SFT-RL pipeline and additional hyperparameters (e.g., ϵ\epsilon and λ\lambda) introduce extra complexity.
  2. The additional computational overhead introduced by RACE is not sufficiently discussed. Although the method significantly reduces reasoning tokens during inference, LoGiC requires additional likelihood evaluations during both data synthesis and RL training. It would be helpful to quantify this overhead and report how much additional computation or wall-clock time RACE introduces relative to the overall training pipeline.
Questions for authors
  1. Given the results, what do the authors consider to be a desirable performance–token trade-off? Under what level of accuracy degradation (or improvement) do the authors believe token savings become practically meaningful?
  2. How reliable is reference-action likelihood decrease as a proxy for reasoning necessity? Although the paper demonstrates an empirical correlation between likelihood decrease and action recovery rate, a small likelihood decrease does not necessarily guarantee that the reference action remains recoverable. Could the authors provide further quantitative analysis of the reliability of this proxy, particularly in cases where the reference-action likelihood is already low?
3 comments
AuthorsOpenReview ↗

Response (1/3)

We sincerely thank the reviewer for the constructive and encouraging feedback. We particularly appreciate your recognition of the motivation behind cross-turn reasoning persistence, the clarity and organization of the paper, and the breadth of our experimental evaluation. Your questions highlight practical aspects of RACE that deserve a clearer explanation. Below, we clarify the roles of the training components, quantify the computational cost, and provide further evidence on the performance–token trade-off and the reliability of the likelihood-based signal.


W1: Complexity of the training pipeline and additional hyperparameters

Thank you for this helpful observation. We agree that the relationship between the core intuition and the training components should be explained more directly. RACE uses one shared reasoning-removal criterion, LoGiC, throughout the SFT–RL pipeline. The components translate this criterion into learning signals at different stages.

SFT provides an initial policy that can either reason or act directly, using trajectories whose removable reasoning has been identified by LoGiC. During RL, the policy and its generated trajectories continue to change, so LoGiC is applied to successful on-policy trajectories to update the supervision accordingly. Within RL, reasoning masking prevents the policy-gradient objective from reinforcing reasoning identified as unnecessary, while closure supervision explicitly teaches the policy to terminate reasoning at those turns.

The component ablations in Table 2 of the revised manuscript support the distinct roles of these training components. Removing closure supervision lowers task success and substantially increases reasoning tokens across all four benchmarks. For example, on WebShop, success decreases from 53.40% to 49.20%, while reasoning tokens increase from 535 to 3,690. On DeepSearch, success decreases from 35.00% to 34.75%, while reasoning tokens increase from 822 to 3,985. These results support the role of closure supervision in learning to terminate unnecessary reasoning while preserving task performance. Removing reasoning masking also lowers task success across all four benchmarks, with success decreasing from 53.40% to 46.00% on WebShop and from 65.13% to 56.75% on AppWorld, supporting its contribution to effective policy optimization. Furthermore, retaining the RACE-SFT initialization but removing LoGiC during RL substantially increases reasoning tokens across all four benchmarks. For example, reasoning tokens increase from 1,900 to 8,574 on AppWorld and from 822 to 3,507 on DeepSearch. This highlights the value of maintaining LoGiC-derived supervision during RL to preserve the reasoning efficiency learned during SFT.

The two additional hyperparameters have direct interpretations: ϵ\epsilon is the likelihood-decrease threshold used to determine whether a reasoning step can be removed, and λ\lambda controls the strength of closure supervision. The SFT–RL structure is shared by the standard training baseline in Figure 4. RACE adds the computation and use of LoGiC-derived supervision within that structure. At inference time, the resulting policy directly decides whether to generate reasoning; it requires no separate LoGiC verification procedure.


W2: Computational overhead during data synthesis and RL training

We appreciate the request for explicit cost measurements. LoGiC introduces additional scoring computation during training, and this cost should be considered alongside the inference-time token savings.

Scoring time breakdown. The table below reports the proportion of time spent on LoGiC scoring in each stage. The SFT and RL percentages are calculated relative to the total SFT data-synthesis time and total RL training time, respectively.

DatasetLoGiC scoring / SFT data-synthesis timeLoGiC scoring / RL training time
ScienceWorld1.58%17.31%
WebShop3.92%14.54%
AppWorld37.24%16.93%
DeepSearch6.08%15.51%

Scoring cost across datasets. LoGiC accounts for 1.58%–6.08% of SFT data-synthesis time on three datasets and 37.24% on AppWorld. AppWorld has substantially fewer training examples, so its overall SFT data-synthesis time is relatively short. LoGiC scoring consequently accounts for a larger fraction of this shorter synthesis stage. During RL, the proportion of total training time spent on LoGiC scoring ranges from 14.54% to 17.31% across the four datasets.

Inference-time execution. LoGiC scoring is performed during data synthesis and RL training. At inference time, the trained policy directly decides whether to generate reasoning, without additional LoGiC scoring.

We have added this LoGiC scoring-time breakdown to Appendix A.4 "Analysis of LoGiC’s Proportion of Training Time".

AuthorsOpenReview ↗

Response (2/3)

Q1: What constitutes a desirable performance–token trade-off.

Thank you for raising this practical question. We consider a desirable trade-off to be a substantial reduction in reasoning computation while keeping task performance close to the corresponding full-reasoning training baseline. The acceptable performance difference depends on the intended application and the consequences of failure.

Quantitative comparison. For the settings studied in this paper, Figure 4 provides the following comparison against standard SFT+GRPO:

BenchmarkSFT+GRPO successRACE successChange (pp)SFT+GRPO reasoning tokensRACE reasoning tokensToken reduction
ScienceWorld74.55%73.28%−1.272,0731,40332.3%
WebShop52.00%53.40%+1.402,70253580.2%
AppWorld65.30%65.13%−0.173,5451,90046.4%
DeepSearch35.75%35.00%−0.752,16382262.0%

Interpretation of the trade-off. Across the four benchmarks, RACE reduces reasoning tokens by 32.3%–80.2%, with a maximum observed success decrease of 1.27 percentage points and a 1.40-point improvement on WebShop, compared with standard SFT+GRPO. On WebShop, RACE improves success by 1.40 percentage points while reducing reasoning tokens by 80.2%. On AppWorld and DeepSearch, it reduces reasoning tokens by 46.4% and 62.0%, with observed success decreases of 0.17 and 0.75 points, respectively. ScienceWorld presents a more noticeable trade-off: a 32.3% token reduction with a 1.27-point success decrease.

For these benchmark settings, we view reductions of roughly one-third to four-fifths in reasoning tokens, accompanied by these small observed performance differences or an improvement, as promising.


Q2: Reliability of reference-action likelihood decrease as a proxy for reasoning necessity — Part 1

Thank you for raising this important question. To more directly evaluate reference-action likelihood decrease as a proxy for reasoning necessity, we conduct paired reference-action recovery experiments and further examine reference-action recovery when the reference-action likelihood is already low. We also trace whether the corresponding candidate reasoning removals are rejected by the complete LoGiC check.

Paired reference-action recovery experiments. We use 100 trajectories per dataset, each generated by Qwen3.5-9B and successfully completing its task. Keep retains the candidate reasoning step, while Skip removes only that step. Both conditions use the same current compressed history, including previously accepted removals, with all other reference actions, observations, and reasoning held fixed. At each target action position, we sample four actions under each condition and measure reference-action recovery. The sampled actions are neither executed in the environment nor inserted into subsequent histories.

The following tables report the average reference-action recovery rates when the candidate reasoning step is retained (Keep) or removed (Skip). The first table shows results when the reference-action likelihood decrease after removal is between 0 and 0.001 (0≤d≤0.0010\leq d\leq0.001). The second shows results when the decrease is greater than 0.001 (d>0.001d>0.001).

Likelihood decrease: 0≤d≤0.0010\leq d\leq0.001.

DatasetKeep recoverySkip recoveryRecovery loss (pp)
ScienceWorld97.97%97.76%0.21
WebShop99.33%99.32%0.01
AppWorld91.19%89.24%1.95
DeepSearch82.64%79.74%2.90

Likelihood decrease: d>0.001d>0.001.

DatasetKeep recoverySkip recoveryRecovery loss (pp)
ScienceWorld86.72%50.62%36.10
WebShop92.14%54.96%37.18
AppWorld81.51%44.37%37.14
DeepSearch60.99%49.09%11.90

Recovery loss denotes the difference between Keep and Skip recovery rates in percentage points. Across all four datasets, small reference-action likelihood decreases correspond to Skip recovery rates close to their Keep counterparts, with additional recovery losses of only 0.01–2.90 percentage points. Skip recovery reaches 97.76% on ScienceWorld, 99.32% on WebShop, 89.24% on AppWorld, and 79.74% on DeepSearch.

When the reference-action likelihood decrease is greater than 0.001, the recovery loss is much larger, ranging from 11.90 to 37.18 percentage points across the four datasets. These results show that larger likelihood decreases correspond to larger reductions in reference-action recovery, supporting the use of likelihood decrease to assess the effect of reasoning removal.

The analysis of recovery when the reference-action likelihood is already low continues in the next comment.

AuthorsOpenReview ↗

Response (3/3)

Q2: Reliability of reference-action likelihood decrease as a proxy for reasoning necessity — Part 2

Recovery when the reference-action likelihood is already low. We group the comparisons according to whether the reference-action likelihood under Keep before removal is at least −0.1-0.1 or below −0.1-0.1. We report the average reference-action recovery rates for each likelihood range and include only comparisons with 0≤d≤0.0010\leq d\leq0.001. Recovery rates are averaged within each trajectory and then equally across trajectories in each likelihood range. Both accepted and rejected candidates are included. A trajectory may contribute comparisons to both likelihood ranges.

DatasetReference-action likelihood before removalTrajectories / ComparisonsKeep recoverySkip recoveryRecovery loss (pp)
ScienceWorld≥−0.1\geq-0.194 / 6,13398.11%97.90%0.21
ScienceWorld<−0.1<-0.13 / 88.33%8.33%0.00
WebShop≥−0.1\geq-0.196 / 73199.36%99.34%0.02
WebShop<−0.1<-0.11 / 10.00%0.00%0.00
AppWorld≥−0.1\geq-0.1100 / 1,71491.54%89.66%1.88
AppWorld<−0.1<-0.12 / 312.50%12.50%0.00
DeepSearch≥−0.1\geq-0.148 / 13887.40%84.40%3.00
DeepSearch<−0.1<-0.115 / 4520.00%20.00%0.00

When the reference-action likelihood before removal is at least −0.1-0.1, Skip recovery ranges from 84.40% to 99.34%, with recovery losses of 0.02–3.00 percentage points relative to Keep. When the reference-action likelihood is below −0.1-0.1, recovery is already low under Keep, with no additional recovery loss observed in these samples. This illustrates the distinction: a small likelihood decrease can correspond to a small additional recovery loss, while the reference action may still be difficult to recover because its likelihood is already low.

We further examine how these low-likelihood comparisons affect the actual reasoning-removal decisions. A small likelihood decrease for one reference action does not by itself establish that the candidate reasoning step can be removed. LoGiC requires the likelihood decreases of the current and all subsequent affected reference actions to remain within the threshold. Across the four datasets, there are 1,014 comparisons in which the reference-action likelihood before removal is below −0.1-0.1. Of these, 57 comparisons (5.62%) have a likelihood decrease between 0 and 0.001. Since one candidate removal may affect multiple reference actions, these 57 comparisons correspond to 50 distinct candidate removals.

Of these 50 candidates, 47 (94%) are rejected by the complete LoGiC check because another affected reference action has a likelihood decrease exceeding 0.001. These results show that a small likelihood decrease for one low-likelihood reference action does not automatically lead to reasoning removal.

Ablation of cross-turn estimation. Table 2 of the revised manuscript compares full RACE with “w/o Cross-Turn Estimation” (current-action-only), which evaluates the likelihood decrease only for the current reference action. We report this comparison below. Each cell reports task success / average reasoning tokens per trajectory.

MethodScienceWorldWebShopAppWorldDeepSearch
Current-action-only72.40% / 80951.20% / 50143.93% / 1,80832.50% / 897
Cross-Turn Estimation (RACE)73.28% / 1,40353.40% / 53565.13% / 1,90035.00% / 822

Removing cross-turn estimation lowers task success across all four datasets, with the largest decrease of 21.20 percentage points on AppWorld. On DeepSearch, current-action-only checking also increases reasoning tokens from 822 to 897 while reducing success from 35.00% to 32.50%. These results suggest that checking only the current action can miss reasoning that supports subsequent decisions. They further support checking all affected reference actions when assessing whether a reasoning step can be removed.

These results show that small reference-action likelihood decreases consistently correspond to small additional recovery losses, the complete LoGiC check rejects most of the examined candidates containing low-likelihood actions with small likelihood decreases, and retaining cross-turn estimation improves task success across all four datasets.

We have added the paired reference-action recovery experiments and the analysis of recovery when the reference-action likelihood is already low to Appendix A.5 "Analysis of Likelihood Decrease and Action Recovery".

Reviewer_xoVYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces RACE, a framework that teaches LLM agents when to reason and when to act directly. The algorithm of RACE, LoGiC, progressively removes reasoning steps when their removal has little impact on the likelihood of current and future reference actions. These signals are then used in both SFT and RL to train agents to skip unnecessary reasoning. Experiments on four agent benchmarks show 32.3%-80.2% fewer reasoning tokens compared to standard SFT+GRPO, while maintaining comparable task performance.

Critical strengths
  • The paper introduces a practical way to train adaptive-reasoning agents without repeated rollouts. Using likelihood changes across future actions to guide reasoning removal is an interesting idea. Incorporating these signals into both SFT and on-policy RL provides useful empirical evidence.

  • RACE achieves substantial reductions in reasoning tokens while maintaining similar success rates across four benchmarks. Removing LoGiC from RL inflates reasoning sharply suggests that the method is useful in practice.

Critical weaknesses
  • The paper argues that earlier reasoning can support future actions, but the analysis does not fully distinguish this from cases where future actions are already predictable from observations alone. Also, LoGiC keeps future reasoning fixed when evaluating removal, so it is unclear whether those future reasoning steps could still be generated after earlier reasoning is removed. More importantly, the experiments don’t isolate the benefits of cross-turn verification and progressive deletion.

  • The positioning against the closest work and efficiency claims are insufficient. Related methods such as Agent-Omit [1] and CogRouter [2] are cited but not directly compared. CogRouter also uses reference-action likelihood, so the differences and advantages of RACE should be explained more clearly. In addition, although the paper emphasizes lightweight verification, but there are no cost measurements and no comparison with generation-based labels in the same pipeline. All later hidden states depend on r_t, so each candidate removal requires re-encoding the remaining trajectory. Therefore, it’s difficult to assess its efficiency advantage over generation-based approaches

[1] Agent-Omit: Adaptive Context Omission for Efficient LLM Agents. ICML 2026.

[2] Think Fast and Slow: Step-Level Cognitive Depth Adaptation for LLM Agents. ICML 2026.

Questions for authors
  • How can we distinguish the benefit of earlier reasoning from cases where future actions are predictable from observations alone?
  • Since future reasoning is kept fixed during LoGiC verification, have the authors evaluated whether removed reasoning affects subsequent free-running rollouts?
  • Could the authors compare cross-turn versus current-turn-only verification, progressive versus independent deletion, and separately evaluate reasoning masking and closure supervision?
  • In RL, which model computes the LoGiC likelihoods: π_θ, π_old, or a frozen reference? If it is the current policy, what prevents a feedback loop where skipping raises no-reasoning likelihoods, which then causes more removals to be accepted?
7 comments
AuthorsOpenReview ↗

Response (1/5)

We sincerely thank the reviewer for the constructive feedback and for recognizing the practical value of adaptive reasoning and the substantial reasoning-token reductions across four benchmarks. Your comments have helped us strengthen the experimental analysis and clarify the scope of our verification signal. Below, we address each concern individually.


W1 & Q1: The contribution of earlier reasoning should be distinguished from predictability based on observations.

Thank you for raising this important distinction. Our work primarily studies when an agent can act without generating additional reasoning, and Section 2.2 provides a preliminary analysis of this question from the perspective of cross-turn reasoning support. We agree that the original analysis alone does not fully separate the effect of retained earlier reasoning from the information already available in the action–observation history. In that analysis, we retain reasoning through a cutoff turn s, remove all subsequent reasoning, and examine whether later reference actions remain recoverable. Since the corresponding action and observation histories are also available, some of this recoverability could indeed come from those histories rather than from the retained reasoning itself. We therefore provide an additional controlled analysis that keeps the task and complete action–observation history fixed, varying only whether the earlier reasoning is retained. This allows us to more directly examine the additional contribution of earlier reasoning to subsequent action recovery.

Controlled comparison. To make this distinction explicit, we extend the analysis in Section 2.2 with an additional control. In the original setting, we retain all reasoning through cutoff turn s and remove reasoning after s. In the control setting, we additionally remove the reasoning through s, leaving no explicit reasoning in the context. Importantly, the task, complete action history, and complete observation history are identical between the two settings. We use the same frozen Qwen3.5-9B model. Thus, the only difference is whether the earlier reasoning is retained.

Action recovery protocol. Following Section 2.2 and Figure 1(a), for each target turn j > s, we sample K = 4 actions without generating additional reasoning. A turn satisfies the recovery condition only when all four sampled actions are canonically equivalent to the reference action (Recovery 4/4).

We define L as the number of consecutive subsequent turns, starting from s + 1, that satisfy this 4/4 recovery condition. The span ends at the first turn that fails the condition. We report P(L ≥ k): the fraction of evaluated cases in which the first k consecutive turns after the cutoff all satisfy the recovery condition.

Experimental results. The tables report P(L ≥ k) as percentages. Improvements are measured relative to removing all reasoning and expressed in percentage points (pp).

ScienceWorld

Reasoning settingk = 1k = 2k = 3k = 4
Remove all reasoning41%23%13%8%
Retain reasoning through cutoff s55%32%21%16%
Improvement+14 pp+9 pp+8 pp+8 pp

WebShop

Reasoning settingk = 1k = 2k = 3k = 4
Remove all reasoning18%7%0%0%
Retain reasoning through cutoff s29%18%7%7%
Improvement+11 pp+11 pp+7 pp+7 pp

Results and analysis. Retaining earlier reasoning increases the survival rate at every reported span, with improvements of 8–14 percentage points on ScienceWorld and 7–11 percentage points on WebShop. The benefit persists across multiple subsequent turns. For at least four consecutive turns, the survival rate increases from 8% to 16% on ScienceWorld and from 0% to 7% on WebShop.

Contribution beyond action–observation history. Some consecutive action recovery remains possible after removing all reasoning, showing that the task and action–observation history already provide useful information. However, retaining earlier reasoning consistently improves action recovery under the same observed histories and evaluation settings. Since the only controlled difference is the presence of earlier reasoning, these results provide controlled evidence that earlier reasoning contributes additional support to subsequent actions beyond what can be explained by the task and action–observation history alone.

We have added this controlled analysis to Appendix A.1 "Controlled Analysis of Cross-Turn Reasoning Support".

AuthorsOpenReview ↗

Response (2/5)

W1 & Q2: Fixed-trajectory verification requires validation through free-running continuations.

We thank the reviewer for highlighting this concern. To examine whether reasoning removal affects subsequent execution beyond fixed-trajectory verification, we conducted additional paired Keep/Skip free-running continuation experiments on all four datasets.

Experimental setup. We select 20 tasks per dataset that are successfully completed by Qwen3.5-9B and for which LoGiC identifies at least one reasoning turn as removable. For each task, we run 4 paired Keep/Skip seeds, yielding 80 paired comparisons per dataset. We select intervention points only from reasoning turns that LoGiC identifies as removable. At each intervention point, the Keep and Skip conditions start from the same task, interaction history, and environment state, differing only in whether the selected reasoning step is retained or removed. Both conditions then proceed with free-running interaction using Qwen3.5-9B: subsequent reasoning and actions are generated anew by the model, and observations are returned by the environment rather than replayed from the reference trajectory.

Experimental results. The table below reports the final task success rate after free-running continuation under the Keep and Skip conditions. The final column reports the difference in success rate, Keep − Skip, in percentage points (pp).

DatasetSuccess Rate (Keep)Success Rate (Skip)Keep − Skip (pp)
ScienceWorld91.25%90.00%1.25
WebShop100.00%97.50%2.50
AppWorld91.25%81.25%10.00
DeepSearch100.00%100.00%0.00

Analysis of free-running results. Across the four datasets, the Keep–Skip gaps range from 0.00 to 10.00 percentage points. On three datasets, the gap is at most 2.50 points, while the largest observed gap is 10.00 points. Overall, reasoning steps identified as removable by LoGiC generally preserve high task success under free-running continuation. This provides evidence beyond fixed-trajectory verification, since all subsequent reasoning, actions, and observations are generated through actual interaction.

We have added this paired free-rollout evaluation to Appendix A.2 "Free Rollout Analysis of Reasoning Removal in LoGiC".

AuthorsOpenReview ↗

Response (3/5)

W1 & Q3: The contributions of cross-turn verification, progressive deletion, reasoning masking, and closure supervision should be evaluated separately.

Thank you for this helpful suggestion. We have expanded the ablation study with separate evaluations of cross-turn estimation, progressive deletion, and closure supervision, complementing the existing ablation of reasoning masking.

Experimental setup. We evaluate the component variants using Qwen3.5-9B on all four benchmarks. For the cross-turn ablation, we replace the full criterion with a current-action-only criterion in both SFT and RL, while retaining progressive deletion, reasoning masking, closure supervision, and the same training hyperparameters. We additionally report variants without progressive deletion, without closure supervision, and without reasoning masking.

Experimental results. Each entry below reports task success rate (%) / average reasoning tokens per trajectory.

MethodScienceWorldWebShopAppWorldDeepSearch
Full RACE73.28 / 1,40353.40 / 53565.13 / 1,90035.00 / 822
w/o Cross-Turn Estimation72.40 / 80951.20 / 50143.93 / 1,80832.50 / 897
w/o Progressive Deletion72.46 / 1,09048.60 / 1,19854.02 / 2,43332.00 / 1,815
w/o Closure Supervision70.97 / 3,14949.20 / 3,69053.50 / 3,26034.75 / 3,985
w/o Reasoning Masking72.95 / 60946.00 / 12456.75 / 3,16132.00 / 2,196

Cross-Turn Verification vs. Current-Turn-Only Verification

VariantScienceWorldWebShopAppWorldDeepSearch
Current-Action-Only Verification72.40 / 80951.20 / 50143.93 / 1,80832.50 / 897
Cross-Turn Verification (Full LoGiC)73.28 / 1,40353.40 / 53565.13 / 1,90035.00 / 822

Progressive Deletion vs. Independent Deletion

VariantScienceWorldWebShopAppWorldDeepSearch
Independent Deletion72.46 / 1,09048.60 / 1,19854.02 / 2,43332.00 / 1,815
Progressive Deletion (Full LoGiC)73.28 / 1,40353.40 / 53565.13 / 1,90035.00 / 822

Reasoning Masking and Closure Supervision

VariantScienceWorldWebShopAppWorldDeepSearch
Full RACE (Ours)73.28 / 1,40353.40 / 53565.13 / 1,90035.00 / 822
w/o Reasoning Masking72.95 / 60946.00 / 12456.75 / 3,16132.00 / 2,196
w/o Closure Supervision70.97 / 3,14949.20 / 3,69053.50 / 3,26034.75 / 3,985

Cross-turn vs. current-turn-only verification. Replacing cross-turn verification with current-turn-only verification reduces task success on all four benchmarks. Full LoGiC improves success by 0.88, 2.20, 21.20, and 2.50 percentage points on ScienceWorld, WebShop, AppWorld, and DeepSearch, respectively. The corresponding reasoning-token changes are 809→1,403, 501→535, 1,808→1,900, and 897→822. These results show that checking only the immediate action can miss reasoning that remains useful for subsequent decisions, supporting the role of cross-turn verification.

Progressive vs. independent deletion. We further compare progressive deletion with independent deletion, where each reasoning-removal decision is evaluated without carrying previously accepted removals into the context. Progressive deletion improves task success on all four benchmarks by 0.82, 4.80, 11.11, and 3.00 percentage points, respectively. It also substantially reduces reasoning tokens on WebShop (1,198→535), AppWorld (2,433→1,900), and DeepSearch (1,815→822), while ScienceWorld shows a higher reasoning cost (1,090→1,403). These results indicate that reasoning-removal decisions are not independent: evaluating later candidates under the progressively compressed context better captures interactions among removal decisions.

Separate effects of reasoning masking and closure supervision. Removing reasoning masking lowers success on all four benchmarks, with particularly clear drops on WebShop (53.40%→46.00%), AppWorld (65.13%→56.75%), and DeepSearch (35.00%→32.00%). Removing closure supervision also lowers success on all four benchmarks and consistently increases reasoning-token use, from 1,403→3,149, 535→3,690, 1,900→3,260, and 822→3,985. These two ablations support complementary roles: reasoning masking prevents removable reasoning from being reinforced by the policy-gradient objective, while closure supervision directly encourages the policy to terminate reasoning at turns where LoGiC identifies additional reasoning as unnecessary.

We have added the separate cross-turn, progressive-deletion, and closure-supervision ablations to Table 2, with the corresponding analysis discussed in Section 4.3 "Quantitative Analysis".

Reviewer_szZgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks how an interactive agent can avoid unnecessary reasoning while preserving task success, using supervision that does not require repeated candidate generation. LoGiC progressively removes reasoning turns while checking the likelihood of current and subsequent reference actions. These signals guide SFT, reasoning-token masking, and reasoning-closure supervision during RL.

Critical strengths

None

Critical weaknesses

The practical objective is clear and supported by the task results. My main reservation is that the benefit of checking subsequent actions, relative to its additional training cost, is not yet isolated. The existing ablations support the LoGiC training approach, but none keeps the rest of the method fixed and compares checking only the current action with checking all subsequent actions. Appendix D gives training settings, and Section 2.3 explains the use of batched teacher-forced scoring, but neither reports its measured overhead. A matched comparison would clarify how much the specifically cross-turn part contributes, beyond the substantial token savings already demonstrated.

Questions for authors
  1. How would a current-action-only LoGiC variant compare with the full version under the same data and training settings, while retaining progressive deletion, masking, and closure supervision? I would be interested in both task success and reasoning-token use.
  2. What fraction of SFT data-synthesis and RL training time is spent on LoGiC scoring, and how does this change between current-action-only and all-future-action checks? A measured breakdown would help assess the proposed computational tradeoff.
3 comments
AuthorsOpenReview ↗

Response (1/2)

Thanks for your helpful comments. We appreciate your recognition of the practical objective and the demonstrated reasoning-token savings. Below, we address your concerns point by point.


W1&Q1: Controlled Comparison of Current-Action-Only and Full LoGiC

Thank you for suggesting this controlled comparison. We agree that it helps isolate the contribution of checking subsequent actions from that of the other training components.

Current-action-only ablation. Following your suggestion, we have added a variant that checks only the likelihood decrease of the current reference action when considering a reasoning removal, whereas Full LoGiC checks the current and all subsequent reference actions. We apply the respective criteria in both the SFT and RL stages, while retaining progressive deletion, reasoning masking, closure supervision, and the same training hyperparameters.

Results and analysis. The table below reports the final results after RL. Success denotes task success rate (%), and Avg. Tok. denotes the average number of reasoning tokens per trajectory.

MethodScienceWorldWebShopAppWorldDeepSearch
Success / Avg. Tok.Success / Avg. Tok.Success / Avg. Tok.Success / Avg. Tok.
Current-action-only LoGiC72.40 / 80951.20 / 50143.93 / 1,80832.50 / 897
Full LoGiC (Ours)73.28 / 1,40353.40 / 53565.13 / 1,90035.00 / 822

Full LoGiC achieves higher task success on all four benchmarks. On WebShop, success improves by 2.20 percentage points with a modest increase in reasoning tokens from 501 to 535. On DeepSearch, success improves by 2.50 points while reasoning tokens decrease from 897 to 822. AppWorld shows the largest improvement, at 21.20 points, with approximately 5.1% more reasoning tokens. On ScienceWorld, the success gain is smaller (+0.88 points) and comes with a larger increase in reasoning tokens, from 809 to 1,403.

These results suggest that preserving the current action’s likelihood alone may be insufficient to identify dispensable reasoning: reasoning that appears unnecessary for the immediate action may still support subsequent decisions. The improvements with modest or reduced token use on WebShop, AppWorld, and DeepSearch further suggest that the benefit is not explained solely by longer reasoning, while ScienceWorld highlights an environment-dependent tradeoff between task success and reasoning-token use.

We have added this comparison as "w/o Cross-Turn Estimation" in Table 2, with the corresponding analysis in Section 4.3 "Quantitative Analysis".

AuthorsOpenReview ↗

Response (2/2)

W2&Q2: Proportion of SFT Data-Synthesis and RL Training Time Spent on LoGiC Scoring

Thank you for this helpful suggestion. Below, we report the fraction of SFT data-synthesis and RL training time spent on LoGiC scoring, comparing current-action-only and full cross-turn checks.

Scoring time breakdown. The following table summarizes the proportion of time spent on LoGiC scoring in each stage. For each variant, the SFT and RL percentages are calculated relative to the total SFT data-synthesis time and total RL training time, respectively.

StageVariantScienceWorldWebShopAppWorldDeepSearch
SFT data synthesisCurrent-action-only3.32%4.50%37.55%2.46%
SFT data synthesisFull LoGiC (Ours)1.58%3.92%37.24%6.08%
RL trainingCurrent-action-only12.63%6.66%10.91%16.70%
RL trainingFull LoGiC (Ours)17.31%14.54%16.93%15.51%

Comparison between the two variants. During RL, LoGiC scoring accounts for 14.54%–17.31% of total training time in the full variant. Compared with current-action-only checking, this proportion increases by 4.68, 7.88, and 6.02 percentage points on ScienceWorld, WebShop, and AppWorld, respectively, and decreases by 1.19 percentage points on DeepSearch. During SFT data synthesis, the proportion of time spent on LoGiC scoring varies more across tasks, with Full LoGiC ranging from 1.58% on ScienceWorld to 37.24% on AppWorld.

Interpreting the timing differences. During SFT data synthesis, Full LoGiC can obtain and reuse multiple action scores from a single teacher-forced pass, so checking more actions does not necessarily increase scoring time proportionally. During RL, both variants use the same scoring backend, but their scoring workloads differ.

We have added the scoring-time breakdown for Full LoGiC to Appendix A.4 "Analysis of LoGiC’s Proportion of Training Time".

AuthorsOpenReview ↗

A Kind Reminder for Reading the Response

Dear Reviewer szZg,

Thank you again for your time and consideration. We sincerely appreciate your constructive feedback, which has helped us strengthen our paper.

As the author–reviewer discussion period will end tomorrow, we would be grateful if you could take a moment to review our responses and revised manuscript. We hope the additional experiments and clarifications have addressed your main concerns, and we would greatly appreciate any further feedback you may have.

Thank you again for your time and consideration.

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response for RACE

We sincerely thank all reviewers for their careful reading and constructive feedback. We appreciate the recognition of RACE's practical motivation, its substantial reasoning-token savings, and its promising performance-efficiency trade-off.

Following the reviewers' suggestions, we have provided point-by-point responses and revised the manuscript. The main updates are summarized below. Key additions are highlighted in red in the revised manuscript.

Focus area and reviewersSupplementary /Revised SectionMain additions and findings
1. Contribution of cross-turn estimation — szZg, xoVYTable 2; Section 4.3 "Quantitative Analysis"We add a current-action-only variant in both SFT and RL, keeping the other components and training settings fixed. Compared with this variant, Full LoGiC achieves higher success on all four benchmarks. On DeepSearch, success increases from 32.50% to 35.00%, while reasoning tokens decrease from 897 to 822.
2. Separate component contributions — xoVY, oeSoTable 2; Section 4.3 "Quantitative Analysis"We add separate ablations of progressive deletion and closure supervision. Removing either component lowers task success across all four benchmarks, while removing closure supervision also consistently increases reasoning-token use.
3. Earlier reasoning beyond observation history — xoVYAppendix A.1 "Controlled Analysis of Cross-Turn Reasoning Support"We supplement the preliminary analysis with a control that removes all reasoning while preserving the same task and complete action-observation history. This comparison helps distinguish the contribution of earlier reasoning from action predictability based on the interaction history alone. Compared with this control, retaining earlier reasoning improves consecutive reference-action recovery by 8–14 pp on ScienceWorld and 7–11 pp on WebShop.
4. Free-rollout evaluation after removal — xoVYAppendix A.2 "Free Rollout Analysis of Reasoning Removal in LoGiC"We conduct free-rollout experiments comparing task success with and without a reasoning step identified as removable by LoGiC, allowing subsequent reasoning and actions to be generated freely through environment interaction. Compared with retaining the reasoning step, removing it decreases success by at most 2.50 pp on ScienceWorld, WebShop, and DeepSearch.
5. Efficiency Comparison with Generation-Based Verification — xoVYAppendix A.3 "Runtime Comparison Between LoGiC and Generation-Based Verification"We compare likelihood scoring with generation-based reference-action verification using four samples per action within the same progressive removal pipeline. Generation-based verification takes 7.16–32.07 times as long as LoGiC.
6. Scoring cost during training — szZg, oeSoAppendix A.4 "Analysis of LoGiC’s Proportion of Training Time"We measure the proportion of total time spent on LoGiC scoring in each stage: 1.58%–6.08% of SFT data-synthesis time on three datasets, 37.24% on AppWorld, and 14.54%–17.31% of RL training time. The response to szZg, Response (2/2), W2&Q2, additionally compares current-action-only and full scoring.
7. Reliability of likelihood decrease — oeSoAppendix A.5 "Analysis of Likelihood Decrease and Action Recovery"We conduct paired action-recovery experiments and examine reference actions with low likelihood before reasoning removal. Likelihood decreases between 0 and 0.001 correspond to 0.01–2.90 pp average recovery loss, compared with 11.90–37.18 pp above 0.001. The complete cross-turn check rejects 47 of 50 examined candidate removals involving low-likelihood actions with small decreases. These results support using likelihood decreases of the current and subsequent reference actions to assess the additional recovery loss associated with reasoning removal.
8. Positioning against related methods — xoVYSection 5 "Related Work"We clarify how RACE differs from Agent-Omit and CogRouter, focusing on its use of likelihood changes for the current and subsequent reference actions under progressive removal and how these signals guide SFT, reasoning masking, and closure supervision.

We have also moved the detailed scaling analysis from the original appendix to Section 4.3 of the main text, with the full scaling results presented in Figure 5. Our responses also clarify the use of the pre-update policy πold\pi_{\mathrm{old}} for LoGiC scoring during RL. Figure 4 shows 32.3%–80.2% fewer reasoning tokens than standard SFT+GRPO, with success changes of −1.27 to +1.40 pp.

We hope these additional results and clarifications help the reviewers assess the contribution of cross-turn estimation, its computational cost, and the empirical reliability of the resulting supervision. We thank the reviewers again for helping us improve the paper and welcome further discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.