acceptodds
Under review as a conference paper at ICLR 2027

Co-Evolving Acting and Forecasting for Language Agents

Abstract

Training language agents is bottlenecked by the high cost of long-horizon rollouts, yet standard agentic reinforcement learning (RL) learns only from a terminal scalar reward, leaving the supervision in intermediate steps largely unused. Prior work exploits this supervision by predicting next observations, either with a separate world model or as an auxiliary loss on the policy. These objectives, however, are ill-suited to agentic settings: observations such as HTML pages and terminal logs are long and noisy, and single-step prediction mismatches the long-horizon RL objective. We propose CoPE, a simple and effective method that adds multi-step action forecasting as an auxiliary loss. On successful on-policy trajectories, CoPE trains the policy backbone on an action plan containing the current action and K future actions predicted from the earlier history, without the intervening observations. Working in the action space sidesteps environmental noise, and forecasting multiple steps ahead better matches the horizon of policy optimization. We show that this loss optimizes success-conditioned action marginals, so acting and forecasting can be learned jointly within a single shared backbone. Across five benchmarks—ALFWorld, ScienceWorld, WebShop, τ²-bench, and AppWorld—CoPE outperforms GRPO and observation-prediction baselines on aggregate metrics in simulated environments and improves the overall metrics on two realistic applications despite a negative Airline result. It is also up to 1.9× and 2.0× more sample-efficient than ECHO and GRPO, respectively, and requires neither a separate model nor a pretraining stage. A separated-forecaster analysis further supports implicit temporal ensembling through the shared backbone.

Primary area:
reinforcement learning
Keywords:
language agent, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_7oYqOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes COPE, which augments GRPO with an auxiliary action-forecasting objective. Using successful on-policy trajectories, a shared policy backbone learns to predict the current action and several future actions without the intervening observations. Experiments on five agent benchmarks show improvements over the authors’ GRPO baseline, although performance decreases on the Airline domain of τ²-bench.

Critical strengths
  1. COPE reuses collected trajectories without requiring expert demonstrations, a separate world model, or additional inference-time computation.
  2. Success rates improve over GRPO from 64.5% to 77.0% on WebShop, from 64.5% to 72.5% on ScienceWorld, and from 41.67% to 55.36% on AppWorld normal (Tables 1 and 3). Improvements with Qwen3-8B provide additional evidence beyond one backbone.
  3. The experiments cover household tasks, scientific experiments, shopping, customer service, and API-based coding. Improvements on AppWorld’s scenario-held-out challenge split and with Qwen3-8B provide evidence of applicability beyond a single task distribution or backbone
Critical weaknesses
  1. Variation across independent training runs and confidence intervals are not reported. This limits confidence in smaller gains, particularly the 1-point improvement over ECHO on ALFWorld and the approximately 1.2-point overall improvement over GRPO on τ²-bench.

  2. The training configurations are incomplete. Table 9 does not include COPE, and complete settings for τ²-bench and AppWorld are not provided. More detail on the update schedule, loss reduction, and checkpoint selection would help assess reproducibility.

  3. Table 5 provides useful K=0K=0 controls, but increasing KK also increases the number of supervised targets and may change the effective auxiliary-loss weight. Clarifying normalization and comparing with a comparably tuned K=0K=0 baseline would better identify the benefit of future-action prediction.

Questions for authors
  1. Does COPE retain its advantage over the K=0K=0 baseline when auxiliary supervision budgets are matched and loss weights are tuned comparably?
  2. How stable are the gains across independent training seeds, particularly on ALFWorld and τ²-bench? Can you provide complete COPE configurations and held-out learning curves against cumulative environment interactions or wall-clock time?
  3. Does the Airline regression persist when tasks requiring no substantive tool actions are evaluated separately or excluded from forecasting supervision?
2 comments

We sincerely thank the reviewer for the careful reading and the constructive questions. We respond to each weakness and question below. We have highlighted all revisions in blue in the updated manuscript, and we would be glad to address any remaining concerns during the discussion period.

[Statistical reliability] The results in Tables 1–3 of the original paper are from a single run. We have now repeated the main experiments with independent seeds and report mean ± std and 95% confidence intervals (Student’s t) below. Due to limited time and API budget, the additional runs cover ALFWorld, ScienceWorld, and the retail and telecom domains of τ²-bench (40 training steps).

Benchmark (Success %)Method# RunsMean ± Std95% CI
AlfWorldGRPO491.38 ± 1.93[88.30, 94.45]
AlfWorldECHO493.25 ± 1.66[90.61, 95.89]
AlfWorldCoPE495.50 ± 0.82[94.20, 96.80]
ScienceWorldGRPO364.33 ± 0.76[62.44, 66.23]
ScienceWorldECHO367.83 ± 0.58[66.40, 69.27]
ScienceWorldCoPE374.63 ± 2.38[68.72, 80.55]
τ²-bench RetailGRPO312.63 ± 4.27[2.03, 23.24]
τ²-bench RetailCoPE323.77 ± 5.12[11.05, 36.48]
τ²-bench TelecomGRPO329.80 ± 5.04[17.29, 42.31]
τ²-bench TelecomCoPE340.63 ± 1.85[36.04, 45.23]

CoPE has the highest mean on every benchmark. On ScienceWorld, the gain over GRPO is clearly separated (CoPE’s lower bound of 68.72 exceeds GRPO’s upper bound of 66.23). On ALFWorld, CoPE outperforms ECHO by 2.3 points on average with lower variance (95.5 ± 0.8 vs. 93.3 ± 1.7). The gap is necessarily small, as ALFWorld is near saturation: GPT-5.5 already reaches 93.5% without training, and CoPE fails only 9 of the 200 test tasks. On τ²-bench the intervals are wide and overlap because of the small number of runs, so we regard these gains (+11.1 on retail, +10.8 on telecom) as consistent in direction but not yet statistically conclusive. Airline is analyzed separately below.

[Sample efficiency under matched budgets] We added Table 4 (Lines 378–396) in the revised version, which reports held-out test success on ScienceWorld at matched cumulative environment interactions, generated tokens, and GPU (wall-clock) time, averaged over two seeds. CoPE leads GRPO throughout training under all three budgets: by +13 to +24 points at matched environment interactions, +11 to +30 at matched generated tokens, and +7 to +17 at matched GPU hours. Under matched training tokens or FLOPs, CoPE leads only in early training, since the forecasting pass trains on every turn of each successful trajectory; this pass accounts for about 70% of CoPE’s FLOPs but only about 40% of its GPU time.

[Analysis on forecasting horizon] We present the test performance of different horizons KK on SciWorld. Among different horizons, performance peaks at an intermediate value, with K=2K=2 achieving the highest score. This reflects a trade-off between foresight and target reliability: too short a horizon provides little multi-step supervision and reaches a slightly lower final score, whereas too long a horizon adds distant targets that are increasingly unpredictable from the current history, which slows early learning without improving the final score. The analysis is also added to the ablation study in the revised manuscript. We apply this setting across all environments for simplicity.

Horizon KSuccess Rate
172.0
275.8
372.8
468.0

[Training configuration details] The complete configurations are provided in the revised manuscript (marked in blue in Appendix C.2). Specifically:

  • CoPE settings. CoPE uses the same settings as GRPO (Table 9 in the original submission, now Table 10), with forecasting horizon K=2K=2 and forecasting coefficient λ=0.01\lambda=0.01 in all environments; this is now stated in the table caption.
  • τ²-bench and AppWorld. Their complete settings are listed in Tables 11 and 12, shared by GRPO (ours) and CoPE.
  • Update schedule. At every training step, the GRPO and forecasting losses are computed on the same rollouts collected in that step and minimized jointly as LGRPO+λLAF\mathcal{L}_{\mathrm{GRPO}} + \lambda \mathcal{L}_{\mathrm{AF}}; all other settings, including PPO epochs and mini-batch sizes, follow GRPO.
  • Loss reduction. The forecasting loss is averaged over its target tokens and over the successful rollouts of each group, so its scale changes neither with KK nor with the number of successes.
  • Checkpoint selection. For all methods, we evaluate the final checkpoint after the training steps listed in the tables.

[K=0 baseline and loss normalization] For each constructed sample of CoPE, the loss is normalized on the token scale. Therefore, the effective auxiliary loss weight is the same across different horizon length. The same clarification is added in our revised version.

As stated in Sec. 8 (Q2), "augmenting GRPO with a success-filtered SFT loss is therefore approximately equivalent to CoPE with K=0K=0", and Table 6 confirms this: without an invalid-action filter, K=0K=0 (66.5) performs on par with GRPO + success-filtered SFT (65.5), and both stay close to GRPO (64.5). K=0K=0 narrows the gap only after we add an invalid-action filter (69.5; Fig. 5), a task-specific heuristic that requires knowing which actions are invalid, and even then it remains below CoPE (72.5), which needs no such filtering. Further tuning the K=0K=0 baseline with such heuristics would therefore not be a like-for-like comparison, and the remaining gap reflects the benefit of future-action prediction. If the reviewer considers it necessary, we are happy to add a K=0K=0 baseline with a separately tuned λ\lambda.

[Analysis of the Airline regression on τ²-bench] We traced the drop to Airline’s task design. 38% of Airline tasks (19/50; 13 of the 30 training tasks) must be refused and leave the database unchanged, compared with only 4.4% in Retail. We therefore evaluated the Airline test set by task type (6 refusal and 14 action tasks, 4 trials each):

MethodRefusal tasksAction tasks
GRPO18/243/56
CoPE13/240/56

Both methods rarely solve action tasks, so the gap comes mainly from refusal tasks. This reveals a limitation of action forecasting. In a successful refusal, the agent looks up the reservation, checks the flight, finds that the policy forbids the request, and only then declines. Most forecasting targets in such a trajectory are lookup steps, while the single declining step carries little weight in the loss. As a result, forecasting teaches the lookup sequence rather than restraint and, together with the action-dominated trajectories, reinforces an “investigate, then act” pattern: on refusal tasks, the agent reliably reaches the point of acting, and then acts.

We have not yet run the variant that excludes refusal tasks from forecasting supervision. Given the explanation above, we expect it to help only partially, since the same pattern is still learned from action tasks; supervising explicit refusals is a more direct remedy, which we leave to future work. We revised Sec. 8 (Q3) accordingly and added the per-type evaluation and a case study with look-alike task pairs to App. C.5 (Table 13).

Reviewer_2VH3OpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CoPE (Co-Evolving Acting and Forecasting), a framework for improving the sample efficiency of reinforcement learning for language agents. Unlike previous methods that rely on single-step observation prediction, CoPE introduces multi-step action forecasting to provide additional supervision from successful trajectories while avoiding noisy environment observations. The method jointly optimizes GRPO and an auxiliary action forecasting loss using a shared LLM backbone, allowing acting and forecasting to improve together. The authors provide theoretical analysis of success-conditioned action prediction and implicit temporal ensembling. Experiments across five agentic benchmarks demonstrate improved overall performance and sample efficiency compared with GRPO and observation-prediction baselines, with moderate computational overhead.

Critical strengths
  1. Important problem and effective solution. This paper addresses a longstanding challenge in agentic reinforcement learning: extracting useful intermediate supervision from sparse terminal rewards. Unlike prior observation-prediction approaches such as RWML and ECHO, CoPE introduces multi-step action forecasting to provide more effective supervision while avoiding noisy environment observations.

  2. Interesting and well-motivated idea. Inspired by multi-horizon action prediction in robotics, such as Diffusion Policy and ACT, the proposed approach offers an interesting adaptation of action forecasting to language agents. The idea is conceptually simple yet effective, and its application to language-agent RL is relatively underexplored.

  3. Well-justified co-evolution mechanism. The co-evolution of GRPO-based policy optimization and action forecasting (AF) is clearly formulated and theoretically motivated. The success-conditioned action marginal analysis and implicit temporal ensembling interpretation provide useful insights, supported by empirical evidence.

  4. Comprehensive empirical evaluation. The paper evaluates CoPE across five diverse benchmarks, complemented by three focused discussion questions and four ablation studies. The results generally support the effectiveness of the proposed approach,

Critical weaknesses

The experiment parts have some flaws:

  1. Insufficient analysis of τ²-bench results. The performance drop on the Airline domain is not well explained. A concrete case study in the appendix would help clarify the failure cases and limitations of the method.

  2. Limited analysis of the forecasting horizon. Table 5 mainly compares K=0 and K=2, while Figure 6 explores different horizons. However, the explanation of why K=2 outperforms larger horizons, such as K=4, is insufficient. Intuitively, increasing K could improve performance up to a certain threshold. For example, Imagine-then-Plan: Agent Learning from Adaptive Lookahead with World Models (EMNLP 2026) identifies K=3 as an effective lookahead horizon. A more detailed analysis of the trade-off between forecasting horizon, computational overhead, and performance would be valuable.

  3. Unclear advantages over test-time ensembling. Table 6 shows that explicit ensembling achieves performance comparable to CoPE. This raises questions about the benefits of joint training, as a simpler test-time approach may offer similar performance with greater flexibility and lower training costs.

Questions for authors

I believe this paper deserves a clear acceptance in the end, so I assign it a rating of 4. However, I have several concerns about the experimental analysis, so I currently rate the soundness as 2. I would be willing to raise my soundness rating if the issues raised in the Critical Weaknesses are adequately addressed. Overall, I consider this a good paper.

1 comment

We sincerely thank the reviewer for the careful reading and the constructive questions. We respond to each weakness and question below. We have highlighted all revisions in blue in the updated manuscript, and we would be glad to address any remaining concerns during the discussion period.

[Analysis of the Airline regression on τ²-bench] We traced the drop to Airline’s task design. 38% of Airline tasks (19/50; 13 of the 30 training tasks) must be refused and leave the database unchanged, compared with only 4.4% in Retail. We therefore evaluated the Airline test set by task type (6 refusal and 14 action tasks, 4 trials each):

MethodRefusal tasksAction tasks
GRPO18/243/56
CoPE13/240/56

Both methods rarely solve action tasks, so the gap comes mainly from refusal tasks. This reveals a limitation of action forecasting. In a successful refusal, the agent looks up the reservation, checks the flight, finds that the policy forbids the request, and only then declines. Most forecasting targets in such a trajectory are lookup steps, while the single declining step carries little weight in the loss. As a result, forecasting teaches the lookup sequence rather than restraint and, together with the action-dominated trajectories, reinforces an “investigate, then act” pattern: on refusal tasks, the agent reliably reaches the point of acting, and then acts. We revised Sec. 8 (Q3) accordingly and added the per-type evaluation and a case study with look-alike task pairs to App. C.5 (Table 13, Fig. 9).

[Analysis on forecasting horizon] We present the test performance of different horizons KK on SciWorld. Among different horizons, performance peaks at an intermediate value, with K=2K=2 achieving the highest score. This reflects a trade-off between foresight and target reliability: too short a horizon provides little multi-step supervision and reaches a slightly lower final score, whereas too long a horizon adds distant targets that are increasingly unpredictable from the current history, which slows early learning without improving the final score. We also note that the auxiliary computation is driven mainly by the number of forecasting targets, which grows with the success rate during training, rather than by the value of KK. The analysis is added to the ablation study in the revised manuscript, and we apply this setting across all environments for simplicity.

Horizon KSuccess Rate
172.0
275.8
372.8
468.0

[Comparison with Imagine-then-Plan] Our trend agrees with Imagine-then-Plan[1] (ITP), which also finds that success peaks at a moderate lookahead horizon (K=3) and declines as longer imagined rollouts become less accurate. The slightly shorter optimum in our case is expected: ITP rolls out intermediate states with a learned world model, whereas CoPE forecasts future actions directly from the current history without intervening observations, so distant targets become unreliable sooner. In our sweep, K=3 (72.8) remains competitive, while K=4 (68.0) is clearly worse.

[Comparison with test-time ensembling] Explicit ensembling catches up with CoPE only at η=0.75\eta=0.75, which requires training a separate LoRA forecaster on the agent's rollouts, so its training cost is not lower; at inference, it further samples 8 candidates and scores each under KK delayed histories. As clarified in Sec. 8 (Q1), only the η=0\eta=0 row is pure test-time scaling: there, GRPO reaches only 60.0 and 66.0 (best of 8), below the 69.0 that CoPE obtains from a single sample (Table 7), and CoPE adds no inference cost (Sec. 5). Note that in this analysis, CoPE is deployed as the base model with a LoRA adapter.

[1] Liu et al. Imagine-then-Plan: Agent Learning from Adaptive Lookahead with World Models. arXiv 2026

Reviewer_W4noOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

COPE adds success-conditioned prediction of the current and future actions to GRPO, using a shared policy backbone. Across five benchmarks, it reports improved aggregate performance and sample efficiency, with moderate training overhead.

Critical strengths
  1. COPE provides a simple auxiliary objective that avoids predicting noisy observations and requires no separate world model.
  2. Evaluation spans five benchmarks and includes useful ablations, including comparisons with observation-prediction methods.
Critical weaknesses
  1. The paper does not report variance across independent runs, making it hard to assess the reliability of the gains.
  2. Sample-efficiency claims would be stronger with comparisons under matched interaction, compute, or token budgets.
Questions for authors
  1. How many independent runs support the main results, and can you report variance or confidence intervals?
  2. Can you clarify the exact forecast-plan length and report efficiency under matched interaction or compute budgets?
  3. How does COPE differ from EnvRL (arXiv:2606.17680) ? Please clarify the conceptual distinction.
1 comment

We sincerely thank the reviewer for the careful reading and the constructive questions. We respond to each weakness and question below. We have highlighted all revisions in blue in the updated manuscript, and we would be glad to address any remaining concerns during the discussion period.

[Statistical reliability] The results in Tables 1–3 of the original paper are from a single run. We have now repeated the main experiments with independent seeds and report mean ± std and 95% confidence intervals (Student’s t) below. Due to limited time and API budget, the additional runs cover ALFWorld, ScienceWorld, and the retail and telecom domains of τ²-bench (40 training steps).

Benchmark (Success %)Method# RunsMean ± Std95% CI
AlfWorldGRPO491.38 ± 1.93[88.30, 94.45]
AlfWorldECHO493.25 ± 1.66[90.61, 95.89]
AlfWorldCoPE495.50 ± 0.82[94.20, 96.80]
ScienceWorldGRPO364.33 ± 0.76[62.44, 66.23]
ScienceWorldECHO367.83 ± 0.58[66.40, 69.27]
ScienceWorldCoPE374.63 ± 2.38[68.72, 80.55]
τ²-bench RetailGRPO312.63 ± 4.27[2.03, 23.24]
τ²-bench RetailCoPE323.77 ± 5.12[11.05, 36.48]
τ²-bench TelecomGRPO329.80 ± 5.04[17.29, 42.31]
τ²-bench TelecomCoPE340.63 ± 1.85[36.04, 45.23]

CoPE has the highest mean on every benchmark. On ScienceWorld the gain over GRPO is clearly separated (CoPE’s lower bound of 68.72 exceeds GRPO’s upper bound of 66.23). On ALFWorld, CoPE outperforms ECHO by 2.3 points on average with lower variance (95.5 ± 0.8 vs. 93.3 ± 1.7). The gap is necessarily small, as ALFWorld is near saturation: GPT-5.5 already reaches 93.5% without training, and CoPE fails only 9 of the 200 test tasks. On τ²-bench the intervals are wide and overlap because of the small number of runs, so we regard these gains (+11.1 on retail, +10.8 on telecom) as consistent in direction but not yet statistically conclusive.

[Sample efficiency under matched budgets] We added Table 4 (Lines 378–396) in the revised version, which reports held-out test success on ScienceWorld at matched cumulative environment interactions, generated tokens, and GPU (wall-clock) time, averaged over two seeds. CoPE leads GRPO throughout training under all three budgets: by +13 to +24 points at matched environment interactions, +11 to +30 at matched generated tokens, and +7 to +17 at matched GPU hours. Under matched training tokens or FLOPs, CoPE leads only in early training, since the forecasting pass trains on every turn of each successful trajectory; this pass accounts for about 70% of CoPE’s FLOPs but only about 40% of its GPU time.

[Forecasting horizon clarification] KK denotes the maximum future-action offset, so the forecast plan contains up to K+1K+1 actions: the current action ata_t and KK future actions at+1:t+Ka_{t+1:t+K}. With the default K=2K=2, the targets are at,at+1,at+2a_t,a_{t+1},a_{t+2}, truncated at the trajectory boundary. All are predicted from hth_t without intervening future observations. We have revised our manuscript with clarification in Appendix C.2 (Lines 942-951). We also note that the auxiliary computation is driven mainly by the number of forecasting targets, which grows with the success rate during training, rather than by the value of KK.

[Difference with EnvRL] EnvRL[1] shares ECHO’s[2] observation-prediction objective and additionally learns inverse dynamics from all trajectories, recovering actions from observed state transitions. CoPE instead forecasts the current and future actions from the current history, without intervening future observations, using successful on-policy trajectories. Both methods share the general auxiliary-training framework; the conceptual distinction lies in CoPE’s success-conditioned, multi-step action forecasting, as opposed to EnvRL’s modeling of environment dynamics.

[1] Wang et al. EnvRL: Learn from Environment Dynamics in Agentic Reinforcement Learning. 2026.

[2] Shrivastava et al. ECHO: Terminal Agents Learn World Models for Free. 2026.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Comment

General Comments

We appreciate the reviewers' efforts in evaluating our paper. All three reviewers recommend acceptance. In the table below, quoted items in bold are concerns for which we provide additional experiments or clarifications, while plain items reflect acknowledged strengths. All revisions are marked in blue in the updated version.

DimensionReviewer W4noReviewer 2VH3Reviewer 7oYqAction/Summary
Novelty & Significance"Simple auxiliary objective... requires no separate world model""Important problem and effective solution... relatively underexplored""Without requiring expert demonstrations, a separate world model, or additional inference-time computation"Summary: All reviewers recognize CoPE as simple and effective.
Evaluation & Presentation"Evaluation spans five benchmarks and includes useful ablations""Comprehensive empirical evaluation""Evidence of applicability beyond a single task distribution or backbone”Summary: All reviewers find the evaluation broad and the paper clear.
Statistical reliability"Does not report variance across independent runs"NA"Confidence intervals are not reported"Rebuttal for W4no & 7oYq: We report mean ± std and 95% CIs over 3–4 independent runs on ALFWorld, ScienceWorld, and τ²-bench Retail/Telecom.
Sample efficiency under matched budgets"Comparisons under matched interaction, compute, or token budgets"NA"Learning curves against... environment interactions or wall-clock time"Rebuttal for W4no & 7oYq: We add new Table 4 (Lines 378-396) to show that CoPE leads GRPO at matched environment interactions, generated tokens, and GPU hours.
Horizon of action forecasting loss"Clarify the exact forecast-plan length""why K=2 outperforms larger horizons"; "test-time ensembling""May change the effective auxiliary-loss weight"Rebuttal for W4no, 2VH3 & 7oYq: We clarify that the plan contains up to K+1 actions with normalization across K and add a horizon sweep (K=1–4) showing that performance peaks at K=2 and degrades at K=4. Test-time scaling alone does not close the gap to CoPE.
τ²-bench analysisNA"The performance drop on the Airline domain is not well explained""Does the Airline regression persist when tasks... are evaluated separately?"Rebuttal for 2VH3 & 7oYq: We add a per-type evaluation on Airline, which shows that the gap comes mainly from refusal tasks (38% of Airline tasks vs. 4.4% in Retail). We also add a case study in App. C.5.

We have also addressed the other points in the individual responses and the revised manuscript: (1) the comparison with a K=0K=0 baseline; (2) the complete CoPE, τ²-bench, and AppWorld configurations, including the update schedule, loss reduction, and checkpoint selection (App. C.2); and (3) the comparison between CoPE and EnvRL and Imagine-then-Plan.

Dear Reviewers and Area Chair,

Thank you for your time and effort. We have responded to all three reviews and uploaded a revised manuscript (changes highlighted in blue). With the discussion deadline (November 18) approaching, we would greatly appreciate it if the reviewers could check our responses and share any remaining concerns.

We summarize our key updates on experiments and clarification below:

  1. Exps on statistical reliability: We added 3–4 independent runs with mean ± std and 95% CIs across four benchmarks.
  2. Exps on sample efficiency: We added matched-budget experiments (Table 4), showing consistent gains over GRPO in environment interactions, generated tokens, and GPU hours.
  3. Exps on forecasting horizon: We conducted a K=1–4 sweep, identifying K=2 as optimal, and clarified plan length and loss normalization.
  4. Exps on τ²-bench airline category: We added per-type analysis and a case study, identifying refusal tasks as the main source of the performance gap.
  5. Additional clarifications: We clarified explicit ensembling vs. test-time scaling, the K=0 baseline, experimental configurations, and differences from EnvRL.

We would be happy to address any further questions before the discussion period ends.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.