acceptodds
Under review as a conference paper at ICLR 2027

Reshaping Tool Calling Policies of Time Series Agents via Reinforced Evaluation

Abstract

Time series reasoning, which involves understanding temporal patterns to answer complex questions, plays a crucial role in a variety of domains such as healthcare, energy, and climate. Modern approaches have tried using large language model (LLM) based agents with analytical tools to reason over time series. However, these methods would always execute tool calls directly without evaluation, which inevitably leads to many low-value tools being executed and uninformative results being generated, thus hindering both the reasoning efficiency and accuracy. One seemingly workable solution inspired by LLM and agent studies is to introduce post-training to guide agents for wiser tool calling. However, tuning these massive parameters of LLM-based agents further imposes additional computational burden. To address these problems, in this paper, we explore a novel direction of evaluating time series reasoning without post-training base agents. The core idea is to bring an evaluation reward model to assess each tool call using accumulated evidence, whose evaluation can further illuminate exploration of time series agents and reshape their action strategies. Naturally, we reformulate the iterative evaluation and action-reshaping process into a Markov decision problem that maximizes the expected reward composed of tool costs and task performance. To optimize the objective, we propose a novel time series agentic framework, TimesEval, which uses value-based reinforcement learning to learn evaluation policies for reshaping agent actions. To estimate rewards for each evaluation step, we propose episode relative advantage learning, which measures the expected return difference among decisions as the advantage for fast convergence of action reshaping. Moreover, to approach the global optimal policy, we utilize evolving policy optimization, which approximates policy iteration with on-policy state sampling to progressively improve evaluation reward model. Notably, the key difference is TimesEval trains a lightweight evaluator through reinforcement learning to reshape agent actions instead of post-training the agent itself. Finally, we conduct extensive experiments on 15 datasets and across four time series tasks, demonstrating that TimesEval consistently achieves state-of-the-art performance, with an average improvement of more than 19% over the strongest baseline and 36% lower tool cost.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Reasoning, Tool Learning, Agent

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ikUTOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

TimesEval addresses tool overuse in LLM agents for time series reasoning. Agents invoke analytical tools and usually execute every proposed tool call. Some calls are redundant given the available evidence, while others produce misleading outputs, leading to wasted computation and reduced accuracy. Instead of post-training the backbone LLMs, TimesEval proposes a lightweight learned module for deciding whether to execute a proposed tool call, reject the tool call, or drop the tool call and answer immediately. The module is trained via paired counterfactual rollouts, learning which decision leads to a better final outcome given the evidence so far. On TSRBench, the final model outperforms baseline tool-use strategies while making fewer tool calls

Critical strengths
  1. The method is practical. A lightweight MLP module learns to gate efficient tool usage with the backbone agents frozen, addressing a problem that otherwise may require expensive RL post-training.
  2. The counterfactual supervision is well-grounded. Branching decisions from the same state isolates each decision’s effect on the final outcome. The paper shows thorough analysis on paired versus unpaired training: unpaired training loses 8.6% (table 2), and paired evaluation turns more wrong cases into right ones than the reverse (fig. 5).
Critical weaknesses
  1. Evaluation is only on the in-distribution TSRBench. The policy is trained and selected on 1,200 TSRBench problems drawn from the same 15 tasks it is tested on, whereas baselines are training-free or zero-shot (including the RL models in sec 5.3.1). The method can prove to be much stronger if it is demonstrated to generalize to OOD tasks and samples.
  2. The ablation in section 5.3 is limited to Qwen3-8B. On other backbone in stable 1, no ablation is reported alongside, so the gains over ReAct, LATS, and ToolTree cannot be attributed solely to the learned gate. The method can be more convincing if similar ablation results show the same conclusion as on Qwen3-8B.
  3. The efficiency claim is not well supported. The main text reports cost only for Qwen3-8B against Always-Commit (table 2), where it drops from 3.90 to 3.33. The cost also excludes the evaluator’s own forward pass through the backbone LLM at every evaluation round. The claim would be more convincing if end-to-end cost, including the evaluator, were reported for all backbones and baselines.
Questions for authors
  1. Can the policy trained on a subset of TSRBench better guide tool usage on some held-out TSRBench samples or in other benchmarks.
  2. The policy is trained on TSRBench training problems while the baselines are not. How do the baselines perform when given the same data, for example as in-context examples or for fine-tuning?
  3. What are the accuracy and cost of TimesEval on other backbone models: TimeOmni-1-7B, Time-MQA-7B, and TS-Reasoner-7B?
  4. Does the reported cost include the evaluator’s forward pass through the backbone LLM, and what are the end-to-end cost and latency of TimesEval relative to Always-Commit?
Reviewer_93Y2OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TimesEval, a framework for improving tool-calling decisions in time-series reasoning agents without fine-tuning the backbone LLM. The key idea is to introduce a lightweight evaluator that assesses each proposed tool call before execution and chooses among three actions: Commit, Replan, and Stop. The evaluator is trained using episode-relative advantages, which compare the returns of alternative decisions through rollouts from the same state. The authors further introduce an evolving policy optimization procedure that iteratively collects new states, refreshes advantage targets, and improves the evaluation policy. The method is evaluated on TSRBench across 15 tasks and four backbone models. The results show improvements over the reported inference-time tool-use baselines, with gains in both task accuracy and interaction efficiency. The paper addresses a practically relevant problem and presents a reasonably complete framework.

Critical strengths
  1. The paper addresses the problem of unnecessary or counterproductive tool calls in LLM-based agents, which is an important and practically relevant problem. Its observation that additional tool executions can introduce redundant evidence and sometimes degrade answer accuracy is well motivated. Improving tool-use efficiency without modifying the backbone model is a promising direction.

  2. The combination of state-dependent evaluation, three explicit intervention actions, and iterative policy improvement provides a clear formulation of tool-execution control. In particular, the episode-relative advantage objective compares alternative decisions from the same state, potentially providing more informative supervision than learning from isolated trajectory returns.

  3. TimesEval outperforms the reported non-training baselines in overall accuracy across all four evaluated backbones. The ablation studies also provide preliminary evidence that paired rollouts, accumulated tool evidence, and policy evolution contribute to the resulting performance. The evaluation across multiple task categories and backbone models is a strength.

Critical weaknesses
  1. Limited methodological novelty. Pre-execution tool evaluation is related to existing approaches such as ToolTree and confidence-based tool-use methods (e.g., Alignment for Efficient Tool Calling of Large Language Models and MICE for CATs), but the paper does not sufficiently establish the added value of its paired relative-advantage learning and evolving policy optimization. Comparisons with confidence-based gating and a static supervised evaluator trained on rollout-derived decision-utility labels, under comparable computational budgets, would help clarify whether the improvements stem from the proposed learning mechanisms or simply from introducing an additional intervention and replanning layer.

  2. Incomplete cost accounting. Although TimesEval avoids fine-tuning the base LLM, its evaluator uses the same model to encode states. Furthermore, the reported interaction cost is only a proxy for actual token consumption, latency, and monetary expense. Training also requires substantial rollout generation, involving approximately 9,540 unique LLM requests and 6,180 trajectories. The authors should report the end-to-end inference overhead and training costs, and clarify the basis for the claimed 36% cost reduction, given the 14.6% reduction reported against Always-Commit in the ablation study.

  3. Insufficient isolation of the proposed mechanisms. TimesEval's performance gains may partly stem from additional LLM invocations and replanning opportunities rather than more effective tool-call evaluation. In particular, Replan allows the base agent to generate new proposals under updated exclusion constraints, potentially improving reasoning through self-correction. The current ablations do not fully disentangle these effects. The authors should compare against simple replanning or retry policies under matched LLM-call budgets and evaluate variants that disable Replan or Stop separately, to better establish the contribution of the learned evaluator.

Questions for authors

Refer to the weaknesses.

1 comment

Thanks for the response. The additional experiements have resolved my main concerns. I'll increase the rating to 3.

Reviewer_GtpnOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors propose TimesEval, a learned tool-execution controller for frozen time-series LLM agents. Given the current evidence and a pending tool call, the controller chooses whether to execute it, request replanning, or stop tool use. A lightweight evaluator is trained to predict return differences relative to executing the proposed tool, using same-state rollouts scored by answer correctness and interaction cost. Training iteratively collects states under the updated controller and refreshes the supervision. The method is evaluated on 15 tasks within TSRBench using four backbones, with reported improvements in overall accuracy and reductions in tool use. Its main contribution is an execution-control framework and its empirical validation without updating the base agent.

Critical strengths
  1. The paper addresses a useful practical problem through a clear control mechanism. Evaluating whether additional tool use is worthwhile can improve both reasoning quality and efficiency. The Commit/Replan/Stop interface provides a straightforward way to change agent behavior while keeping the base model frozen.

  2. The reported results suggest that the approach can benefit multiple backbones. Overall accuracy improves across all four evaluated models, and the ablations investigate the contributions of accumulated evidence, paired rollout supervision, and iterative policy updates. If supported under a consistent evaluation protocol, these results would constitute a useful empirical contribution.

Critical weaknesses
  1. Important inconsistencies make the experimental results difficult to assess under the stated protocol. Appendix C and Table 7 specify at most two executed tool calls per problem, whereas Table 3 reports 2.49 calls per sample for TimesEval with Time-MQA-7B on Reasoning. Figure 3 also shows eight evolution rounds, while Appendix D specifies three. These may reflect different experimental configurations rather than incorrect results, but the manuscript does not explain the distinction. Because the methodology largely combines established ideas in value estimation and approximate policy iteration, reliable empirical validation carries substantial weight in assessing its contribution.

  2. The reported interaction-cost reductions do not yet establish the complete computational advantage. The evaluator uses the base LLM as a frozen state encoder, but the cost definition counts tool executions and base-agent invocations without clearly accounting for evaluator encoding. It is unclear whether these representations are reused or require additional forward passes. Consequently, fewer tool calls may not translate into lower total inference cost. This matters because achieving a better accuracy–computation tradeoff is a central justification for the framework.

Questions for authors

Please see the question proposed in weaknesses section.

Comments on the paper

1 comment
Area_Chair_BFADOpenReview ↗

Discussion period ending soon

Dear Reviewers,

Thank you for your reviews of this submission. The authors have posted responses addressing the concerns raised.

With limited time remaining in the discussion period, please take a moment to review the authors' responses and post a brief follow-up indicating whether your concerns have been addressed or what remains outstanding. If your assessment changes as a result, please update your score accordingly.

Active engagement at this stage is essential for a fair and well-substantiated final decision. Thank you for your time and effort in reviewing this submission.

Best regards, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.