acceptodds
Under review as a conference paper at ICLR 2027

Reasoning through Verifiable Forecast Actions: Consistency-Grounded RL for Financial LLMs

Abstract

Financial markets are characterized by extreme non-stationarity, low signal-to-noise ratios, and strong dependence on external information such as news, company fundamentals, and macroeconomic signals. Yet, existing approaches either abstract time-series into text or decouple forecasting from language-based reasoning, leading to a fundamental mismatch between qualitative reasoning and quantitative outcomes. To address this, we introduce Stock-R1, a time-series–enhanced LLM that unifies stock forecasting and financial reasoning through a verifiable forecast action. Based on a tool-call design, the model first emits a forecast action, which is a structured and interpretable representation of its qualitative market outlook. It then invokes a time-series decoder conditioned on this action to generate distributional future trajectories, leading to more informed question answering and financial reasoning. We optimize the full pipeline with reinforcement learning, where rewards jointly reflect answer validity, forecast accuracy, and consistency between generated actions and observed time-series dynamics. In addition, rewards are reweighted by a sample-level uncertainty scalar, encouraging the model to accommodate varying uncertainty in market dynamics. We evaluate Stock-R1on financial question answering and stock forecasting over a large-scale 10-year benchmark. Stock-R1 achieves strong performance across both tasks and improves question-answering accuracy over its corresponding direct-prompt Qwen3 backbones by 17.7 and 25.9 percentage points at the 4B and 8B scales, respectively. These findings demonstrate that structured forecast actions provide an effective interface between language reasoning and temporal prediction, enabling LLMs to reason through verifiable, interpretable, and numerically grounded decisions.

Primary area:
learning on time series and dynamical systems
Keywords:
financial reasoning, time series, large language model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_hJzmOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes STOCK-R1, a Qwen3-based financial LLM that does not write future prices directly. Instead, it emits a structured JSON "forecast action" (direction, end-point change, volatility and drawdown bins, turning points, and so on), which is passed as a tool call to a separately pretrained time-series decoder. The decoder conditions on OHLCV history and the action and returns a 10-day trajectory, which the LLM then uses to answer the question. Training has three stages: probabilistic pretraining of the time-series module, SFT on GPT-5 distilled traces, and GRPO with rewards for answer correctness, action accuracy, trajectory precision, and action/trajectory consistency, with samples reweighted by predicted uncertainty. The model is evaluated on a self-built S&P 500 QA benchmark, on forecasting metrics against time-series and LLM baselines, with ablations, and with a backtest.

Critical strengths

[1] The action interface is a sensible design idea.

Asking the LLM to commit to a small set of checkable trajectory attributes, and then letting a dedicated decoder produce the numbers, is a cleaner division of labor than having the LLM write long numeric strings. It also gives field-level reward signals that free-form rationales do not. The action-field masking in Table 11 and the conditioned vs. unconditioned tool comparison in Table 3 point in the expected direction, and the SBUX case in Appendix C.6 shows the intended behavior concretely.

[2] The data pipeline is described with some care, and the set of time-series baselines is fairly wide.

Appendix A.1 explains how news and fundamentals are cut off before the forecast date, which is the right concern for this kind of benchmark. Table 2 also includes both zero-shot foundation models and models trained from scratch, which makes the forecasting comparison more informative than in many similar papers.

Critical weaknesses

[1] The main QA comparison is not fair, and the control for the action interface is not well documented.

Table 1 compares STOCK-R1, which was trained with SFT and RL on 40K questions from the same GPT-5 generator and answer format as the test set, against zero-shot prompting of general and financial LLMs. I think most of the headline gain comes from this. Forecast QA makes it clear: GPT-5 gets 8.16 and o3 gets 2.78, which looks like a scoring or format effect rather than a reasoning gap. Qwen3-8B also scores below Qwen3-4B, which again suggests answer-format failures. The missing baseline is the same Qwen3 backbone trained with the same SFT and GRPO on the same data, but without the decoder and action interface. Table 9 already shows that giving Qwen3-4B the unconditioned tool brings most of the gain, so the remaining gap mixes in-distribution training with the action mechanism.

The evidence for the mechanism itself (Tables 3 and 10) is weakened by Appendix D, which says the controls lack seeds and confidence intervals and that the records do not establish whether the unconditioned arm was retrained or simply had conditioning removed at inference. In the second case, the drop is a distribution shift, not evidence for the interface. I appreciate the honesty, but the central ablation needs a clearly defined control. Table 1 also has no majority-class or base-rate reference, even though most items are binary questions about future returns or volatility in a test window that went mostly up.

[2] Several numbers are inconsistent, and some claims do not match the paper's own figures.

Table 1 gives STOCK-R1-4B News Analysis 61.83, but Figure 3a and Table 6 (which match every other cell of that row) give 51.83. Ten points is not a rounding issue. The 67.5 for u-GRPO in Table 4 matches neither model in Table 1, and the differences between GRPO variants in Table 4 are within about one standard deviation with no seed count, so the uncertainty reweighting claim rests mainly on an entropy curve. Section 4.3 says time-series tokens give consistent gains for all categories, but Figure 3a shows several categories where the model without them is better.

The backtest in Table 7 has impossible combinations of Sharpe ratio and return (for example, positive Sharpe with negative return for ARIMA, and the reverse for DLinear). There is also no buy-and-hold benchmark and no transaction costs, and STOCK-R1 has the highest beta in a rising market, so part of its return is just market exposure. Thus, I cannot take the backtest as evidence of trading value.

[Smaller issues]

In Table 2, STOCK-R1 is not the best on several metrics and its directional accuracy is almost tied with plain Qwen3-8B, and there are no naive baselines or significance tests. The SFT teacher sees the realized future action, which may teach post-hoc rationalization. The paper does not say whether test questions are temporally disjoint from training questions. Notation for reward weights clashes with other symbols, and some details in Table 5 and the baseline configurations are missing.

Questions for authors

Q1) How exactly was the unconditioned-tool arm in Tables 3 and 10 produced? Was it retrained with SFT and RL without actions, or is it the same model with conditioning removed at inference? If it is the latter, a rerun with matched training and a few seeds would be needed before the comparison says anything about the action interface.

Q2) Could you report Qwen3-4B and 8B trained with the same SFT and GRPO on the same 40K questions and context, but with no decoder and no action, together with majority-class and always-up references for each QA category? This is the comparison that would tell me how much of the gain comes from the proposed mechanism, and it would change my assessment.

Q3) Which value is correct for STOCK-R1-4B News Analysis, 61.83 or 51.83, and which model produces the 67.5 in Table 4? Could you also explain the Sharpe and return signs in Table 7?

Q4) Is the QA train/test split temporal, with all test questions dated after every training question? Also, can the policy game the consistency reward by emitting actions that the frozen decoder reproduces easily, such as near-flat profiles? The distribution of emitted directions and volatility bins before and after RL would help to check this.

Reviewer_NCpuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes STOCK-R1, which connects LLM-based financial reasoning with time-series forecasting through structured Forecast Actions and uses consistency rewards and uncertainty-weighted reinforcement learning to optimize the model.

Critical strengths

I find the research motivation clear. Connecting language reasoning with numerical forecasting through verifiable structured Actions is a meaningful design. The overall method is fairly complete, and the experiments cover multiple financial tasks and provide some ablation studies.

Critical weaknesses
  1. Although the full model achieves substantial gains, a large portion of the improvement comes from the additional time-series tool. Appendix D.3 also does not establish whether the Unconditioned Tool was trained under fully matched conditions. I therefore believe that the current experiments do not sufficiently demonstrate the independent contribution of Forecast Actions.
  2. The authors derive target Actions from realized future trajectories and use them to construct teacher reasoning. This may introduce hindsight rationalization. In addition, the paper does not explain the temporal split between financial QA training and test data. I hope the authors can provide stricter controls to show that the model learns genuine forward-looking predictive ability.
  3. Uncertainty weighting yields relatively limited gains over standard GRPO. The paper also lacks direct validation of the reliability of its uncertainty estimates. I do not think the current experiments sufficiently support the robustness of this component across different market conditions.
  4. The average QA results differ substantially across tables, including Table 1 and Table 4. These results may correspond to different settings, but the paper does not clearly explain the differences.
Questions for authors
  1. Was the unconditioned forecasting tool retrained under the same SFT and RL settings as the full model?
  2. How are hindsight bias and temporal overlap prevented when constructing outcome-conditioned supervision?
  3. Could the authors clarify the different QA accuracy results reported in Tables 1 and 4?
Reviewer_2yAJOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces STOCK-R1, a time-series-enhanced financial LLM that couples language reasoning with numerical forecasting through a structured forecast action. A policy LLM reads multimodal market context (OHLCV log-returns, macro indicators, fundamentals, news) plus a financial question, and emits a field-factorized action (direction, volatility level, turning-point count, peak timing, …). That action is passed to a dedicated action-conditioned time-series decoder, which returns a distributional trajectory plus an uncertainty scalar; the LLM then answers conditioned on context, action, and trajectory. Training runs in three stages: pretrain the time-series module with a Gaussian-NLL + quantile objective, joint SFT on GPT-5-distilled tool-call traces (with the target action computed from the realized future, used only for supervision), then GRPO-style RL with a reward R = R_ans + αR_act + βR_prec + γR_cons, reweighted by a sample-level uncertainty scalar w(U_q). Evaluation covers five financial-QA categories on a new S&P 500 benchmark (2015–2025) and time-series forecasting against foundation models, from-scratch forecasters, and multimodal LLMs.

Critical strengths
  1. The interface idea is genuinely well-posed. Making the LLM emit a compact, field-factorized action rather than dense future values gives a checkable intermediate object: each field can be scored against the trajectory the decoder actually produced and against the realized future. The paper is explicit about why that matters — conditioning alone does not guarantee the decoder uses the action.

  2. The action-vs-no-action control is the right experiment and it is run. Table 3 compares an unconditioned tool p(y|H_ts) against the action-conditioned one under the same interface, and the trajectory–action alignment jumps 12.7 → 53.8 — a large, mechanism-relevant separation rather than a bare accuracy delta.

  3. The honesty in the appendix is high. D.1 runs an identity-masking control (ticker/company/date anonymised, −0.20/+0.24/−0.3 pp) and then states plainly that it "does not exclude teacher-side hindsight or implicit event identification". D.2 states that the prompting/tool comparison "does not establish equal inference compute or optimized prompting", and that the retained records lacked exact prompts and decoding budgets.

Critical weaknesses
  1. The QA split is never defined temporally. A.2 gives "40,000 training / 4,000 evaluation questions" and guarantees only that labels use no post-t information — not that eval questions come after training ones. Since the corpus spans 2015-01-01→2025-10-01 and context is the ten most recent headlines, a random split makes eval questions near-duplicates of training ones (same ticker, adjacent dates, overlapping news). Fig. 7 shows a fixed "test set" exists but never states its boundary. State the boundary and confirm no eval decision time precedes a training decision time per asset.

  2. The headline gap is mostly tool access, not the action interface the abstract credits. Abstract: "improves QA accuracy … by 17.7 and 25.9 pp" (verified: 60.81−43.07, 63.99−38.04), then concludes "structured forecast actions provide an effective interface". But D.2 shows Qwen3-4B goes 43.0 → 55.6 with an unconditioned tool: +12.6 of 17.7 (≈71%) is tool access, the action adds +5.2 (Table 3: +3.8). D.2 says this plainly; the abstract does not. Report the decomposition in the abstract.

  3. The two nearest competing works are uncited, so the delta is left implicit. Cast-R1 (teacher-trajectory distillation → multi-turn GRPO agentic forecaster) and TS-Agent (financial TS + LLM agent + stepwise RL + finance-aware reward) as the closest works; neither is in the bibliography. The paper does cite the generic "RL for time-series reasoning" line once — "(Guan et al., 2025; Luo et al., 2025)" in §1 — but only as a passing instance, and neither is differentiated from; notably Luo et al. 2025 already pairs SFT→RL with a fine-grained multi-objective reward specifically for forecasting, so the reward design is not itself the novelty.

Questions for authors

See weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.