acceptodds
Under review as a conference paper at ICLR 2027

REATS: LLM Reasoning-based Ensemble Learning for Adaptive Time Series Forecasting

Abstract

Recent advances in time series forecasting (TSF) have produced models with complementary design strengths, but no single model handles the diverse patterns of all samples. Existing ensembles rely on fixed rules or numerical-only black-box models, failing to exploit LLM reasoning or explain weighting decisions. We propose REATS, an intelligent LLM ensemble router that jointly processes textual temporal-pattern descriptions and numerical features to generate interpretable, sample-adaptive weights through chain-of-thought reasoning. To enable effective LLM-based ensembling, we study its key design choices and propose: (i) a structured input pipeline that converts raw series into hybrid textual–numerical representations with token cost fixed across input lengths, supports LLM reasoning and API-free rule-based chain-of-thought construction, and incorporates retrieved similar-sample priors; (ii) diverse multi-row weight supervision to enrich training signals, together with a token-efficient percentage-table format that reduces numerical complexity and LLM hallucinations; and (iii) a two-stage fine-tuning framework combining SFT for structured reasoning and GRPO with a reciprocal reward mapping that maps continuous, unbounded regression rewards to bounded signals, amplifies near-oracle sensitivity, and addresses uniform sensitivity and outlier-dominated advantage compression in mixed-quality rollout groups. Experiments on eight benchmarks show that REATS outperforms competitive ensemble baselines for both foundation-model and small-model candidate groups, while providing natural-language explanations and strong transfer learning and out-of-domain generalization to unseen candidate models.

Primary area:
learning on time series and dynamical systems
Keywords:
Time series forecasting, Ensemble Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_pjyBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

REATS uses an LLM to assign sample-dependent weights to candidate forecasters. Its inputs combine textual and numerical temporal features with weight priors retrieved from training examples. Rule-generated reasoning supervision supports SFT, followed by forecasting-oriented GRPO. The evaluation covers eight datasets, specialized and foundation-model candidate pools, changes in the candidate set, and transfer settings. The reported improvements are promising. The remaining questions concern which training objective produced the reported results and what LLM reasoning adds over alternatives with the same information.

Critical strengths
  1. REATS targets adaptive weighting of candidate forecasts. The method specifies a hybrid temporal representation, a training-only retrieval pool, API-free construction of reasoning supervision, and two-stage training. Using the LLM as an ensemble router gives these components a defined forecasting role.

  2. The evaluation covers eight datasets and two classes of candidate forecasters, with candidate-set changes, transfer, and CoT/RAG ablations. Table 2 reports average MSE of 0.1384 versus 0.1597 for the foundation-model OptWval comparison, and 0.1080 versus 0.1352 for the small-model OptWtr comparison. The component ablations support the usefulness of CoT and retrieval within this pipeline. Together, these results make the overall supervised router worth considering.

Critical weaknesses
  1. The implemented reward needs to be connected to the reported results. Table 2(b) reports Electricity/Weather MSE of 0.1476/0.0206, whereas Table 8 reports 0.1463/0.0061. Different candidate pools, seeds, or settings could explain the difference, but their correspondence is unspecified. Section 3.4 assumes a nonnegative row-specific MSE gap, although nine reference rows are deliberately nonoptimal. The prior static audit summarized in the supplied revision record reports that the code clips negative gaps to zero. Clipping avoids the previously suspected negative-gap reciprocal singularity. The analysis still needs to account for saturation after an auxiliary reference is beaten. The same audit records percentage-parsing, column-order, and exact-row-count validation issues. Their frequency and effect in the reported runs remain unknown. An immutable source snapshot was unavailable in that reassessment. Please identify the version used and provide representative parsed rollouts and reward records from the reported runs. A parser patch or a possible explanation would not, by itself, connect the implemented objective to the existing results.

  2. The experiments support the complete routing pipeline more clearly than the incremental contribution of LLM reasoning. Appendix A.1.3 states that RLMC receives raw histories, whereas REATS receives engineered features and retrieved weight priors. A direct retrieved-weight baseline and a lightweight router with the same features, supervision, and retrieval information would help isolate that contribution. Algorithm 1 explicitly constructs reasoning targets from oracle weights, which is disclosed training supervision. Prediction gains and plausible explanations nevertheless do not establish explanation faithfulness. Candidate transfer also rebuilds retrieval pools from the new candidates' historical training predictions. This historical-calibration requirement should be stated explicitly.

Questions for authors
  1. Which candidate pools, seeds, split boundaries, and training configurations produced Table 2(b), Table 8, and Figure 6? Please map these results to their runs. Explain the differing Electricity/Weather entries and how the stated 7:1:2 split corresponds to the reported ETT sample counts.

  2. Please identify the reward and output-validation version used in the reported GRPO runs, with its version identifier and configuration. State how it handles gap clipping, percentage parsing, column order, required row count, and inference-time row selection. Include representative parsed rollouts and reward records, and any summaries showing how often the identified cases occurred. If those cases materially affected training, a targeted comparison with corrected behavior would help assess their impact. This concern remains open until the implemented behavior is connected to the reported results.

  3. Are there results for direct retrieved weights and a lightweight router with the same candidate predictions, engineered features, training labels, and nearest-neighbor weight information? If absent, these controls would address the contribution-attribution question. Please state the historical calibration required for candidate transfer. A controlled faithfulness test would be needed to support the stronger explanation claim; otherwise, the interpretation should be limited to generated, inspectable rationales.

Additional presentation feedback Figure 2's oracle-weight block lists nonnegative simplex constraints and then gives the inverse-Gram closed form. Section 3.3 states that this closed form applies without the nonnegativity constraint and that the actual oracle uses a constrained quadratic program. Please distinguish these two problems in the figure. For Figures 1–2, please supply a corrected, editable SVG, drawn directly or produced with AI assistance and manually checked. Check the mathematical notation, labels, and signal paths against the method. Empirical plots should be regenerated from their numerical records and plotting code.

6 comments
AuthorsOpenReview ↗

Response to Critical Weakness 1

Critical Weaknesses 1:

Weakness point (1): The implemented reward needs to be connected to the reported results. Table 2(b) reports Electricity/Weather MSE of 0.1476/0.0206, whereas Table 8 reports 0.1463/0.0061. Different candidate pools, seeds, or settings could explain the difference, but their correspondence is unspecified.

Answer: Table 2(b) uses the default reward coefficients in Eq. (4), G1 with (λ1,λ2,λ3)=(0.8,0,0.2)(\lambda_1,\lambda_2,\lambda_3)=(0.8,0,0.2), for all datasets. The “GRPO with Reward Mapping” column in Table 8 reports the lowest observed test MSE for each dataset across the six reward-coefficient settings in Figure 6.

As the values highlighted in red in Figure 6 of the paper, G5 (1,0,0)(1,0,0) gives the lowest MSE on Electricity and Weather (0.1463 and 0.0061), compared with 0.1476 and 0.0206 under G1. For Exchange, G1 gives the lowest MSE (0.1587), so both tables report the same value. We will clarify this distinction in the Table 8 caption.

Weakness point (2): Section 3.4 assumes a nonnegative row-specific MSE gap, although nine reference rows are deliberately nonoptimal. The prior static audit summarized in the supplied revision record reports that the code clips negative gaps to zero. Clipping avoids the previously suspected negative-gap reciprocal singularity. The analysis still needs to account for saturation after an auxiliary reference is beaten.

Answer: Thank you for pointing this out. Our implementation clips negative row-wise MSE gaps to zero, so the MSE reward for a row saturates at 1 when the prediction matches or beats its reference. We will make this explicit in Section 3.4. The highest logged aggregate rewards were in the 0.90–0.99 range. Designing a reward that continues to distinguish improvements after a prediction reaches the maximum reward is an interesting direction for future work.

Weakness point (3): The same audit records percentage-parsing, column-order, and exact-row-count validation issues. Their frequency and effect in the reported runs remain unknown. An immutable source snapshot was unavailable in that reassessment. Please identify the version used and provide representative parsed rollouts and reward records from the reported runs. A parser patch or a possible explanation would not, by itself, connect the implemented objective to the existing results.

Answer: Thanks for the valuable advice. We would like to clarify a few points in the hope of addressing your concerns.

(1) The anonymous repository submitted with the paper contains the same code used for all reported experiments.

(2) We implemented a fallback mechanism for parsing failures: when valid weights cannot be recovered, they are replaced with a uniform allocation, which is generally less competitive than the learned adaptive weights. Such cases are extremely rare, accounting for less than 0.5% of samples, and thus have negligible impact on the reported results.

(3) We also examined the rollout-level reward records. The rewards generally increase from approximately 0.2 at the beginning of training to around 0.95 as training progresses.

AuthorsOpenReview ↗

Response to Critical Weakness 2

Critical Weaknesses 2:

Weakness point (1): The experiments support the complete routing pipeline more clearly than the incremental contribution of LLM reasoning. Appendix A.1.3 states that RLMC receives raw histories, whereas REATS receives engineered features and retrieved weight priors. A direct retrieved-weight baseline and a lightweight router with the same features, supervision, and retrieval information would help isolate that contribution.

Answer: We add the lightweight router baselines, including GBDT, LightGBM, and MLP, which are provided with the same engineered features, supervision, and retrieved weight information as REATS. We also add the direct retrieved-weight baseline with the same training retrieval pool and Euclidean distance as REATS (retrieval top 3 priors).

Results are shown in Table 1, REATS achieves the best overall performance, demonstrating the benefits of its LLM-based design beyond the shared input information, including CoT reasoning, GRPO-based reward optimization, and structured weight generation.

Table 1. Ensemble learning on small model groups. The experimental settings are identical to those used in the main paper. We report per-dataset test MSE for direct retrieval, lightweight routers, and REATS.

MethodExchangeETTh1ETTh2ETTm1ETTm2WeatherElectricityTrafficAvg. ↓
Direct retrieval Top-10.26820.17780.20170.10660.09430.28530.15650.11090.1752
Direct retrieval Top-3 mean0.22680.16120.18460.08530.09020.08620.15590.10360.1367
kNN model selection0.21170.15240.17670.09080.07220.27950.14640.06670.1495
GBDT0.22130.15590.18050.08180.08220.11380.15290.10180.1363
LightGBM0.22130.15580.18060.08170.08240.11850.15270.10190.1369
MLP0.23330.16690.18320.08360.08280.08750.15310.10240.1366
RLMC (raw history)0.38750.15720.30290.09700.08380.16660.15490.06660.1771
RLMC (hybrid features)0.44470.16290.19380.09480.08210.17400.14780.06660.1708
REATS-GRPO0.15870.14090.17000.07250.07200.02060.14760.08190.1080

Weakness point (2): Algorithm 1 explicitly constructs reasoning targets from oracle weights, which is disclosed training supervision. Prediction gains and plausible explanations nevertheless do not establish explanation faithfulness. Candidate transfer also rebuilds retrieval pools from the new candidates' historical training predictions. This historical-calibration requirement should be stated explicitly.

Answer: Thanks for the valuable insights. We evaluated rationale–weight consistency using 80 test outputs per model, sampled from 8 datasets (10 per dataset). In both REATS-SFT and REATS-GRPO, the candidate prioritized in the rationale received the highest weight in the first output row in all 80 cases (80/80). The Weather example in the Appendix further illustrates this consistency. While these results provide empirical evidence of rationale–weight consistency, the causal faithfulness of the explanations is still worth further investigation, and we leave the development of faithfulness metrics to future work. Besides, we will clarify in the revised manuscript that candidate transfer requires historical forecasts from the new candidates on the training split to rebuild the retrieval pool, while the router remains fixed.

AuthorsOpenReview ↗

Response to Questions For Authors

Questions For Authors 1:

Question point (1): Which candidate pools, seeds, split boundaries, and training configurations produced Table 2(b), Table 8, and Figure 6? Please map these results to their runs. Explain the differing Electricity/Weather entries.

Answer:

(1) The difference between the results in Table 2(b) and Table 8 arises from the different loss coefficients used in Eq. (4). Table 2(b) uses the G1 configuration, whereas Table 8 reports the best-performing configuration among G1–G6.

(2) The candidate pools, random seeds, data-split boundaries, and all other training settings are identical across Table 2(b), Table 8, and Figure 6. The only difference is the choice of loss coefficients in Eq. (4).

Please refer to the answer of Weakness point (1) in the response to Critical Weaknesses 1 for details.

Question point (2): how the stated 7:1:2 split corresponds to the reported ETT sample counts.

Answer: By adjusting the split boundaries, we can achieve an approximately 7:1:2 ratio of valid sliding windows across the training, validation, and test sets while preserving temporal order.

Questions For Authors 2:

Please see our response to Critical Weakness 1.

Questions For Authors 3:

Please see our response to Critical Weakness 2.

Finally, we thank the reviewer for the additional presentation suggestions. We will revise Figures 1–2, particularly the mathematical formulations, to clearly distinguish the unconstrained closed-form solution from the simplex-constrained QP used to obtain the actual oracle weights. We will also check the labels and signal paths and provide the editable SVG files in the project repository. In addition, we have verified that all empirical plots are generated from the corresponding numerical records and plotting code.

Reviewer_F97QOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In time series forecasting, the best-performing model may vary across samples and application scenarios. Ensemble learning can mitigate this limitation by combining the complementary strengths of different models. Existing neural routers enable dynamic weight allocation, but typically use numerical inputs and are trained for a fixed pool of candidate models. This paper proposes REATS, an LLM-based routing framework that uses the LLM’s ability to process and generate text to incorporate textual information into routing decisions and generate textual explanations of weight allocations. Specifically, REATS takes extracted time series features, candidate-model descriptions, and reference weights from similar historical samples as inputs, constructs CoT using rules, and generates multiple weight rows as supervision targets for SFT. It then further improves forecasting performance through GRPO with a reciprocal reward mapping. The paper evaluates REATS on eight datasets and reports forecasting performance improvements over the compared baselines.

Critical strengths
  1. The paper proposes an LLM-based ensemble routing method that combines structured inputs with two-stage training. REATS takes time series features, candidate-model descriptions, and reference weights from similar historical samples as inputs. It performs SFT using rule-generated CoT and multiple weight rows, followed by GRPO with a reciprocal reward mapping to improve forecasting performance.
  2. The method improves forecasting performance under the evaluated settings. The paper evaluates REATS on eight datasets and two types of candidate-model pools, reporting lower average errors than the compared baselines. Adding GRPO after SFT further improves the results.
  3. The design choices are evaluated through comprehensive ablation studies. The paper provides ablations on CoT, RAG, input representation, output format, the number of weight rows, and reward mapping, and also examines unseen candidate models and cross-dataset transfer.
Critical weaknesses
  1. The core motivation of this paper is to use LLMs to achieve Multimodal information fusion, Flexible scalability, and Inherent interpretability. However, prior work, “Conversational Time Series Foundation Models: Towards Explainable and Effective Forecasting” [1], has already explored related directions. The authors should provide a comparison and discussion to clarify this paper’s differences and contributions.

  2. The absence of an oracle-weight routing baseline based on similar historical samples makes it difficult to attribute the core gains. The input to REATS already includes oracle weights from similar historical samples. Therefore, a straightforward and necessary baseline is to use the same retrieval pool and similarity metric and directly adopt the nearest neighbors’ weights or combine them using rule-based weighting. This would assess how much incremental benefit the LLM provides beyond the retrieved weight priors. This issue also affects the interpretation of “flexibility.” In the unseen-model experiments in section 4.2, the RAG pool is reconstructed using the new models’ predictions on the training set, meaning that the system already has access to information about these new candidate models’ historical performance. The current experiments cannot distinguish whether the flexibility comes from the LLM or from the new models’ historical performance priors.

  3. The ablation experiments with and without CoT support the predictive benefits of CoT training, but they do not establish interpretability as an advantage unique to using LLMs. The paper’s CoT is constructed using rules. Traditional ensemble methods could also generate explanations using similar rules based on temporal features and prediction weights.

  4. The authors should report metrics such as actual wall-clock time and GPU memory usage for different methods to more clearly demonstrate the impact of introducing an LLM on inference efficiency and resource requirements.

  5. Several statements in the paper overstate the limitations of existing methods or attribute empirical gains to mechanisms that are not directly supported by the presented evidence. For example, GRPO does not require binary rewards. Thus, describing the continuous MSE objective as mismatched with GRPO is inaccurate. In section 4.5, the paper claims that removing CoT reduces the diversity of weight generation and leads to “sparse reward variance,” but the corresponding experiments report neither generation diversity nor within-group reward variance. Similarly, section 4.4 describes multi-row weight supervision as allowing the reward to reflect a “simplex region,” thereby providing “richer directional guidance,” whereas the paper actually aggregates row-wise scores into a response-level scalar reward. Such statements make the engineering choices appear to offer mechanistic advantages that have not been substantiated.

  6. There is a potential risk of violating double-blind anonymity: the provided anonymous code link contains an author’s local path on line 11 of ‘generate_base_preds.py’.

[1] Cao, Defu, et al. "Conversational time series foundation models: Towards explainable and effective forecasting." arXiv preprint arXiv:2512.16022 (2025).

Questions for authors

Please see the weaknesses.

4 comments
AuthorsOpenReview ↗

Response to Critical Weakness 1 and Critical Weakness 2.

Critical Weaknesses 1:

The core motivation of this paper is to use LLMs to achieve Multimodal information fusion, Flexible scalability, and Inherent interpretability. However, prior work, “Conversational Time Series Foundation Models: Towards Explainable and Effective Forecasting” [1], has already explored related directions. The authors should provide a comparison and discussion to clarify this paper’s differences and contributions.

Answer:

Thank you for pointing out this related work.

We want to emphasize that the two approaches differ fundamentally in their roles of LLMs and weight optimization mechanisms. Cao et al. (arXiv preprint) use an LLM to evaluate optimizer-proposed weights and guide iterative traditional SLSQP optimization, with a GRPO reward incorporating SHAP-based faithfulness. In contrast, REATS directly predicts sample-adaptive ensemble weights through CoT reasoning over hybrid textual-numerical features and retrieved priors, optimized with a bounded reciprocal reward based on forecasting MSE gaps.

Moreover, REATS supports candidate-pool changes through prompt-level model descriptions. We demonstrate this through model-level OOD experiments, training on small-model candidates and evaluating on unseen foundation models (as well as mixed and within-group unseen candidates), and cross-dataset transfer experiments, training on one dataset and testing on the other seven.

We will cite this work and clarify these distinctions in Related Work.

Critical Weaknesses 2:

Weakness point (1): The absence of an oracle-weight routing baseline based on similar historical samples makes it difficult to attribute the core gains. The input to REATS already includes oracle weights from similar historical samples. Therefore, a straightforward and necessary baseline is to use the same retrieval pool and similarity metric and directly adopt the nearest neighbors’ weights or combine them using rule-based weighting. This would assess how much incremental benefit the LLM provides beyond the retrieved weight priors.

Answer point (1): Thanks for the valuable insights. We added direct oracle-weight transfer baselines using the same training retrieval pool and Euclidean distance as REATS (retrieval top 3 priors). We compared nearest-neighbor weight transfer (0.1752) with similarity-weighted aggregation (0.1367). The latter performs better and is adopted as the final RAG-only strategy. REATS outperforms this baseline in both in-distribution (ID) and out-of-distribution (OOD) settings, demonstrating gains beyond retrieved historical weight priors alone. We use the same ID and OOD experimental settings as those in the original paper.

Weakness point (2): This issue also affects the interpretation of “flexibility.” In the unseen-model experiments in section 4.2, the RAG pool is reconstructed using the new models’ predictions on the training set, meaning that the system already has access to information about these new candidate models’ historical performance. The current experiments cannot distinguish whether the flexibility comes from the LLM or from the new models’ historical performance priors.

Answer point (2): As shown in Table 1, REATS consistently outperforms the RAG-only baseline in OOD settings, where the candidate pool is replaced with previously unseen small and foundation models. This suggests that REATS's OOD gains stem not only from the new candidates' historical performance priors but also from its feature-based reasoning and adaptive weight prediction.

Table 1: Test MSE comparisons with RAG-only and GRPO. RAG-only uses the same training retrieval pool and Euclidean distance as REATS. ID indicates In Domain and OOD indicates Out Of Domain.

SettingMethodExchangeETTh1ETTh2ETTm1ETTm2WeatherElectricityTrafficAvg. ↓
Small models in-distributionDirect retrieval0.22680.16120.18460.08530.09020.08620.15590.10360.1367
Small models in-distributionREATS-GRPO0.15870.14090.17000.07250.07200.02060.14760.08190.1080
Small models→ out-of-distributionDirect retrieval0.14440.21250.31590.12760.11150.06450.16910.09270.1548
Small models→ out-of-distributionREATS-GRPO0.13700.17980.25680.08160.07610.02180.15220.07910.1231
AuthorsOpenReview ↗

Response to Critical Weakness 3 and Critical Weakness 4

Critical Weaknesses 3:

The ablation experiments with and without CoT support the predictive benefits of CoT training, but they do not establish interpretability as an advantage unique to using LLMs. The paper’s CoT is constructed using rules. Traditional ensemble methods could also generate explanations using similar rules based on temporal features and prediction weights.

Answer:

Thank you for this insightful comment. Rule-based templates are used only to construct offline Rule-CoT supervision. Ground-truth labels provide outcome-based evidence for evaluating candidate forecasts and constructing CoT targets. Without such ground-truth supervision, explanations based solely on temporal features may sound plausible but lack grounding in actual forecasting performance. At inference, REATS requires no ground-truth labels or oracle weights, as the trained LLM generates CoT explanations and sample-adaptive weights from observable inputs and retrieved priors. Thus, Rule-CoT enables the model to learn generalizable, outcome-grounded reasoning rather than merely generate post-hoc explanations.

Besides, REATS integrates sample-level weight prediction with natural-language reasoning in a unified framework.

Moreover, our CoT ablation reduces average test MSE from 0.1151 to 0.1080, demonstrating its predictive benefit.

Critical Weaknesses 4:

The authors should report metrics such as actual wall-clock time and GPU memory usage for different methods to more clearly demonstrate the impact of introducing an LLM on inference efficiency and resource requirements.

Answer:

Thanks for the valuable comments. As an exploration of the potential of LLMs for time-series ensemble forecasting, REATS inevitably involves additional computational costs and inference latency compared with lightweight traditional methods.

Following the reviewer's suggestion, we report REATS's inference latency (about 1.1 s/sample) and estimated model-weight memory (about 3.4 GB for the 1.7B Qwen3 backbone in BF16/FP16). We acknowledge that REATS introduces additional computational costs and inference latency compared with lightweight baselines such as OptW and RLMC. Nevertheless, these costs bring substantial improvements in forecasting accuracy, interpretability, and cross-model and cross-dataset generalization, highlighting the potential of LLMs for interpretable ensemble forecasting.

AuthorsOpenReview ↗

Response to Critical Weakness 5 and Critical Weakness 6

Critical Weaknesses 5:

Weakness point (1): Several statements in the paper overstate the limitations of existing methods or attribute empirical gains to mechanisms that are not directly supported by the presented evidence.

Answer point (1):

We thank the reviewer for this valuable comment. In general, our intention was to provide insights into the potential contributions of these design choices, rather than claim experimentally established mechanisms. We will detail each below.

Weakness point (2): For example, GRPO does not require binary rewards. Thus, describing the continuous MSE objective as mismatched with GRPO is inaccurate.

Answer point (2): Our original intention was to highlight that standard GRPO is "unsuitable", rather than entirely inapplicable, for fine-grained continuous MSE optimization. We will revise the wording to clarify that, under group-standard-deviation normalization, extreme MSE gaps may inflate the group standard deviation and compress advantage differences among other rollouts. We therefore propose the Reciprocal Reward to mitigate this issue while preserving near-oracle discrimination, achieving better empirical performance than directly using raw negative MSE rewards.

Weakness point (3): In section 4.5, the paper claims that removing CoT reduces the diversity of weight generation and leads to “sparse reward variance,” but the corresponding experiments report neither generation diversity nor within-group reward variance.

Answer point (3): Our intention was to suggest that CoT provides additional contextual guidance for adaptive weight generation, potentially promoting more diverse weight configurations and informative reward signals. Our CoT ablation confirms its predictive benefit. We will revise the wording to present these effects as potential explanations rather than experimentally established mechanisms.

Weakness point (4): Similarly, section 4.4 describes multi-row weight supervision as allowing the reward to reflect a “simplex region,” thereby providing “richer directional guidance,” whereas the paper actually aggregates row-wise scores into a response-level scalar reward. Such statements make the engineering choices appear to offer mechanistic advantages that have not been substantiated.

Answer point (4): Our intention was to leverage multiple predicted weight configurations for more comprehensive reward evaluation. We score each weight row separately and aggregate the scores into a response-level scalar reward. Our G5–G6 ablation in Fig.6 shows that row-wise scoring achieves lower test MSE, compared with averaging weight vectors before scoring. We will replace the "simplex region" and "richer directional guidance" claims with this more precise description.

Critical Weaknesses 6:

There is a potential risk of violating double-blind anonymity: the provided anonymous code link contains an author’s local path on line 11 of ‘generate_base_preds.py’.

Answer:

We thank the reviewer for pointing this out. The directory name is merely a local folder label used in the development environment, rather than an explicit author identifier. Nevertheless, to avoid any potential ambiguity regarding anonymity, we will remove all hard-coded local paths from the anonymous code repository and replace them with user-configurable path inputs.

Thank you for the additional baselines. They strengthen the paper, but I still have the following concerns.

  1. The contribution of LLM reasoning itself remains unclear. The RAG-only comparison does not distinguish semantic reasoning from learned corrections to retrieved weight priors.
  2. Explanation faithfulness remains insufficiently validated. Oracle-based supervision and improved MSE do not establish that the stated rationales faithfully explain the weighting decisions.
  3. The practical accuracy–cost trade-off remains unevaluated. No computational measurements are provided for competing baselines, preventing a direct comparison of computational costs.

Therefore, I maintain the score.

Reviewer_LUsuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

REATS fine-tunes Qwen3-1.7B as a sample-adaptive ensemble router over 4 candidate forecasters. Its inputs are a fixed-length textual–numerical feature summary, a retrieved prototype with neighbor oracle weights, and candidate descriptions. It first outputs a rule-generated CoT, then a 10-row integer weight table. Training is SFT followed by GRPO with a reciprocal reward on the MSE gap to the per-sample QP oracle. The paper reports gains over static, neural (RLMC), and zero-shot LLM ensembles on 8 univariate benchmarks, and transfer to unseen candidates.

Critical strengths
  1. The ablations are thorough and answer real design questions.

  2. The reward-shaping analysis is practically useful for regression-style GRPO.

Critical weaknesses
  1. In the small-model group, TimeXer has an average MSE of 1.67 and up to 1.94. This is roughly 10× the other candidates in the same table, which suggests a misconfigured candidate. Two other candidates have MSE above 2 on Traffic. Much of the "20.1% reduction" therefore comes from learning to zero out broken models, which a simple per-dataset validation filter would also achieve. The Weather numbers (e.g., MSE 0.0030 and individual-model values from 0.0039 to 0.5683) also call for a scale check.

  2. The paper does not report the per-sample oracle MSE (the attainable ceiling), the best single model selected per sample by a kNN on the same features, or a gradient-boosted regressor on the same 8 feature groups plus retrieved weights. Without these, it is unclear how much of the gain needs an LLM at all. RLMC is the only learned baseline, and it uses raw inputs only.

  3. Rule-CoT is reverse-engineered from oracle weights through templates, so the SFT targets are post-hoc rationalizations by construction. No faithfulness test is reported, such as intervening on stated features or checking weight changes when the claimed rationale is altered. The closest prior work, an LLM-as-judge ensemble of TSFMs with SFT + GRPO and SHAP-based faithfulness rewards (arXiv 2512.16022), is neither cited nor compared.

  4. The approach takes about 1.1 s/sample of LLM inference on top of running all candidates, vs. milliseconds for OptW/RLMC. Evaluation covers only univariate settings with L = H = 96 and N = 4. Significance rests on a sign test over 8 datasets, and no seeds or variance are reported.

Questions for authors
  1. What is the per-sample oracle MSE for each table, and what fraction of the oracle–OptW gap does REATS close? How does a non-LLM learner (e.g., LightGBM or MLP) on exactly the same hybrid features plus retrieved weights perform?

  2. Can you re-run Table 2(b) with a correctly tuned TimeXer, or with a pool that excludes candidates whose validation MSE is more than 3× the best? Does the 20.1% margin survive?

  3. For the reciprocal reward, how sensitive are results to k in the absence of outliers, i.e., with a standard-deviation-free (Dr.GRPO-style) normalization combined with the reciprocal mapping?

5 comments
AuthorsOpenReview ↗

Response to Critical Weaknesses 1

Response to Critical Weaknesses 1:

Weakness point (1): In the small-model group, TimeXer has an average MSE of 1.67 and up to 1.94. This is roughly 10× the other candidates in the same table, which suggests a misconfigured candidate. Two other candidates have MSE above 2 on Traffic.

Answer point (1): The candidate small modes are all based on the authors’ released implementation. The observed performance fluctuations are attributable to the data characteristics rather than model misconfiguration. To keep the runtime and computational cost of LLM-based ensembling manageable, we conduct experiments in a univariate forecasting setting with a relatively limited number of training and test windows. The relatively limited training data may make it difficult for some candidate models to adequately fit the dataset-specific data distribution. Meanwhile, a few test windows with exceptionally large prediction errors can independently and disproportionately increase the overall MSE.

For example, we examined the per-window errors of TimeXer on the reported evaluation windows. On Weather, the highest-error 1% of windows accounted for 99.2% of the total squared error. In fact, after excluding model misconfiguration as a possible cause, such variations in candidate reliability are common in practical forecasting and provide a useful test of ensemble methods’ ability to mitigate occasional large prediction errors. All ensemble methods use the same candidate forecasts and evaluation targets, ensuring a fair comparison.

Weakness point (2): Much of the "20.1% reduction" therefore comes from learning to zero out broken models, which a simple per-dataset validation filter would also achieve.

Answer point (2): The paper already includes InvMSE and OptW fitted on validation data. As shown in Table 2 here, OptW can assign zero weight to individual candidates. We also evaluated a simple per-dataset validation filter here. For each dataset, we ranked the four candidates by validation MSE and chose �0��0� to minimize the validation MSE of the equal-weight ensemble. We discarded all candidates ranked below the top kk and averaged the retained models’ forecasts equally on every test window.

As shown in Table 1, REATS-GRPO achieves the lowest average test MSE on the eight datasets and the lowest average MSE.

Weakness point (3): The Weather numbers (e.g., MSE 0.0030 and individual-model values from 0.0039 to 0.5683) also call for a scale check.

Answer point (3): Please refer to above Answer point (1) and Answer point (2).

Table 1: Comparisons with simple per-dataset validation filter.

MethodExchangeETTh1ETTh2ETTm1ETTm2WeatherElectricityTrafficAvg. ↓
InvMSE (validation)0.30640.15910.20580.08660.07830.19320.18690.07330.1612
OptW (validation)0.26290.15990.19110.09280.07330.19280.15400.08580.1516
Validation Top-k + equal weights0.53190.16470.19440.09790.07290.19140.14720.06670.1834
REATS-GRPO0.15870.14090.17000.07250.07200.02060.14760.08190.1080

Table 2: OptW weights for the small-model candidates. OptW can assign zero or near-zero weights to high-error candidates.

DatasetCARDLSINetPDFTimeXer
Exchange16.63%83.37%≈0%0%
ETTh189.92%2.69%5.39%2.00%
ETTh295.69%2.22%0.24%1.85%
ETTm192.42%3.74%2.60%1.24%
ETTm20%≈0%99.00%1.00%
Weather29.61%28.02%16.22%26.16%
Electricity9.58%14.23%76.19%≈0%
Traffic98.37%0.45%0.67%0.51%
AuthorsOpenReview ↗

Response to Critical Weakness 2 and Weakness 3

Critical Weakness 2:

The paper does not report the per-sample oracle MSE (the attainable ceiling), the best single model selected per sample by a kNN on the same features, or a gradient-boosted regressor on the same 8 feature groups plus retrieved weights. Without these, it is unclear how much of the gain needs an LLM at all. RLMC is the only learned baseline, and it uses raw inputs only.

Response to Critical Weakness 2:

Following the reviewer’s suggestion, we added a per-window oracle and several baselines, including kNN, a gradient-boosted regressor, LightGBM, MLP, and feature-based RLMC. We use k=3k=3 for kNN to match REATS’s retrieval size. For a fair comparison, the gradient-boosted regressor, LightGBM, and MLP use exactly the same query-window hybrid features as REATS, plus the aggregated top-3 retrieved weights, with categorical fields one-hot encoded. As shown in Table 1, REATS-GRPO achieves the lowest average test MSE among predictive methods in the small-model group (0.1080), compared with 0.1363 for the gradient-boosted regressor, 0.1369 for LightGBM, and 0.1366 for MLP.

Table 1. Per-window oracle and non-LLM baselines for the small-model group. We report test MSE and its mean across eight datasets. Bold marks the best predictive result at the displayed precision, excluding the oracle.

MethodExchangeETTh1ETTh2ETTm1ETTm2WeatherElectricityTrafficAvg. ↓
Per-window oracle (MSE lower bound)0.06990.09820.12570.04350.04530.00080.12170.06050.0707
kNN model selection0.21170.15240.17670.09080.07220.27950.14640.06670.1495
Gradient-boosted regressor0.22130.15590.18050.08180.08220.11380.15290.10180.1363
LightGBM0.22130.15580.18060.08170.08240.11850.15270.10190.1369
MLP0.23330.16690.18320.08360.08280.08750.15310.10240.1366
RLMC (raw series)0.38750.15720.30290.09700.08380.16660.15490.06660.1771
RLMC (feature based)0.44470.16290.19380.09480.08210.17400.14780.06660.1708
REATS-GRPO0.15870.14090.17000.07250.07200.02060.14760.08190.1080

Critical Weaknesses 3:

Weakness point (1): Rule-CoT is reverse-engineered from oracle weights through templates, so the SFT targets are post-hoc rationalizations by construction.

Answer point (1): Rule-CoT derives its SFT targets from oracle weights using templates. In our experiments, training on these targets achieved accuracy comparable to training on GPT-generated CoT, with negligible target-generation cost. This substantially reduces the cost of building SFT datasets for large-scale time-series tasks.

Weakness point (2): No faithfulness test is reported, such as intervening on stated features or checking weight changes when the claimed rationale is altered.

Answer point (2): In our held-out examples, we observed that the model adapted both its stated rationale and its weight allocation to the input, and that the two remained consistent. For example, across two held-out ETTh1 windows with different features, both REATS-SFT and REATS-GRPO generated different rationales and weight allocations. Both assigned 94% to CARD in one window and 97% to PDF in the other, illustrating how the reasoning and weights vary across test inputs.

Weakness point (3): The closest prior work, an LLM-as-judge ensemble of TSFMs with SFT + GRPO and SHAP-based faithfulness rewards (arXiv 2512.16022), is neither cited nor compared.

Answer point (3): Unlike Cao et al. (arXiv preprint: 2512.16022), who use an LLM to evaluate optimizer-proposed weights with a SHAP-based reward, REATS directly predicts per-window ensemble weights using CoT and a bounded reciprocal reward. We will cite and discuss this distinction in Related Work.

AuthorsOpenReview ↗

Response to Critical Weakness 4

Critical Weaknesses 4:

Weakness point (1): The approach takes about 1.1 s/sample of LLM inference on top of running all candidates, vs. milliseconds for OptW/RLMC.

Answer point (1): Although LLM inference introduces additional overhead, REATS provides significant gains in forecasting accuracy, interpretability, and cross-model/dataset generalization over OptW and RLMC, justifying the additional cost.

Weakness point (2): Evaluation covers only univariate settings with L = H = 96 and N = 4.

Answer point (2): We want to emphasize that beyond N=4, our REATS-SFT experiments with N=2, 6, and 8 also consistently outperform traditional baselines (as shown in paper figure 3 or Appendix Table 18). We adopt a fixed forecasting setting (L=H=96) to control experimental costs, following common practice in LLM and time-series foundation model evaluation (e.g., Timer). REATS is not inherently tied to a specific forecasting configuration and can be applied across different forecasting settings (e.g., different L, H and N) with multiple candidate forecasters.

Weakness point (3): Significance rests on a sign test over 8 datasets, and no seeds or variance are reported.

Answer point (3): As shown in Table 1 and Table 2, we conduct three random-seed experiments to compare our method with the ensemble baselines. InvMSE and OptW are deterministic given fixed candidate predictions and data splits, so we report a single MSE for each. Overall, our method outperforms the ensemble baselines across random seeds, demonstrating its robustness.

Table 1. Test MSE by dataset with TSFM candidates (MOIRAI, MOMENT, TimeMoE, TimesFM).

MethodExchangeETTh1ETTh2ETTm1ETTm2WeatherElectricityTraffic
Equal-weight ensemble0.19320.16240.20770.09440.17710.07660.45930.1508
InvMSE-tr0.18310.16210.20610.09400.16050.05600.43830.0831
InvMSE-val0.19150.16210.20620.09400.16020.00820.43790.0825
RLMC0.16690 ± 0.000430.16130 ± 0.000200.20389 ± 0.000890.09165 ± 0.000100.16760 ± 0.005260.00363 ± 0.000350.39012 ± 0.000120.08264 ± 0.00047
OptW-tr0.17030.16130.19880.09290.14290.04330.39320.0821
OptW-val0.19670.16180.20130.09280.14290.00690.39300.0823
REATS-SFT0.14803 ± 0.001180.14060 ± 0.000150.18658 ± 0.000220.08327 ± 0.000130.13830 ± 0.000250.00508 ± 0.000070.38506 ± 0.000830.08089 ± 0.00018
REATS-GRPO0.13807 ± 0.001150.13379 ± 0.000210.17641 ± 0.000050.08092 ± 0.000330.13210 ± 0.000050.00318 ± 0.000200.36912 ± 0.000420.07648 ± 0.00054

Table 2. Test MSE by dataset with small-model candidates (CARD, LSINet, PDF, TimeXer).

MethodExchangeETTh1ETTh2ETTm1ETTm2WeatherElectricityTraffic
Equal-weight ensemble0.31460.24200.27300.17530.18920.19260.31680.8353
InvMSE-tr0.18790.15170.18100.07810.07830.20810.17570.0750
InvMSE-val0.30640.15910.20580.08660.07830.19320.18690.0733
RLMC0.16089 ± 0.004730.15245 ± 0.000850.18386 ± 3.1e−80.07980 ± 0.001430.08385 ± 0.000080.24365 ± 0.095040.15021 ± 0.000020.06664 ± 0.00004
OptW-tr0.15610.15490.17950.07530.07400.20720.14860.0859
OptW-val0.26290.15990.19110.09280.07330.19280.15400.0858
REATS-SFT0.20967 ± 0.018330.15204 ± 0.002430.17794 ± 0.001450.07893 ± 0.000490.07944 ± 0.000210.04493 ± 0.002950.16079 ± 0.000320.09354 ± 0.00105
REATS-GRPO0.16435 ± 0.004960.14114 ± 0.000330.17115 ± 0.001130.07279 ± 0.000230.07282 ± 0.000750.03548 ± 0.013540.14766 ± 0.000050.08192 ± 0.00000

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.