acceptodds
Withdrawn from ICLR 2027

TSCOMP-Agent: Automated Decomposition and Reassembly of Time Series Forecasting Models via LLM Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

The time-series forecasting literature is expanding rapidly, with new methods spanning an increasingly broad range of architectural paradigms. Many advances arise from specific mechanisms ranging from series normalization and decomposition to specialized architecture design, whose effectiveness often varies across diverse forecasting tasks. This motivates the question of whether effective mechanisms from different models can be identified as reusable components and composed into stronger forecasting architectures. Recent studies investigate this modular perspective, yet remain confined to manual expert curation, rendering continuous expansion unsustainable as the forecasting literature rapidly evolves. We introduce TSCOMP-AGENT, an LLM-based multi-agent framework that automatically decomposes forecasting models into verified and reusable components for automated reassembly. Building on a taxonomy-structured knowledge base, TSCOMP-AGENT synthesizes proposals from independent exploration agents into an executable plan. The implementation agent then translates this plan into modular components through iterative review and repair. Global consolidation finally verifies their compatibility and reusability before admission into the shared component pool. Experimental results verify that TSCOMP-AGENT reliably extends the component pool from diverse forecasting models across varied LLM instantiations. Models reassembled from this pool match or outperform human-designed baselines on multiple datasets, even under historical-cutoff and backbone-constrained settings, providing a scalable mechanism to accumulate design knowledge of time-series forecasting and facilitate automated architecture discovery.

Primary area:
learning on time series and dynamical systems
Keywords:
Time-Series Forecasting, LLM Agents, Multi-Agent Systems, Model Decomposition, Model Reassembly, Automated Model Design

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_RMaLOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes TSCOMP-Agent, a multi-agent LLM workflow built with LangGraph. It reads a forecasting model's paper and code, proposes how to split the model into components, implements those components as modules that follow the slot contracts of an existing component framework (TSCOMP / TSGym, Liang et al. 2026), and admits them to a shared pool after deterministic contract checks, reviewer-guided repair and a global consolidation/repair stage (Sec. 3.2, Alg. 1, App. B). The resulting pool is searched with random screening, pruning and TPE, and the top-5 validation configurations are ensembled (Sec. 3.3, App. E.3). The authors make three claims. (i) The workflow reliably extends the pool. Stage-wise ablations are in Sec. 4.2, and a comparison with "Prompt Only" and "Dynamic Workflow" baselines across three LLM backbones is in Sec. 4.3 and Table 18. (ii) The workflow generalizes to newly introduced models (Table 2). (iii) Reassembled models match or beat 13 reference models reproduced inside the framework and 4 AutoML/automated baselines on 8 LTSF datasets. These results hold up under historical-cutoff and backbone-constrained settings, both tested on ILI only (Tables 3-6). In total, 26 new components were extracted from 30 models (Table 11).

Critical strengths
  • The problem is real and well motivated. Component-level analysis of forecasting models (TSCOMP, TimeRecipe, CombinationTS) depends on manually curated component spaces that age quickly. Automating the step of turning a paper and its code into slot-conforming components is a useful, practical goal.
  • The engineering design is careful. Deterministic code owns all writes to the shared workspace. Compatibility contracts include rationales for each field (Eq. 8, Table 12). Repairs bind each fix to a specific failure, with no-progress detection. Failure attribution is separated from agent repair (App. D.5). Tokens and wall-clock time are logged precisely. The prompts are documented in App. B, which helps reproducibility.
  • The stage-wise ablations are honest about trade-offs and report negative findings. Exhaustive review lowers yield (Table 14). Local memory lowers novelty. Distinctiveness feedback lowers recall (Table 16). Standard deviations over 5 runs are given in the appendix.
Critical weaknesses
  1. The reassembly gains are not attributed to the agent. The central claim is that the agent-extracted components enable better reassembled models. But no experiment compares the same search and ensemble pipeline on the seed pool C0 alone with the pipeline on C0 plus the 26 new components. Without this ablation, Tables 3-6 could largely reflect (a) per-dataset/horizon search over a large configuration space and (b) top-5 ensembling, neither of which needs the agent. A report of how many of the selected top-5 configurations use agent-added components would also help. This is the most important missing experiment.

  2. The baseline comparisons are not like-for-like, and the "outperform SOTA" claims in the conclusion are overstated. (a) "Ours" is an equal-weight ensemble of 5 configurations chosen from about 200 trials per dataset-horizon cell. The baselines are single models with fixed reproduced configurations (App. E.2-E.3). (b) All trials are trained for at most five epochs with a shared training setup (App. E.3), and the baselines appear to use the same protocol. Several reproduced numbers are well below the values usually reported for these models. For example, Table 3 gives iTransformer Traffic at 0.624 and ECL at 0.227 average MSE, and PatchTST Traffic at 0.569. The comparison is therefore against weakened reproductions. (c) Only one random seed is used (seed 1, App. E.3), and the forecasting tables have no variance. On ILI, the validation-selected model is unstable: H=48 MSE is 4.467 against 1.834 for the oracle (Table 21). The conclusion says reassembly "discovers novel architectures that outperform SOTA baselines". It would be more accurate to say the pipeline beats in-framework reproductions under a short, shared training budget.

Questions for authors
  1. Can you report Table 3 (or at least a subset of datasets) with the same search and ensemble pipeline run on the seed pool C0 alone? How many of the selected top-5 configurations contain agent-added components?
  2. How many epochs and how much hyperparameter tuning do the reference baselines receive? Why are reproduced numbers such as iTransformer and PatchTST on Traffic and ECL far from commonly reported values? How do the results change for a single validation-selected configuration (no ensemble), or when baselines get the same ensembling?
  3. Did you check agreement between the LLM judge (MiniMax-M2.7) and humans on Consistency and Novelty? Were the "ground-truth decompositions" in App. D.2 human-made or LLM-assisted (App. D.4)?
  4. What evidence supports "outperforms manual curation" and "matches human-expert decomposition quality"?
Reviewer_FtEHOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes an LLM-agent workflow for extracting reusable components from time-series forecasting models and incorporating them into a modular forecasting library. The workflow includes exploration, proposal review, implementation, repair, and compatibility checking. The extracted components are subsequently used in model search and ensemble construction, with evaluations of both component extraction and forecasting performance.

Critical strengths
  1. It is a practical problem that turns new forecasting methods into reusable components for time series tasks, given the rising paradigm of TSFM. The paper presents a workflow provideing a structured way to identify, implement, and integrate mechanisms from existing models. This could reduce the manual effort required to keep a modular forecasting library current.

  2. The paper examines the trade-offs within the extraction process rather than reporting only final forecasting accuracy. The experiments investigate exploration, review, and repair, showing how these choices affect component quality, implementation success, and cost. This analysis is useful for understanding the design of an automated model-development workflow.

Critical weaknesses
  1. The evaluation part needs significant improvement. For example, the results section does not clearly isolate the value of the automatically extracted components. The final system combines library expansion with architecture search and ensembling, while already starting from a manually constructed component library. Without a comparison using only the original library under the same procedure, it is difficult to attribute the gains to automated extraction.

  2. The evaluation provides limited assurance that extracted components faithfully preserve the source mechanisms. Runtime compatibility establishes that a component can execute, but does not establish that it implements the intended computation. The reliance on LLM-assisted judgments, together with differing descriptions of the reference annotations, leaves uncertainty about the strength of the fidelity evidence.

Questions for authors
  1. I wonder how much of the forecasting improvement comes from the automatically extracted components? What happens when the original component library is evaluated using the same search budget, training procedure, and ensemble selection? Which newly extracted components appear in the final selected models, and how consistently are they useful across datasets?

  2. How reliably does the extraction process preserve the intended mechanisms of the source models? Specifically, what human verification supports the reference annotations and the LLM-assisted fidelity judgments? Please reconcile the description of “manually verified” references with Appendix D.4’s qualification that they are not an independently validated ground truth, and clarify whether functional equivalence is checked beyond successful execution.

  3. For the forecasting baselines to the proposed system, how do we understand the fairness? Table 3 has presented reassembled reference models. How closely do these reproduce the original implementations and their performance? Do the baselines receive comparable hyperparameter tuning and ensemble opportunities, or could these differences account for part of the reported advantage?

  4. How thoroughly do the compatibility checks cover the combinations used during model search? The described checks combine an individual new component with legacy components. Are combinations containing multiple newly extracted components also tested? What happens when components pass the initial checks but interact poorly during training or subsequent reassembly?

  5. How stable are the final forecasting gains across repeated searches? The decomposition experiments include multiple runs, while the forecasting search uses a fixed seed. Is there evidence that the selected architectures and their performance remain consistent across different search and training seeds, particularly on datasets where the gains over the strongest baseline are small?

Reviewer_uGGkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TSCOMP-AGENT uses LLM agents to extract reusable components from forecasting papers and source code, adapt them to a common interface, and combine them into forecasting models. Its workflow includes independent exploration, plan review, implementation and repair, and final compatibility checks. Starting from the existing TSCOMP component library, the system adds 26 components from 30 source models. It then evaluates candidate architectures and averages the predictions of five models selected using validation performance. Experiments examine the workflow stages, different LLM backbones, and forecasting performance across eight datasets, with additional historical-cutoff and fixed-backbone analyses on ILI.

Critical strengths
  1. The framework addresses a meaningful practical problem of incorporating new forecasting mechanisms into a shared library normally requires considerable manual analysis and implementation. The proposed workflow is carefully designed around explicit interfaces, source-code evidence, review, repair, and execution checks. The integration of these items into an automated decomposition and reuse pipeline serves as a useful and well engineered contribution.

  2. The evaluation gives useful and scientific evidence that the workflow matters. In Figure 5 and Table 18, TSCOMP-AGENT remains effective with MiniMax-M2.7 when Prompt Only and Dynamic Workflow extract few accepted components. Together with the comparisons across three LLM backbones and the stage-wise ablations, this supports the claim that structured orchestration can partly compensate for weaker underlying models. The validation-selected forecasts improve on the reassembled references on six of eight datasets and achieve the best MSE and MAE on four of six datasets in the external-method comparison in Table 4. These are meaningful demonstrations of practical usefulness beyond the oracle results.

Critical weaknesses

My main concern is whether the accepted components faithfully implement the mechanisms they are meant to extract. Appendix B.1 states that semantic review is advisory and final acceptance depends on runtime validation. These checks establish execution, expected output shape, and finite values. The fidelity to source computation is primarily assessed through LLM judgments only. Though Appendix D.2 describes manually verified reference decompositions, Appendix D.4 qualifies the proposal references as involving LLM-assisted alignment and not through independently validated ground truth. The manuscript should clearly state and describe an independent extert audit or equivalent computational validation of the final accepted implementations.

The custom metrics are useful diagnostics, but some metrics such as normalized compatibility measures runtime success relative to existing components, and consistency averages the ordinal LLM ratings. None of these establish a correctness rate. Their agreement with direct checks of the implemented mechanisms remains unclear. Since reliable automated extraction is the main and distinguishing contribution of the paper, this uncertainty limits confidence in the claims of matching human-expert decomposition quality. A representative expert audit or suitable computational equivalence tests should be described in the paper.

Questions for authors
  1. What validation establishes that the final accepted components preserve their intended source computations? Please clarify whether the manual verification described in Appendix D.2 covers these implementations. An expert audit of a representative sample or direct equivalence tests could resolve the main concern; agreement with the automated consistency and nonredundancy judgments would also be informative.

  2. How closely do the reassembled baselines in Table 3 reproduce the original implementations? Is there an MSE/MAE comparison under comparable settings, or other validation of this correspondence? Reassembly within a common framework is reasonable, but this clarification would help establish the scope of the model-specific performance claims.

  3. Is there evidence showing how much the 26 added components contribute to forecasting performance? Applying the same model-selection procedure, training budget, and five-model averaging to the starting component collection would help distinguish their contribution from the benefits of architecture selection and averaging. Existing results on a representative subset would be useful.

As a suggestion (perhaps for future work), extending Tables 5 and 6 beyond ILI would help establish how broadly those findings hold.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.