acceptodds
Under review as a conference paper at ICLR 2027

TimeCoder: LLM-Driven Synthesis and Refinement for Intervention-Aware Time Series Modeling

Abstract

Time series modeling in real-world systems often involves sparse and heterogeneous external interventions that abruptly alter underlying dynamics, posing significant challenges for both classical mechanistic models and purely data-driven approaches. While mechanistic models provide strong domain knowledge and interpretability, they lack flexibility under uncertainty, whereas deep learning models struggle to generalize intervention effects under limited supervision. We propose TimeCoder, an automated framework that integrates large language models (LLMs) reasoning, domain-informed mechanistic knowledge, and data-driven optimization to iteratively construct intervention-aware time series models. TimeCoder operates through a closed-loop generate–calibrate–refine process, where an LLM proposes explicit intervention-aware model structures grounded in domain knowledge and statistical evidence, model parameters are calibrated on observed data, and structured diagnostics of predictive errors guide subsequent structural refinement. We evaluate TimeCoder on four challenging benchmarks spanning healthcare, hydrology, and epidemics, demonstrating consistent improvements in forecasting and classification accuracy, generalization under sparse interventions, and sample efficiency compared to state-of-the-art baselines. Code is available at https://anonymous.4open.science/r/TimeCoder/.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series, Large Language Model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_mK9WOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TimeCoder, an LLM-based framework for intervention-aware time series modeling. The main idea is to use an LLM to automatically generate and refine time series model structures rather than relying on fixed architectures. TimeCoder first generates executable models based on domain knowledge and statistical information, then trains their parameters using observed data. Then, it improves the models by analyzing prediction errors and using feedback from previous iterations. The authors evaluate TimeCoder on four datasets from healthcare, hydrology, and epidemiology. The results show that TimeCoder outperforms existing deep learning and LLM-based methods in forecasting accuracy.

Critical strengths
  • S1. TimeCoder achieves strong forecasting performance across four datasets. It consistently outperforms existing deep learning and LLM-based baselines.
  • S2. The framework provides a flexible way to adapt model structures to different time series characteristics.
Critical weaknesses
  • W1. The authors argue that fixed model structures cannot adequately capture intervention-induced changes, but does not provide direct empirical evidence to support this claim.
  • W2. It remains unclear whether structural refinement is necessary for capturing intervention-induced changes. It should be discussed or compared against similar or even simpler approaches, such as intervention-aware training objectives, regularization, or data augmentation, can achieve same improvements.
  • W3. Although TimeCoder outperforms existing baselines, the experiments do not clearly isolate the contribution of structural refinement itself.
  • W4. Since intervention-aware modeling is the main motivation of the paper, the evaluation should more directly examine how well TimeCoder captures intervention-induced changes. Most results focus on overall forecasting accuracy.
  • W5. The paper does not sufficiently clarify what methodological advances TimeCoder introduces beyond adapting ideas in LLM-SR and HDTwin.
  • W6. The paper does not sufficiently analyze the computational overhead or performance–cost trade-off compared with existing methods.
Questions for authors
  • Q1. Could the authors provide empirical evidence that fixed model structures cannot sufficiently capture intervention-induced changes?
  • Q2. Why is structural refinement necessary for intervention-aware time series modeling? Could similar improvements be achieved using fixed architectures with intervention-aware training objectives, regularization, or data augmentation?
  • Q3. Could the authors provide an ablation that isolates the contribution of iterative structural refinement?
  • Q4. Could the authors provide a more detailed evaluation of intervention-specific performance?
  • Q5. What are the main methodological differences between TimeCoder and existing LLM-based model synthesis approaches (e.g., LLM-Sr and HDTwin)?
  • Q6. What is the computational cost of TimeCoder in terms of LLM calls, training time, and overall inference or development cost? How does this compare with existing methods?
3 comments
AuthorsOpenReview ↗

Author Response to Reviewer mK9W

Thank you for your thoughtful and constructive comments. We appreciate this valuable suggestion and provide additional discussion below.

Q3.1: What shows that fixed structures are insufficient?

A3.1: Table 7 provides empirical evidence through the white box only configuration, which uses the same data and calibration procedure but restricts model construction to white box instructions. Its RMSE is 0.372, 0.110, 0.515, and 0.014 on SF²Bench, Lung Cancer, VitalDB, and COVID-19, compared with 0.328, 0.092, 0.445, and 0.009 for the complete TimeCoder. This corresponds to relative RMSE reductions of 11.8%, 16.4%, 13.6%, and 35.7%. The initial SEIR structure captures the coarse epidemic dynamics, while subsequent refinements introduce latent memory, neural residual corrections, and more expressive intervention response mechanisms, leading to further reductions in the best historical RMSE.

Q3.2: Could simpler objectives, regularization, or augmentation achieve similar gains?

A3.2: As formulated in Section 3.2 and Equation 1, TimeCoder jointly searches over the model structure h and calibrates its parameters . A modified objective or regularizer optimizes within a predefined structure, while data augmentation expands the observed samples without changing the underlying functional form. They therefore cannot directly introduce missing mechanisms such as delayed intervention effects, latent memory, variable coupling, or nonlinear residual dynamics. Empirically, the white box only configuration in Table 7 remains consistently worse than the complete TimeCoder, despite using the same data and calibration procedure. Thus, these simpler techniques are complementary to TimeCoder, but they do not replace adaptive structural refinement

Q3.3: How will iterative structural refinement be isolated?

A3.3: We already isolate the two key mechanisms of iterative structural refinement through controlled ablations with the same backbone, data, calibration procedure, and generation budget. Table 5 removes historical refinement memory while retaining the same number of LLM generations. This increases RMSE by 6.6% to 35.7% across the four datasets, demonstrating the contribution of conditioning structural updates on previously successful hypotheses. Table 7 removes statistical residual descriptions under the same experimental setting, increasing RMSE by 5.7% to 18.2%, which demonstrates the contribution of error guided structural revision. Because the search budget is unchanged in both ablations, these improvements arise from iterative feedback and history conditioned refinement rather than additional LLM calls or candidate evaluations.

Q3.4: Is intervention-specific performance evaluated?

A3.4: Yes. Section 5.3 and Table 2 report intervention-specific results on VitalDB using drug-only test subsets. We evaluate both propofol, a common intervention, and remifentanil, a rare intervention, after removing all training samples involving the evaluated drug. On the propofol-only subset, TimeCoder achieves an RMSE/MAE of 0.972/0.368, compared with 1.034/0.395 for Time-LLM and 1.107/0.512 for iTransformer. On the remifentanil-only subset, TimeCoder achieves 1.061/0.462, compared with 1.126/0.498 and 1.184/0.526, respectively. Appendix Table 3 reports the intervention frequencies used to define the common and rare settings. Appendix Figure 4 further provides a qualitative post-intervention analysis on SF²Bench.

Q3.5: How does TimeCoder differ from LLM-SR and HDTwin?

A3.5: LLM-SR discovers symbolic equations using data fitting feedback, while HDTwin evolves general hybrid digital twins using overall and component level validation losses. TimeCoder is specifically designed for intervention-aware time series. It identifies high loss intervention-response prototypes and uses temporal lag, trend, periodicity, abrupt change, and bias diagnostics to refine stateful models with intervention mechanisms, latent memory, and neural residuals. This time-series-specific refinement is supported by the held out intervention results in Table 2 and the ablations in Tables 5 and 7.

Q3.6: What is the computational cost?

A3.6: Computational cost is analyzed in Section 5.4 and Appendix E.7. Figure 5 reports the mean and variance of token consumption at every refinement generation across all four datasets. Each run uses at most T=10 refinement generations on a single NVIDIA H200 NVL GPU. Token usage is concentrated in the early structural exploration stage and stabilizes as the model converges. Table 8 further shows that successful code repair requires only 1.4 attempts on COVID-19 and 1.1 attempts on Lung Cancer on average. This cost is incurred only during offline model discovery.

Thank you for the responses. However, I am still unconvinced about the necessity and contribution of structural refinement, as the current ablations do not directly isolate its benefits. I will maintain my rating.

AuthorsOpenReview ↗

Response to Reviewer mK9W

Thank you for the valuable comments! The motivation for structural refinement is stated in the Introduction and formalized in Section 3.2 and Equation 1. Sparse and heterogeneous interventions make temporal dynamics non-stationary, while limited data make it difficult for a single fixed architecture to represent diverse intervention-response mechanisms. Equation 1 therefore formulates learning as joint hypothesis search over h∈Hh \in \mathcal{H} and parameter calibration over θ∈Θh\theta \in \Theta_h, subject to empirical and knowledge alignment. Because H\mathcal{H} is discrete and combinatorial, direct optimization is intractable. TimeCoder approximates this search through iterative structural refinement, where each step proposes a new hypothesis or structural edit and then recalibrates its parameters.

This contribution is directly evaluated in Table 7. Under the same data, calibration procedure, and experimental protocol, restricting TimeCoder to white-box structures increases RMSE from 0.328 to 0.372, 0.092 to 0.110, 0.445 to 0.515, and 0.009 to 0.014 across the four datasets. Figure 2 and Appendix E.8.1 further show that, after the initial SEIR structure is calibrated, introducing latent memory and neural residual mechanisms produces additional reductions in the best historical RMSE. These results demonstrate improvements from explicit structural refinement that parameter recalibration alone cannot achieve.

Reviewer_mrMkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents TimeCoder, an LLM-driven framework for constructing intervention-aware time-series models. It integrates domain knowledge, statistical descriptors, calibration, and iterative refinement into a generate–calibrate–refine pipeline. The framework uses an LLM to propose executable model structures, calibrates their parameters using observed data, and employs counterexample selection and historical model memory to guide subsequent revisions. Experiments on four datasets including healthcare, hydrology, and epidemiology report improvements in forecasting and classification performance. The author also did additional analyses of unseen interventions and model refinement. However, there exist major consistencies in experimental results.

Critical strengths
  1. Relevant and practically motivated problem. Modeling temporal dynamics under sparse and heterogeneous interventions is important in some scientific applications. Combining mechanistic knowledge with data-driven learning is a promising direction.

  2. Reasonable integration of domain knowledge and empirical feedback. The use of temporal residual descriptors to inform executable model refinement is intuitively motivated and potentially useful. The generated models can also be deployed without additional LLM calls at inference time.

  3. Broad empirical evaluation. The paper compares against numerous forecasting baselines and includes experiments on held-out interventions, different LLM backbones, memory configurations, and individual framework components. The reported improvements can be potentially meaningful, if the experimental inconsistencies (see below) can be resolved.

Critical weaknesses

1. Inconsistency in the main experimental results.

Table 1 states that TimeCoder uses Qwen3-32B-Instruct, but its results exactly match those of Qwen3-Coder-Flash in Table 6. The reported Qwen3-32B results in Table 6 are noticeably worse. This discrepancy raises serious concerns about the validity of the paper's experimental claims.

2. Inconsistencies in experimental reporting.

Appendix E.8.2 reports 10 independent code-generation trials, yet Table 8 shows validity rates of 0.95 and 0.92, which are inconsistent with the stated sample size. Additionally, the maximum refinement iteration is set to 10, while Figure 2 and the appendix show up to 14 iterations. This needs clarification.

3. Limited methodological novelty.

The overall generate–calibrate–refine framework shares substantial similarities with existing LLM-based model discovery methods. The main additions appear to be time-series-specific diagnostic feedback and prototype selection. However, the paper does not clearly compare how their methodology advances over prior approaches.

4. Insufficient ablation of the core refinement mechanism.

While the paper includes several component ablations, it does not directly isolate the contributions of iterative structural refinement and prototype-based counterexample selection. It is also unclear how much of the improvement comes from additional model search rather than the proposed refinement strategy.

5. Limited evaluation of generalization to unseen interventions.

The held-out intervention experiments in Table 2 consider only two drugs on VitalDB and compare against two baselines. The results also lack uncertainty estimates and detailed sample counts.

Questions for authors
  1. Which LLM backbone was actually used for Table 1? Can the authors reconcile Tables 1 and 6? This is the most important issue affecting my assessment.

  2. What are the actual sample counts and refinement configurations? In particular, how were the valid rates in Table 8 computed from 10 trials, and why do the refinement trajectories exceed the stated maximum number of iterations?

  3. What is the measurable contribution of iterative structural refinement itself? Can the authors provide controlled ablations isolating prototype-based feedback and iterative structure updates?

  4. How are intervention forecasts and dataset splits constructed? Are future interventions known at prediction time or inferred by the generated model? Are clinical data split by patient rather than overlapping temporal windows?

2 comments
AuthorsOpenReview ↗

Author Response to Reviewer mrMk

Thank you for your thoughtful and constructive comments. We appreciate this valuable suggestion and provide additional discussion below.

Q2.1: Which backbone produced Table 1?

A2.1: The values currently shown in Table 1 were produced using Qwen3-Coder-Flash and were inadvertently copied from the backbone ablation. The intended fixed backbone for the main comparison was Qwen3-32B. The corresponding RMSE/MAE values are 0.364/0.185, 0.120/0.084, 0.476/0.331, and 0.042/0.035 for SF²Bench, Lung Cancer, VitalDB, and COVID-19, respectively. We will replace the current values with these Qwen3-32B results in the revised Table 1.

Q2.2: What are the actual trial counts and refinement settings?

A2.2: We apologize for the misunderstanding. In Appendix E.8.2, each trial refers to an independent batch containing 10 generated programs, rather than a single program. The 10 trial batches therefore contain 100 initial programs per dataset. Accordingly, the first pass validity rates are 30/100 for COVID-19 and 40/100 for Lung Cancer, while the final validity rates after repair are 95/100 and 92/100, respectively. Each initially invalid program was allowed up to K=10 execution guided repair attempts, and the averages of 1.4 and 1.1 were computed over successfully repaired programs. The main experiments use T=10 structural refinement iterations with the early stopping criterion in Equation 5. Figure 2 and Appendix E.8.1 present a separate extended 14 iteration COVID-19 case study for analyzing the refinement trajectory; it was not used to compute the main experimental results.

Q2.3: Does iterative refinement help beyond additional search?

A2.3: Yes. The component ablations use the same backbone, calibration procedure, and number of LLM generations, thereby controlling for the total search budget. As shown in Table 5, removing historical refinement memory increases RMSE from 0.328 to 0.351 on SF²Bench, from 0.092 to 0.112 on Lung Cancer, from 0.445 to 0.518 on VitalDB, and from 0.009 to 0.014 on COVID-19. Table 7 further shows that removing the statistical residual descriptions consistently degrades performance under the same generation budget. These results demonstrate that the gains arise from conditioning structural updates on diagnosed errors and previously successful hypotheses, rather than merely evaluating additional candidates. Figure 2 and Appendix E.8.4 further illustrate how specific residual diagnostics lead to targeted structural revisions and subsequent error reduction.

Q2.4: Are future interventions known, and how are clinical splits constructed?

A2.4: These settings are specified in the manuscript. Section 3.1 defines the forecasting input as a historical context window of observations and interventions, so future intervention values are not provided at prediction time. Appendix E.1 further specifies that, for VitalDB, the preceding 20 minutes of vital signs and medication records are used to predict the subsequent 10 minute vital trajectory. The generated model therefore rolls forward from the observed intervention history and models persistent or delayed treatment effects through its intervention and latent memory states. As described in Section 5.1 and Appendix E.1, the 6,388 VitalDB surgical cases are split into training, validation, and test sets with a 6:2:2 ratio before temporal windows are constructed. Thus, windows from the same patient do not cross data splits. Table 2 further evaluates held out interventions by removing samples involving the selected drug from training and evaluating them on the corresponding drug specific test subset.

Q2.5: What supports generalization to unseen interventions?

A2.5: Section 5.3 and Table 2 provide controlled held out intervention experiments on VitalDB. We remove all training samples involving either propofol, a common intervention, or remifentanil, a rare intervention, and evaluate on both the full test set and the corresponding drug specific test subset. On the common drug subset, TimeCoder achieves an RMSE/MAE of 0.972/0.368, compared with 1.034/0.395 for Time LLM and 1.107/0.512 for iTransformer. On the rare drug subset, TimeCoder achieves 1.061/0.462, compared with 1.126/0.498 and 1.184/0.526, respectively. These results show that TimeCoder generalizes to interventions excluded entirely from training, including both common and rare drugs. Appendix Table 3 reports the medication frequencies used to define these two settings.

Thanks for the clarifications which help to resolve some of my questions. However, the corrected Table 1 results weaken the main empirical claims. With Qwen3-32B TimeCoder only beats all the baselines on one of the four forecasting datasets.

I am still not convinced by the core methodological contributions. Tables 5 and 7 still have the Qwen3-Coder-Flash results in them and the ablations as currently presented don't actually provide a clean comparison of iterative structural refinement against an equal budget search baseline. I'll still keep my score of 2.

Reviewer_1s2COpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TimeCoder, an LLM-driven framework for automated model design in interventional time-series modeling. TimeCoder integrates LLM reasoning, domain knowledge, and data-driven methods to iteratively generate, calibrate, and refine the proposed models. TimeCoder is evaluated on several challenging benchmarks and compared against a large number of baselines to show its advantages on forecasting and classification tasks.

Critical strengths

The paper compares against a sufficient number of baselines, and the experimental workload is substantial.

Critical weaknesses
  1. The methodological contribution does not appear to be significant. LLM-driven iterative generate–calibrate–refine frameworks have already appeared in many related works, like HDTwinGen. The novelty of the paper may lie in the "Empirical Misalignment Identification with Prototypical Counterexamples" component. However, the paper does not provide sufficient detail to understand this part, and there is no corresponding ablation study. In addition, the ablation of the Statistical Evidence Description was not conducted carefully enough. Table 7 removes all five descriptors at once and reports only RMSE, without per-descriptor results.
  2. The paper describes its framework as "intervention-aware", but at first glance this appears to simply mean using data that contain interventions, together with intervention-related prompts and statistical descriptors. Moreover, the paper only addresses prediction under the existing interventions and does not consider counterfactual prediction when the interventions are altered, which is what matters most in causal decision-making.
Questions for authors
  1. In Section 4.3, how do you construct the prototypical samples that represent diverse intervention–response patterns? Please provide more details and an ablation study for this part.
  2. Since you consider interventional time series, how does your framework deal with time-varying confounding, which may affect counterfactual prediction, including on your benchmark datasets?
  3. There are many data inconsistencies throughout the paper. The most serious one is that the TimeCoder results in Tables 1, 5, and 7 are based on Qwen3-coder-flash (Table 6) rather than on Qwen3-32B, as claimed in the paper. In addition, there are several places where the text and the reported numbers conflict; for example, Section 5.1 sets T_max = 10, but Figure 2 shows 14 iterations, and Appendix E.8.2 reports 10 trials per dataset, yet Table 8 gives final valid rates of 0.95 and 0.92. Could you carefully review these parts?
  4. I cannot access the code through the provided anonymous link. Is there a way to resolve this issue?
  5. There are many \vspace commands in the main text, which already make the paper somewhat hard to read.
  6. Does the paper report the parameter calibration results? These are important for the model's interpretability. Could you please provide them?
2 comments
AuthorsOpenReview ↗

Author Response to Reviewer 1s2C

Thank you for your thoughtful and constructive comments. We appreciate this valuable suggestion and provide additional discussion below.

Q1.1: How are prototypes constructed, what is novel, and what ablation supports them?

A1.1: We cluster intervention signals and last-step dynamics using K-means (K=20), select the centroid-nearest sample per cluster, and return the five highest-loss prototypes, including predictions, targets, and residual summaries, to the LLM. Prototypes only compress diagnostic context; calibration and model selection still use the full training and validation sets. The method-specific contribution is this intervention-conditioned diagnostic signal, combined with temporal residual descriptors and memory-guided revision, rather than the generic cycle of generation, calibration, and refinement. We will isolate it using no-feedback, random-sample, loss-top-k, and clustered-prototype variants, plus per-descriptor ablations.

Q1.2: Does TimeCoder support counterfactual prediction or time-varying confounding adjustment?

A1.2: No. TimeCoder is designed for intervention-conditioned forecasting. Given observed histories of covariates and interventions, it predicts future outcomes. It does not identify causal effects, estimate counterfactual outcomes under alternative intervention policies, or adjust for time-varying confounding.

Q1.3: Can the backbone, trial-count, and iteration inconsistencies be reconciled?

A1.3: We apologize for the ambiguity. All main experiments were conducted using Qwen3-32B as the fixed backbone. Qwen3-Coder-Flash was evaluated separately in the backbone ablation and achieved better performance, likely because TimeCoder repeatedly generates, modifies, and verifies executable code. Table 1 inadvertently copied the Qwen3-Coder-Flash results into the main comparison. The correct Qwen3-32B RMSE/MAE values are 0.364/0.185, 0.120/0.084, 0.476/0.331, and 0.042/0.035 for SF²Bench, Lung Cancer, VitalDB, and COVID-19, respectively. We will correct Table 1, clearly separate the backbone ablation, and report the trial count and maximum refinement iterations consistently.

Q1.4: Why is the code inaccessible?

A1.4: The anonymous repository temporarily lost access to the source repository because its authorization had expired. We have restored access and made the code publicly accessible through the same anonymous URL.

Q1.5: Why are manual \vspace commands used?

A1.5: \vspace commands were introduced during final typesetting to meet the page limit by reducing local whitespace. They do not remove content or affect the reported results.

Q1.6: Are calibrated parameters reported?

A1.6: Yes. Section 4.2 describes how the parameters of each generated model are calibrated on the observed data using Adam. Appendix E.8.1 provides a complete COVID-19 case study of the resulting model evolution and calibrated parameters across refinement iterations. In particular, Figures 32 and 33 present the hybrid SEIR structure at iteration 10 together with its explicit parameter values, including =0.461, =0.098, =0.194, intervention effect =0.342, and the learned memory parameters. This example illustrates both how the model structure evolves and how its interpretable parameters are instantiated through calibration.

Thank you for the response. However, my previous concerns have not been fully resolved:

  1. The authors corrected Table 1, but Tables 5 and 7 still contradict Table 6: their full-model rows are the Qwen3-Coder-Flash results in Table 6, whereas the paper states that Qwen3-32B is the default backbone. The response does not address these two tables; since the authors acknowledged the problem in Table 1, it should be resolved here as well.

  2. Both the intuitive and the empirical evidence for why the Prototypical Counterexamples are effective remain insufficient. In particular, a single evaluation of the model involves thousands of samples, whereas there are only 20 prototypes, selected by clustering, and only the five with the highest loss are passed to the LLM. Why this is effective requires further validation.

  3. What the method can actually do is much more limited than the claim of "Intervention-Aware Time Series Modeling" suggests. At a minimum, the authors need to explicitly discuss the limitations of the method in the paper.

  4. For parameter calibration, the appendix only gives initial values for the COVID-19 case study, and the authors say this case study was not used for the main results. The paper does not report the optimized parameter values, and it does not compare them with the true values on the two simulated datasets. I think this is not sufficient.

Since the above concerns remain, I am inclined to keep my current score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.