acceptodds
Under review as a conference paper at ICLR 2027

A Foundation Model for Instruction-Conditioned In-Context Time Series Tasks

Abstract

In-context learning (ICL) enables task adaptation at inference time by conditioning on demonstrations rather than updating model parameters. Although recent time-series foundation models incorporate contextual conditioning, retrieval, or example-based prompting, they typically rely on implicit positional structure or task-specific objectives rather than explicit instruction-conditioned input-output demonstrations. We introduce iAmTime, a time-series foundation model trained with instruction-conditioned amortized meta-learning to infer tasks directly from example demonstrations. iAmTime represents each episode as a structured prompt over historical context and future-known variables using specialized semantic tokens that attend to designated time-series regions, exchange information across demonstrations, and inject task information into the query representation. The model combines a Hierarchical Multi-Scope Transformer Encoder, which captures temporal and covariate dynamics while inferring latent task structure from demonstrated input-output mappings, with a Task-Conditioned Patch Decoder, which adapts decoding through expert-based routing. We train iAmTime on large-scale real and synthetic corpora using supervised and self-supervised instruction-conditioned tasks, including forecasting, imputation, reconstruction, classification, anomaly detection, and source de-mixing. Across diverse domains, frequencies, and horizons, iAmTime improves zero-shot adaptation over strong time-series foundation baselines on probabilistic and point forecasting benchmarks, while achieving competitive or superior performance on four non-forecasting tasks.

Primary area:
learning on time series and dynamical systems
Keywords:
in-context learning, time-series foundation models, instruction-conditioned modeling, probabilistic forecasting, hierarchical attention, example-conditioned decoding, quantile regression, self-supervised learning, multi-task learning, zero-shot forecasting, few-shot learning, anomaly detection, time-series classification

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_pRFKOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes an unified in-context learning framework for diverse time series tasks, including forecasting, classification, anomaly detection, etc. The framework can be utilized for a specific task with only a support set of contexts rather than fine-tuning a task-specific head. Specifically, the framework is consisted by a Hierarchical Multi-Scope Transformer Encoder for temporal and covariate information capturing and a Task-Conditioned Patch Decoder for aggregating information among support and query samples. Trained by a meta-learning strategy, the method's effectiveness is demonstrated on several benchmarks across diverse time series tasks.

Critical strengths
  1. The paper is well-written and easy to follow.

  2. The method is well motivated.

  3. The idea of a unified framework for diverse time series tasks without fine-tuning task-specific head is interesting.

  4. Experiments on several benchmarks across diverse time series tasks demonstrate the effectiveness of the proposed method.

Critical weaknesses
  1. Limited comparisons and discussions with ICL-based methods. Similar ideas about use in-context learning for time series forecasting were already proposed in [1]. Moreover, the idea of ICL with within-sample self-attention and compressed cross-example interface has also been widely used in Tabular Foundation Models such as TabPFN-3 and TabICLv2. Clarifying the connections and differences with these methods would help distinguish the specific contributions of the proposed method. Also, comparing the existing TSFMs that without ICL looks not fair. The comparisons with ICL-based methods is also expected.

  2. For the experiments, I'm really curious about the relationship among different training tasks. Are they mostly positively transferring the knowledge or negatively? I'd like to see some empirical results of ICL training of each single task (for example forecasting) compared with the joint one. Moreover, in table 3, ICL gains not such obviously for forecasting tasks, especially for multivariate and covariate settings. Is there any analysis? Without ICL, the method still outperforms the existing TSFMs. What's the reason? Is this brought by the diverse training tasks?

  3. I'm also curious about the training and inference costs of the proposed method with ICL, compare with the baseline methods. Not only the inference time, but also the FLOPs and memory usage.

[1] In-Context Fine-Tuning for Time-Series Foundation Models

Questions for authors
  1. The proposed method do the information interaction between support and query samples with only the summary tokens, which reduce the computational costs but may also introduce an information bottleneck. I'm wondering would a total self-attention between all support and query tokens be better than only do cross-sample attention of the summary tokens. And would a larger amount of summary tokens be better?

  2. During training, the method includes diverse training tasks. How does the proposed method balance the training steps, losses, and gradients of different tasks?

6 comments
AuthorsOpenReview ↗

Rebuttal by Authors

Dear Reviewer pRFK,

Thank you for pointing us to related ICL work and for asking us to test the summary bottleneck directly. We answer each point below and report additional experiments that support the paper's claims. This part gives our answers and the key results; Parts 2–4 give the full tables.

W1 — ICL-based methods (Part 2). iAmTime shares the principle of within-sample attention with a compressed cross-example interface with TabICLv2 and TabPFN-3, and we will cite this connection. The difference from these methods and from TimesFM-ICF [1] is what the examples do: in iAmTime each demonstration is an explicit input–output pair, so the demonstrations define the task for one fixed checkpoint. The submission already compares with ICL methods (Chronos-2 and TabPFN-TS/-3 for forecasting; TiCT, TimEE and RocketPFN for classification). We added three comparisons with independent demonstration-based methods:

New comparisonReferenceiAmTime
TimesFM-ICF: ETT forecasting, average MAE at horizons 96 / 1920.318 / 0.355 (50 examples)0.271 / 0.300 (4 examples)
ICTP: held-out imputation, MSE on ETTh1 / ETTm1 / Weather0.785 / 0.934 / 0.231 (TimesFM+ICTP)0.759 / 0.910 / 0.224 (trained without imputation)
Flat Transformer, same parameters and demonstrations: UCR ICL acc. / Block-8 MSE72.5% / 0.099181.3% / 0.0727

W2 — Relationship among tasks, and the role of ICL in forecasting (Part 3). Transfer is positive overall: joint training preserves forecasting accuracy (46.8 vs 46.4 fev skill for forecasting-only training), and the joint model beats an imputation-only specialist (Block-8 MSE 0.0727 vs 0.0820). On Table 3: the query already specifies the forecasting task, so the default same-series demonstrations add little; with related-series demonstrations, skill improves in all three subsets. Strength without ICL comes from the diverse training tasks (+1.1 skill points) and, above all, from per-example fusion and the curriculum (9.0 and 16.6 points).

W3 — Cost (Part 4). On a common workload with four demonstrations, iAmTime and Chronos-2 have comparable latency (23.5 vs 23.7 ms per query); iAmTime uses 31% more FLOPs and about 0.5 GiB more memory. Four demonstrations cost 6.2× the FLOPs of none.

Q1 — Full self-attention, and more summary tokens (Part 4). We ran both, with four demonstrations.

Cross-example interfacefev skill (%) ↑GIFT CRPS ↓UCR ICL acc. (%) ↑Block-8 MSE ↓Anomaly F1 (%) ↑GFLOPs/query
iAmTime, 2 summary tokens (submission)46.80.46581.30.072793.57109.47
iAmTime, 4 summary tokens46.70.46481.70.071592.59111.77
iAmTime, 8 summary tokens46.90.46282.00.072194.06116.39
Flat Transformer: full self-attention, 300M44.50.48772.50.099188.71265.30

Full attention between all support and query tokens, at the same parameter count, is worse on all five metrics at 2.4× the FLOPs. More summary tokens give small, non-monotonic gains (at most 0.7 accuracy points).

Q2 — Balancing tasks (Part 4). We use no gradient surgery or loss weighting. Balance comes from curriculum sampling probabilities, one shared quantile loss in normalized target space, and demonstration-conditioned FiLM and expert routing. Sampling controls task exposure and joint AdamW optimizes the shared loss.

We will add these results and attributions to the paper and are glad to answer any follow-up questions.

AuthorsOpenReview ↗

Continued Rebuttals (2/4): Details for W1 - relation to ICL methods and the new comparisons

Connections and differences. iAmTime specializes the compression-based ICL principle of TabICLv2 and TabPFN-3 for demonstration-conditioned temporal tasks. Semantic tokens summarize designated regions (target history, demonstrated outputs, historical and future-known covariates) through masked Token Read; [START]/[MID] tokens exchange information across examples; and Token Write injects the inferred task into the query patches through FiLM. This read–exchange–write step repeats at every encoder layer. TimesFM-ICF [1] conditions a forecaster on related series placed in its context, and the task remains forecasting. Our contribution is the structured temporal interface and its episodic multi-task training.

ICL comparisons already in the submission.

  • Forecasting. Chronos-2 (cross-series conditioning) and the TabPFN-based forecasters, including TabPFN-TS-3 (labeled feature–target rows). On GIFT-Eval, iAmTime, Chronos-2 and TabPFN-TS score 0.680, 0.698 and 0.771 MASE (Table 19).
  • Classification. Three ICL classifiers that predict from labeled support examples without weight updates (Table 12): iAmTime obtains 71.03% mean accuracy, versus 60.66% for TiCT, 70.10% for TimEE and 72.86% for RocketPFN. Two of the ten datasets are in-domain for iAmTime; the eight-dataset means are in our response to Reviewer tHBy (Part 4).
  • Mechanism. Demonstration interventions (Table 4), the NoCEA and NoTWrite ablations (Table 5, Appendix J) and representation analyses (Appendix K) examine how the interface conveys task information.

TimesFM-ICF (forecasting). TimesFM-ICF has no public checkpoint, so we follow its rolling-window ETT protocol (Zhou et al., 2021) and compare with its published results (MAE; 512-step query history; TimesFM-ICF with 50 in-context examples, iAmTime with a random four sampled from the pool of 50 constructed using their selection procedure).

DatasetETTh1ETTh1ETTh2ETTh2ETTm1ETTm1ETTm2ETTm2AvgAvg
Horizon9619296192961929619296192
TimesFM-ICF (50 examples)0.3740.4000.3270.3710.3310.3640.2390.2830.3180.355
iAmTime (4 examples)0.2930.3380.3090.3150.2950.3040.1860.2420.2710.300

iAmTime with four examples is better on all eight settings, by 15% on average at both horizons.

ICTP (held-out imputation). ICTP is a reference for held-out-task transfer, so we compare on imputation and not on forecasting. iAmTime-NoImp, trained without imputation, is evaluated under ICTP's protocol (details in our response to Reviewer M6wx, Part 2). Entries are standardized MSE / MAE.

Held-out imputationETTh1ETTm1Weather
TimesFM+ICTP (reported)0.785 / 0.5920.934 / 0.6920.231 / 0.279
LPTM+ICTP (reported)1.141 / 0.7791.096 / 0.7640.331 / 0.411
iAmTime-NoImp, 4 demos0.759 / 0.5800.910 / 0.6570.224 / 0.275

iAmTime-NoImp has lower MSE and MAE than both published references on all three datasets (2.6–3.3% lower MSE than TimesFM+ICTP and 17–34% lower than LPTM+ICTP).

Flat Transformer (all four task families). Comparable ICL baselines for imputation and anomaly detection are lacking, so we trained a control under the same support–query formulation: a vanilla Transformer with 12 layers, hidden size 1024 and 16 heads (~300M parameters; parameter- and budget-matched), with the same explicit role tokens, a shared quantile head and staged training. The prompt is flattened into one sequence, so query patches attend directly to the demonstrations. CB-ICL, the transfer of our mechanism to a Chronos-Bolt backbone, is from Table 21.

Modelfev skill (%) ↑GIFT CRPS ↓UCR ICL acc. (%) ↑Block-8 MSE ↓Anomaly F1 (%) ↑
Flat Transformer (new)44.50.48772.50.099188.71
CB-ICL, 0 demos41.70.534–0.326177.83
CB-ICL, 4 demos43.20.512–0.200285.16
Full iAmTime46.80.46581.30.072793.57

CB-ICL classification is reported on the ten multivariate datasets of Table 12 (69.35%, against 71.03% for iAmTime). With prompt information matched, full iAmTime leads the flat Transformer on every metric, including 8.8 points higher UCR accuracy and 26.6% lower Block-8 MSE. The CB-ICL pair and the flat Transformer show that demonstration gains also transfer across backbones.

AuthorsOpenReview ↗

Continued Rebuttals (3/4): Details for W2 - task relationships and the role of demonstrations in forecasting

Single-task versus joint training. The forecasting-only comparison is reported in Tables 5 and 17 (NoMeta, NoExmp, Meta-NoExmp):

Variant (Table 5)Training tasksTraining demosfev skill (%) ↑GIFT CRPS ↓UCR acc. ↑Block-8 MSE ↓Anomaly F1 ↑
NoExmpforecasting onlyno46.40.4710.2380.193572.10
NoMetaforecasting onlyyes46.40.4680.2240.174074.30
Meta-NoExmpall tasksno47.50.4580.2400.081492.20
Full iAmTimeall tasksyes46.80.4650.8130.072793.57
  • Forecasting. Joint training does not hurt forecasting. With training demonstrations absent, adding the auxiliary tasks raises fev skill from 46.4% to 47.5% and improves CRPS from 0.471 to 0.458, so diverse tasks strengthen forecasting even with empty support.
  • Removing one task family (Table 5). The effect is concentrated on the removed task: NoCls lowers UCR accuracy from 81.3% to 27.4%, NoImp raises Block-8 MSE from 0.0727 to 0.1421, and NoAD lowers anomaly F1 from 93.57% to 80.64%. Effects on retained tasks are small and go either way; dropping synthetic classification, for example, improves GIFT CRPS from 0.465 to 0.460 and lowers UCR accuracy to 61.1%.
  • Single-task specialist (new). Block-8 imputation, MSE / MAE, same training budget:
Training task mix0 examples ↓4 examples ↓
NoImp (joint, imputation held out)0.1891 / 0.20900.1421 / 0.1784
Imputation-only ICL training0.0982 / 0.14500.0820 / 0.1311
iAmTime (full joint ICL)0.1730 / 0.19670.0727 / 0.1265

With four demonstrations, the joint model beats the specialist, which supports positive transfer. The specialist scores well at zero demonstrations because its training fixes the task. NoImp still improves with four demonstrations, showing adaptation to a held-out task.

Why demonstrations matter less for forecasting than for other tasks. In Table 3, four demonstrations improve SQL skill by 0.7 points on univariate tasks and leave multivariate and covariate-informed skill unchanged; average win rate changes by +5.6, +3.8 and −0.2 points. The query already specifies the forecasting task and supplies the history, related variates and covariates, while the default demonstrations are earlier windows of the same series (Appendix E.1) and largely repeat that information. Classification needs the examples to define the label mapping: accuracy rises from 20.6% to 81.3% when they are supplied (Table 4). Forecasting does respond to example quality: perturbing the demonstrations on held-out domains lowers win rate from 71.3% to 66.2% (Appendix J.8).

More informative demonstrations (new). We supplied related series from the same dataset, found by nearest-neighbor search over iAmTime encoder embeddings; related-series conditioning has a precedent in TimesFM-ICF. Entries are SQL skill / win rate (%).

fev-bench subset0 examples4: same-series history (Table 3)4: random cross-series4: related series
Univariate37.10 / 75.9037.80 / 81.5037.85 / 81.9138.35 / 82.60
Multivariate58.20 / 74.5058.20 / 78.3058.10 / 77.8058.85 / 79.00
With covariates48.40 / 73.5048.40 / 73.3048.35 / 73.4548.93 / 74.50

Related examples are best in every subset, improving skill by 0.5–0.7 points over the default. The gain also appears in the multivariate and covariate subsets, where the default demonstrations leave skill unchanged.

Why forecasting remains strong with zero demonstrations. The auxiliary tasks contribute (Meta-NoExmp vs NoExmp above). The larger factors are per-example fusion and the staged curriculum, whose removal costs 9.0 and 16.6 fev skill points (Table 5).

EvaluationMetricFull, 0 exFull, 4 exNoMeta, 0 exNoMeta, 4 ex
fev-benchSQL skill (%) ↑46.646.845.246.4
GIFT-EvalCRPS ↓0.4670.4650.4770.468

At zero demonstrations the full model is better than forecasting-only NoMeta, which shows the benefit of multi-task training for query-only forecasting. Demonstrations add little to the full checkpoint but recover almost all of the gap for NoMeta. The paper's central claim is demonstration-specified task adaptation; we will state that the forecasting benefit of demonstrations depends on the setting, and add the related-series result.

Reviewer_tHByOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors investigated the establishment of time-series foundation model which supports multi-task in-context learning. The authors proposed that current TSFMs are without multitask capability since they're missing demonstration-based contexts. The authors address this problem by proposing iAmTime, a TSFM pretrained via unified tokenization upon task contexts and data summaries, which achieves in-context capabilities by additionally pulling per-example summaries into tokens surrounding inputs. The authors provided extensive experiments on four tasks and iAmTime leads most of the baselines.

Critical strengths
  1. The paper starts from a clear motivation and frames their contribution clearly.

  2. The paper is with extensive experiments, supported by considerable amounts of details and ablations. The results justified the contribution of cross-example attention well. Furthermore, a mitigation of the ICL pipeline on Chronos-Bolt also shows good performance.

Critical weaknesses

1. The contribution on intellectual side is marginal

While the paper establishes a time-series foundation model architecture with considerable details, a large potion of them (tokenization design, demonstration-based inputs, temporal self-attention, token write) is either commonly used by previous TSFM or LLM designs or adopted from previous methods. There is novelty, though, in the start and mid token attention as justified by ablations, so this is not heavily negative to my recommendation.

2. The methodology and experiment are missing details and with conflicts

Although already equipped with extensive details and ablations, some details remain unclear in this paper, and some of them are in conflicts with each other. For example:

  • how is the examples of baseline method constructed? Are they sharing the same input of iAmTime or not? The authors clarified that they took forecasting performance off the leaderboard but didn't specify for the rest of the tasks.
  • Whatif the input or output sequence exceeds horizon limit? The authors mentioned that they're doing direct multi-horizon prediction (line 247) but also claimed "Longer horizons use blockwise autoregression" (line 1379). Also, the authors claimed to use one checkpoint for all, but didn't specify how to deal with various input or example length.
  • How are the ablations in table 17 and 18 conducted? Is the model maintaining the same input length? And is the removal done by zero-masking the parts? For example, NoToks-WExmp removes token stream but maintains examples. How is cross-example done then? (since B.3 says token stream is basically the four layers)
  • Some datasets in Table 23 also appears in the evaluation set as shown in table 25, like Electricity, Solar, and M4. Also, JapaneseVowels and SpokenArabicDigits are not belonging to any subset in Table 27, implying that they are used to supply pretraining, as mentioned by table 27 captions. Yet they both appears in the evaluation results in Table 13
Questions for authors

Please see weaknesses

9 comments
AuthorsOpenReview ↗

Response to the weaknesses and questions (1/4)

Dear Reviewer tHBy,

Thank you for the detailed review, and for recognizing the cross-example attention as a contribution supported by the ablations. This part gives our answers and the key results; Parts 2–4 give the details.

W1 - Contribution. iAmTime builds on established components (tokenization, demonstration-based inputs, temporal self-attention, FiLM modulation). Its architectural contribution is the role-aware pathway that carries a demonstrated input–output mapping to the query: Token Read summarizes role-specific regions, the query's [START] and [MID] attend to the support summaries, and Token Write conditions the query patches. We will state this explicitly in the contributions. The evidence that this pathway, and not the mere presence of demonstrations, drives adaptation:

  • Removing cross-example attention or Token Write drops UCR in-context accuracy from 81.3% to 26.3% and 24.6% (Table 5).
  • On a fixed checkpoint, reordering intact demonstrations leaves accuracy at 81.2%, while shuffling outputs across inputs reduces it to 11.0% (Table 4).
  • In a new control (run for Reviewer M6wx), a parameter-matched flat Transformer keeps the same role tokens and lets query patches attend directly to all demonstrations:
ModelRole tokensCross-example interactionfev skill (%) ↑UCR ICL acc. (%) ↑Block-8 MSE ↓Anomaly F1 (%) ↑
NoToks-WExmp (Table 5)noPatch fusion at aligned positions38.650.20.104881.55
Flat Transformer (new)yesFull self-attention44.572.50.099188.71
iAmTimeyes[START]/[MID] cross-attention, then FiLM46.881.30.072793.57

With prompt information matched, the pathway still adds 8.8 accuracy points and 26.6% lower imputation error.

W2a - Baseline inputs (Part 2). Every rerun baseline is evaluated on the same splits and test queries as iAmTime, in its native input format; demonstrations are shared wherever a baseline can take them. Part 2 lists, per task, which scores are published and which were rerun.

W2b - Horizon limit and input lengths (Part 2). The two statements refer to different regimes. The checkpoint predicts up to 1,024 steps (64 patches × 16) in one pass, and the longest evaluated horizon is 900, so every reported forecast is direct; autoregression applies only beyond 1,024 steps and was not used. One checkpoint handles different lengths by capping (4,096 steps of query history; 1,024 per demonstration segment) and masking NaN padding.

W2c - Ablations (Part 3). All variants keep the same input lengths, and components are removed from the architecture, not zero-masked. In NoToks-WExmp, cross-example exchange goes through the patch-fusion attention, widened to all supports and the query at aligned patch indices. Part 3 specifies every variant.

W2d - Datasets in both pretraining and evaluation (Part 4). Tables 23–25 list the source datasets with their original summary statistics before filtering and truncation. Our preprocessing truncates Electricity and M4 Monthly before the held-out target periods on shared series and removes the overlapping Solar series; we will add the series and time ranges used. JapaneseVowels and SpokenArabicDigits are outside the Table 27 subsets because their training splits contribute to pretraining. Table 13 reports them on their held-out test splits, to match the ten-dataset suite of prior work; Part 4 also gives the mean over the other eight datasets.

We will add these clarifications to the paper and are glad to answer any follow-up questions.

AuthorsOpenReview ↗

(2/4) Evaluation protocol - baseline inputs, horizons and input lengths (W2a, W2b)

Which baselines share iAmTime's inputs. We will mark in each table which scores were rerun or published, and where demonstrations were shared.

TaskPublished scoresRerun on matched queriesDemonstrations
Forecasting (App. E.1)Most fev-bench and GIFT-Eval baselines (official leaderboards)Chronos-2, TabPFN-TS, TabPFN-3iAmTime: four, from earlier windows of the query series
Classification, direct ICL (App. F)MOMENT and MantisV2 (from the TimEE paper), RocketPFN (its own paper)TimEE, TiCT (public weights and code); classical baselines via aeonUp to eight labeled examples from the official training split
Classification, linear probe (App. F)noneiAmTime, Chronos-2 and other encoders; ROCKET, MiniROCKET; statistical methods (aeon defaults)none (probe fit on training embeddings)
Imputation, anomaly detection (App. G–I)UniTS (same evaluation setting)UniTS and MOMENT as checks; Chronos-Bolt-ICLiAmTime and Chronos-Bolt-ICL share four per query
  • Forecasting. We reran Chronos-2, TabPFN-TS and TabPFN-3 because they support a form of ICL, aligning the history length up to each checkpoint's limit. Chronos-2 uses its native time-series inputs; the TabPFN variants encode the history as tabular context. iAmTime's four demonstrations are earlier windows of the query series (64–1,024 history steps); each window, including its target, ends before the query forecast origin. In the added ETT comparison with TimesFM-ICF (response to Reviewer pRFK, Part 2), TimesFM-ICF uses 50 demonstrations per query built following its approach and iAmTime a random four of them, with query history capped at 512 steps.
  • Classification, direct ICL. On test subset 3 of Table 27, each prompt has up to eight labeled examples from the official training split, at least one per represented class; series are capped at 1,024 steps. TimEE was rerun with its public weights and inference code on the matched queries and caps, and the same queries were reused for TiCT's public checkpoint. Classical baselines such as DTW-1NN use aeon implementations without demonstrations. Because our query sets match TimEE's, MOMENT and MantisV2 scores are taken from the TimEE paper and RocketPFN scores from its own paper; these retain their source protocols.
  • Classification, linear probes. On test subsets 1 and 2 (up to 37 classes, 1,024-step cap), all rerun methods use the same training and test examples; probes are fit on frozen training embeddings and scored on test embeddings.
  • Imputation and anomaly detection. Fixed test windows from the public UniTS pipeline are used for iAmTime and all rerun baselines, with matched masking settings and point-adjusted scoring with anomaly-ratio thresholds. The supervised baselines use a shared backbone with dataset-specific input projections and task-specific output heads (Appendix I). For imputation, iAmTime and Chronos-Bolt-ICL share the same four demonstrations per query from the training split, each pairing a masked input with its restored values; UniTS and MOMENT are evaluated on the same queries. For anomaly detection, iAmTime and Chronos-Bolt-ICL share four synthetic demonstrations per query, with context capped at 2,048 steps.
  • ICTP comparison (response to Reviewer M6wx, Parts 1–2). iAmTime-NoImp follows ICTP's protocol (channel-independent setup, chronological 60/20/20 split, training-split scaling, 192-step windows with 96 masked targets) and its four-demonstration budget.

For all ICL-capable baselines, the matched comparison uses the same support identities, support labels or outputs, query instances and random seeds wherever the interface permits.

Horizons and input lengths with one checkpoint.

  • Horizon. A 720-step forecast, for example, uses 45 future patches in one call (Appendix C.6). Beyond 1,024 steps, blockwise autoregression (line 1379) feeds each predicted block back into the history; it was not used for any reported number, and no model is trained per horizon.
  • Caps. 4,096 steps of query history; 1,024 steps each for the history and the future of a demonstration. These are configuration values (Table 9b), used in all evaluations.
  • Padding and masks. Shorter histories are left-padded and shorter demonstration futures right-padded with NaNs, up to the longest corresponding segment in the batch. Padding and missing values are masked. Covariates follow the same alignment; past-only covariates contain NaNs in their future segments.
  • Variable counts. Each target series is a (num_variates, length) tensor, so one pipeline handles univariate and multivariate inputs.

We will state these conventions, the handling of the number of examples and all evaluation-time caps in the paper, and relabel "Max output steps: 64" in Table 9b as "maximum output patches: 64".

AuthorsOpenReview ↗

(3/4) How the ablations in Tables 17–18 are implemented (W2c)

We match the lengths of retained inputs. Structural ablations remove the specified modules, and no-example variants mask support inputs during training. Components are removed from the architecture, not zero-masked; the only masking is of example inputs in the variants trained with K=0K=0. We will expand Table 17a with the details below for every ablation.

Full model. For each support or query block, temporal attention connects patches within a series, and per-example fusion (PEF) connects its target and covariates at the same patch index. Token Read and Token Self-Attention also stay within a block. Cross-example transfer occurs only when the query's [START] and [MID] attend to the corresponding support tokens; it reaches the prediction through FiLM and the decoder gate.

NoToks-WExmp: cross-example exchange without the token stream.

  • Removed: the role-token slots and embeddings, Token Read, Token Self-Attention, token-level CEA, Token Write and the token FFN. Retained: normalization, observation masks, temporal encoding and input lengths.

  • Replacement. After temporal attention within each series, the fusion group is widened from one example's variates to the variates of all supports and the query at aligned patch index kk:

    Zk=concat⁡(H1,:,k,:,…,HK,:,k,:,Hq,:,k,:),Zk←Attn⁡shared(Zk,Zk,Zk).Z_k=\operatorname{concat}\left(H_{1,:,k,:},\ldots,H_{K,:,k,:},H_{q,:,k,:}\right),\qquad Z_k\leftarrow\operatorname{Attn}_{\mathrm{shared}}(Z_k,Z_k,Z_k).

    Query patches attend directly to support patches at the same index; stacked temporal and shared-fusion layers propagate information across time and examples.

  • Decoder. Experts are routed by the masked mean of the query output-patch representations; the rest is unchanged.

  • Alignment example. With four supports of 1,024 history steps and a query of 4,096 history steps and horizon 256, every series has 272 patch positions: each support has 192 padding positions, 64 history patches and 16 output patches; the query has 256 history patches and 16 prediction placeholders.

  • NoToks-NoExmp is the same architecture with K=0K=0 (within-query fusion only), trained on NoExmp's forecasting episodes.

Other variants. Unless stated, a variant keeps the full architecture, the multi-task episodes and the demonstrations.

VariantImplementation
NoMetaTraining restricted to the forecasting episode families, with sampling probabilities renormalized within each phase.
NoExmpNoMeta's episode distribution with no example inputs in training (NaNs; K=0K=0); the example attention modules are masked out.
Meta-NoExmpMulti-task query inputs with no examples in training (NaNs; K=0K=0): example-side attention is masked out, the query-side Token Stream stays active.
NoPEFPEF removed; patches pass directly onward. Target and covariate information combine only through Token Read and Token Self-Attention.
NoCEACross-attention from support summaries to the query's [START]/[MID] removed. By construction, the query prediction is independent of support content.
NoTWriteBoth FiLM operations removed (Token Write is an identity map). Support information reaches the prediction only through expert-routing weights.
SharedToksEvery semantic slot initialized from one shared trainable embedding.
UniMoELearned routing replaced by the arithmetic mean of expert predictions; the routing projection and expert keys are removed. FiLM still conveys demonstrations.
NoClsReal-data and synthetic classification episodes removed, including classification branches of ambiguity episodes; remaining probabilities renormalized.
NoImpImputation/reconstruction episodes removed, with the ambiguity branches that supervise masked-input-to-clean-output mappings.
NoADAnomaly-mask prediction episodes and their ambiguity branches removed.
NoSynthClsOnly synthetic classification episodes removed; synthetic data used by other task generators retained.
NoCurriculumComplete episodes shuffled across the run with the same total counts, optimizer schedule and budget, so only the order of task exposure changes.

Common settings. Wherever retained, variants use the same sampled lengths, horizons, covariates and support selections (caps: 4,096 steps of query history, 1,024 of support history, 1,024 output steps). Evaluation is identical to the full model; without a Token Stream, evaluation demonstrations communicate through shared fusion. All variants share the initialization of common modules, 500,000 updates, batch size 256, loss, precision, optimizer schedule and checkpoint-selection rule. The training, output-specialization, task-family and curriculum variants have 300M parameters, NoCEA and NoPEF 270M, and the NoToks variants 160M. To complement the NoToks variants at matched capacity, we added the parameter-matched flat Transformer control shown in Part 1.

Reviewer_M6wxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

iAmTime uses input-output examples to specify time-series tasks at inference time. Its encoder combines temporal and variable attention, role tokens, attention across examples, and FiLM conditioning. A shared mixture-of-experts decoder supports forecasting, classification, imputation, anomaly detection, and source separation. Episodic pretraining encourages the model to learn the mappings shown in the examples.

Critical strengths
  1. The model provides a common interface for several time-series tasks. The role tokens and training episodes are designed to help it interpret the demonstrated task.
  2. The paper includes detailed ablations and recent forecasting baselines. Shuffling example outputs and removing demonstrations provide useful checks that the model actually uses the examples.
Critical weaknesses
  1. The improvement over the closest demonstration-based method, ICTP, is not clear. The added modules use familiar operations, and the NoToks-WExmp ablation also removes role and example-boundary information. Could you compare with ICTP on shared tasks, or with a simpler Transformer that keeps this information?
  2. The separation between pretraining and evaluation is not documented clearly enough. Table 23 includes Electricity at 15-minute and hourly resolutions in pretraining, while Table 25 includes Electricity in GIFT-Eval. This alone does not establish data overlap. Please provide the series and time ranges used in each stage and explain how evaluation targets are excluded from pretraining.
  3. Classification becomes expensive when the number of classes exceeds eight. Appendix F.1 runs inference over every eight-class subset, which requires 45 passes for ten classes. It also does not explain how confidence is calculated and compared across subsets. Please clarify this decision rule and report the inference cost, since both affect the comparison with other classifiers.
Questions for authors

Please see weaknesses.

7 comments
AuthorsOpenReview ↗

Part 1 of 4: Main response

Dear Reviewer M6wx,

Thank you for a careful and constructive review. We address each of your three points below, with additional experiments that support the paper's claims. This part gives our answers and the key results; Parts 2–4 give protocols and supporting data.

W1 — Improvement over ICTP, and a simpler Transformer that keeps role and boundary information. We ran both comparisons you asked for.

(a) ICTP on a shared task. We evaluated iAmTime-NoImp (Table 5; trained without any imputation) under ICTP's own held-out imputation protocol. Entries are standardized MSE / MAE; ICTP numbers are as published.

Held-out imputationETTh1ETTm1Weather
TimesFM+ICTP (reported)0.785 / 0.5920.934 / 0.6920.231 / 0.279
LPTM+ICTP (reported)1.141 / 0.7791.096 / 0.7640.331 / 0.411
iAmTime-NoImp, 0 demos1.401 / 1.1101.601 / 1.2710.623 / 0.702
iAmTime-NoImp, 4 demos0.759 / 0.5800.910 / 0.6570.224 / 0.275

Four demonstrations reduce the error by 43–64% and place the checkpoint ahead of both ICTP references on all three datasets. ICTP adapts its backbone on the training splits of these datasets, whereas they are not in iAmTime's pretraining corpus.

(b) A simpler Transformer. We trained a parameter-matched (~300M) flat Transformer that keeps the role and boundary tokens and lets query patches attend directly to all demonstrations.

ModelRole tokensfev skill (%) ↑GIFT CRPS ↓UCR ICL acc. (%) ↑Block-8 MSE ↓Anomaly F1 (%) ↑
NoToks-WExmp (Table 5)no38.60.56950.20.104881.55
Flat Transformer (new)yes44.50.48772.50.099188.71
Full iAmTimeyes46.80.46581.30.072793.57

The control confirms the value of the explicit role and boundary tokens in our design, and it isolates the architecture: with prompt information matched, the hierarchical design is better on all five metrics, by 8.8 accuracy points and by 26.6% in imputation MSE. Details are in Part 2.

W2 — Separation between pretraining and evaluation (Part 3). Our preprocessing follows TiRex's data preparation and Chronos-2's exclusion of benchmark targets across sampling frequencies. It removes the overlapping Solar, KDD Cup 2022 and Favorita series, and truncates Electricity and M4 Monthly to observations that precede held-out target periods on shared series. Tables 23–25 list the source datasets with their original summary statistics before filtering and truncation; we will add the series and timestamp ranges used in each stage.

W3 — Classification with more than eight classes (Part 4). The model predicts the class whose code is nearest to its output, with a softmax over code distances as confidence; across subsets we take the most confident label. This affects 3 of the 10 datasets in Table 13. A grouped procedure with 3 prompt evaluations per query, instead of 9 and 45, changes accuracy by 0.0 and −0.4 points, so the comparison with other classifiers does not depend on exhaustive enumeration.

We will add these results to the paper and are glad to answer any follow-up questions.

AuthorsOpenReview ↗

Part 2 of 4: Details for W1 — ICTP protocol, flat Transformer, role of demonstrations

ICTP comparison (table in Part 1).

  • Protocol. Following the ICTP paper and public code, we matched the channel-independent setup, the chronological 60/20/20 split, training-split scaling, and the test-window and masking rules for 192 input steps and 96 masked targets. Weights are fixed and Both NoImp runs use the same masked queries. The ICTP rows report published results under the cited protocol.
  • Margins. With four demonstrations, NoImp has lower MSE and MAE than both references on all three datasets: 2.6–3.3% lower MSE than TimesFM+ICTP and 17–34% lower than LPTM+ICTP.
  • Different settings. ICTP adapts an existing backbone on the benchmark training splits while withholding the evaluation task. ETTh1, ETTm1 and Weather are not in iAmTime's pretraining corpus (Tables 23–24), so NoImp is held out on both the task and the datasets.
  • Why this is the like-for-like comparison. iAmTime's primary setting trains jointly on its intended tasks, whereas ICTP deliberately withholds the evaluation task; withholding has a cost (Table 5: Block-8 MSE rises from 0.0727 to 0.1421 without imputation episodes). A forecasting comparison would mix architecture with task exposure and benchmark adaptation, which is why the paper discusses ICTP in prose. We will add this held-out comparison.

Flat Transformer control (table in Part 1).

  • Specification. A vanilla Transformer with 12 layers, hidden size 1024 and 16 heads (~300M parameters; parameter- and budget-matched), with the same explicit role tokens, a shared quantile head and staged training. The prompt is flattened into one sequence, so query patches attend directly to the demonstrations. NoToks-WExmp tests the joint removal of the explicit cues and the token stream; this control keeps the cues and changes only the architecture.
  • Reading. The flat Transformer recovers 22.3 of the 31.1 UCR accuracy points between NoToks-WExmp and the full model, which confirms the value of explicit role tokens. With prompt information matched, full iAmTime adds 8.8 UCR accuracy points, 4.9 anomaly F1 points and 2.3 fev skill points, with 26.6% lower Block-8 MSE and 4.5% lower GIFT CRPS.
  • On "familiar operations". The primitives are standard by design; the contribution is their composition into a role-aware read–exchange–write pathway. Each element is necessary for task adaptation (Table 5): UCR accuracy is 26.3% without cross-example attention, 24.6% without token write, and 68.9% with tied role embeddings, against 81.3% for the full model.
  • Curriculum. Multi-task episodes, the ingredient shared with ICTP, are not sufficient alone: UCR accuracy is 22.4% for forecasting-only NoMeta, 52.4% without the curriculum and 81.3% with it.
  • Transfer. The pipeline also transfers to Chronos-Bolt (Table 21): on one augmented checkpoint, four demonstrations versus zero give fev skill 43.2% vs 41.7%, GIFT CRPS 0.512 vs 0.534, Block-8 MSE 0.2002 vs 0.3261 and anomaly F1 85.16% vs 77.83%.

How much demonstrations contribute. Tables 4 and 14–16 vary only the demonstrations given to one fixed iAmTime checkpoint:

iAmTimeForecasting fev univ. skill (%) ↑Classification UCR acc. (%) ↑Anomaly F1 (%) ↑Block-8 imputation MSE / MAE ↓
Without ICL37.1020.678.090.1730 / 0.1967
Unrelated-task examples20.3618.467.820.2386 / 0.2471
With ICL37.8081.393.570.0727 / 0.1265

Demonstrations matter most when they specify the output mapping; shuffling only the example outputs drops UCR accuracy to 11.0% (Table 4).

Against other in-context classifiers on the ten multivariate datasets (Table 12, Appendix F.3), iAmTime obtains 71.03% mean accuracy, versus 60.66% for TiCT, 70.10% for TimEE and 72.86% for the classification-only RocketPFN. Two of the ten datasets are in-domain for iAmTime; the eight-dataset means are in our response to Reviewer tHBy (Part 4).

AuthorsOpenReview ↗

Part 3 of 4: Details for W2 - separation between pretraining and evaluation

Tables 23–25 list the source datasets with their original summary statistics. Before training, we preprocess them following TiRex's data preparation and Chronos-2's approach of excluding benchmark targets across sampling frequencies. For the data used in the reported results, we verified the following.

Shared sources are handled in one of two ways: the evaluated series are removed from pretraining, or they are truncated so that pretraining uses only observations that precede the held-out target periods.

SourcePretrainingEvaluationHandling
Electricity15-minute and hourly (Table 23)GIFT-Eval, 370 series at 15-minute, hourly, daily and weekly resolutionsTruncated: only time steps preceding the held-out target periods are retained
M4 Monthly48,000 seriesGIFT-Eval short-horizon setting, 2,400 series, 18-month horizonTruncated: for the evaluated series, only observations before that window are used
Solar5,029 series at each of the 5-minute and hourly resolutionsGIFT-EvalRemoved: 137 potentially overlapping series dropped at both resolutions, following TiRex
KDD Cup 2022nonefev-benchRemoved: excluded from the GiftEvalPretrain subset we used
Favorita Salesnonefev-benchRemoved: excluded from the GiftEvalPretrain subset we used

For Solar, the removal covers the hourly resolution shared with GIFT-Eval; the other GIFT-Eval Solar configurations use a different frequency.

We will extend Tables 23–25 to show the data as used after preprocessing. For shared sources, we will report the timestamp ranges of both the retained pretraining observations and the held-out evaluation targets, as described above.

Comments on the paper

1 comment
Area_Chair_xgiPOpenReview ↗

Discussion period ending soon

Dear Reviewers,

Thank you for your reviews of this submission. The authors have posted responses addressing the concerns raised.

With limited time remaining in the discussion period, please take a moment to review the authors' responses and post a brief follow-up indicating whether your concerns have been addressed or what remains outstanding. If your assessment changes as a result, please update your score accordingly.

Active engagement at this stage is essential for a fair and well-substantiated final decision. Thank you for your time and effort in reviewing this submission.

Best regards, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.