acceptodds
Under review as a conference paper at ICLR 2027

The Price of a Label: Static Identity in Heads on Frozen Time-Series Models

Abstract

Adding a store or platform label to a frozen time-series foundation model raises three separate questions: whether the label reaches the model (access), whether a fitted head gains from it (fitted value), and whether the gain carries to new data (transfer). We give an exact erasure condition: a static channel that enters only through centered values, with no bypass, loses its identity exactly. On 70 public retail series, three of five released forecasters give identical forecasts under every tested relabelling, and a fourth model's response is traced to unobserved entries and removed by neutralizing them. The standard permuted-label ablation can mislead: on one public retail subset at a frozen checkpoint, true store codes beat permuted codes over ten paired seeds yet lose to a label-free input code shared by all series, which keeps the lower mean loss in five further seeds, where true versus permuted is unresolved. On 3,087 development-seen fundraising campaigns, equal-width heads using true labels improve later seven-day path loss beyond group-mean and dispersion calibration by 0.0154 and 0.0048 nats on two frozen representations, with favorable adjusted seed intervals; for transfer, the same law yields no store-code contrast with a favorable adjusted interval on a public retail panel, so fitted value is population-specific. On 3,022 public crowdfunding campaigns, ordinary history recalibration is a strong reference, and adding centered category and country indicators lowers the mean loss by a further 0.03254 nats. A favorable true-versus-permuted contrast is therefore not by itself evidence that identity helps: label ablations need shared-code and ordinary-calibration references.

Primary area:
learning on time series and dynamical systems
Keywords:
time series foundation models, static covariates, label ablation, permutation baselines, frozen representations, forecasting heads, probabilistic forecasting

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_UwwCOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper asks whether static group labels (store, platform, category) help forecasters built on frozen time-series foundation models. It splits the question into access, fitted value and transfer.

  • Access. The authors give a sufficient condition (Proposition 1) under which per-channel normalization exactly erases a constant covariate. Tests of five released forecasters on 70 M5 series agree:
    • Chronos‑2, TimesFM‑3 and TabPFN‑TS give bit-identical forecasts under relabelling.
    • Moirai‑2's response is traced to unobserved padding entries.
    • TimesFM‑2.5 responds only negligibly.
  • Fitted value. On a proprietary fundraising panel, a count head on frozen Chronos‑2 or Moirai features reads a platform-code embedding. Over ten seeds, the true code beats constant and permuted codes, and also group mean-and-dispersion calibration. This holds only for campaigns seen during development.
  • Transfer.
    • Gains are unresolved for unseen campaigns and absent for M5 store labels.
    • On one Moirai‑2 checkpoint, the true code beats a permuted code but loses to a label-free shared code.
    • On MDCC, ordinary history recalibration is already a strong baseline.

The authors conclude that true-versus-permuted label ablations need shared-code and ordinary-calibration references. They propose a three-step check: relabel the input bit for bit, refit with the controls, and report results per population.

Critical strengths
  1. Clear framing of the question. Splitting it into access, fitted value and transfer is a sensible way to organize label ablations, and the paper is the first to address them systematically for frozen time-series foundation models.
  2. Exact, reproducible interface census. Testing five released forecasters bit for bit, with time-varying liveness controls, gives a concrete, cheap and public check. Tracing Moirai‑2's response to its padding entries, then removing it by intervention, is careful mechanistic work.
  3. Useful methodological caution. The paper shows that a favorable true-versus-permuted result can coexist with a loss to a label-free shared code. This is a concrete reason to add shared-code and calibration baselines to label ablations.
  4. Practically relevant side finding. Passing a covariate channel at all can change forecasts substantially, even when its value is erased. Practitioners comparing runs with and without covariates should know this.
  5. Transparency. Results are prespecified, every arm and population is reported (including null and adverse results), the claim status table (Table 56) is honest, and saved scores are reconstructed independently.
Critical weaknesses
  1. Limited novelty of the theory.
    • Proposition 1 amounts to (c − c)/s = 0.
    • Eq. 1 is the classical conditional-entropy identity, and the authors concede it "neither predicts nor bounds" the fitted gain.
    • The partition lemma and Gaussian-smoother examples restate known cautions on refit-based importance and covariance penalties.
    • No theory explains the paper's own key findings: the shared code beating true codes, the ordering flip between validation and later windows, and the lack of transfer.
  2. Weak, fragile empirical support for fitted value.
    • The gains are 0.005–0.015 nats per path.
    • Their intervals use optimization seeds as the unit of uncertainty and mostly include zero when campaigns are resampled.
    • On Moirai, the gain is concentrated in one 72-campaign group.
    • The ordering differs on the earlier validation split, which the authors cannot explain.
  3. Evaluation data reused and many analyses post hoc. The later window was scored under several protocols without adjustment across them, and several decompositions and resampling analyses were added after results were known.
  4. No transfer, and the main result cannot be verified.
    • The positive result is limited to development-seen campaigns in a private dataset.
    • It is unresolved for unseen campaigns and absent for M5 store labels.
    • The shared-code counterexample comes from one checkpoint with hand-injected codes, and the five-seed follow-up no longer resolves true versus permuted.
  5. Controls do not isolate information. The calibration baselines do not nest with the head, use a finite search, differ in capacity (588 vs. 840 parameters in MDCC), and lack confirmed convergence. "Beats calibration" therefore mixes information with function class and optimization.
  6. Limited practical payoff.
    • Results are reported only in nats, with no standard accuracy metric or decision-level impact.
    • No method, fix or rule for deciding in advance whether to use a label is offered; Table 56 marks this "Not evaluated."
    • The proposed protocol is essentially standard good practice.
  7. Presentation. Heavy hedging, 13 appendices and many separately adjusted experiment families make the core argument hard to reconstruct.
Questions for authors
  1. Can you report the fundraising results in a standard forecasting metric (e.g., MASE, CRPS or weighted quantile loss) and explain what a gain of 0.005–0.015 nats per path means in practice?
  2. Do the main fitted-value contrasts remain significant when uncertainty accounts for campaign and seed variation together?
  3. Why does the ordering of the true, constant and permuted arms differ between the development validation split and the later window?
  4. What mechanism do you propose for the shared code beating the true code, and does that result hold with more seeds or on other checkpoints?
  5. Can a matched recalibration control (equal capacity, nested with the head, converged fits) be added, so that "beats calibration" isolates label information?
  6. Beyond the relabelling check, what decision rule do you recommend for when a practitioner should include a static label?
1 comment

Thanks for the comments. What is new (W1, Q4). Proposition 2 isolates the cost a shuffle leaves out. It applies to balanced groups, an equiangular codebook, a linear readout and orthogonal nonlabel inputs. With lower risk better, true minus shuffled risk equals Λ−Γ, true minus shared equals Σ−Γ and shuffled minus shared equals Σ−Λ. Γ is label signal, Λ is signal that survives shuffling, and Σ is the cost of estimating group components rather than one common component. A shuffle cancels that cost. Thus true beats shuffled yet loses to shared precisely when Λ<Γ<Σ; the shuffle table bounds leakage before fitting (Appendix H). Centering a constant is Observation 1. Appendix B credits the classical information identity. In the nonlinear M5 encoder, the shared code also beats an erased input, 7.99774 versus 8.01432 nats; a linear model with a free intercept would give equality. Unequal code distances and the count loss also limit the analogy.

Fitted gains, uncertainty and transfer (W2–W4, Q1–Q3). On the fixed private cohort the mean log-loss gains over sequential calibration correspond to 1.55% and 0.48% on the geometric likelihood scale. Varying campaigns and seeds together, true minus calibration gives adjusted intervals [−0.06079,+0.00412] (Chronos-2) and [−0.01592,+0.00449] (Moirai-large). Crossing 27 encoded platforms and seeds gives [−0.07380,+0.00472] and [−0.03978,+0.01045]; even ordinary 95% intervals span zero. True versus constant also spans zero. Matched permutations have no corresponding crossed analysis (Appendix J.1). In the later cohort, every single-platform omission preserves the gain, but taking out one 72-campaign group leaves only 16% and 2% of the Moirai gains over shuffled and constant codes. All adjusted ordinary-calibration comparisons for 607 training-unseen campaigns span zero. Four earlier protocols used the later window, and our post-review analyses reuse it. These are opened cohorts.

For counts, daily CRPS with true labels versus sequential calibration is 0.01072 versus 0.01163 on Chronos-2 and 0.00958 versus 0.00934 on Moirai-large. All fitted arms beat zero (0.01569) in point estimates; fitted-reference intervals span zero. Medians beat zero on daily MAE; means lost (Table 4). On validation, constant beat true, and true beat shuffled. For 1,767 identical campaigns under the same heads, the true-minus-constant contrast decreases by 0.0074 from validation to later scoring (Table 19). Composition alone cannot explain that within-campaign change; its cause remains open.

On these same campaigns the Chronos-2 change stays negative after every platform omission. Moirai-large is near zero and 11 of 29 omissions reverse it (Table 45).

How strong is the public result (W4, W6, Q4)? On one M5 checkpoint, true beats shuffled in the original ten-seed log score and loses to a shared code. With five further seeds the true-minus-shuffled interval is [−0.00824,+0.00508]; a reserved shared code equidistant from all stores retains its advantage. The adjusted true-minus-shared CRPS interval is [+0.00126,+0.00446]; against shuffled it is [−0.00151,+0.00094] (Table 43). Across eight primary retail store transfer contrasts, none supports true. Only one checkpoint was tested.

Jointly resampling paired seeds and series within retail strata reinforces this distinction. All nine percentile shared-code intervals favor sharing. Seed-only Student intervals retain seven, a Student multiplier on crossed uncertainty three, and a small-seed correction two (J.7). Before extending to thirty new seeds we repeated twenty old fits. Their maximum mean discrepancy, 0.00153 nats, exceeded the 0.00050 tolerance, so no new-seed fit was run. All five shuffle intervals cross zero after adjustment across 14 contrasts on one checkpoint (Table 44). Resampling whole stores instead, under the original ten seeds, preserves the advantage over true codes: sharing wins in nine of ten individual stores, and the pooled shuffle contrast spans zero (Appendix J.7).

Controls and use (W5–W7, Q5–Q6). We have no equal-capacity, nested and converged private calibration control. Joint fitting raises gradient convergence from 100/740 to 734/740 groups; the later margins move by under 0.0001 and both intervals still span zero (Appendix D.3). On MDCC, history has 588 coefficients and the true metadata fit 840. The added shuffled comparison holds both arms at 840. True beats five shuffled fits by 0.03757 nats, under the penalty chosen on real labels. Neither meets the strict gradient threshold. The percentile date interval favors true over shuffled; a weighted Student sensitivity spans zero, as does the percentile comparison against history (Table 5, Section 5). The usual shuffle test omits the shared code, which on M5 changes the decision. For W4, public aggregate replays reproduce the contrasts, not the proprietary fits. For W7, Appendix A.1 maps the studies and Table 6 names the sampling units.

Reviewer_BionOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies whether providing extra labels (e.g., platform or store) to a frozen time-series foundation model will help the prediction or not. It finds that: 1. the label might be normalized out at the beginning so the TSFM may not have access to the label. 2. The regular practice that randomize the labels vs. true labels is not strict causal analysis. There also must be placebo test and ordinary calibration. 3. Providing the label to the head may have marginal but stable improvement, it is also highly population dependent. They finally suggest a better way to evaluate the value of additional labels.

Critical strengths
  1. The paper has a nice decomposition of the problem: access, fitted, value, and transfer. It might be meaningful insights for the TSFM community.
  2. They have a long limitation section, improving the general soundness, and show honest reporting
  3. Extra analysis on abnormal observations on Moirai-2.
Critical weaknesses
  1. Part of the experiments are on proprietary data, making it hard to reproduce. The only positive results seem to depend on this proprietary data.
Questions for authors

N/A

1 comment

Thanks for the comments. In an earlier count model on 30,490 public M5 series, the secondary department label reduces seven-day log loss for Chronos-2 by 0.00770 nats against a constant code and by 0.01107 against shuffled departments. All ten training seeds agree across seven departments and ten observed stores (Table 24). A new check resamples entire stores along with those seeds. The two adjusted intervals still favor true departments, both on the full panel and on the 6,165 series excluded from head fitting (Table 46). These forecasts share ten stores and the same dates. Under the later negative-binomial model, only the shuffled comparison excludes zero over seeds. The interval against group-rate calibration is [−0.0130,+0.0046] (Table 22). Neither adjusted department interval for Moirai-large excludes zero in the new store/seed comparison.

The opposite practical decision arises on 1,750 M5 series with a different frozen encoder. Under the original ten-seed log-loss comparison, real store codes beat shuffled codes. Yet every store can be given a single shared code and the loss falls further. Sharing wins in nine of ten individual stores, and the pooled advantage survives resampling stores and seeds together (Appendix J.7). The same ten fitted seeds favor shared over true when scored in sales-count units (CRPS, Table 43). A shuffle by itself misses the better choice.

We also report public crowdfunding results for 3,022 campaigns. True category and country metadata improve on five shuffled fits with the same 840 coefficients. True minus shuffled log loss is −0.03757 nats. Resampling nine launch dates gives a descriptive, unadjusted interval of [−0.06588, −0.00091]. Seven of nine consecutive dates favor true over shuffled, but a size-weighted Student interval across dates spans zero [−0.07668, +0.00154]. The date interval versus history also spans zero (Table 5 and Section 5). The shuffled fits reuse the penalty selected on real labels. The true recalibrators and the shuffled fits both miss the strict gradient threshold (Appendix I.3).

The private campaign data cannot be redistributed. Here the estimated gains over ordinary calibration span zero when campaigns or platforms and training seeds vary together. The public examples and their controls carry the practical claim. In the revision the opening example, Proposition 2 and the guide in Appendix A.1 explain how to read those comparisons. The accompanying guide points directly to the programs that recompute the public date intervals, retail scores and store sensitivity from aggregates. These reproduce the comparisons, not model training.

Reviewer_r7NcOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper looks at what actually happens when you add static labels (like store ID, department, etc.) to frozen time-series foundation models, and whether or not they help performance. The authors break down the evaluation into three parts - does the label even reach the model, does it improve it, and if that performance transfers.


The paper checks this on 70 public retail series across five released forecasters common in the community (Chronos-2, TimesFM-3, TabPFN-TS, TimesFM-2.5, Moirai-2) and finds that three of them output bit-identical forecasts no matter what static label you give them.

They then show that the standard "permuted-label ablation" used to prove a label's value when it is used in the model is flawed, because a shared code or just an ordinary calibration can often beat the true labels. The authors finish by proposing a straightforward three-part test for evaluating static labels.

Critical strengths
  1. Methodological checks are valuable to the community. Exposing the flaw in the standard permuted-label ablation is a great sanity check for the field.
  2. Interface auditing. Digging in and finding out that Moirai-2 was only responding to labels because of a bug with unobserved padding entries is great debugging work.
  3. Strong Baselines. Testing against ordinary history recalibration (adjusting for group-mean and dispersion) is a much better, more robust baseline than just relying on permuted codes.
Critical weaknesses
  1. Motivation for the interface tests needs better placement: The paper tests these models for static label erasure, but none of these interfaces actually have a native static channel. You do explain in Appendix A that practitioners encode static information into dynamic channels as a workaround, and on page 4 mention that “none of the five interfaces has a static channel into the pretrained network” but to avoid any confusion I feel it is more appropriate to state this front and center. Adding a sentence in the main introduction and/or abstract - "While these models don't officially support static labels, practitioners routinely encode them into dynamic channels as a workaround—so we tested whether that actually works” - would clear this up right away and ground the motivation much better.

  2. Proprietary Data: The main fundraising dataset with 21,625 campaigns is proprietary. This makes the core negative-binomial count head experiments impossible to fully reproduce end-to-end for the rest of the community.

  3. Opaque Metrics: Evaluating everything in "nats" (negative log-likelihood) makes the results really hard to ground in reality. A median campaign gain of 0.00166 nats means very little practically. The paper would be improved by metrics closer to the physical domain to show if these differences actually matter for downstream forecasting tasks.

Questions for authors
  1. Can you include some standard deterministic metrics (MAE, CRPS) alongside nats so we can see the actual physical impact of the label ablations?

  2. Preposition 1 necessary? Are we not just saying that subtracting the mean from a constant value equals 0

  3. The paper focuses on frozen foundation models and readouts. It doesn't tell us what happens during full end-to-end fine-tuning, where the trunk might just learn to bypass the normalization parameters to keep the identity info intact. Can finetuning enable label information to be useful in the cases where it is not just normalized out at input.

  4. Any better suggestions on how to include paths for such labels from the start in FM training?

1 comment

We agree that the median gain of 0.00166 nats is small. It corresponds to about 0.17% more probability on the observed path at the median campaign, with no measured financial benefit. We therefore scored the daily number of additional platform-reported participants on its own count scale.

What changes in count units (W3, Q1)? On the 3,087 development-seen campaigns, daily CRPS for true labels, the constant code and sequential calibration is 0.01072, 0.01153 and 0.01163 on Chronos-2. On Moirai-large the same scores are 0.00958, 0.00921 and 0.00934. Zero scores 0.01569. Section 4 (p. 5) gives all seven values. The four adjusted true-versus-fitted-reference intervals across platforms and seeds span zero (Appendix J.3). In this sparse panel predicted means lose to zero in daily absolute error. The median minimizes expected absolute error and every fitted median improves on zero. Chronos-2 has the lowest fitted median error with true labels (0.01333), Moirai-large with a constant code (0.01207, Table 4). We retained the unfavorable mean results and every later median, quantile and CRPS comparison. Table 43 gives public retail scores on the same count scale.

Does the static observation need a theorem (W1, Q2)? You are right that subtracting a constant's mean is elementary. We demoted it to Observation 1 and state the sufficient condition that makes it useful: every path from the repeated label to the predictor must discard that label. None of the five released interfaces supplies a dedicated static input to the pretrained network. On 70 public retail series three forecasters returned exactly the same forecast bytes when we changed the repeated label. In Moirai-2 the fill values of eleven unobserved entries still carried it. Zeroing just those values removed the static response while a time-varying control remained live.

What can be reproduced (W2)? The commercial records cannot be redistributed. The public retail and crowdfunding studies are separate experiments. The supplement guide points to replays of retail score contrasts and the three public launch-date intervals from aggregates, including the original compared means and the sampling units. These do not train the models or reconstruct the proprietary experiments.

What could a new architecture do (Q3, Q4)? If the inputs reaching the predictor are identical, changing downstream weights alone cannot distinguish the labels. One could learn a static embedding after normalization and a missing-label token, refit true, shuffled and shared embeddings, and withhold categories to test transfer (Limitations). QKCV and MetaTST illustrate relevant designs. In our separate UniCA adapter experiment on Chronos-Bolt-base, the path was live but the series-cluster interval for real minus reassigned departments included zero (Appendix B). This model differs from the head-side Chronos-2 department experiment. Neither is a test of full-backbone fine-tuning.

Figure 1 shows the decision at one public retail checkpoint. All nine adjusted seed-and-series percentile intervals favor sharing on the tested Moirai-2 checkpoint; all five adjusted true-versus-shuffled intervals span zero (Table 44). Sharing also beats true in nine of ten stores and the pooled contrast keeps its sign when whole stores and seeds vary together (Appendix J.7). Seed-only Student intervals favor sharing in seven of those nine comparisons, and a small-seed correction with joint variance retains two. Before extending to thirty new seeds we repeated twenty old fits. Their maximum mean discrepancy, 0.00153 nats, exceeded the 0.00050 tolerance, so no new-seed fit was run. A second injection-compatible checkpoint is still needed to test that finding elsewhere.

Comments on the paper

1 comment

Thank you very much to all the reviewers.

Two retail experiments lead to different choices. In a secondary Chronos-2 comparison under the earlier count model on all 30,490 M5 series, the real department reduced seven-day log loss by 0.00770 nats versus a constant code and by 0.01107 versus shuffled departments. All ten training seeds agreed on this panel of seven departments and ten stores. Table 24 gives the paired-seed intervals. Under the later count model, only the shuffled comparison excluded zero. The comparison with group-rate calibration included zero (Table 22). For store identity on 1,750 M5 series at one Moirai-2 checkpoint, the original ten-seed log loss favored true over shuffled labels, yet assigning every store the same code gave the lowest loss. Figure 1 shows how both comparisons change when entire stores and seeds vary. When we resample whole stores along with training seeds, both Chronos-2 contrasts for the seven departments retain their direction after adjustment across sixteen comparisons, including on the 6,165 series excluded from head fitting (Table 46). Both adjusted Moirai-large department intervals include zero on each population.

The new analyses ask whether this store result depends on the score or the sample. For daily distributional error measured in sales-count units (CRPS), a shared code scored 0.59268, real labels 0.59554 and shuffled labels 0.59582. Every seed favored the shared code over real labels. Under CRPS and scaled quantile loss, the adjusted true-minus-shuffled intervals included zero (Table 43). When we jointly resampled paired seeds and retail series, all nine adjusted percentile log-loss intervals favored a shared code and all five shuffle intervals included zero (Table 44). Resampling whole stores instead, under the original ten seeds, preserves the advantage over true codes: sharing wins in nine of ten individual stores, and the pooled shuffle contrast spans zero. A seed-only Student sensitivity supports seven of the nine shared-code contrasts; a small-seed correction with joint variance supports two (Appendix J.7). Before extending to thirty new seeds we repeated twenty old fits. Their maximum mean discrepancy, 0.00153 nats, exceeded the 0.00050 tolerance, so no new-seed fit was run.

On 3,087 private campaigns the full-distribution analysis gives opposite directions for the two representations. The real label has lower daily CRPS than ordinary recalibration on Chronos-2 and higher CRPS on Moirai-large. Crossing platform groups and seeds leaves all four true-versus-fitted-control intervals spanning zero. We also report appropriate medians for absolute error, ordinary and adjusted platform-by-seed log-loss intervals and the matched-campaign change between validation and later scoring. Section 4 and Limitations state this private-panel uncertainty.

Observation 1 states where an interface erases the label. Proposition 2 gives the linear conditions under which a shuffle conceals the cost of fitting distinct groups. The public crowdfunding comparison holds the true and five shuffled fits at 840 coefficients and reports both campaign and launch-date uncertainty (Table 5). The percentile comparison narrowly favors true across nine dates, while a size-weighted Student interval spans zero. Readers can recompute the retail score contrasts and the crowdfunding date sensitivity from public aggregate tables in the supplement. The paper lists the AI-assisted tasks and identifies the manuscripts that share data or code. The five matched public crowdfunding fits were specified after submission and completed before reviews were released. The new resampling analyses are retrospective. The paper identifies the sampling unit and data shared across studies.

For readers returning from the submission, its claims table (Table 56) is now Table 8, its public same-checkpoint study is in Appendix G, and its original public crowdfunding Table 4 is now Table 39. The new matched-permutation comparison is Table 5.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.