acceptodds
Under review as a conference paper at ICLR 2027

ONE-Core: Evidence-Gated Coordinate Search for Time-Series Inductive Biases

Abstract

Architecture comparisons bundle observation, temporal, channel, capacity, and objective-side inductive biases, making whole-model winners difficult to interpret or transfer. We introduce ONE-CORE, a factorized testbed that audits one mechanism-linked bias coordinate at a time. Matched interventions measure its response, direct training-data descriptors propose where that response may change, and nested leave-one-dataset-out evaluation admits conditioning only when it improves on a fixed prior. Unsupported updates abstain locally; joint updates require a measured endpoint. Forecasting and remaining useful life (RUL) exhibit different compression preferences within the audited settings: a six-dataset forecasting audit retains a hierarchical prior, whereas C-MAPSS favors a compact single bottleneck before conditioning capacity. For MSE forecasting, a normalization gate reduces mean regret from 10.15% with fixed RevIN to 2.11% across twelve nested-LODO folds, with selective improvements on two folds. An in-family RUL capacity gate reaches 0.00% regret versus 2.66% for fixed dim16 across four C-MAPSS subsets; external bearing audits provide only directional corroboration. Anomaly detection (AD) instead audits readout, evidence source, and calibration. Its six-source diagnostic envelope reaches 0.438 macro oracle Best-F1 versus 0.412 for the best fixed source, but five datasets do not support a routing gate. These outcomes provide local design evidence with explicit scope and uncertainty, rather than a global architecture oracle.

Primary area:
learning on time series and dynamical systems
Keywords:
time-series modeling, inductive bias, model selection, dataset descriptors, evidence-gated model design, forecasting, remaining useful life, anomaly detection

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_iG5JOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies the inductive biases in time series modeling. It designs a methodology to analyze the impact of each design component on the performance of the model. Its studied design components range from normalization to local patching, and its analyzed use cases include forecasting, RUL, and anomaly detection. The findings are used to design time series models with the appropriate inductive bias for a given task.

Critical strengths
  • The paper touches on an important question in time series tasks, since the inductive biases in time series modeling play a very significant role, which is not necessarily involved in other domains, making the analysis very relevant.
  • The paper covers a relatively comprehensive list of aspects in the design of time series models, and the analysis has been shown on not only for forecasting tasks, but also for applications beyond.
Critical weaknesses
  • I feel that the presentation of the paper needs to be improved, as well as its positioning. In particular, it should be made clearer whether the major contribution of the paper is a method to analyze the inductive biases in time series modeling, or the insights derived from it. In terms of the results in section 4, I find them too densely presented. The paper needs to better highlight the main takeaways.
  • The paper mainly focuses on ablating every single design choice in the model, but there are two major questions that need to be answered more clearly:
    • How does that differ from a standard ablation study that people have already been doing?
    • Can the method handle cases where more than one inductive bias interacts with each other, showing a high correlation?
Questions for authors
  1. How specific is the proposed method for time series modeling? That is, can a similar method/framework be adopted in other domains, such as computer vision and natural language processing? An answer to this question can better position the significance of One-Core.
  2. Does the analysis apply only to a specific problem with a specific dataset? For example, can a similar analysis be done on a time series foundation model to reveal the inductive biases of the model, and potentially relate it to the properties of its downstream tasks?
2 comments
AuthorsOpenReview ↗

Reply to Reviewer iG5J

Thank you for asking us to clarify the relationship between the method and the findings. ONE-Core's primary contribution is a controlled procedure for determining which design decisions are supported by matched interventions and held-out evaluation within a stated scope. In v8, Figure 1 shows the workflow from descriptors and matched interventions to admission decisions. Section 3 describes the shared architecture before the experiments. Table 2 connects the evidence to each bias decision, and Section 4 opens each task's results with FIX, CONDITION, or ABSTAIN.

From matched ablations to design decisions

A matched ablation measures the response to a design change with other model choices held fixed. ONE-Core uses that evidence to determine which design decision is supported within a stated model family, candidate set, and evaluation protocol. A supported fixed choice (FIX), an admitted conditional override (CONDITION), and retaining the prior when evidence is insufficient (ABSTAIN) are three outcomes of the same procedure. For a specified descriptor and rule class, we fit the threshold and leaf actions outside each held-out dataset or subset. A conditional override must lower mean held-out regret relative to the best fixed choice and match or improve on that choice in at least two-thirds of folds.

In the original six-dataset H96 abstraction audit, the matched comparison identified 32→8 as the lower-regret fixed choice (0.49% versus 1.09% for Single-8). The context-memory rule tied it at 0.49%, so the testbed yields a fixed starting configuration for that protocol and candidate set (FIX). For RevIN, the selected descriptor rule yielded 2.11% mean regret versus 10.15% for fixed RevIN and met the fold criterion (CONDITION). The RevIN evidence is conditional on the selected descriptor and rule class within the original 12-dataset audit. Across two selected external targets, the frozen RevIN rule had higher mean regret than fixed RevIN. The anomaly detection evidence-source audit recorded heterogeneous responses and abstained from routing (ABSTAIN). Each decision records what the available evidence supports. Performance on a new dataset requires a separate evaluation.

V8 also adds a deterministic reuse diagnostic (Table 5; Appendix E). At budget four, evidence-based revalidation priority yields 0.431% mean regret versus 1.405% for random allocation. All budgets are reported, including a loss at budget one. This retrospective cross-fitted replay uses oracle correction and shared development rows; its three-budget adjusted tail fraction is 0.105. It illustrates a bounded downstream use of the response record, without establishing transfer or a newly admitted rule.

Interactions among design choices

For two binary choices, a matched 2×2 design measures each intervention at both settings of the other. In the Weather H96 experiment, removing RevIN lowered mean MSE from 0.1711 to 0.1634 with Patch and from 0.1741 to 0.1670 with LocalFull. The estimated interaction is about 0.0006 MSE in magnitude, using the rounded means (v8 §3, p. 7 and Table 19).

Before adopting multiple changes together, ONE-Core compares the measured joint configuration with the retained one. The RUL audit tested a joint change to locality, abstraction, and width and retained the capacity policy. On FD002, the mean paired gain against that policy was +0.11% (95% CI [−5.08%, +5.30%]). The interval includes zero and the gain falls below the 0.5% practical margin, so the audit records a tie (v8 Table 36). The one-axis audits measure each change separately. The joint comparison measures their combined effect. Estimating pairwise interactions would require intermediate configurations.

Applicability beyond the evaluated model family

The admission protocol could be instantiated in vision or language using matched interventions, domain-specific metrics and held-out units, and training-data descriptors for any proposed conditional rule. For a time-series foundation model, an initial application could hold the pretrained backbone fixed and compare two adapters or readouts under the same downstream task and tuning budget. A question about the bias of the pretrained architecture itself would require matched pretraining comparisons. The descriptor catalogue and empirical findings in this paper cover the evaluated forecasting, RUL, and anomaly detection settings within the ONE-Core model family.

AuthorsOpenReview ↗

Follow-up on our response

Thank you again for your review. Our response and revised manuscript clarify the contribution, the distinction from standard ablations, interactions between design choices, and potential applicability to other domains and foundation models.

As the discussion period draws to a close, we would appreciate any feedback on whether any of your original concerns remain unresolved.

Reviewer_92xDOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2(+1)
Soundness
2
Read review
Paper summary

ONE-Core changes one inductive-bias choice at a time inside one small model family. The choices are RevIN, interface, capacity, hierarchy and channel treatment, plus readout, evidence source and calibration for AD. A dataset-conditioned rule (a one-feature decision stump on a training-data descriptor) is admitted only if it beats the best fixed choice under nested leave-one-dataset-out (LODO) validation. There are three main results: a RevIN gate on window-scale drift (12 datasets, regret 10.15% to 2.11%), a capacity gate on C-MAPSS (2.66% to 0%), and no gate for AD.

Critical strengths

I like the attitude of the paper. The reporting is careful and honest: paired seeds, equivalence margins, explicit abstention and negative results. The authors also disclose the Nature15 concentration themselves. Matched one-factor contrasts are better evidence than whole-model leaderboards.

Critical weaknesses

I lean towards rejection. The protocol is sound in spirit, but the evidence for its two positive results is thin, and the paper is hard to read.

The admitted rules are not shown to transfer. I recomputed the gate from Table 18. Only Nature15 and Human3 fall below the 0.301 threshold. Nature15 gives 95.5% of the regret reduction, and without it the gate is worse than fixed RevIN (3.16% vs 2.69%, Table 39). The descriptor was also chosen on the same 12 datasets (L191-192, L679-683, L736-738). With the released descriptor file, D_shift would fail admission (11.02% vs 10.15%), and only 2 of the 10 descriptors pass, so the nested LODO estimate is optimistic. The RUL gate is fit on four subsets. Its split matches the single vs. multiple operating-condition split (L443-444), and 8 of 24 permutations also reach zero regret. The abstract's numbers suggest more generality than this supports.

Beyond that, I learned little that is new. The fixed priors the testbed recovers (RevIN by default, channel independence on ETT) are known. The obvious practical alternative is to train both options and pick one by validation on the target dataset, and the paper does not compare against it. The backbone trails strong published models in forecasting and AD (Tables 9 and 11), so I am not sure the measured responses hold for strong models. The main text relies on many coined terms and leaves the architecture and the experiment matrices to the appendix, which made the paper hard to follow.

Questions for authors

What is the regret if RevIN on/off is chosen per dataset by validation MSE? Which descriptors and rule classes did you try before freezing Dσ96, and what was their nested regret? Alternatively, could you test the frozen rule on datasets that no exploratory analysis has used? Strong answers to these two questions would move me towards acceptance.

Two smaller points. What is the bootstrap uncertainty of Dσ96 for ETTh1, Human3 and Nature15, given that ETTh1 and Human3 are each only about 0.008 from the threshold? Can the RUL gate be separated from the number of operating conditions?

Related work that may be of interest: van Rijn & Hutter (KDD 2018) and Probst et al. (JMLR 2019) study the same "fix vs. tune per dataset" question.

3 comments
AuthorsOpenReview ↗

Reply to Reviewer 92xD

Thank you for asking us to compare the RevIN rule with target validation and test its transfer. ONE-Core separates measured responses to matched interventions from decisions about reuse. Conditional admission requires lower mean held-out regret and performance at least as good as the best fixed choice in at least two-thirds of folds.

To improve readability, v8 explains the workflow in Figure 1, introduces the shared architecture before the experiments, and uses Table 2 and the opening of each task's results to state the FIX, CONDITION, or ABSTAIN decision.

Target-dataset validation

We reran RevIN on/off on all 12 forecasting datasets with five paired seeds and the original 50-epoch cap and checkpoint protocol. We selected the setting with lower five-seed mean MSE on the full validation split. All policies use the same rerun test results (v8 Table 4, p. 8).

PolicyMean test regretTest-oracle matchesTarget fits
Target validation MSE1.22%10/1210
Nested LODO descriptor gate2.11%8/125
Fixed RevIN on10.15%7/125

Validation selection recovered the better setting on Weather, AQShunyi, and AQWan, while the gate recovered it on Human3. Fit counts compare target costs for an already fitted gate and exclude gate development.

Descriptor selection and uncertainty

The historical exploration considered an amplitude-bearing trend score and predictive relative scale drift (P_scale). A P_scale threshold of 0.229 matched NYSE and Human3 and missed Nature15 and Finance77. On the original 12-dataset matrix, depth-one/depth-two rules gave 2.11%/9.64% for direct window-scale drift and 11.02%/9.63% for context-to-future shift. This comparison selected the scale-drift stump. In the separate post-review ten-descriptor screen, only scale drift (2.11%) and level-change correlation (2.47%) improved on fixed RevIN (10.15%). Refitted P_scale gave 11.08% (v8 Appendix A).

The original outcomes informed descriptor selection. LODO refits the threshold and leaf actions with the chosen descriptor and rule class fixed, so 2.11% is conditional evidence within this corpus. The gate improves two folds, ties nine, and worsens one. Nature15 accounts for 95.54% of the total reduction. Removing it reverses the comparison to 3.16% for the gate versus 2.69% for fixed RevIN.

We recomputed the descriptor using 10,000 moving-block bootstrap draws of training windows, with blocks of 16 windows and all channels resampled together. Holding each fitted LODO rule fixed, the action flipped in 26.55% of draws for ETTh1, 45.92% for Human3, and 3.38% for Nature15. Human3 was 0.00116 below its fold-specific threshold. Table 8 gives the intervals and thresholds. The flip-rate ordering persists at block lengths 8, 32, and 64.

Frozen forecasting transfer

We selected CzeLan and USBirths1 using training descriptors and locked both targets, predictions, and the full-corpus threshold (0.301218, off below) before observing their paired outcomes. We retained the backbone, 96-step input and horizon, 50-epoch cap, and five paired seeds, using the standard TFB rolling split. Both policies selected ON on CzeLan (test MSE 0.20272 on vs. 0.83424 off). The gate selected OFF on USBirths1 (0.35336 on vs. 0.48964 off), and all five seeds favored ON. With ON as the reference, the paired gain was −38.68% (95% interval [−45.40%, −31.95%]). Mean regret across the two targets was 19.28% for the frozen gate versus 0.00% for fixed RevIN. Appendix A reports the protocol and separates this check from the retrospective replay that includes KDD211.

RUL operating conditions and transfer

On four C-MAPSS subsets, the spectral gate (0.00% regret versus 2.66% for fixed dim16) exactly matches the single- versus multiple-operating-condition split. The matrix leaves the incremental value of the spectral descriptor unresolved, and 8 of 24 descriptor-row permutations attain zero regret. Applying the C-MAPSS-only threshold to all nine N-CMAPSS subsets retrospectively yielded 4.73% versus 1.04% for fixed dim16. Separately, the threshold frozen after the DS02/DS03 pilot yielded 3.51% versus 1.34% for fixed dim16 on the seven remaining subsets (v8 §4, p. 8).

The frozen rules had higher regret than the fixed comparators on their respective transfer sets. These results limit support for transferring those rules beyond the original RevIN and C-MAPSS audits. The testbed also identifies fixed choices within an audit. Across six H96 datasets, it favored fixed 32→8 over fixed Single-8 (0.49% versus 1.09% mean regret), and the descriptor rule tied fixed 32→8 at 0.49% (v8 §4). A conditional rule for a new corpus requires its own held-out evaluation.

Appendix E adds a retrospective rule-based allocation replay with shared folds and oracle correction: budget-4 regret 0.431% vs. 1.405% random; the three-budget adjusted tail fraction is 0.105.

Thanks for the thorough reply and the new experiments. v8 is a lot easier to read, so I've raised presentation to 2. I also accept that the original abstract was already hedged.

That said, the new results mostly confirm my concern. Target validation beats the gate (1.22% vs 2.11%), and in the new frozen checks both admitted rules do worse than the fixed choice (19.28% vs 0% for fixed RevIN, from the miss on USBirths1, and 3.51% vs 1.34% for fixed dim16 on N-CMAPSS). v8 is upfront about this, but now that the procedure is the main contribution, it weakens the case for the admission criterion itself, not just the two rules.

I appreciate how honestly all of this is reported, but I'm keeping my score at 2.

AuthorsOpenReview ↗

Follow-up to Reviewer 92xD

Thank you for clarifying that your concern is whether passing the admission criterion predicts useful transfer. We agree that the frozen results weaken the case for using the original admission decisions to justify conditional overrides on new corpora.

In forecasting, nested LODO evaluates the rule conditional on a descriptor selected using the same development corpus. We interpret each CONDITION result within its respective development evaluation. Applying either rule to a new corpus would require a separate comparison with the fixed baseline in that setting.

ONE-Core also reports what the evidence supports when a conditional rule is unwarranted. The forecasting abstraction audit supports retaining the fixed 32→8 hierarchy (FIX) under the evaluated protocol. With five datasets, the anomaly detection audit was limited to diagnostic comparisons and recorded ABSTAIN without fitting or evaluating a conditional routing rule.

Reviewer_E4J3OpenReview ↗
Rating
3(+1)
Confidence
1(−1)
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes ONE-Core, a modular testbed for studying architectural choices in time-series forecasting, remaining useful life (RUL) estimation, and anomaly detection. It measures individual design changes, uses training-data descriptors to propose simple conditional selection rules, and retains those rules only when leave-one-dataset-out evaluation improves upon a fixed configuration. Its main positive findings are a forecasting normalization rule and a capacity-selection rule within C-MAPSS; anomaly detection yields diagnostic comparisons without a validated selection rule.

Critical strengths

Careful internal evaluation and documentation. The paper uses paired multi-seed comparisons, training-only descriptors, explicit fixed-configuration baselines, dataset/family holdouts, and permutation diagnostics. These controls make its implementation-specific findings reasonably auditable.

Critical weaknesses

The methodological and mechanistic novelty is limited. The evaluated procedure largely combines modular ablations, manually chosen descriptors, threshold rules, and cross-validation; a sequential coordinate-search procedure is never actually executed (Sec. 1). I think the paper needs stronger evidence that its framework produces new, reusable design knowledge.

For instance, the highlighted “Single-8 versus hierarchy” comparison simultaneously changes attention/FFN depth, accessible tokens, and parameter count (Appendix A). It therefore establishes a preference between particular implementations, while leaving the contribution of hierarchical compression itself unresolved.

Questions for authors

See weaknesses.

1 comment
AuthorsOpenReview ↗

Reply to Reviewer E4J3

Thank you for pointing out the confounds in the original abstraction comparison and for asking what design guidance the procedure yields. V8 adds a two-stage control and clarifies the scope of the architectural results and the search procedure.

Abstraction control

The original Single-8 vs. 32→8 comparison changes the number of Q1 attention/FFN stages, the parameter count, and the encoder-token access span. We trained an 8→8 control on the same six H96 forecasting datasets with the same five seeds and 50-epoch cap. This audit uses the unpatched LocalFull encoder, which produces 96 encoder tokens from a 96-step input. The 12-token count in the appendix applies to the P16/S8 Patch configuration. Single-8 accesses eight tokens once (29,703 parameters). The two-stage paths both have 33,015 parameters and access 8 then 8 or 32 then 8 tokens. These spans describe access to encoder outputs, which can already mix information across input time steps.

We paired the new 8→8 scores by seed with the archived Single-8 and 32→8 scores. The archived ETT scores have four-decimal precision. For ETT 32→8, the available record is the official-split per-seed summary. V8 Table 3 (p. 8) reports the mean paired relative test MSE reductions and descriptive 95% paired t intervals below. Positive values favor the second configuration in each column.

DatasetSingle-8 vs. 8→88→8 vs. 32→8
ETTh1−3.38% [−10.83%, +4.07%]+3.03% [−1.60%, +7.66%]
ETTh2−0.10% [−0.79%, +0.58%]−0.97% [−2.63%, +0.70%]
ETTm1+1.00% [−0.46%, +2.46%]+2.44% [+1.89%, +3.00%]
ETTm2+0.17% [−1.83%, +2.17%]+0.79% [−2.55%, +4.13%]
Weather+1.63% [+0.60%, +2.67%]+0.35% [−0.60%, +1.30%]
Exchange+1.76% [−0.61%, +4.13%]−3.62% [−6.09%, −1.16%]

At fixed eight-token access, adding a second stage helped on Weather under our paired interval and 0.5% practical-margin criterion. This contrast changes depth and parameter count together. At fixed depth and parameter count, widening the first stage helped on ETTm1 and hurt on Exchange. The other contrasts were unresolved. V8 reports the original result as a comparison of complete abstraction paths and separates these controls. Appendix B gives the token counts for LocalFull and Patch.

Reusable design decisions and search scope

ONE-Core's primary contribution is a controlled procedure for determining which design decisions are supported by matched interventions and held-out evaluation within a stated scope. A conditional override must lower mean held-out regret relative to the best fixed choice and match or improve on that choice in at least two-thirds of folds.

Across the six H96 datasets, the matched comparison identified 32→8 as the lower-regret fixed choice (0.49% versus 1.09% for Single-8). The context-memory rule selected 32→8 in all six held-out folds and tied it at 0.49%. Retaining 32→8 is therefore a supported FIX decision for this model family, protocol, and candidate set. Recommending it for a new dataset requires its own evidence.

V8 adds a rule-based reuse diagnostic (Table 5; Appendix E). With the same initial LODO decisions, candidate set, and four-query budget, nearby source counterexamples prioritize revalidation and yield 0.431% mean regret versus 1.405% for random allocation and 1.426% for shuffled pairings. At budget one the policy is worse than random. This is a retrospective cross-fitted archive replay with ideal oracle correction and overlapping development folds; the three-budget adjusted tail fraction is 0.105. It provides a bounded example of consuming the response record to allocate revalidation, with the observation assumptions stated explicitly.

Our experiments perform coordinate-wise audits around task-specific reference configurations and evaluate selected joint endpoints. V8 describes this scope in the Introduction and Method and marks a later audit in Algorithm 1 (p. 6) as an optional extension. Evaluating sequential search would require adopting a change and rerunning the remaining coordinate tests around the updated reference.

To check whether the budget-four result in Table 5 depends on oracle correction, we kept the original query order and initial LODO decisions fixed and used the lower five-seed mean validation MSE to select the RevIN setting on each queried dataset. With four datasets queried, mean test regret was 0.431% for evidence-based allocation versus 1.811% in expectation for random allocation at the same budget. Random allocation performed better at budget one. This is a retrospective replay on the same 12-dataset corpus using saved full-run validation scores for both settings.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.