acceptodds
Under review as a conference paper at ICLR 2027

Mechanism-Equivalence Routing for Time-Series Foundation Models

Abstract

Sparse mixture-of-experts models scale capacity by conditionally reusing a small subset of parameters, yet the relation that should govern this reuse remains unspecified in time-series foundation models. Routing by trajectories or learned embeddings can merge similar observations with different predictive responses and split a common response across different operating conditions. We formulate mechanism equivalence as a support-conditioned operational reuse criterion: two target-window predictors may share response parameters when their baseline-subtracted multi-horizon responses agree on declared domains under admissible affine coordinate changes. Under canonical squared-error risk and a well-specified equivariant expert family, this relation admits a shared population-optimal response component with a unit-specific horizon baseline. MER operationalizes the relation with finite response probes, a canonical fingerprint, and prototype routing. A controlled CausalTime audit tests routing alignment under designed response families. In frozen target-stage evaluation, MER gives lower mean MSE than pattern routing in all ten directions and than the protocol-matched sparse baseline in all 40 dataset–horizon cells. These end-to-end results support frozen target-stage reuse under the evaluated same-backbone protocol.

Primary area:
learning on time series and dynamical systems
Keywords:
time-series forecasting, time-series foundation models, mixture of experts, sparse routing, mechanism equivalence, functional response fingerprints

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_zKbVOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper addresses the limitations of existing MoE routing methods for time-series foundation models, which assign experts based on observed patterns or learned representations without considering whether different time series share similar predictive mechanisms.

To address this challenge, the authors propose MER, which characterizes predictive relationships through normalized multi-horizon response fingerprints generated by perturbing relevant historical inputs. These fingerprints are clustered into shared prototypes to guide expert selection, which enables time series with equivalent predictive responses to reuse model parameters. Extensive experiments demonstrate improved forecasting accuracy and zero-shot generalization compared with existing MoE methods.

Critical strengths
  1. The major novelty lies in the definition of predictive response equivalence as the criterion for expert reuse, rather than the fingerprint encoding or clustering techniques themselves as indicated by the authors. Pertubrbation, representation learning, clustering, and Top-K routing techniques are all commonly used. Therefore, the novelty of the paper is moderate, though not minimal.
  2. The paper reports strong numerical results, consistently achieving lower forecasting MSE than the strong baseline Moirai-MoE across multiple datasets and forecasting horizons.
  3. The authors also conduct controlled synthetic experiments, ablation studies, and cross-dataset transfer evaluations, providing further evidence of the proposed method's effectiveness and generalization capability.
Critical weaknesses
  1. Even though the performance is clearly improved, MER includes additional response-model training, fignerprint calibration, and prototype construction. The extra training effort incurred should be quatified.
  2. The paper argues that similar observable patterns can correspond to different predictive mechanisms. However, if those mechanisms cannot be distinguished using the proposed method, no routing function can reliably identify the mechanism. To further improve the soundness of the work, it will be helpful to analyze which observable information enables MER to distinguish these mechanisms. I understand this can be a tough task especially for an academic paper. Therefore, this weakness does not impact my decision about this paper.
  3. MER relies on a pretrained model to estimate predictive relationships, but errors in these estimates may lead to incorrect expert selection, especially under domain shifts. Although the paper includes an oracle routing baseline, the authors could further compare routing based on true and predicted responses in synthetic experiments to better understand how estimation errors affect performance.
Questions for authors

Refer the the weaknesses

1 comment
AuthorsOpenReview ↗

Learned versus generator responses, observable information, and source cost

Thank you for suggesting the direct comparison with generator responses. It lets us measure how often the selected experts change when we improve the responses supplied to a frozen router. We report this comparison in §6.4 and Table 22, alongside a separate test of fingerprint sensitivity. We address the three weaknesses in the order listed in your review.

W1 · Added training cost. On a named source, support–response fitting took 1.50 h, fingerprint calibration 1.00 h, offline clustering of 100,000 128-dimensional fingerprints 0.12 h, and forecast co-adaptation 2.73 h. The total was 5.35 elapsed hours including clustering. The respective update ceilings for the three trained stages were 62,100, 38,100 and 89,300; validation-based stopping may finish earlier (Appendix H.1, p. 27).

Raising the Stage-4 ceiling to 189,500 improves frozen ETT macro MSE from .33876 to .33210 for Pattern and .31641 to .31284 for Graph. MER stays at .30775, a .00509 (1.63%) advantage over extended Graph, which took 5.79 elapsed hours (§6.4, p. 10; Table 25). This is a wall-clock comparison; training FLOPs were not matched. At inference, the full procedure raises per-window latency from 17.5 to 20.4 ms and compute from 3.3 to 3.5 GFLOPs, with 2.1M auxiliary parameters (Table 1).

W2 · Observable information about the regime. In Proposition 3 (§5, p. 7), the two regimes share marginals and second-order summaries. With G(x)=x3−3x/5G(x)=x^3-3x/5, the covariance term vanishes because E[XsG(Xs)]=0\mathbb{E}[X_sG(X_s)]=0. But at forecast origin tt, a completed historical pair with s+1≤ts+1\le t carries the nonlinear signal

E[G(Xs)Ys+1∣Z]=Za E[G(Xs)2].\mathbb{E}[G(X_s)Y_{s+1}\mid Z]=Za\,\mathbb{E}[G(X_s)^2].

Its sign follows the persistent regime ZZ. MER can condition its response estimator on already observed multichannel history and query it at canonical input values. This example identifies information available in principle. If the balanced regimes have identical laws over everything the router observes, a classifier using those observations cannot exceed 50% population accuracy (Corollary 1).

W3 · Learned versus generator responses. On 4,000 held-out CausalTime windows, we fix one checkpoint (training seed 17) and replace only its learned canonical-query responses. We estimate the generator's conditional means from 4,096 trajectories per query. Both branches share the six selected parent–lag roles, queries, fingerprint encoder, prototypes, experts, and forecast head. Each applies the original single-parent normalization to its own responses.

Response supplied to router96-step MSE ↓MRC ↑MSS ↑
Learned.09831.8017.7879
Generator conditional mean.08674.8746.8834

The substitution lowers MSE by 11.77% without retraining. The unordered Top-2 sets agree on 3,427 windows (85.675%) and change on 573. MRC and MSS use the same 120 same-class and 160 different-class pairs in both branches (§6.4, p. 9; Table 22, p. 26). For comparison, Table 19 reports a five-run MER mean of .097 and an oracle MSE of .076. That oracle receives the true reuse class directly. The generator-response branch above still selects experts through the learned fingerprint and prototypes.

We also perturb the normalized 128-dimensional generator-response fingerprints. At displacements .05/.10/.20/.40 measured after renormalization, Top-2 agreement is 96.2/85.7/71.4/52.8% and MSE is .08912/.09804/.11263/.13820 (Table 23). The .10 setting resembles the learned branch in its aggregate agreement and MSE; it does not estimate that branch's per-window errors. Proposition 5 (p. 15) gives a sufficient condition for each window: displacement below half the second-to-third cosine-score margin preserves the set. Mixture weights can still change.

Separately, all eight ETT/PEMS directions excluding the target from MER source training favor MER over Graph. ILI/Weather establish frozen target evaluation with unknown inclusion in the source mixture (Tables 16–17). These transfers measure forecast utility under shift; response-estimation error itself was measured in the controlled study.

Reviewer_bWJqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Mechanism-Equivalence Routing (MER), which assigns time-series data to different experts based on their predictive responses rather than similarities in their observed patterns. The method builds response-based fingerprints to identify which inputs can share the same experts. Experiments on synthetic and real-world datasets show that MER improves routing consistency and forecasting accuracy compared with existing baselines.

Critical strengths
  1. The response-based routing strategy offers a somewhat new perspective on expert selection.
  2. The method shows promising zero-shot performance across different datasets.
Critical weaknesses
  1. The fingerprint is normalized using single-parent responses, which can be zero for pure interactions such as (xz), making the interaction features unstable.

  2. In my understanding, the method only considers the previous eight lags across datasets with very different sampling frequencies. This choice may miss important longer-range dependencies.

Questions for authors
  1. How sensitive is MER to the number of prototypes (K=32)? Would fewer or more prototypes significantly affect routing performance?

  2. In Table 2B, how is the alignment between prototypes and expert indices handled when combining banks and routing pipelines from different methods?

  3. For ILI and Weather, the source-mixture checkpoint may contain target data, which is a bit confusing to me, could the authors clarify how this potential data exposure affects the zero-shot evaluation?

1 comment
AuthorsOpenReview ↗

Interaction-scale correction, lag/prototype sensitivity, and routing alignment

Thank you for pointing out the pure-interaction case. Your example exposed a failure of the original denominator. We revised Eq. (10) and Proposition 4, and Table 4 (p. 10) now reports where the correction changes forecasting and routing. Numbering follows the order of your review.

W1 · Interaction normalization. For g(x,z)=xzg(x,z)=xz at centered probes, both single-parent contrasts vanish while the raw pairwise contrast is abab. The original scale therefore reaches its floor. Equation (10) (§4.2, p. 5) now defines one run-specific denominator, σˉr,u=max⁡(σr,u,ϵflat)\bar\sigma_{r,u}=\max(\sigma_{r,u},\epsilon_{\mathrm{flat}}). The default takes σr,u=σsingle,u\sigma_{r,u}=\sigma_{\mathrm{single},u}. The interaction-aware variant takes σr,u=σsingle,u2+σpair,u2\sigma_{r,u}=\sqrt{\sigma_{\mathrm{single},u}^2+\sigma_{\mathrm{pair},u}^2}. Equations (11), (12), and (16) all use the chosen denominator.

The pair term is the RMS of the raw second-order contrasts in the numerator of Eq. (12), taken over the three pairs among the top three roles, both endpoints (−1,−1)(-1,-1) and (1,1)(1,1), and the selected horizons. Since the interaction-aware denominator is at least the pair RMS, a vanishing single-parent RMS alone cannot arbitrarily magnify these sampled interactions. This bound applies only to the sampled probes. Both RMS terms scale with the same positive gain. Proposition 4 (§5, p. 7) and its proof (Appendix B.5) state the conditions for invariance, including exact responses on the query domain, equivariant normalization, corresponding roles and tie decisions, and the same scale rule in both windows with both pre-floor scales above the floor.

Targeted familyOriginal MRC / MSECorrected MRC / MSE
xzxz.512 / .181.786 / .104
xz2xz^2.498 / .187.771 / .109

In the standard five-run audit (Table 4), MSE is .097 under both scales at the reported precision; MRC and MSS each decrease by .001 (.804 to .803 and .793 to .792). The largest gains occur on pure interactions. On held-out x2+xzx^2+xz, MSE also improves from .079 to .078 and constituent Top-2 recovery from 81.5% to 81.9%. All real-data forecasting and transfer results use the original scale. Figure 1, §4.2 and §6 identify that default. Table 4 shows where the alternative helps under controlled conditions; its effect on real data remains open.

W2 · Explicit lag range. The model observes 512 steps. The default limit of eight lags restricts which parent–lag coordinates can be individually perturbed; older observations can enter the encoded context. With six selected roles, 32 experts and Top-2 activation fixed, retraining the affected pipeline at lag limits 8, 16 and 24 yields frozen ETT macro MSE of .30775, .30702 and .30758. At 16 and 24, respectively, 11.3% and 14.8% of windows select a lag beyond eight. Thus wider probes change selected roles, with a maximum MSE reduction of .00073 on ETT. The same lag spans different physical durations across datasets (§6.4, p. 10; Table 24).

Q1 · Number of prototypes. Holding capacity at 32 expert slots and two active experts, 8, 16, 32 and 64 prototypes give ETT macro MSE of .31639, .30992, .30775 and .30910, respectively. The lowest observed MSE is at 32; doubling prototype resolution gives no further gain in this comparison (Table 24, p. 27).

Q2 · Expert alignment in Table 2B. Each row fixes the expert bank, backbone, and forecast head. For each candidate router, source forecasting losses determine the prototype-to-expert lookup for that bank. At 32 prototypes this is a one-to-one assignment with ties resolved by expert index. We freeze the lookup before target evaluation. Thus .31641 and .31128 compare Graph and MER routing against the same Graph bank. In the 64-prototype sweep, several addresses may map to one slot; duplicate slots are removed so that two distinct experts execute. The slot-to-class mapping used to score constituent recovery is separate (Appendix C, pp. 19–20; Appendix H.1, p. 27).

Q3 · ILI and Weather exposure. Table 16 records foundation-model pretraining exposure as Yes for both datasets and inclusion in MER's inherited source mixture as Unknown. Table 17 therefore describes their result as frozen target evaluation: all components are fixed and there are zero target-stage updates. Target exclusion from MER source training is verified for the eight ETT/PEMS directions, all of which favor MER over Pattern and Graph. The 4,800-update schedule applies only to the separate few-shot evaluation.

Reviewer_K1UyOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper introduces the Mechanism-Equivalence Routing framework that assigns inputs to experts in time-series foundation models based on similarities in their predictive response functions rather than similarities in their observed temporal patterns. It defines a notion of mechanism equivalence under affine transformations and proposes a routing procedure that uses learned predictive supports, normalized response probes, functional fingerprints, and prototype-based clustering to identify reusable expert representations. The performance of the framework is evaluated both on experiments with synthetic data and on a number of real world data sets. The results indicate that the proposed framework achieves improved routing consistency in the synthetic experiments and lower PMSE across the data sets considered.

Critical strengths
  • The proposed framework has an interesting conceptual motivation and it provides a mathematical definition of response equivalence under affine transformations. The formulation distinguishes between observed data similarity and similarity of the predictive functions, which can prove useful for domain generalization and transfer learning.

  • The empirical results show overall good performance against competing approaches. In addition, the controlled experiments provide insights into the differences between similar observed patterns with different predictive responses and different observed patterns with similar responses.

  • The paper considers computational aspects and reports both inference cost and latency.

Critical weaknesses
  • The proposed mechanism equivalence principle is closely related to a large body of literature on invariant prediction, functional similarity, and parameter sharing in modular neural networks; the authors provide numerous references to these concepts, including Peters et al., 2016; Parascandolo et al., 2018; Rosenbaum et al., 2018; Goyal et al., 2021. The paper formalizes equivalence through normalized predictive responses under affine transformations; however, it remains unclear how this fundamentally differs from established functional similarity based approaches. Further, the implementation largely combines existing techniques, including feature selection, neural response estimation, Deep Sets, clustering, and prototype-based routing.

  • The authors explicitly mention and acknowledge that their proposed framework does not identify causal mechanisms. Nevertheless, the proposed equivalence relation is essentially based on similarities between learned predictive functions. Hence, it remains unclear what the notion of mechanism equivalence contributes beyond established concepts of functional similarity and invariant predictive representations.

  • The theoretical justification of the proposed framework is based on fairly stringent assumptions and thus it is unclear whether it support parameter reuse in practical settings.

  • It is somewhat unclear how robust the routing procedure is to errors in the learned response functions, particularly under distribution shifts.

  • The controlled experiments use predefined response families and equivalence classes related with the proposed definition; hence, there is concern about the generalizability of the proposed framework to more complex, partially overlapping, or non-affine relationships in real time series data.

Questions for authors
  • How sensitive is the proposed framework to errors in the learned response model?

  • Since the framework introduces additional elements, it is unclear whether the posited performance improvements are primarily due to the equivalence principle or these additional elements?

  • How well does the approach generalize beyond the designed synthetic mechanisms?

  • How significant are the improvements over the Graph router?

  • What is the computational cost and predictive improvements of the proposed framework versus simpler routing approaches under equivalent computational budgets?

  • How does the proposed mechanism equivalence framework fundamentally differ from established approaches to functional similarity, invariant prediction and parameter sharing?

1 comment
AuthorsOpenReview ↗

A window-level reuse criterion: evidence, boundaries, and budget

Thank you for asking what the reuse criterion adds to functional similarity and how much of the forecasting gain it explains (§§2.3–3, 6.3–6.4). Q and W number your questions and weaknesses in their original order.

Q1 / W4 · Response error. In §6.4 (p. 9), we hold a seed-17 checkpoint fixed on 4,000 held-out CausalTime windows. Replacing only the learned query responses with generator conditional means estimated from 4,096 trajectories per query lowers 96-step MSE from .09831 to .08674 (11.77%). The branches agree on the Top-2 set in 3,427 windows. In the fingerprint sweep, agreement falls from 96.2% at ε=.05 to 52.8% at ε=.40. At ε=.10, MSE and agreement (.09804, 85.7%) are close to the learned branch (.09831, 85.675%) in aggregate (Tables 22–23). Proposition 5 guarantees set stability when a window’s displacement is below half its second-to-third cosine margin. The aggregate match does not establish which windows satisfy that bound. Errors under real-data shift remain unknown.

Q2 / W1 · Added components. With the Graph expert bank fixed, switching from Graph to MER routing lowers ETT macro MSE from .31641 to .31128. With Pattern routing fixed and indices aligned to each bank on source data, switching from the Pattern bank to the MER bank lowers MSE from .33876 to .31452. Full MER reaches .30775 (Table 2B, p. 9). The closest response-informed control, Local-response, gives MRC/MSS/MSE of .698/.663/.121 versus .804/.793/.097 for original-scale MER (Table 19). The 19.8% MSE reduction compares complete pipelines. These controls give evidence for both bank formation and routing choice, but cannot assign the entire gain to the mathematical criterion.

Q3 / W5 · Generalization. With the combination x²+xz held out but both constituents seen during training, MER obtains .079 MSE and 81.5% constituent recovery versus .130 and 44.2% for Pattern (Table 2A, original scale). The interaction-aware scale gives .078 MSE and 81.9% recovery on that composition (Table 4, p. 10). Both versions restrict coordinate equivalence to positive-affine maps. Classes are unobserved on real series, and alignment with partially overlapping mechanisms remains open (§7).

Q4 · Gains over Graph. Across all ten frozen directions, MER lowers MSE by 2.38–3.05% against original-protocol Graph (mean absolute reduction .00609; Table 17). Eight ETT/PEMS directions exclude the target from MER source training; ILI/Weather use a frozen checkpoint with unknown source-mixture inclusion. Table 18 gives 8.9–9.5% against Pattern on those directions. Table 21 gives 4.2–22.3% against native Moirai-MoE across 40/40 dataset–horizon cells. Table 17 reports per-method SDs over five runs in each direction; Table 1 gives SDs of seed-level family macros. Directions share data and runs were unpaired, so the comparisons are descriptive.

Q5 · Training budget and cost. The main comparison matches the backbone, two active experts, and the Stage-4 ceiling of 89,300 updates. MER also fits its response and fingerprint models and clusters the fingerprints. At a 189,500-update ceiling, Pattern reaches .33210 ETT macro MSE and Graph improves by .00357 (.31641 to .31284). MER stays at .30775. Its .00509 advantage (1.63%) is relative to longer-trained Graph on the ETT macro (§6.4, p. 10; Table 25); the 2.38–3.05% above uses the original Graph value for each transfer direction. MER took 5.35 elapsed hours versus 5.79 for extended Graph. This compares similar elapsed time; training FLOPs were not matched. At inference, MER raises latency from 17.5 to 20.4 ms and cost from 3.3 to 3.5 GFLOPs per window, with 2.1M auxiliary parameters (Table 1).

Q6 / W1–W2 · Functional similarity. Invariant prediction asks which conditionals remain stable across environments. CKA and Task2Vec compare representations and tasks. We treat mechanism equivalence as a form of functional similarity: it specifies when two target windows can share a response expert. PMDA tests whether routing follows this criterion on windows with known response classes. The criterion concerns predictive responses (§1); causal structure remains unidentified. Definition 1 permits a lag-preserving correspondence of input roles, positive-affine input coordinates, horizon-specific baselines, and one positive output gain shared across horizons. Thus a change of units is admissible, while a sign reversal or a change confined to the long-horizon amplitude generally changes the reuse class. Deep Sets and clustering implement a finite-probe approximation to this criterion.

W3 · Assumptions for reuse. Proposition 2 (§3, p. 4) gives a sufficient condition for sharing under squared loss, equivariant normalization, and a well-specified, role-equivariant expert family. When those assumptions fail, the guarantee falls away. Optimal sharing of hidden-layer parameters remains an empirical question.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.