acceptodds
Under review as a conference paper at ICLR 2027

Rethinking Federated Time Series Pretraining Through Shared Latent Dynamics

Abstract

Pretraining a time series model typically relies on centralizing large and diverse collections of data, an assumption incompatible with privacy-sensitive domains such as power grids, healthcare, finance, and distributed sensor networks. Beyond privacy, Federated learning (FL) addresses client heterogeneity by combining personalized local models with a shared global model, placing the choice of shared representation at the center of the model design. Existing FL approaches for time series make it encode observation-level temporal patterns, which vary substantially across heterogeneous clients. We instead identify latent state transitions as a more transferable object for federation and propose FedState, a federated pretraining framework that keeps client models private and shares only a latent transition operator. The operator is indexed by elapsed physical time rather than patch position and is learned entirely in latent state space, allowing clients with different sampling intervals, channel counts, value scales, and local architectures to learn shared dynamics without aligning their observations. Only the transition operator is exchanged, a small fraction of the model parameters and of the per-round communication of full-model federation. Extensive experiments across eight benchmarks demonstrate that FedState outperforms full-model federation, advanced federated time series pretraining methods, and centralized models in both deterministic and probabilistic forecasting, including zero-shot transfer to unseen domains without target-domain adaptation.

Primary area:
learning on time series and dynamical systems
Keywords:
Federated time series modeling, time series pretrianing, time series forecasting

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_TY9QOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FedState, a federated time-series pretraining framework that addresses client heterogeneity by sharing latent temporal dynamics rather than observation-level representations or entire models. FedState maintains client-specific encoders and decoders while federating a shared latent transition operator conditioned on elapsed physical time. This design accommodates differences in sampling frequencies, channel dimensions, data scales, and local model architectures. The framework further introduces latent-state pretraining with multi-scale representations and heterogeneity-aware aggregation that assigns greater weights to clients with larger transition errors. The paper provides theoretical analysis of physical-time conditioning and evaluates FedState across multiple benchmarks, including deterministic and probabilistic forecasting, few-shot adaptation, zero-shot transfer, and communication efficiency. The results generally demonstrate competitive or superior forecasting performance with substantially reduced communication overhead.

Critical strengths

S1. Sharing latent temporal dynamics instead of observation-dependent representations offers a meaningful alternative to conventional federated time-series learning, particularly when clients have substantially different data distributions and sampling frequencies.

S2. The combination of physical-time conditioning, latent-state prediction, and personalized observation models provides a coherent approach to handling multiple forms of client heterogeneity.

S3. The analysis of grid-indexed versus physical-time-indexed transitions provides theoretical motivation for the proposed design. The aggregation mechanism is also supported by a distributionally robust optimization formulation.

S4. The paper evaluates FedState across several benchmarks and settings, including in-domain forecasting, probabilistic forecasting, few-shot learning, zero-shot transfer, heterogeneous architectures, and communication efficiency.

S5. Federating only the transition operator substantially reduces the number of communicated parameters compared with full-model federated learning.

Critical weaknesses

W1. The approach assumes that heterogeneous clients can learn compatible latent representations in which a common transition operator is meaningful. However, the client encoders are independently trained and remain private. Although the empirical results support the usefulness of the shared operator, the paper does not fully establish when independently learned latent spaces become sufficiently aligned or when this assumption may fail.

W2. Although FedState achieves strong overall performance, the improvements are not consistent across all datasets. For example, in Table 3, FedState underperforms several baselines on ECL and Exchange. Similarly, Table 8 identifies BuildingsBench as a challenging zero-shot transfer setting. These results suggest that the benefits of shared latent dynamics depend on the characteristics of the participating domains.

W3. The proposed framework combines a shared transition operator, physical-time conditioning, multi-scale representations, additional pretraining objectives, and specialized aggregation. Although the paper includes component ablations, a more direct comparison between sharing observation representations and sharing latent dynamics under otherwise identical architectures and training objectives would provide stronger evidence for the central hypothesis.

Questions for authors

Q1. In Section 4.2, FedState uses all 46 TIME clients and 52 GIFT-Eval clients, whereas full-model federated baselines use only 34 and 38 clients, respectively, because of context-length constraints. Consequently, the improvements may partially reflect access to more participating clients rather than solely the proposed shared representation. An additional comparison using identical client subsets would strengthen the conclusions.

Q2. Equation (2) allows each client to learn its own encoder while using the same transition operator. However, the paper does not clearly explain how latent representations from different encoders acquire compatible semantics without explicit cross-client alignment. The shared operator may encourage implicit alignment, but the mechanism and its limitations deserve further explanation.

Q3. The theorem establishes an advantage for physical-time indexing when clients share the same underlying transition kernel. However, FedState is motivated by heterogeneous clients whose dynamics may differ. The theoretical analysis does not establish that physical-time conditioning necessarily reduces excess risk under arbitrary heterogeneous dynamics. The paper should more clearly distinguish the theorem’s assumptions from the broader empirical claims.

Q4. The aggregation mechanism assigns larger weights to clients with higher latent prediction errors. However, larger errors may reflect intrinsically difficult datasets, noisy observations, or poorly learned representations rather than underrepresented dynamics. The paper could better justify when prioritizing these clients improves the shared operator instead of introducing bias toward noisy or difficult domains.

Q5. The discussion reports 1.73M uploaded parameters for full-model federation and 1.44M for FFTS, whereas Table 7 reports 2.92M and 5.63M, respectively. The authors should clarify whether these numbers correspond to different configurations or correct the inconsistency.

Q6. Algorithm 1 states that unseen clients use an average of the private encoders. Since clients may have different architectures, channel dimensions, and sampling configurations, it is unclear how their encoder parameters can be averaged consistently. The paper should clarify the exact procedure used for constructing observation mappings for unseen clients.

7 comments
AuthorsOpenReview ↗

Authors’ Rebuttal (1/4)

Thank you for your feedback and valuable comments. Below is our point-by-point response.


W1 & Q2 [Compatibility of independently learned latent spaces, and its boundary]

Training private encoders against a shared transition operator drives their latent coordinates toward compatibility without explicit alignment, as supported by our theoretical analysis and direct substitution experiments across clients and architectures. We address this from four aspects below:

(1) Mechanism of the Implicit Alignment. Consider a client whose latent transition satisfies z′=fϕ(z,Δ)z'=f_\phi(z,\Delta). An invertible coordinate transformation TT preserves the same law only if fϕ(Tz,Δ)=Tfϕ(z,Δ)f_\phi(Tz,\Delta)=Tf_\phi(z,\Delta) on the relevant state support. A private operator can accommodate arbitrary transformations through conjugation, TfϕT−1Tf_\phi T^{-1}, whereas a shared operator admits only those compatible with the same dynamics. Thm. 1(a) characterizes the corresponding excess risk through the dispersion DD of client latent transition kernels over overlapping state distributions. Minimizing the federated loss therefore drives latent coordinates toward agreement, while the variance and grounding terms in Eq. 18 discourage degenerate representations.

(2) Conditions and Boundary of the Alignment. The shared operator constrains latent coordinates where client state distributions overlap, since it must model their transitions at the same physical intervals. When clients occupy disjoint latent regions, the operator can fit their dynamics without conflict. Clients in disjoint regions benefit less from cross-client transfer but do not impair the rest of the federation.

(3) Empirical Evidence of Compatibility. Existing results (Fig. 1) show that cross-domain displacement agreement increases from 0.41 in observation space to 0.70 in latent space. Domain classification from latent transition directions achieves 16% accuracy, close to the chance level of 17%, compared with 78% and 83% using spectral and increment features (Fig. 8).

(4) Direct Measurement across Clients and Architectures. We add substitution experiments on TSLib and a 48-client zero-shot federation spanning 10 sampling periods from 10 seconds to one day. We measure pairwise representation similarity using linear CKA on common validation windows and evaluate encoder substitution across clients and replacement with the aggregated observation model, all after pretraining without further optimization. Table R1 reports the results.

Table R1. Functional compatibility of independently initialized private encoders, averaged over three seeds. Linear CKA ranges from 0 to 1 and is invariant to rotation and isotropic scaling.

FederationTrainingLinear CKAΔMSE, cross-clientΔMSE, aggregate
TSLib, 7 clientsPrivate operators0.41+58.7%+51.6%
TSLib, 7 clientsFedState0.84+6.2%+4.6%
Zero-shot, 48 clientsPrivate operators0.33+71.5%+63.8%
Zero-shot, 48 clientsFedState0.79+8.4%+6.1%

The shared law substantially improves representation similarity and functional compatibility. This compatibility extends across heterogeneous encoder architectures (Tab. 9), where encoder substitution increases MSE by 6.3% under the shared law versus 73.9% under private operators. We have added the mechanism, its conditions, and Table R1 to the revised manuscript.


W2 [Consistency across datasets: ECL, Exchange, and BuildingsBench]

The reported exceptions arise from evaluation conditions rather than a limitation of shared latent dynamics. FedState ranks first on both ECL and Exchange with a 512-step look-back, and on BuildingsBench it remains the most accurate among methods sharing its backbone. We address this from three aspects below:

(1) ECL. To examine the effect of context length, we retrain the compared methods on the ECL client of Tab. 3 using look-backs of 96 and 512, with all other settings unchanged. Table R2 reports the results. With a 512-step context, which covers about three weeks of hourly electricity demand rather than four days, FedState achieves the lowest MSE. The gap in Tab. 3 therefore stems from the short 96-step context.

Table R2. MSE on ECL under the in-federation protocol, with one federation per method at each look-back.

MethodLook-back 96Look-back 512
Scratch0.1780.166
Pooled0.1780.164
FedAvg0.1800.167
FedRep0.1800.166
FedState0.1890.160

(2) Exchange. With the 96-step look-back of Tab. 3, FedState achieves comparable performance to training from scratch. Exchange rates are near-random walks, so short contexts carry little predictive information and favor simple models such as DLinear. With a 512-step look-back, FedState ranks first with an MSE of 0.389 (Tab. 2).


Please see the subsequent Authors' Rebuttal (2/4) for remaining responses.

AuthorsOpenReview ↗

Authors’ Rebuttal (2/4)

Continued: Authors’ Rebuttal (2/4)


(3) Zero-Shot on BuildingsBench. BuildingsBench contains a single sampling frequency, with strong weekly repetition in commercial building loads. This favors seasonal-naive forecasting, which achieves a CVRMSE of 12.4 on commercial buildings and outperforms all learned methods (Tab. 20). On residential buildings, where weekly repetition is less effective, FedState outperforms seasonal naive and all full-model and personalized federated baselines. Overall, FedState performs best among methods sharing the same backbone, including centralized alternatives. When buildings are included in pretraining, FedState ranks first on four of seven collections and second on BDG-2 (Tab. 2). With larger-scale LOTSA pretraining, it achieves the best BuildingsBench CVRMSE of all evaluated methods (Tab. 24). We have added Table R2 to App. C.4.


W3 [Sharing observation representations vs. latent dynamics under identical architectures and objectives]

Under identical architectures and training objectives, sharing only the transition law matches or outperforms sharing the encoder or full model at substantially lower communication cost. We provide this controlled comparison in Fig. 3a and extend it to the in-federation protocol through Table R3 below, with further evidence from supervision-space comparisons. We address this from three aspects below:

(1) Choice of Shared Component on TIME. Sharing only the law matches the CRPS$_n$ of sharing the encoder or both the encoder and law, while uploading only 0.35M parameters per round compared with 0.62M and 0.97M. It also outperforms sharing the decoder or full model, the latter requiring 1.86M parameters.

(2) Choice of Shared Component under the In-Federation Protocol. Under the protocol of Tab. 3, we vary only the shared component. Table R3 reports the results. Sharing only the law outperforms sharing the encoder, decoder, or full model on both metrics. It also stays within one standard deviation of sharing the encoder and law jointly while requiring only 37% of the upload.

Table R3. In-federation comparison of shared components on seven TSLib datasets under a consistent setting (96-step look-back). Results are averaged over three seeds.

Shared componentUpload/RoundAvg MSEAvg MAE
None (Scratch)00.3347±0.00050.3647±0.0004
Full model1.13M0.3346±0.00130.3629±0.0009
Decoder0.15M0.3281±0.00070.3612±0.0008
Encoder0.62M0.3214±0.00110.3589±0.0005
Encoder and law0.98M0.3166±0.00080.3567±0.0006
Law (FedState)0.36M0.3170±0.00060.3570±0.0004

(3) Latent vs. Observation-Level Objectives. Under matched architectures and training budgets, latent-state prediction reduces MSE by 14.7%, MAE by 7.8%, and worst-client MSE by 40.9% compared with observation-space prediction (Fig. 3b). The same trend holds against observation-level federated self-supervised baselines with a shared backbone, where FedState leads in 17 settings versus at most three for any baseline (Tab. 2).

We have added Table R3 to App. C.4.


Q1 [Identical client subsets on TIME and GIFT-Eval]

On identical client subsets of TIME and GIFT-Eval, FedState maintains its advantage, which therefore does not depend on the additional clients. We retrain FedState on the 34 TIME and 38 GIFT-Eval clients available to the full-model baselines and compare one federation per method. Table R4 reports the results. FedState achieves the best performance on seven of the eight TIME metrics, consistent with Tab. 5, and reduces GIFT-Eval WQL$_n$ by 3.2% relative to the strongest baseline. These improvements require only 19% of FedAvgM's per-round upload.

Table R4. TIME (34 clients) and GIFT-Eval (38 clients) results with identical client populations and one federation per method.

Benchmark, metricFedAvgMFedBNFedRepFedState
TIME, MSE$_n$0.6240.6370.6220.616
TIME, MAE$_n$0.8160.8210.8200.815
TIME, RMSE$_n$0.7940.8100.8050.792
TIME, MAPE$_n$0.7660.7650.7700.747
TIME, sMAPE$_n$0.8620.8800.8790.855
TIME, MASE$_n$0.8200.8320.8360.823
TIME, ND$_n$0.7930.7990.7890.780
TIME, CRPS$_n$0.7510.7500.7570.741
GIFT-Eval, MASE$_n$0.9150.9280.9550.920
GIFT-Eval, WQL$_n$0.7530.7280.7550.705

We have added Table R4 to App. C.4.


Please see the subsequent Authors' Rebuttal (3/4) for remaining responses.

AuthorsOpenReview ↗

Authors’ Rebuttal (3/4)

Continued: Authors’ Rebuttal (3/4)


Q3 [Assumptions of the theorem under heterogeneous dynamics]

Thm. 1(a) and (b) hold exactly for arbitrary heterogeneous kernels, and only the strict separation of Cor. 1 requires a shared kernel. We further extend this separation to heterogeneous kernels under an explicit sufficient condition. We address this from two aspects below:

(1) Assumptions of the Theorem. App. A assumes no relationship among client transition kernels. Thm. 1(a) characterizes the excess risk of the optimal interval-indexed operator by the kernel dispersion DD at equal physical intervals, whereas Thm. 1(b) characterizes that of the optimal slot-indexed operator by DslotD^{\rm slot} at client-specific intervals kτik\tau_i. Their relative magnitudes depend on whether client dynamics agree more closely at equal elapsed times or equal sample offsets. The condition below formalizes the former regime, which is natural for physical processes such as weather, energy load, and traffic.

(2) A Sufficient Condition for Heterogeneous Kernels. Under Assumption 1, choose any Borel reference family Φ\Phi on ⋃iGi\bigcup_iG_i for which the energies below are finite, write χi=Φi−Φ\chi_i=\Phi_i-\Phi, and let D0slotD_0^{\rm slot} be the slot dispersion of Φ\Phi as in Cor. 1(ii), with the same mixing measure and densities. Let εint=∑Δ∈G∫∑i∈A(Δ)ri∥χi(⋅,Δ)∥2 dmΔ,εslot=1K∑k≤K∫∑iρi∥χi(⋅,kτi)∥2 dm.\varepsilon_{\rm int}=\sum_{\Delta\in\mathcal G}\int\sum_{i\in A(\Delta)}r_i\Vert\chi_i(\cdot,\Delta)\Vert^2\ dm_\Delta,\qquad\varepsilon_{\rm slot}=\frac{1}{K}\sum_{k\le K}\int\sum_i\rho_i\Vert\chi_i(\cdot,k\tau_i)\Vert^2\ dm . Proposition. D≤εintD\le\varepsilon_{\rm int} and Dslot≥[D0slot−εslot]+\sqrt{D^{\rm slot}}\ge\big[\sqrt{D_0^{\rm slot}}-\sqrt{\varepsilon_{\rm slot}}\big] _ +. Consequently, D<DslotD<D^{\rm slot} whenever εint+εslot<D0slot\sqrt{\varepsilon_{\rm int}}+\sqrt{\varepsilon_{\rm slot}}<\sqrt{D_0^{\rm slot}}.

Proof. At each interval, the common reference cancels in the dispersion, which leaves the weighted dispersion of {χi}\lbrace\chi_i\rbrace. Lemma 2 bounds it by the weighted second moment ∑iri∥χi∥2\sum_i r_i\Vert\chi_i\Vert^2, and integration gives D≤εintD\le\varepsilon_{\rm int}. For the slot term, consider the weighted L2L^2 space of client-slot families with inner product 1K∑k∫∑iρi⟨aik,bik⟩ dm\frac{1}{K}\sum_k\int\sum_i\rho_i\langle a_{ik},b_{ik}\rangle\ dm, modulo null sets. Let PP be the orthogonal projection onto families that do not depend on the client index. By Lemma 2, (Pa)ik=∑jρjajk(Pa) _ {ik}=\sum_j\rho_ja_{jk}, and Dslot=∥(I−P)A∥2D^{\rm slot}=\Vert(I-P)A\Vert^2 for Aik=Φi(⋅,kτi)A_{ik}=\Phi_i(\cdot,k\tau_i). Writing A=A0+ΞA=A^0+\Xi with Aik0=Φ(⋅,kτi)A^0_{ik}=\Phi(\cdot,k\tau_i) and Ξik=χi(⋅,kτi)\Xi_{ik}=\chi_i(\cdot,k\tau_i), the triangle inequality and ∥I−P∥≤1\Vert I-P\Vert\le1 give Dslot≥∥(I−P)A0∥−∥Ξ∥=D0slot−εslot\sqrt{D^{\rm slot}}\ge\Vert(I-P)A^0\Vert-\Vert\Xi\Vert=\sqrt{D_0^{\rm slot}}-\sqrt{\varepsilon_{\rm slot}}. □\square

When both operators are trained on the same physical intervals, with μi\mu_i uniform over {kτi}k≤K\lbrace k\tau_i\rbrace_{k\le K}, their Bayes risks coincide, yielding εint=εslot=ε\varepsilon_{\rm int}=\varepsilon_{\rm slot}=\varepsilon. Physical-time conditioning achieves strictly lower optimal risk when ε<D0slot/4\varepsilon<D_0^{\rm slot}/4, where cross-client deviations from a common physical-time kernel are smaller than one quarter of the dispersion induced by sampling-grid mismatch. Cor. 1 corresponds to the special case ε=0\varepsilon=0. Consistent with this prediction, replacing physical-time conditioning with slot indexing increases forecasting errors by 2.9% to 4.0% on TIME and GIFT-Eval. We have added this proposition after Cor. 1 and stated its condition in Sec. 3.


Q4 [Tilted aggregation and noisy or difficult clients]

Tilted aggregation prioritizes clients with underrepresented learnable dynamics, while standardized losses and bounded reweighting limit the influence of noisy or poorly fitted clients. Our additional experiments on TIME confirm that these safeguards preserve the performance of other clients under data corruption and encoder failure. We address this from three aspects below:

(1) Benefit of Prioritization. When high transition errors reflect learnable dynamics underweighted by data-size aggregation, the tilt increases their contribution to the shared operator. On the 46 TIME clients, removing the tilt increases median CRPS$_n$ by 0.9% (Sec. 4.4).

(2) Sufficiency of Bounded Reweighting. Large transition errors may also arise from intrinsically difficult datasets, noisy observations, or poorly learned representations. Rather than distinguishing these causes explicitly, the tilt standardizes client losses and clips their logits, bounding any client's aggregation weight (Prop. 1(b)).


Please see the subsequent Authors' Rebuttal (4/4) for remaining responses.

Reviewer_JixBOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work studies the problem of time series in federated learning. The authors propose FEDSTATE, a federated pre-training framework that shares only the latent state transition operator. They train this operator in a latent state space. They propose three mechanisms to further leverage operators. A physical-time condition is used to align temporal semantics across different sampling grids; latent-state supervision decouples shared parameters from information at the observation level; and heterogeneity-aware aggregation reweights client updates based on state-transition errors. Extensive experiments conducted across eight benchmarks demonstrate that FEDSTATE outperforms baselines in both deterministic and probabilistic forecasting.

Critical strengths
  1. The paper is overall well-written.

  2. The shared transfer module has approximately 0.36M parameters and offers the potential to reduce communication overhead.

  3. The experiments are quite comprehensive, covering in-domain representation learning, few-shot transfer, zero-shot prediction, and others.

Critical weaknesses
  1. Where are the encoder, decoder, prediction branches, and combination weights for the new client obtained?

  2. The ablation of key components in Figure 4 does not directly show the results of removing the physical time condition.

  3. How are the latent coordinates of private encoders aligned?

  4. When the patch length is fixed at 16, the actual duration covered by a single fast state for different clients is 16$\tau_i$. The coverage ranges for mid states and slow states also vary depending on the sampling rate and window length.

  5. Although the paper explicitly introduces FedTRL’s domain-invariant representation, prototype alignment, and domain-aware aggregation, both in the related work section and in Table 1, it fails to include a comparison with this method in the experiments.

Questions for authors

Please see the weaknesses.

5 comments
AuthorsOpenReview ↗

Authors’ Rebuttal (1/2)

Thank you for providing us with your valuable feedback. We have carefully considered your comments and would like to address them as below.


W1 [Source of the encoder, decoder, prediction branches, and combination weights for a new client]

A zero-shot client uses the final federated transition law and a one-time aggregate of the participating clients' observation models, without target-domain training or additional combination weights. The aggregate is weighted by each client's training-window count. Table R1 summarizes each component.

Table R1. Source and aggregation of each component for a zero-shot client.

ComponentSourceAggregation
Transition law fϕf_\phifederated law after the final roundfederated in every round
Encoder EθE_\thetaclients' encoders after pretrainingweighted average, once
Decoder gψg_\psi and prediction branchesclients' decoders and branches after pretrainingweighted average, once
Per-step combination weightsnot usednot aggregated

The per-step combination weights are fitted on validation data only during participating-client fine-tuning. We have added Table R1 to App. C.4.


W2 [Removing the physical-time condition]

We have added a direct ablation that replaces physical-time conditioning with slot indexing. This increases forecasting errors by 2.9% to 4.0% on TIME and GIFT-Eval, where clients exhibit substantial sampling-period heterogeneity. All other components are unchanged. Table R2 reports the comparison.

Table R2. Ablation of physical-time conditioning on TIME and GIFT-Eval.

Operator indexed byTIME MSE$_n$TIME MASE$_n$TIME CRPS$_n$GIFT-Eval MASE$_n$GIFT-Eval WQL$_n$
Slot index kk0.6340.8510.7620.9460.723
Physical interval Δ=kτi\Delta=k\tau_i (FedState)0.6100.8210.7350.9190.695

Under slot indexing, the same index kk represents different physical intervals across clients, inducing the kernel dispersion characterized by Thm. 1(b). We have added the TIME results to Fig. 4b.


W3 [Alignment of latent coordinates across private encoders]

Training private encoders against a shared transition operator drives their latent coordinates toward compatibility, as supported by our theoretical analysis and new substitution experiments. We address this from three aspects below:

(1) Compatibility Induced by the Shared Law. Consider a client whose latent transition satisfies z′=fϕ(z,Δ)z'=f_\phi(z,\Delta). An invertible coordinate transformation TT preserves the same law only if fϕ(Tz,Δ)=Tfϕ(z,Δ)f_\phi(Tz,\Delta)=Tf_\phi(z,\Delta) on the relevant state support. A private operator can accommodate arbitrary transformations through conjugation, TfϕT−1Tf_\phi T^{-1}, whereas a shared operator admits only those compatible with the same dynamics. Thm. 1(a) characterizes the corresponding excess risk through the dispersion DD of client latent transition kernels over overlapping state distributions. Minimizing the federated loss therefore drives latent coordinates toward agreement, while the variance and grounding terms in Eq. 18 discourage degenerate representations.

(2) Empirical Evidence of Compatibility. Existing results (Fig. 1) show that cross-domain displacement agreement increases from 0.41 in observation space to 0.70 in latent space. Domain classification from latent transition directions achieves 16% accuracy, close to the chance level of 17%, compared with 78% and 83% using spectral and increment features (Fig. 8).

(3) Direct Measurement on the Zero-Shot Federation. We further conduct substitution experiments on TSLib and a 48-client zero-shot federation spanning 10 sampling periods from 10 seconds to one day, with all private encoders initialized independently. We measure pairwise representation similarity using linear CKA on common validation windows and evaluate encoder substitution across clients and replacement with the aggregated observation model. All substitutions are performed after pretraining without further optimization. Table R3 reports the results.

Table R3. Functional compatibility of independently initialized private encoders, averaged over three seeds. Linear CKA ranges from 0 to 1 and is invariant to rotation and isotropic scaling.

FederationTrainingLinear CKAΔMSE, cross-clientΔMSE, aggregate
TSLib, 7 clientsPrivate operators0.41+58.7%+51.6%
TSLib, 7 clientsFedState0.84+6.2%+4.6%
Zero-shot, 48 clientsPrivate operators0.33+71.5%+63.8%
Zero-shot, 48 clientsFedState0.79+8.4%+6.1%

The shared law substantially improves representation similarity and functional compatibility.

We have incorporated the theoretical clarification and Table R3 into the revised manuscript.


Please see the subsequent Authors' Rebuttal (2/2) for remaining responses.

AuthorsOpenReview ↗

Authors’ Rebuttal (2/2)

Continued: Authors’ Rebuttal (2/2)


W4 [Physical duration covered by fast, mid, and slow states]

Each scale summarizes a client-dependent physical duration, but the shared transition law is conditioned on elapsed physical time rather than state coverage, so this variation does not affect the law. Our additional ablation confirms this, as explicitly providing the durations changes no metric by more than 0.001. We address this from two aspects below:

(1) State Coverage and Temporal Queries. All temporal intervals are defined relative to the forecast origin t0t_0. Table R4 summarizes the duration covered by each state (Sec. 3, App. B.2). For each target, the shared law receives its physical-time query Δ=kτi\Delta=k\tau_i, while the state coverage remains determined by the private encoder.

Table R4. Temporal convention of the states on client ii.

ScaleContext stateTarget stateQuery interval
Fastone patch, 16τi16\tau_ithe kk-th future patch, 16τi16\tau_iΔk\Delta_k, the start of that patch after t0t_0
Midfour overlapping patches, 40τi40\tau_ithe kk-th future block, 40τi40\tau_iΔk\Delta_k of its first patch
Slowthe whole context, [t0−Liτi,t0)[t_0-L_i\tau_i,t_0)the whole future window, [t0,t0+Hiτi)[t_0,t_0+H_i\tau_i)the first interval Δ1\Delta_1

(2) Effect of the Client-Dependent Durations. We augment Δ\Delta with the patch duration PτiP\tau_i, which also determines mid-state coverage, and then also include the context and future window durations LiτiL_i\tau_i and HiτiH_i\tau_i. All durations are encoded as in Eq. 3. Table R5 reports the results.

Table R5. Effect of explicit state-duration conditioning on TIME under the protocol of Tab. 5.

Law inputMSE$_n$MASE$_n$CRPS$_n$
Δ\Delta, PτiP\tau_i, LiτiL_i\tau_i, HiτiH_i\tau_i0.6100.8220.736
Δ\Delta, PτiP\tau_i0.6110.8200.736
Δ\Delta (FedState)0.6100.8210.735

Although sampling periods on TIME span six orders of magnitude, the elapsed-time query Δ\Delta alone provides sufficient temporal conditioning. We have added Tables R4 and R5 to App. C.4.


W5 [Comparison with FedTRL]

We have included FedTRL in the zero-shot, in-federation, and in-domain evaluations, where FedState consistently outperforms it. Since FedTRL's source code is not publicly available, we reproduce it following the original paper, retaining its causal Transformer encoder (3 layers, width 128, patch length 16) and denoising decoder (1 layer). All methods use the same training setting. Table R6 shows that FedState outperforms FedTRL on seven of the eight zero-shot metrics, and Table R7 shows that it also leads within the training federation.

Table R6. Zero-shot results of FedState and FedTRL under the protocol of Tab. 8.

BenchmarkMetricFedStateFedTRL
TSLibMSE / MAE0.407 / 0.4180.460 / 0.458
WEATHER-5K$_{512}$MAE17.10917.236
WEATHER-5K$_{48}$MAE18.02717.854
Unseen stationsMASE / MAE1.078 / 0.5401.210 / 0.578
LiveHouse-TSMASE3.1794.465
BuildingsBenchCVRMSE51.454.1

Table R7. In-federation results on TSLib under the protocol of Tab. 3, averaged over the seven datasets.

MethodAvg MSEAvg MAE
FedRep0.3300.362
FFTS0.3340.369
FeDaL0.3340.363
FedTRL0.3330.365
FedState0.3150.355

FedState achieves the lowest average error while uploading only 0.36M parameters per round, compared with 0.64M for FedTRL's encoder and prototypes. Table R8 further evaluates both methods under the in-domain protocol.

Table R8. In-domain MSE / MAE under the protocol of Tab. 2, averaged over four horizons, and BuildingsBench CVRMSE averaged over seven collections.

DatasetFedStateFedTRL
ETTh10.422 / 0.4350.478 / 0.470
ETTh20.339 / 0.3860.413 / 0.433
ETTm10.360 / 0.3830.356 / 0.390
ETTm20.254 / 0.3130.274 / 0.330
ECL0.165 / 0.2580.167 / 0.261
Traffic0.406 / 0.2790.420 / 0.297
Weather0.235 / 0.2710.233 / 0.271
Exchange0.389 / 0.4160.527 / 0.464
Avg0.321 / 0.3430.359 / 0.364
BuildingsBench (CVRMSE)56.1256.41

FedState reduces average MSE and MAE by 10.6% and 5.8% relative to FedTRL, with lower errors in 13 of the 16 dataset-level metrics, and also attains a lower average CVRMSE on BuildingsBench. We have added FedTRL to Tabs. 2, 3, and 8 and the corresponding full results, with its reproduction settings in App. C.2.


We hope this fully addresses your questions and comments, and remain happy to engage in further discussion for the rest of the rebuttal period.

Dear Reviewer JixB,

Thank you for your positive assessment of our work and your constructive feedback.

As the discussion period draws to a close, we would appreciate knowing whether our responses have adequately addressed your concerns. We would also be happy to provide any further clarification during the remaining discussion period.

Best regards,
The Authors

Reviewer_Z1gMOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents FedState, a federated time series pretraining framework that shares latent transition dynamics instead of full model parameters. The method introduces physical time conditioning, latent state prediction, and heterogeneity aware aggregation. Experiments across multiple benchmarks demonstrate improvements in forecasting accuracy, communication efficiency, and cross domain transfer.

Critical strengths
  1. Meaningful design motivation. Sharing latent dynamics instead of observation dependent representations is an interesting approach to heterogeneous federated learning. Physical time conditioning also provides a reasonable mechanism for handling different sampling frequencies.
  2. Broad experimental evaluation. The paper evaluates several forecasting settings and includes extensive ablations. The reported communication savings are particularly appealing for distributed applications.
Critical weaknesses
  1. Insufficient justification of the shared latent representation. The method assumes that independently trained private encoders can produce compatible latent states. However, the theoretical analysis does not establish this alignment. More importantly, Appendix B.3 averages private encoders for zero shot inference, which raises questions about the claim that only the transition operator is exchanged.
  2. Concerns about experimental fairness and generalization. In Section 4.2, FedState uses more training clients than the full model federated baselines on TIME and GIFT-Eval. Appendix B.3 also reports averaging multiple FedState federations without clearly establishing equivalent ensemble settings for the baselines. Furthermore, the zero shot results on BuildingsBench are substantially weaker than several alternatives. These issues weaken the broad superiority claims.
Questions for authors

Q1. How are private encoders averaged for zero shot inference, especially when clients have different architectures? Does this process introduce additional communication or privacy requirements?

Q2. Can the authors provide comparisons using identical client populations and ensemble budgets? Such controls would help determine whether the reported improvements arise from the proposed method rather than differences in experimental settings.

6 comments
AuthorsOpenReview ↗

Authors' Rebuttal (1/3)

Thank you for providing us with your valuable feedback. We have carefully considered your comments and would like to address them as below.


W1 [Alignment of independently trained private encoders]

Training private encoders against a shared transition operator drives their latent coordinates toward compatibility, as supported by our theoretical analysis and new substitution experiments. We address this from four aspects below:

(1) Compatibility Induced by the Shared Law. Consider a client whose latent transition satisfies z′=fϕ(z,Δ)z'=f_\phi(z,\Delta). An invertible coordinate transformation TT preserves the same law only if fϕ(Tz,Δ)=Tfϕ(z,Δ)f_\phi(Tz,\Delta)=Tf_\phi(z,\Delta) on the relevant state support. A private operator can accommodate arbitrary transformations through conjugation, TfϕT−1Tf_\phi T^{-1}, whereas a shared operator admits only those compatible with the same dynamics. Thm. 1(a) characterizes the corresponding excess risk through the dispersion DD of client latent transition kernels over overlapping state distributions. Minimizing the federated loss therefore drives latent coordinates toward agreement, while the variance and grounding terms in Eq. 18 discourage degenerate representations.

(2) Empirical Evidence of Compatibility. Existing results (Fig. 1) show that cross-domain displacement agreement increases from 0.41 in observation space to 0.70 in latent space. Domain classification from latent transition directions achieves 16% accuracy, close to the chance level of 17%, compared with 78% and 83% using spectral and increment features (Fig. 8).

(3) Direct Measurement on the Zero-Shot Federation. We further conduct substitution experiments on TSLib and a 48-client zero-shot federation spanning 10 sampling periods from 10 seconds to one day, with all private encoders initialized independently. We measure pairwise representation similarity using linear CKA on common validation windows and evaluate encoder substitution across clients and replacement with the aggregated observation model. All substitutions are performed after pretraining without further optimization. Table R1 reports the results.

Table R1. Functional compatibility of independently initialized private encoders, averaged over three seeds. Linear CKA ranges from 0 to 1 and is invariant to rotation and isotropic scaling.

FederationTrainingLinear CKAΔMSE, cross-clientΔMSE, aggregate
TSLib, 7 clientsPrivate operators0.41+58.7%+51.6%
TSLib, 7 clientsFedState0.84+6.2%+4.6%
Zero-shot, 48 clientsPrivate operators0.33+71.5%+63.8%
Zero-shot, 48 clientsFedState0.79+8.4%+6.1%

The shared law substantially improves representation similarity and functional compatibility. In the 48-client federation, replacing a client's observation model with the aggregate increases MSE by only 6.1%, compared with 63.8% under private operators.

(4) One-Time Bootstrap versus Iterative Communication. During federated pretraining, clients exchange only the transition law. After the final round, private observation models are aggregated once to initialize unseen clients (Alg. 1, App. B.3). Participating clients retain their private observation models for all in-domain, in-federation, and heterogeneous-architecture evaluations. Zero-shot clients use the aggregate without target-domain training, while full- and few-shot clients outside the Monash federation fine-tune from it, with FedState performing best in most settings (Tab. 7). Unlike full-model federation, which exchanges observation mappings every round, FedState transmits them only once for deployment.

We have incorporated the theoretical clarification and Table R1 into the revised manuscript.


Q1 [Averaging private encoders for zero-shot inference: procedure, architectures, communication, and privacy]

The server aggregates client observation models once after pretraining, with separate aggregates for different architectures. This requires only one additional upload per client and supports secure aggregation and client-level differential privacy. We address this from three aspects below:

(1) Construction of the Zero-Shot Model. The server averages client encoders, decoders, and prediction branches, weighted by their training-window counts, while retaining the federated transition law. In the zero-shot evaluation, all clients use the same architecture and parameter shapes, allowing direct parameter averaging. No aggregation is required in the heterogeneous-architecture experiments (Tab. 9), where clients retain their private encoders. All federated baselines follow the same zero-shot deployment.


Please see the subsequent Authors' Rebuttal (2/3) for remaining responses.

AuthorsOpenReview ↗

Authors' Rebuttal (2/3)

Continued: Authors’ Rebuttal (2/3)


(2) Communication Cost. The one-time bootstrap requires an additional upload of 2.10M parameters per client, less than a single round of full-model federation (2.46M). Including 60 rounds of transition-law updates, the total upload per FedState client is 60×0.36M+2.10M=23.7M60\times0.36\text{M}+2.10\text{M}=23.7\text{M} parameters, only 16.1% of the 147.6M required by full-model federation.

(3) Privacy Guarantees. Since aggregation requires only the sum of client models, secure aggregation prevents the server from observing individual models. For a release with client-level differential privacy, we use equal weighting and clip each observation model to an L2 norm of C=1.0C=1.0. Each client adds Gaussian noise N(0,σ2C2I/N)\mathcal{N}(0,\sigma^2C^2I/N) before secure aggregation, yielding aggregate noise N(0,σ2C2I)\mathcal{N}(0,\sigma^2C^2I) calibrated to the sum's sensitivity CC. With δ=10−5\delta=10^{-5}, this provides (ε,δ)(\varepsilon,\delta)-differential privacy at ε=4.4\varepsilon=4.4 for σ=1\sigma=1 and ε=2.0\varepsilon=2.0 for σ=2\sigma=2. Table R2 reports the resulting zero-shot performance.

Table R2. Zero-shot performance with client-level differential privacy (δ=10−5\delta=10^{-5}).

ReleaseUnseen stations MASEWEATHER-5K$_{512}$ MAELiveHouse-TS MASE
Weighted average without noise1.07817.1093.179
Clipped, equal weights, w/o. noise1.08017.1313.183
Clipped, equal weights, σ=1\sigma=1, ε=4.4\varepsilon=4.41.12117.4783.421
Clipped, equal weights, σ=2\sigma=2, ε=2.0\varepsilon=2.01.23518.2443.687
FedAvgM w/o. protection1.47720.8543.982

Clipping and equal weighting change the results by at most 0.2%. Even at ε=2.0\varepsilon=2.0, the private release remains substantially more accurate than unprotected full-model federation on every metric. We have added the aggregation procedure to App. B.3 and Table R2 to App. C.4.

W2 [Experimental fairness and generalization]

FedState retains its advantage under identical client populations and equal ensemble sizes. A single FedState federation also outperforms ensembles of the strongest baselines, while its BuildingsBench results confirm its superiority among methods using the same backbone. We address this from four aspects below:

(1) Identical Client Populations with One Federation per Method. FedState supports client-specific look-back lengths and therefore federates all 46 TIME and 52 GIFT-Eval clients, whereas full-model baselines are restricted to 34 and 38 clients with a common context length of 512. To ensure identical client populations, we retrain FedState on these subsets and compare one federation per method. Table R3 reports the results.

Table R3. TIME (34 clients) and GIFT-Eval (38 clients) results with identical populations and one federation per method.

Benchmark, metricFedAvgMFedBNFedRepFedState
TIME, MSE$_n$0.6240.6370.6220.616
TIME, MAE$_n$0.8160.8210.8200.815
TIME, RMSE$_n$0.7940.8100.8050.792
TIME, MAPE$_n$0.7660.7650.7700.747
TIME, sMAPE$_n$0.8620.8800.8790.855
TIME, MASE$_n$0.8200.8320.8360.823
TIME, ND$_n$0.7930.7990.7890.780
TIME, CRPS$_n$0.7510.7500.7570.741
GIFT-Eval, MASE$_n$0.9150.9280.9550.920
GIFT-Eval, WQL$_n$0.7530.7280.7550.705

FedState achieves the best performance on seven of the eight TIME metrics, consistent with Tab. 5, and reduces GIFT-Eval WQL$_n$ by 3.2% relative to the strongest baseline. These improvements require only 19% of FedAvgM's per-round upload.

(2) Identical Client Populations and Equal Ensemble Sizes. App. B.3 averages predictions from independently trained federations that differ only in random seed. The per-round upload of each federation remains unchanged, while total communication scales with the ensemble size kk. We therefore match both client populations and ensemble sizes, comparing FedState against the strongest baseline for each metric in Table R4.

Table R4. Results with seven TSLib clients (k=3k=3), 34 TIME clients (k=6k=6), and 38 GIFT-Eval clients (k=6k=6). Single-model results are averaged over kk runs.

Benchmark, metricStrongest baselineBaseline, singleBaseline, ensembleFedState, singleFedState, ensemble
TSLib, Avg MSEFedRep0.3310.3280.3170.315
TSLib, Avg MAEFedRep0.3620.3600.3570.355
TIME, MSE$_n$FedRep0.6230.6190.6160.611
TIME, CRPS$_n$FedBN0.7500.7440.7410.737
GIFT-Eval, WQL$_n$FedBN0.7270.7200.7050.700

A single FedState federation outperforms the strongest baseline ensemble on every metric, and this advantage persists when ensemble sizes are matched. On TIME, it uploads 21.6M parameters per client, only 6.1% and 3.2% of the 356M and 673M required by six-member FedRep and FedBN ensembles.


Please see the subsequent Authors' Rebuttal (3/3) for remaining responses.

AuthorsOpenReview ↗

Authors’ Rebuttal (3/3)

Continued: Authors’ Rebuttal (3/3)


(3) Zero-Shot on BuildingsBench. BuildingsBench contains a single sampling frequency, with strong weekly repetition in commercial building loads. This favors seasonal-naive forecasting, which achieves a CVRMSE of 12.4 on commercial buildings and outperforms all learned methods (Tab. 20). On residential buildings, where weekly repetition is less effective, FedState outperforms seasonal naive and all full-model and personalized federated baselines. Overall, FedState performs best among methods sharing the same backbone, including centralized alternatives. When buildings are included in pretraining, FedState ranks first on four of seven collections and second on BDG-2 (Tab. 2). With larger-scale LOTSA pretraining, it achieves the best BuildingsBench CVRMSE of all evaluated methods (Tab. 24).

(4) Scope of the Superiority Claims. FedState outperforms all full-model federated baselines across eight zero-shot settings and ranks first overall in six (Tab. 8). It also leads in 17 in-domain settings (Tab. 2), 10 of 14 settings in each of deterministic and probabilistic in-federation forecasting (Tabs. 3 and 4), and 20 of 30 full- and few-shot transfer settings (Tab. 7).

We have clarified ensemble sizes in App. B.3, added Tables R3 and R4 to App. C.4, and specified the comparison set of each claim in Sec. 4.3.


Q2 [Identical client populations and ensemble budgets]

We have added both comparisons, and FedState retains its advantage with identical client populations and equal ensemble budgets. Please refer to our response to W2 (1) and (2), with results in Tables R3 and R4.


We hope this fully addresses your questions and comments, and remain happy to engage in further discussion for the rest of the rebuttal period.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

Dear Reviewers, AC, SAC, and PC,

Thank you for your time and effort in reviewing and handling our paper. We have posted point-by-point responses to each reviewer and summarize the main points below.


Paper Overview and Strengths

[Paper Overview]: This paper proposes FedState, a federated time series pretraining framework that keeps each client's observation model private and federates only a lightweight interval-conditioned latent transition operator. It outperforms full-model federation across eight benchmarks at substantially lower communication cost and transfers zero-shot to unseen domains.

[Strengths]: We are encouraged that the reviewers consider sharing latent dynamics instead of observation-dependent representations a meaningful direction (Z1gM, TY9Q), regard physical-time conditioning as a reasonable mechanism for different sampling frequencies (Z1gM), find the design coherent and theoretically motivated (TY9Q), and recognize our broad evaluation (Z1gM, JixB, TY9Q), communication savings (Z1gM, TY9Q), and the compact shared module and clear writing (JixB).


Additional Experiments for Concerns

  • [Encoder Compatibility] (Z1gM, JixB, TY9Q): Substitution experiments (Table R1 for Z1gM, R3 for JixB, R1 for TY9Q) show that substituting the aggregate for a client's observation model raises MSE by 6.1% under the shared law against 63.8% under private operators. Across architectures, encoder substitution raises it by 6.3% against 73.9% (TY9Q).
  • [Identical Client Populations] (Z1gM, TY9Q): On the same 34 TIME and 38 GIFT-Eval clients with one federation per method (Table R3 for Z1gM, R4 for TY9Q), FedState performs best on seven of the eight TIME metrics and lowers GIFT-Eval WQL$_n$ by 3.2%.
  • [Matched Ensemble Sizes] (Z1gM): A single FedState federation outperforms the strongest baseline ensemble on every metric (Table R4 for Z1gM).
  • [Differential Privacy] (Z1gM): Released with client-level differential privacy at ε=2.0\varepsilon=2.0 (Table R2 for Z1gM), the one-time aggregate remains substantially more accurate than unprotected full-model federation.
  • [Physical-Time Conditioning and Durations] (JixB): Replacing the physical interval with the slot index increases errors by 2.9% to 4.0% (Table R2 for JixB), whereas adding client-dependent state durations changes no metric by more than 0.001 (Tables R4 and R5 for JixB).
  • [Comparison with FedTRL] (JixB): FedState outperforms our reproduction of FedTRL (Tables R6 to R8 for JixB) on seven of the eight zero-shot metrics, within the training federation, and in-domain by 10.6% in MSE and 5.8% in MAE.
  • [Context Length on ECL] (TY9Q): With a 512-step look-back (Table R2 for TY9Q), FedState achieves the lowest MSE, so the gap in Tab. 3 stems from the short context.
  • [Shared Component] (TY9Q): Varying only the shared component (Table R3 for TY9Q), sharing the law alone outperforms sharing the encoder, decoder, or full model, and matches sharing the encoder and law jointly with 37% of their upload.
  • [Robustness of the Tilt] (TY9Q): Under data corruption or a frozen encoder on the largest hourly TIME client (Table R5 for TY9Q), the error of other clients increases by at most 0.6%, while the tilt improves the perturbed client.

Summary of Responses to Reviewers' Questions

  • [Aggregation for Unseen Clients] (Z1gM, JixB, TY9Q): Only the transition law is exchanged during pretraining. After the final round, client observation models are aggregated once, separately within each architecture, to initialize unseen clients (Table R1 for JixB, R6 for TY9Q), which adds one upload per client and supports secure aggregation (Z1gM).
  • [Alignment Boundary and Theorem Scope] (TY9Q): Alignment is enforced only where client state distributions overlap, and disjoint clients do not impair the federation. Thm. 1(a) and (b) hold for arbitrary heterogeneous kernels, and a new proposition extends the separation of Cor. 1 to them under a sufficient condition.
  • [Dataset-Level Exceptions] (Z1gM, TY9Q): On BuildingsBench, where weekly repetition favors seasonal-naive forecasting, FedState remains the most accurate among methods sharing its backbone and the best overall with LOTSA pretraining. It ranks first on Exchange with a 512-step look-back (TY9Q) and outperforms all full-model federated baselines in all eight zero-shot settings (Z1gM).

Revisions During Rebuttal

All changes are highlighted in blue: the compatibility mechanism and the tilt discussion (Sec. 3), the comparison sets and corrected communication numbers (Sec. 4.3), the slot-index ablation (Fig. 4b), FedTRL (Tabs. 2, 3, 8, 14, 18 to 22, and 30, App. C.2), the heterogeneous-kernel proposition (App. A, Sec. 3), the one-time aggregation and ensemble sizes (App. B.3), and all other new experiments (Tabs. 33 to 42, App. C.4).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.