- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes FedState, a federated time-series pretraining framework that addresses client heterogeneity by sharing latent temporal dynamics rather than observation-level representations or entire models. FedState maintains client-specific encoders and decoders while federating a shared latent transition operator conditioned on elapsed physical time. This design accommodates differences in sampling frequencies, channel dimensions, data scales, and local model architectures. The framework further introduces latent-state pretraining with multi-scale representations and heterogeneity-aware aggregation that assigns greater weights to clients with larger transition errors. The paper provides theoretical analysis of physical-time conditioning and evaluates FedState across multiple benchmarks, including deterministic and probabilistic forecasting, few-shot adaptation, zero-shot transfer, and communication efficiency. The results generally demonstrate competitive or superior forecasting performance with substantially reduced communication overhead.
Critical strengths
S1. Sharing latent temporal dynamics instead of observation-dependent representations offers a meaningful alternative to conventional federated time-series learning, particularly when clients have substantially different data distributions and sampling frequencies.
S2. The combination of physical-time conditioning, latent-state prediction, and personalized observation models provides a coherent approach to handling multiple forms of client heterogeneity.
S3. The analysis of grid-indexed versus physical-time-indexed transitions provides theoretical motivation for the proposed design. The aggregation mechanism is also supported by a distributionally robust optimization formulation.
S4. The paper evaluates FedState across several benchmarks and settings, including in-domain forecasting, probabilistic forecasting, few-shot learning, zero-shot transfer, heterogeneous architectures, and communication efficiency.
S5. Federating only the transition operator substantially reduces the number of communicated parameters compared with full-model federated learning.
Critical weaknesses
W1. The approach assumes that heterogeneous clients can learn compatible latent representations in which a common transition operator is meaningful. However, the client encoders are independently trained and remain private. Although the empirical results support the usefulness of the shared operator, the paper does not fully establish when independently learned latent spaces become sufficiently aligned or when this assumption may fail.
W2. Although FedState achieves strong overall performance, the improvements are not consistent across all datasets. For example, in Table 3, FedState underperforms several baselines on ECL and Exchange. Similarly, Table 8 identifies BuildingsBench as a challenging zero-shot transfer setting. These results suggest that the benefits of shared latent dynamics depend on the characteristics of the participating domains.
W3. The proposed framework combines a shared transition operator, physical-time conditioning, multi-scale representations, additional pretraining objectives, and specialized aggregation. Although the paper includes component ablations, a more direct comparison between sharing observation representations and sharing latent dynamics under otherwise identical architectures and training objectives would provide stronger evidence for the central hypothesis.
Questions for authors
Q1. In Section 4.2, FedState uses all 46 TIME clients and 52 GIFT-Eval clients, whereas full-model federated baselines use only 34 and 38 clients, respectively, because of context-length constraints. Consequently, the improvements may partially reflect access to more participating clients rather than solely the proposed shared representation. An additional comparison using identical client subsets would strengthen the conclusions.
Q2. Equation (2) allows each client to learn its own encoder while using the same transition operator. However, the paper does not clearly explain how latent representations from different encoders acquire compatible semantics without explicit cross-client alignment. The shared operator may encourage implicit alignment, but the mechanism and its limitations deserve further explanation.
Q3. The theorem establishes an advantage for physical-time indexing when clients share the same underlying transition kernel. However, FedState is motivated by heterogeneous clients whose dynamics may differ. The theoretical analysis does not establish that physical-time conditioning necessarily reduces excess risk under arbitrary heterogeneous dynamics. The paper should more clearly distinguish the theorem’s assumptions from the broader empirical claims.
Q4. The aggregation mechanism assigns larger weights to clients with higher latent prediction errors. However, larger errors may reflect intrinsically difficult datasets, noisy observations, or poorly learned representations rather than underrepresented dynamics. The paper could better justify when prioritizing these clients improves the shared operator instead of introducing bias toward noisy or difficult domains.
Q5. The discussion reports 1.73M uploaded parameters for full-model federation and 1.44M for FFTS, whereas Table 7 reports 2.92M and 5.63M, respectively. The authors should clarify whether these numbers correspond to different configurations or correct the inconsistency.
Q6. Algorithm 1 states that unseen clients use an average of the private encoders. Since clients may have different architectures, channel dimensions, and sampling configurations, it is unclear how their encoder parameters can be averaged consistently. The paper should clarify the exact procedure used for constructing observation mappings for unseen clients.
7 comments
Authors’ Rebuttal (2/4)
Continued: Authors’ Rebuttal (2/4)
(3) Zero-Shot on BuildingsBench. BuildingsBench contains a single sampling frequency, with strong weekly repetition in commercial building loads. This favors seasonal-naive forecasting, which achieves a CVRMSE of 12.4 on commercial buildings and outperforms all learned methods (Tab. 20). On residential buildings, where weekly repetition is less effective, FedState outperforms seasonal naive and all full-model and personalized federated baselines. Overall, FedState performs best among methods sharing the same backbone, including centralized alternatives. When buildings are included in pretraining, FedState ranks first on four of seven collections and second on BDG-2 (Tab. 2). With larger-scale LOTSA pretraining, it achieves the best BuildingsBench CVRMSE of all evaluated methods (Tab. 24). We have added Table R2 to App. C.4.
W3 [Sharing observation representations vs. latent dynamics under identical architectures and objectives]
Under identical architectures and training objectives, sharing only the transition law matches or outperforms sharing the encoder or full model at substantially lower communication cost. We provide this controlled comparison in Fig. 3a and extend it to the in-federation protocol through Table R3 below, with further evidence from supervision-space comparisons. We address this from three aspects below:
(1) Choice of Shared Component on TIME. Sharing only the law matches the CRPS$_n$ of sharing the encoder or both the encoder and law, while uploading only 0.35M parameters per round compared with 0.62M and 0.97M. It also outperforms sharing the decoder or full model, the latter requiring 1.86M parameters.
(2) Choice of Shared Component under the In-Federation Protocol. Under the protocol of Tab. 3, we vary only the shared component. Table R3 reports the results. Sharing only the law outperforms sharing the encoder, decoder, or full model on both metrics. It also stays within one standard deviation of sharing the encoder and law jointly while requiring only 37% of the upload.
Table R3. In-federation comparison of shared components on seven TSLib datasets under a consistent setting (96-step look-back). Results are averaged over three seeds.
| Shared component | Upload/Round | Avg MSE | Avg MAE |
|---|---|---|---|
| None (Scratch) | 0 | 0.3347±0.0005 | 0.3647±0.0004 |
| Full model | 1.13M | 0.3346±0.0013 | 0.3629±0.0009 |
| Decoder | 0.15M | 0.3281±0.0007 | 0.3612±0.0008 |
| Encoder | 0.62M | 0.3214±0.0011 | 0.3589±0.0005 |
| Encoder and law | 0.98M | 0.3166±0.0008 | 0.3567±0.0006 |
| Law (FedState) | 0.36M | 0.3170±0.0006 | 0.3570±0.0004 |
(3) Latent vs. Observation-Level Objectives. Under matched architectures and training budgets, latent-state prediction reduces MSE by 14.7%, MAE by 7.8%, and worst-client MSE by 40.9% compared with observation-space prediction (Fig. 3b). The same trend holds against observation-level federated self-supervised baselines with a shared backbone, where FedState leads in 17 settings versus at most three for any baseline (Tab. 2).
We have added Table R3 to App. C.4.
Q1 [Identical client subsets on TIME and GIFT-Eval]
On identical client subsets of TIME and GIFT-Eval, FedState maintains its advantage, which therefore does not depend on the additional clients. We retrain FedState on the 34 TIME and 38 GIFT-Eval clients available to the full-model baselines and compare one federation per method. Table R4 reports the results. FedState achieves the best performance on seven of the eight TIME metrics, consistent with Tab. 5, and reduces GIFT-Eval WQL$_n$ by 3.2% relative to the strongest baseline. These improvements require only 19% of FedAvgM's per-round upload.
Table R4. TIME (34 clients) and GIFT-Eval (38 clients) results with identical client populations and one federation per method.
| Benchmark, metric | FedAvgM | FedBN | FedRep | FedState |
|---|---|---|---|---|
| TIME, MSE$_n$ | 0.624 | 0.637 | 0.622 | 0.616 |
| TIME, MAE$_n$ | 0.816 | 0.821 | 0.820 | 0.815 |
| TIME, RMSE$_n$ | 0.794 | 0.810 | 0.805 | 0.792 |
| TIME, MAPE$_n$ | 0.766 | 0.765 | 0.770 | 0.747 |
| TIME, sMAPE$_n$ | 0.862 | 0.880 | 0.879 | 0.855 |
| TIME, MASE$_n$ | 0.820 | 0.832 | 0.836 | 0.823 |
| TIME, ND$_n$ | 0.793 | 0.799 | 0.789 | 0.780 |
| TIME, CRPS$_n$ | 0.751 | 0.750 | 0.757 | 0.741 |
| GIFT-Eval, MASE$_n$ | 0.915 | 0.928 | 0.955 | 0.920 |
| GIFT-Eval, WQL$_n$ | 0.753 | 0.728 | 0.755 | 0.705 |
We have added Table R4 to App. C.4.
Please see the subsequent Authors' Rebuttal (3/4) for remaining responses.
Authors’ Rebuttal (3/4)
Continued: Authors’ Rebuttal (3/4)
Q3 [Assumptions of the theorem under heterogeneous dynamics]
Thm. 1(a) and (b) hold exactly for arbitrary heterogeneous kernels, and only the strict separation of Cor. 1 requires a shared kernel. We further extend this separation to heterogeneous kernels under an explicit sufficient condition. We address this from two aspects below:
(1) Assumptions of the Theorem. App. A assumes no relationship among client transition kernels. Thm. 1(a) characterizes the excess risk of the optimal interval-indexed operator by the kernel dispersion at equal physical intervals, whereas Thm. 1(b) characterizes that of the optimal slot-indexed operator by at client-specific intervals . Their relative magnitudes depend on whether client dynamics agree more closely at equal elapsed times or equal sample offsets. The condition below formalizes the former regime, which is natural for physical processes such as weather, energy load, and traffic.
(2) A Sufficient Condition for Heterogeneous Kernels. Under Assumption 1, choose any Borel reference family on for which the energies below are finite, write , and let be the slot dispersion of as in Cor. 1(ii), with the same mixing measure and densities. Let Proposition. and . Consequently, whenever .
Proof. At each interval, the common reference cancels in the dispersion, which leaves the weighted dispersion of . Lemma 2 bounds it by the weighted second moment , and integration gives . For the slot term, consider the weighted space of client-slot families with inner product , modulo null sets. Let be the orthogonal projection onto families that do not depend on the client index. By Lemma 2, , and for . Writing with and , the triangle inequality and give .
When both operators are trained on the same physical intervals, with uniform over , their Bayes risks coincide, yielding . Physical-time conditioning achieves strictly lower optimal risk when , where cross-client deviations from a common physical-time kernel are smaller than one quarter of the dispersion induced by sampling-grid mismatch. Cor. 1 corresponds to the special case . Consistent with this prediction, replacing physical-time conditioning with slot indexing increases forecasting errors by 2.9% to 4.0% on TIME and GIFT-Eval. We have added this proposition after Cor. 1 and stated its condition in Sec. 3.
Q4 [Tilted aggregation and noisy or difficult clients]
Tilted aggregation prioritizes clients with underrepresented learnable dynamics, while standardized losses and bounded reweighting limit the influence of noisy or poorly fitted clients. Our additional experiments on TIME confirm that these safeguards preserve the performance of other clients under data corruption and encoder failure. We address this from three aspects below:
(1) Benefit of Prioritization. When high transition errors reflect learnable dynamics underweighted by data-size aggregation, the tilt increases their contribution to the shared operator. On the 46 TIME clients, removing the tilt increases median CRPS$_n$ by 0.9% (Sec. 4.4).
(2) Sufficiency of Bounded Reweighting. Large transition errors may also arise from intrinsically difficult datasets, noisy observations, or poorly learned representations. Rather than distinguishing these causes explicitly, the tilt standardizes client losses and clips their logits, bounding any client's aggregation weight (Prop. 1(b)).
Please see the subsequent Authors' Rebuttal (4/4) for remaining responses.
Authors’ Rebuttal (1/4)
Thank you for your feedback and valuable comments. Below is our point-by-point response.
W1 & Q2 [Compatibility of independently learned latent spaces, and its boundary]
Training private encoders against a shared transition operator drives their latent coordinates toward compatibility without explicit alignment, as supported by our theoretical analysis and direct substitution experiments across clients and architectures. We address this from four aspects below:
(1) Mechanism of the Implicit Alignment. Consider a client whose latent transition satisfies z′=fϕ(z,Δ). An invertible coordinate transformation T preserves the same law only if fϕ(Tz,Δ)=Tfϕ(z,Δ) on the relevant state support. A private operator can accommodate arbitrary transformations through conjugation, TfϕT−1, whereas a shared operator admits only those compatible with the same dynamics. Thm. 1(a) characterizes the corresponding excess risk through the dispersion D of client latent transition kernels over overlapping state distributions. Minimizing the federated loss therefore drives latent coordinates toward agreement, while the variance and grounding terms in Eq. 18 discourage degenerate representations.
(2) Conditions and Boundary of the Alignment. The shared operator constrains latent coordinates where client state distributions overlap, since it must model their transitions at the same physical intervals. When clients occupy disjoint latent regions, the operator can fit their dynamics without conflict. Clients in disjoint regions benefit less from cross-client transfer but do not impair the rest of the federation.
(3) Empirical Evidence of Compatibility. Existing results (Fig. 1) show that cross-domain displacement agreement increases from 0.41 in observation space to 0.70 in latent space. Domain classification from latent transition directions achieves 16% accuracy, close to the chance level of 17%, compared with 78% and 83% using spectral and increment features (Fig. 8).
(4) Direct Measurement across Clients and Architectures. We add substitution experiments on TSLib and a 48-client zero-shot federation spanning 10 sampling periods from 10 seconds to one day. We measure pairwise representation similarity using linear CKA on common validation windows and evaluate encoder substitution across clients and replacement with the aggregated observation model, all after pretraining without further optimization. Table R1 reports the results.
Table R1. Functional compatibility of independently initialized private encoders, averaged over three seeds. Linear CKA ranges from 0 to 1 and is invariant to rotation and isotropic scaling.
The shared law substantially improves representation similarity and functional compatibility. This compatibility extends across heterogeneous encoder architectures (Tab. 9), where encoder substitution increases MSE by 6.3% under the shared law versus 73.9% under private operators. We have added the mechanism, its conditions, and Table R1 to the revised manuscript.
W2 [Consistency across datasets: ECL, Exchange, and BuildingsBench]
The reported exceptions arise from evaluation conditions rather than a limitation of shared latent dynamics. FedState ranks first on both ECL and Exchange with a 512-step look-back, and on BuildingsBench it remains the most accurate among methods sharing its backbone. We address this from three aspects below:
(1) ECL. To examine the effect of context length, we retrain the compared methods on the ECL client of Tab. 3 using look-backs of 96 and 512, with all other settings unchanged. Table R2 reports the results. With a 512-step context, which covers about three weeks of hourly electricity demand rather than four days, FedState achieves the lowest MSE. The gap in Tab. 3 therefore stems from the short 96-step context.
Table R2. MSE on ECL under the in-federation protocol, with one federation per method at each look-back.
(2) Exchange. With the 96-step look-back of Tab. 3, FedState achieves comparable performance to training from scratch. Exchange rates are near-random walks, so short contexts carry little predictive information and favor simple models such as DLinear. With a 512-step look-back, FedState ranks first with an MSE of 0.389 (Tab. 2).
Please see the subsequent Authors' Rebuttal (2/4) for remaining responses.