- Rating
- 2
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
FedTRL aims to train a shared time series model across different data sources. It reduces differences within each source and uses feature-based weights when combining the models, so both intra and inter domain heterogeneity. The experimental setups cover point and probabilistic forecasting on several benchmarks.
Critical strengths
The experiments cover several benchmarks and forecasting setups and show improvements over the federated baselines, also tested on top of some existing backbones
Critical weaknesses
The method uses domain classification loss as a heuristic measure of source reliability. But a source may be easy to identify simply because of source specific patterns or shortcuts. Although the weighting improves the results, the link between this score and reliability can still benefit from better analysis and evidence.
Questions for authors
Clarification: Under what conditions can combining different data sources hurt prediction, and how well does FedTRL handle these cases?
Clarification: Are the main comparisons matched in terms of total data seen and training budget?
3 comments
Authors' Rebuttal (2/2)
Continued: Authors’ Rebuttal (2/2)
Q2 [Matched total data seen and training budget in the main comparisons]
Yes. The comparisons behind our claim that federated pretraining matches or outperforms centralized pretraining are matched in data, backbone, and number of updates. In the in-domain setting, all methods use the same eight datasets and Transformer backbone, the federated methods follow one protocol (App. C.1), and All Mixed pools these data in a single centralized run. In the large-scale setting, all methods use the same 302M backbone, the Time-MoE-300B corpus, and the same total number of sequences and updates across clients (App. C.6). We have added the GPU-hours of every method, and FedTRL uses only 2.7% more than All Mixed (1,102 vs. 1,073), so the budgets also match in computation. Centralized TSFMs are compared through their public results, so their pretraining data and budgets are fixed by their original training. Among them, Time-MoE is pretrained on the same Time-MoE-300B corpus (Tab. 11).
We have added Tables R1 and R2 to App. E and stated the matched training budgets in App. C.6 of the revised manuscript.
We hope this fully addresses your questions and comments, and remain happy to engage in further discussion for the rest of the rebuttal period.
Dear Reviewer egXg,
As the discussion period draws to a close, we would appreciate knowing whether our responses have addressed your concerns. If so, we would be grateful if you could reconsider your rating accordingly. We would also be happy to address any remaining questions during the remaining discussion period.
Best regards,
The Authors
Authors' Rebuttal (1/2)
Thank you for providing us with your valuable feedback. We have carefully considered your comments and would like to address them as below.
W1 [Whether ℓk reflects source reliability or source-specific shortcuts]
DaG does not rely on the server-side classification loss ℓk of a client's prototype alone. Instead, it combines ℓk with the prototype deviation δk through sk=−αℓk−βδk. Imagine a client whose series share a periodic pattern absent from all other clients. This client is easy to identify (low ℓk), but its prototype lies far from the global prototype (high δk), allowing δk to offset its low ℓk. We have added two analyses showing that DaG weights align more closely with measured client utility than ℓk alone or sample size, while suppressing the weight of such a shortcut-driven client. Specifically, we address the link to reliability from two aspects below:
(1) Weights versus Measured Utility. We measure each client's utility as the increase in TSLib zero-shot MSE when that client is excluded from 500 additional rounds of uniform aggregation, which start at round 9,500 of the large-scale model and keep the total number of updates fixed. Uniform aggregation keeps this measurement independent of the weighting rule, and Tables R1 and R1b below report the results. The three clients with the largest DaG weights are also the three with the highest utility, and the only client whose removal lowers the error (C8) receives the second-smallest weight.
Table R1. Spearman correlation between aggregation weight and client utility.
Table R1b. DaG weight and utility of each client.
(2) Shortcut Client. We add the client of the example above as a tenth client that holds 10% of all samples. It is the easiest to identify ($\ell_k=0.05vs.amedianof0.29)buthasalargeprototypedeviation(\delta_k=$2.28 vs. 0.24). Table R2 below compares aggregation rules differing only in their client weights, with scores and weights averaged over the last 1,000 rounds. The ℓk-only rule assigns this client more weight than its sample share and incurs the largest RW-Bench MSE increase. In contrast, DaG assigns it less than half its sample share and reduces the increase to +0.004.
Table R2. Results with the added shortcut client. RW-Bench change is relative to the same rule without this client.
We have clarified the role of ℓk in Sec. 3.2 and App. B.2 of the revised manuscript.
Q1 [Conditions under which combining sources hurts, and how well FedTRL handles them]
Combining sources can hurt prediction when their temporal dynamics conflict, leading to negative transfer under joint training. For example, exchange rates evolve almost as a random walk, whereas electricity load follows daily cycles, so fitting both with a single model can degrade prediction accuracy. We have added a comparison against training on each dataset alone (Table R3 below): FedAvg exhibits negative transfer on five of eight datasets, whereas FedTRL improves on all eight. Each local-only model is trained in the same way as FedAvg.
Table R3. In-domain MSE of local-only and joint training.
Consistent with the example above, both All Mixed and FFTS underperform local-only training on Exchange. Such conflicting dynamics manifest as opposing gradients across domains. Tab. 24 shows that gradient conflict increases with the number of mixed domains under centralized training, but grows much more slowly under FedTRL. FedTRL also achieves the lowest error at every tested level of domain imbalance (Tab. 25) and when a related domain is withheld from pretraining (Tab. 22).
Please see the subsequent Authors' Rebuttal (2/2) for remaining responses.