- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2(+1)
- Soundness
- 2
Read review
Paper summary
The paper consider a temporal heterogeneity setting in Federated Continual Learning (FCL), where clients have asynchronous task progress. The paper proposed TARDIS, an algorithm that leverage the neural collapse to achieve the Equiangular Tight Frame (ETF) state, and make them as memory for the FCL system.
Critical strengths
- The idea of shared ETF anchors and the utilization of neural collapse is some how interesting and worth reading from my perspective.
- The results seem to be strong in class incremental learning.
- The paper apply a comprehensive ablation studies to prove the paper's technicality.
Critical weaknesses
- Aggregation bias and knowledge drift are long-standing FL/FCL issues, and the paper does not directly show that heterogeneous workloads cause or worsen them. Observations I–II are asserted rather than measured, and there is no synchronous schedule with the same partition as a control.
- It seems that the temporal heterogeneity is close to the asynchronous FCL setting [R1], The authors should consider about this more carefully to make a better positioning to the current paper scope of works.
- The ETF seems to be closely related to the current series of gradient projections in Continual Learning, especially the series of works in FCL, e.g., FOT [R2]. This part need a careful discussion to prove the novelty of ETF contribution. Furthermore, the discussion of model inversion seems to be incremental and disconnected from the whole storyline of ETF geometry. It should be noted that the current literature of applying model inversion in FCL has been considered, e.g., LANDER [R3] and TARGER [R4].
- The method is class-incremental by design, since expansion requires new, disjoint classes, so the general “FCL” framing in the title overclaims.
- The current presentation of the paper requires significant improvement. For instance, the neural collapse and the ETF appear without motivation in the introduction. Several core objects are never defined, e.g., the feature extractor , the auxiliary head, the global model and the forgetting metric.
- [R1]: Asynchronous federated continual learning, CVPRW.
- [R2]: Federated orthogonal training: Mitigating global catastrophic forgetting in continual federated learning, ICLR 2024.
- [R3]: Text-Enhanced Data-free Approach for Federated Class-Incremental Learning, CVPR 2024.
- [R4]: TARGET: Federated Class-Continual Learning via Exemplar-Free Distillation, ICCV 2023.
Questions for authors
- How the "Heterogeneous Workloads" are important? How it related to the model performance?
- As the task-stream misalignment, how it related to the different learning states and distinct task-specific knowledge?
- How to recover a global ETF from the learned representation space? do we just train it and overfit to converge?
- According to the experimental setup, how the temporal heterogeneity is made? it is not detailed clearly in the paper. How are task-IL and class-IL setting is made according to those dataset?
- Although the historical data is not saved. however, please report about the memory of the storage of ETF?
- As dirichlet alpha are proposed. how is the robustness according to different type of heterogeneity? For example , beside the class skew, how about the task-order mismatch [R5]?
- [R5]: HERO: A Heterogeneity-Aware Benchmark Library for Federated Continual Learning, NeurIPS 2026.
8 comments
Response to Reviewer 9FZn (part 2)
Comment 2:
"It seems that the temporal heterogeneity is close to the asynchronous FCL setting [R1], The authors should consider about this more carefully to make a better positioning to the current paper scope of works."
Response:
We thank the reviewer for bringing up this important question for better positioning the scope of work and pointing us to "Asynchronous Federated Continual Learning (AFCL) [Shenaj et al., 2023]". Although both AFCL and our temporal heterogeneity (TH) setting involve asynchronous aggregation and clients training different tasks at the same time, the asynchrony arises from fundamentally different sources, and the two settings therefore pose different problems.
In AFCL, each client receives its tasks at times and in an order set by its own circumstances, for example when its data become available or when the device can participate. The task schedule itself differs across clients and is independent of the learning problem. FedSpace accordingly focuses on keeping the global model stable when client updates arrive out of step with one another.
In TH, all clients follow the same task sequence, but each device can only process a limited amount of data between two communication rounds, due to its compute budget and the cost of communicating with the server. A task therefore occupies a client for as many rounds as its local workload requires. Because local data workload differ, clients spend different numbers of rounds on the same task and gradually fall out of step, even though they start together. At any given round, the server therefore receives updates from clients at different stages of the task sequence, and the clients with the most data on a task are still sending updates for it after most others have moved on. The asynchrony in TH thus comes from how much each task asks of a resource-constrained client, instead of from when the task arrives.
In the revision, we cite AFCL and add a detailed discussion of related work on asynchronous federated continual learning in Appendix A.1.1. We further compare TARDIS with FedSpace (proposed by Shenaj et al. in AFCL), when applied under temporal heterogeneity across all benchmarks.
Table R1. Comparison with FedSpace (Shenaj et al., CVPRW 2023), the method proposed for asynchronous FCL, when applied under temporal heterogeneity. Mean ± std over three random seeds. Acc: accuracy (%, ↑); F: forgetting (%, ↓). Bold indicates the better accuracy.
| Benchmark | Method | Task-IL Acc ↑ | Task-IL F ↓ | Class-IL Acc ↑ | Class-IL F ↓ |
|---|---|---|---|---|---|
| Split-CIFAR-10 | FedSpace | 68.23 ± 0.95 | 7.98 ± 1.00 | 29.35 ± 2.50 | 31.67 ± 0.63 |
| TARDIS | 93.79 ± 0.28 | 0.46 ± 0.20 | 65.61 ± 0.56 | 13.52 ± 0.87 | |
| Split-CIFAR-100 | FedSpace | 35.11 ± 1.56 | 49.40 ± 1.45 | 14.01 ± 1.12 | 60.44 ± 2.19 |
| TARDIS | 76.06 ± 0.16 | 4.00 ± 0.30 | 38.83 ± 0.54 | 10.91 ± 0.48 | |
| Split-EMNIST | FedSpace | 54.20 ± 1.74 | 34.39 ± 2.76 | 20.01 ± 0.77 | 57.74 ± 1.60 |
| TARDIS | 97.96 ± 0.01 | 0.32 ± 0.05 | 85.62 ± 0.63 | 5.61 ± 1.22 | |
| Split-TinyImageNet | FedSpace | 15.76 ± 0.15 | 56.63 ± 0.20 | 7.88 ± 0.16 | 66.26 ± 0.33 |
| TARDIS | 42.68 ± 0.70 | 6.41 ± 0.55 | 18.76 ± 1.10 | 5.13 ± 1.24 |
As shown in Table above, TARDIS outperforms FedSpace on all four benchmarks under both protocols, with substantially lower forgetting. A per-task inspection shows that FedSpace learns the most recent task well (e.g., Task-IL is 98.6% on the last Split-EMNIST task, 67.40% on the last Split-TinyImageNet) but drops to chance level on all earlier tasks. This follows from its design assumptions. FedSpace preserves old classes through prototypes aggregated from the classes that clients are currently training, which is effective in AFCL, where random task orders keep every class in circulation. Under TH, however, all clients eventually move past a task, after which its classes are no longer covered by the global prototypes and their classifier receives no supervision. TARDIS does not depend on old classes remaining active: their representations are tied to fixed ETF anchors shared by all clients and preserved by feature distillation, so knowledge of a finished task is retained even after no client trains it.
Response to Reviewer 9FZn (part 3)
Comment 3:
"The ETF seems to be closely related to the current series of gradient projections in Continual Learning, especially the series of works in FCL, e.g., FOT [R2]. This part need a careful discussion to prove the novelty of ETF contribution. Furthermore, the discussion of model inversion seems to be incremental and disconnected from the whole storyline of ETF geometry. It should be noted that the current literature of applying model inversion in FCL has been considered, e.g., LANDER [R3] and TARGER [R4]."
Response:
We thank the reviewer for highlighting these relevant studies. We provide a careful discussion as below to clarify their relationship to TARDIS below. We also added this discussion to the revised manuscript in Appendix A.1.3.
ETF anchoring vs. gradient projection.
Gradient-projection methods such as GPM [1] and the FCL study FOT [2] protect old tasks by restricting how parameters may change: FOT extracts a per-layer subspace of old-task activations after each task and projects the aggregated update of the next task to be orthogonal to it. ETF anchoring instead fixes where class features should be: each class is assigned a fixed vertex shared by all clients, and features are aligned to it, without restricting the direction of parameter updates. They are differ in three respects. First, FOT protected subspace grows with every task and progressively reduces the space available for new tasks, whereas adding classes to the ETF only adds target directions and leaves the parameter space unconstrained. Second, the orthogonality in TARDIS lies in the class-target space, not in the parameter space. Third, which is most relevant to our setting, FOT relies on synchronized task boundaries. In this case, the subspace is built once all clients have finished a task, and the projection is applied to the aggregated update under the assumption that all clients train the same new task. While under temporal heterogeneity, however, some clients are still training task while others have moved on, so removing the task directions from the aggregated update would also discard the progress of the clients still learning task . ETF alignment acts per sample on each client’s own classes and needs no shared task boundary, because consistency across clients comes from the shared anchors themselves. The two mechanisms are complementary rather than competing, and combining them is a possible direction for future work.
Model inversion and its role in TARDIS.
We acknowledge that model inversion and data-free knowledge transfer have been investigated in LANDER [3] and TARGET [4]. We do not claim novelty in model inversion itself. Instead, TARDIS$_{DF}$ is an optional extension for data-sensitive or memory-constrained environments where storing historical exemplars is undesirable. Synthetic samples allow previously learned classes to continue participating in ETF alignment and representation distillation without retaining raw replay data. The core TARDIS framework remains independent of this extension, as demonstrated by our buffer-based experiments.
We will revise the manuscript to state this explicitly in the contributions and present the variant in a separate extension subsection, so that it is not read as a core part of the method. We will also expand the related-work discussion in Appendix A.1.3 to clarify these distinctions and better separate the central contribution of temporal geometric anchoring from the optional data-free implementation.
References
[1] Saha, Gobinda, Isha Garg, and Kaushik Roy. "Gradient projection memory for continual learning." arXiv preprint arXiv:2103.09762 (2021).
[2] Bakman, Yavuz Faruk, et al. "Federated orthogonal training: Mitigating global catastrophic forgetting in continual federated learning." International Conference on Learning Representations. Vol. 2024. 2024.
[3] Tran, Minh-Tuan, et al. "Text-enhanced data-free approach for federated class-incremental learning." 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2024.
[4] Zhang, Jie, et al. "Target: Federated class-continual learning via exemplar-free distillation." 2023 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2023.
Response to Reviewer 9FZn (part 1)
Dear Reviewer 9FZn,
We would like to thank the reviewer for taking the time to review our paper and provide valuable feedback. We conduct extensive experiments and address your concerns point by point below. We also updated the manuscript with changes highlighted in blue.
Comment 1:
Response:
We thank the reviewer for raising this important question. To examine the effect of temporal misalignment, we follow your suggestion and compare FedAvg under synchronized task progression (Sync) and workload-induced temporal heterogeneity (TH) at α=0.5. As shown below, TH consistently yields lower accuracy and higher forgetting across the four benchmarks. These results provide further evidence that heterogeneous task progression can exacerbate forgetting in FCL.
Table 1. Effect of temporal misalignment on FedAvg (α=0.5). Acc: accuracy (%, ↑); F: forgetting (%, ↓). The Δ row shows the relative accuracy decrease under TH (single run, seed 0).
We further extend this experiment by varying the Dirichlet concentration α to examine FedAvg under different levels of temporal heterogeneity. As shown in the below, stronger temporal heterogeneity generally reduces accuracy.
Table 2. Effect of workload heterogeneity on FedAvg. Smaller α induces greater imbalance in client task workloads. Acc: accuracy (%, ↑); F: forgetting (%, ↓). Single run, seed 0.