- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates Federated Continual Learning (FCL) and identifies a limitation of existing prompt-based FCL methods: they select and aggregate historical prompts as a whole, overlooking the fact that different directions may play distinct roles, with some capturing shared knowledge and others introducing interference during knowledge integration. To address this issue, the authors propose SDPrompt, which decomposes historical prompts from a spectral perspective to selectively preserve useful knowledge while suppressing interference, thereby improving federated continual learning performance. Experiments conducted on ImageNet-R, DomainNet, Cars196, and CIFAR-100 demonstrate that the proposed method outperforms the compared baselines.
Critical strengths
-
This paper identifies a specific limitation of existing prompt-based federated continual learning methods: historical prompts are typically selected and aggregated as a whole, overlooking the fact that different structural directions may serve distinct roles. Some directions capture knowledge shared across tasks, while others may interfere with the integration of current knowledge. This is a concrete issue in FCL, and addressing it could significantly improve the reuse of historical knowledge across tasks and clients.
-
The proposed SDPrompt method is novel. It leverages historical prompt knowledge from a spectral perspective. Specifically, the server extracts a dominant spectral subspace to preserve knowledge, while each client applies soft correction to weak spectral components and extracts a task-conditioned subspace. This decomposition goes beyond a simple combination of existing techniques and may inspire future research on spectral- or subspace-based prompt aggregation in federated continual learning.
Critical weaknesses
- The performance improvements on some datasets are relatively small. However, the paper does not explicitly report whether repeated experiments were conducted using multiple random seeds, making it difficult to assess the stability of these improvements. I suggest that the authors conduct additional experiments with multiple random seeds and report the mean and standard deviation to strengthen the reliability of the experimental results.
- Although the authors compare the overall performance across four datasets, the ablation studies of the three core modules (G/T/D) are conducted only on ImageNet-R. I suggest extending the ablation studies to the other datasets to provide a more comprehensive evaluation.
- This paper improves federated continual learning performance by introducing spectral decomposition, spectral knowledge correction, and dual-subspace constraints. However, it lacks a comparison of computational costs and other overheads with existing methods, making it difficult to comprehensively evaluate the trade-off between performance gains and additional resource consumption. I suggest that the authors provide an analysis of these overheads to further demonstrate the practicality of the proposed method.
Questions for authors
- The performance improvements on some datasets are relatively small. Have the authors conducted independent repeated experiments using multiple random seeds? If so, could the authors provide the corresponding mean and standard deviation to further demonstrate the stability of these improvements?
- The ablation studies of the G/T/D modules are conducted only on ImageNet-R. Could the authors clarify whether each of these three modules also provides performance improvements on DomainNet, Cars196, and CIFAR-100?
- SDPrompt introduces additional spectral decomposition, spectral knowledge correction, and dual-subspace constraints. How much additional computational and communication overhead do these operations introduce? Compared with baseline methods such as C2Prompt, how do the actual training time and resource consumption change? Flag For Ethics Review