Shared Prompt Carriers with Bounded Task Residuals for Federated Continual Learning
Abstract
Federated continual learning (FCL) requires clients to learn from evolving and heterogeneous tasks while transferring knowledge without sharing data. Prompt-based adaptation is parameter-efficient, but fully shared prompts can cause task interference, whereas isolated prompts limit reuse of common structure. We introduce FedSCR, a prompt parameterization built on a shared carrier with bounded low-rank task residuals. The shared carrier remains trainable across clients, while each task retains a compact offset whose norm is bounded relative to the carrier norm captured at warmup. Experiments on ImageNet-R and DomainNet show competitive accuracy and transfer compared with representative federated continual learners under heterogeneous task arrivals. Analyses indicate that most of the accuracy observed in our experiments is already present in the shared carrier, while residuals provide compact task state for controlled retention. FedSCR offers a simple way to balance cross-client knowledge transfer and controlled task retention in FCL.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.