FedNASC: Noise-Aware Semantic Control for Federated LoRA Adaptation

Federated LoRA adaptation of pretrained Vision Transformers must distinguish genuine cross-client semantic variation from finite-sample prototype noise. This is difficult when local class support is small, because unstable prototypes can mimic genuine within-class domain modes. We propose FedNASC, a noise-aware semantic controller that explicitly estimates this instability from split-half prototype disagreement. Each client augments its class prototype with a split-half instability statistic, and the server forms a matched class-wise noise floor, subtracts it from observed dispersion, and maps the calibrated excess to a score controlling prototype-bank capacity, classifier anchoring, semantic regularization, and prototype-readout fusion. Under a fixed-map perturbation model, we show that the untruncated calibrated difference estimates population representation dispersion in expectation up to controlled noise-dependent terms. Population dispersion in turn determines the weighted Euclidean single-prototype consensus error and bounds its unit-norm counterpart. We further derive a stationarity bound for the implemented optimization and server pipeline with explicit implementation residuals. Across CIFAR-100, HAM10000, DomainNet, and NICO++, FedNASC outperforms the strongest of six federated adaptation baselines by 1.4-2.4 percentage points across all ten settings. Ablations show that noise calibration contributes most under label skew, whereas adaptive bank capacity contributes most under domain shift, consistent with the intended roles of the two mechanisms.