- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
FedSCR adapts a frozen pretrained vision transformer through a shared prompt carrier and bounded low-rank residuals associated with client-task slots. Each residual uses a carrier norm captured after warmup, receives two adaptation exposures, and is then frozen under the ordinary stream. Owner-bound updates avoid dilution of slot-specific state. A separate anchor-recovery branch adjusts old residuals when a carrier-drift condition is met before a revisit. Experiments use slot-aware ImageNet-R and DomainNet protocols, with carrier-only and lifecycle ablations, a controlled drift-revisit test, and resource accounting.
Critical strengths
-
The paper is unusually explicit about its evaluation assumptions, including task-slot metadata, class masks, single-owner residual updates, and the distinction between tensor payload and measured network communication. This transparency makes the results easier to assess.
-
The shared-carrier design achieves competitive accuracy with relatively compact trainable state. The carrier-only control, paired differences, and per-seed results provide useful evidence about which parts of the design account for performance.
Critical weaknesses
- The reported evidence does not establish a consequential benefit from the bounded residual mechanism that distinguishes the method. In Table 2, full FedSCR reaches 86.874% AIA versus 86.784% for the shared carrier alone. Appendix C, Table 10 gives the paired improvement as only 0.091 percentage points, with a 95% paired interval of [-0.236, +0.417]. The corresponding BT values in Table 2 are almost identical (1.358 versus 1.357). The reported intervals for removing warmup, using an unbounded rank-one residual, and diluting owner updates also cross zero. These results do not prove that the mechanisms are useless, but they leave their practical advantage unresolved.
- The ordinary main protocols follow the no-trigger path, so they do not demonstrate the usefulness of the anchor-recovery mechanism during natural learning. The positive recovery result instead comes from a prescribed 10% earlier-task-gradient-aligned carrier perturbation, followed by direct application of recovery (Section 4.4 and Appendix C.3). That test does not establish whether the calibrated trigger detects harmful drift, avoids unnecessary interventions, or improves retention over a naturally evolving long stream. Every nonempty residual update group has exactly one owner, and all five clients participate synchronously for only 15 rounds. The multi-contributor factor-aggregation extension is explicitly unevaluated. Consequently, the empirical evidence does not yet establish robustness to unknown task identity, realistic participation, or prolonged task accumulation.
Questions for authors
-
Under matched seeds, rank, tuning budget, and optimization, what is the gain of bounded rank-two residuals over both a carrier-only model and unbounded rank-two residuals? Is the advantage reproduced on DomainNet?
-
Can you demonstrate the complete trigger-and-recovery pipeline on naturally induced carrier drift, reporting trigger frequency, false interventions, earlier-task accuracy, and comparison with always-on recovery? The current constructed perturbation test does not evaluate the trigger itself.
-
How much of the improvement remains without oracle client-task output-block selection, or under a substantially longer stream with partial participation? Please distinguish evidence for the current slot-aware setting from claims about broader FCL deployment.
2 comments
- Removing oracle output-block selection and deployment scope
We evaluated inference without sample client/task/output-block IDs, keeping the trained models unchanged. Image-only readout uses all seen prompt rows and trained output blocks, merging repeated physical classes by maximum logit. ImageNet-R stream AIA is:
| Readout | FedSCR (%) | POWDER (%) |
|---|---|---|
| Slot-aware | 86.751 | 85.793 |
| Raw image-only | 73.918 | 75.333 |
Expanding the candidate outputs changes both absolute accuracy and the method comparison: FedSCR's AIA difference from POWDER changes from +0.957 pp under slot-aware readout to -1.415 pp under raw image-only readout. To understand this change, we examined errors at frozen endpoint checkpoints. Even when the correct task head predicts the right class, a higher score for a wrong class from another head can change the final prediction when all blocks compete.
We therefore evaluate an optional calibration that learns a temperature and offset for each task head before merging scores by physical class. This preserves class rankings within each head while adjusting relative scores across heads, directly targeting the competition identified above. The backbone, prompts, residuals, and classifier remain frozen, and all three methods receive the same calibration procedure. The separate endpoint FAA results are:
| Method | Raw image-only FAA (%) | + Calibration (%) |
|---|---|---|
| FedSCR | 72.029 | 78.056 |
| Carrier only | 72.190 | 77.795 |
| POWDER | 73.566 | 77.993 |
Calibration raises FedSCR's endpoint FAA by 6.027 pp with representations fixed, showing the value of aligning scores across heads. All three methods benefit, supporting comparison against identically calibrated baselines. They use the same training-only data split and calibration budget; training-side validation selects the calibration, and test data are excluded from fitting and selection. This optional readout extension requires historical training images. The calibrated readout supports inference without output-block IDs under the original synchronous, full-participation training protocol.
Thank you for recognizing the shared-carrier results and explicit evaluation setting. We address the residual controls, natural recovery behavior and inference without oracle output-block selection below.
To assess the contribution of task residuals and their amplitude bound, we compare bounded rank-two residuals with a carrier-only model and a same-rank unbounded control. The unbounded control removes radial saturation while matching rank, local gain, initialization, data order, optimization, and lifecycle. Both residual designs use the same fixed training configuration without separate retuning. On ImageNet-R:
Paired AIA differences are +0.186 pp over unbounded (95% paired-t interval [-0.187, +0.558]) and +0.091 pp over carrier only [-0.236, +0.417]. The corresponding warmup and owner-update ablation differences are +0.083 pp [-0.185, +0.352] and +0.154 pp [-0.064, +0.372]. The close AIA values indicate that the shared learning path supplies much of the observed performance, while the residual adds an explicit budget for task-specific state. Freezing preserves the learned task offset, warmup fixes its scale reference, and ownership governs its writes during adaptation. These roles distinguish the state-management design from its incremental accuracy contribution in this comparison.
A separate paired DomainNet control study gives the following means ± sample SD:
The paired AIA differences are -0.065/-0.060 pp over unbounded/carrier only. The shared carrier attains similar AIA. Residual magnitudes are normalized by the carrier norm captured after each task's warmup. Their logged maxima are 0.094-0.095 with bounding and 2.471-2.557 without saturation. The unbounded model therefore uses substantially larger task offsets while attaining similar AIA, making the bound's control of task-state magnitude directly observable.
In a 36-round ImageNet-R stream revisiting five fixed tasks under ordinary synchronous, full-participation updates, AUTO triggered 0 times across 18 eligible rounds; AUTO and OFF both reached 86.822% earlier-task accuracy. Always-on recovery executed twice and reached 86.635%. Forced recovery from identical OFF checkpoints changed earlier-task accuracy by -0.020/+0.032 pp.
AUTO followed the same trajectory as OFF because the two-consecutive-exceedance condition was never met. ALWAYS measures the cumulative effect of intervening at both revisit boundaries, while forced recovery from identical OFF checkpoints isolates the immediate effect of each correction. The small immediate changes show why the intervention decision matters alongside the correction rule. The trigger uses carrier movement with a threshold fixed from a separate calibration run; test accuracy is used to evaluate outcomes. Appendix C.6 gives the full comparison.
The separate automatic-path test under amplified federated updates produced a -0.018 pp final earlier-task difference. The paper's +1.328 pp result directly invokes recovery after a prescribed gradient-aligned perturbation. Old-task gradients construct that perturbation; recovery uses saved prompt targets. This controlled result demonstrates correction through old-task residual factors while the shared carrier remains fixed. The natural stream evaluates when the automatic policy intervenes under ordinary updates.