- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes TriVA (Tri-State Value Adaptation) for parameter-efficient federated adaptation of CLIP-style vision-language models. TriVA updates only native value-projection biases in the visual and textual encoders. Its motivation is that, for fixed attention-layer inputs, value-bias updates introduce additive corrections that are preserved exactly under parameter averaging. To accommodate heterogeneous clients, a one-shot calibration procedure assigns attention-head bias groups to three persistent states: Frozen, Shared, or Private. Assignments depend on the estimated utility of local adaptation and the loss incurred by replacing local parameters with their globally averaged counterparts. Only Shared groups participate in subsequent communication. Experiments cover few-shot classification, domain heterogeneity, larger client populations, and a CamVid segmentation extension. The paper reports competitive accuracy with a small adaptation interface and low steady-state communication costs.
Critical strengths
(1) The paper correctly separates whether parameter averaging reproduces functional averaging from whether sharing is beneficial for heterogeneous clients. This distinction provides a coherent motivation for combining the adaptation interface with selective synchronization.
(2) Equation (1) establishes an exact additive property for fixed layer inputs under the stated attention assumptions. The manuscript appropriately acknowledges that this property does not imply end-to-end linearity when adapting multiple layers and encoders.
(3) Updating existing value biases avoids inserting prompts or adapters. This offers a practical deployment advantage, particularly when model architecture and inference implementation should remain unchanged.
(4) The Frozen/Shared/Private assignments distinguish adaptation utility from synchronization suitability. Evaluating replacements on local probes is more interpretable than assigning roles solely through parameter magnitude or predetermined layer partitions.
(5) Experiments include label-distribution heterogeneity, domain shifts, different supervision budgets, and 100-client settings. The paper also acknowledges settings where TriVA does not achieve the best accuracy, including CIFAR-100 and some synchronization ablations.
Critical weaknesses
(1) Figure 1 provides precise merge-error values for prompts, adapters, and value biases, but the manuscript does not adequately specify the diagnostic dataset, client configurations, optimization settings, adaptation budgets, or variability underlying these measurements. The normalized metric also does not fully eliminate differences in adaptation magnitude. Under a locally smooth parameter-to-function mapping, smaller updates can yield smaller normalized nonlinear aggregation errors. Comparisons should therefore control for functional displacement or local adaptation quality, rather than relying on the reported normalized scores alone.
(2) On Stanford Cars, removing Private parameters improves accuracy from 79.34% to 79.50%, while removing Shared parameters yields 79.07% with no steady-state communication. The complete method offers a possible accuracy–communication compromise, but these results do not establish that all three states are consistently necessary. Broader comparisons against local-only value-bias tuning, binary Frozen/Private and Shared/Private assignments, random assignments with matched state proportions, and budget-matched alternatives are needed.
(3) Roles are determined using three probe batches after initial calibration and remain fixed throughout training. Their reliability is especially uncertain under very limited local supervision. The paper does not report sensitivity to probe sampling, calibration duration, the active-ratio cap, or the stability of role assignments across seeds. There is also a mismatch between calibration and subsequent aggregation: calibration evaluates replacement by an average over all clients, whereas later updates average only clients that mark a group Shared. The initial synchronization-cost estimate therefore evaluates a different composition from the one used during training.
(4) Proposition 3.1 follows from the Lipschitz continuity of cross-entropy and a decomposition involving the functional-average risk. It clarifies two sources of synchronization error, but does not establish that value-bias adaptation minimizes either term or that TriVA improves population risk. The joint-assignment analysis bounds cross-group interactions through mixed Hessian terms, but these quantities are not estimated empirically. Consequently, the sufficient condition does not establish that the actual assignments are safe, particularly when useful groups are frozen by the active-ratio cap.
(5) Important details remain unclear, including learning rates, optimizer settings, client participation, probe construction, repeated runs, and the CamVid adaptation pipeline and partitioning protocol. These omissions matter because several reported margins are small: on DomainNet-100, TriVA exceeds pFedMMA by only 0.06 % . Communication figures explicitly measure steady-state parameter payloads, which is reasonable, but they exclude the initial all-group calibration exchange and other setup costs. Calibration also requires repeated counterfactual evaluations across bias groups. Total communication, calibration runtime, and training memory should be reported alongside steady-state costs.
Questions for authors
(1) What exact protocol produces the aggregation-fidelity results in Figure 1? Please report diagnostic data, optimization settings, update magnitudes, local performance, and variability.
(2) Does the fidelity advantage persist when interfaces are matched by functional displacement, local accuracy, or trainable parameter count?
(3) How does TriVA compare with local-only value-bias adaptation and random or binary role assignments across all benchmarks?
(4) How stable are the assignments across probe samples and random seeds? How sensitive are results to calibration duration and the active-ratio cap?
(5) Why should a synchronization probe using the all-client average predict the benefit of later aggregation over only Shared clients?
(6) Can the authors measure cross-group interaction effects and the actual joint loss change induced by calibration assignments?
(7) What are the complete communication and computational costs, including calibration? Please also specify the segmentation pipeline, client participation protocol, and independent-run statistics.
Clarifications on Aggregation Fidelity, Tri-State Ablations, Cross-Group Interactions, and Calibration Reliability
W1/Q1/Q2: Aggregation fidelity and matched adaptation comparison
Figure 1 uses Stanford Cars with 10 clients, 16 shots per class, and 8,041 test images. All interfaces use SGD (LR 0.004, momentum 0.9, weight decay 0.0005), five local epochs, and data-weighted FedAvg. Each round uses the global model before local training as the reference. We report medians over 50 client-round measurements from rounds 46–50, with interquartile ranges (IQR).
We further compare four interfaces under matched local CE improvement using 1,635 separate validation images.
Value-bias achieves the lowest normalized merge error in both comparisons, supporting its aggregation-fidelity advantage under comparable local adaptation quality.
W2/Q3: Tri-state ablation comparisons
We extended the tri-state ablations to all five single-domain benchmarks, comparing TriVA with local-only adaptation, binary role assignments, and matched-budget random allocation.
TriVA achieves the highest accuracy on four of the five benchmarks and consistently outperforms both binary role assignments, supporting the effectiveness of counterfactual tri-state synchronization.
W3/Q4/Q5: Calibration reliability and synchronization consistency
We evaluated calibration sensitivity to probe size, sampling seeds, calibration duration, and the active-ratio cap on Stanford Cars.
Role agreement ranges from 50.33% to 55.21% across probe sizes and from 50.6% to 56.3% across sampling seeds. Although individual assignments vary, accuracy remains relatively stable under the tested probe configurations.
To examine the difference between all-client calibration and subsequent Shared-only aggregation, we recomputed replacement costs using actual selectively aggregated parameters. At rounds 10, 25, and 50, 62.05%–68.41% of initially Shared groups retain non-positive replacement costs. Selected-client recalibration achieves 79.38% accuracy, compared with 79.34% for the original one-shot calibration.
One-shot calibration achieves comparable accuracy to selected-client recalibration without additional overhead.
W4/Q6: Joint assignment and cross-group interactions
Proposition 3.1 characterizes synchronization risk rather than guaranteeing non-increasing joint loss. We empirically evaluated cross-group interactions and actual joint calibration loss changes on SUN397.
For the five clients affected by the active-ratio cap, restoring 165 frozen groups with positive adaptation utility reduces their mean joint ΔCE from +0.00659 to just +0.00013.
This result supports the utility of TriVA's adaptation-utility estimates in identifying beneficial parameter groups.
W5/Q7: Reproducibility and total overhead
Cars: 10/10 clients participate; probes sample 96 training images/client without replacement. Profiling: 18.57 min calibration (15.45 min probing), 180.27 min total, 5.03 GiB peak GPU memory. Separate run: 10.83 MB bidirectional traffic (1.53 MB calibration; 9.30 MB later), including metadata.
CamVid: 367/101/233 train/val/test images, 11 classes, 10 clients (Dirichlet β=0.5 by dominant non-void class); CLIP ViT-B/16 + shared convolutional decoder, 224×224 inputs, SGD LR 0.004, 50 rounds × 5 epochs, batches 8/16, pixel CE ignoring void; last-10 mIoU.
Appendix Table 8 reports three-seed mean±SD for five single-domain datasets. Due to compute limits, DomainNet-100 uses one seed; three-seed results will be included in the final version.