- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes PFEDREP, a method to improve the security of federated prompt learning when some clients may upload malicious prompts. It combines prompt screening, client feedback, reputation scores, diverse expert selection, and limits on each expert's influence. The authors also provide theoretical analysis and evaluate the method on several datasets and under different attack settings.
Critical strengths
- This paper addresses an important security problem in federated prompt learning. The proposed method can improve security without sharing raw client data.
- Selecting diverse experts and limiting their influence are useful ideas. They can reduce the risk of one malicious prompt having too much influence on the final prediction.
- The author/authors discuss important limitations and assumptions, which helps readers understand when the proposed method may work well.
Critical weaknesses
- The evaluation does not clearly show how well the proposed method works against adaptive attackers that change their behavior to avoid detection.
- Malicious prompts may still enter the retrieved expert pool. More experiments are needed to check whether the defense remains effective when attackers have more influence or work together.
- Some theoretical guarantees depend on fixed retrieval and ordering conditions. It is unclear whether these guarantees still hold when the expert pool changes during operation.
- The proposed method assumes that the server is honest and client identities are authenticated. These assumptions may not always hold in real-world federated learning systems.
Questions for authors
- Can the author/authors evaluate PFEDREP against stronger adaptive and colluding attackers to check whether it remains effective when malicious clients change their prompts or coordinate their attacks?
- How well does the proposed method scale when the number of clients and candidate experts increases significantly?
- Since malicious experts may still be retrieved during inference, can the authors provide more experiments to measure their impact under different attack settings and show which defense components are most effective in limiting their influence?
- How sensitive is PFEDREP to its screening thresholds and reputation parameters when the proportion of malicious clients increases?
- Can the author/authors evaluate the computational and communication overhead of PFEDREP compared with standard federated prompt learning, especially as the number of clients increases?
- Can you clarify the assumptions behind the theoretical guarantees and explain whether they still hold when the retrieved experts or their ordering change dynamically during inference?
8 comments
Response to Question 1
Response. We completed 48 runs comparing full pFedREP with a governance-free control under adaptive and colluding attacks. The eight settings combine 20%/40% malicious clients, proximity radii 0.1/1.0, and independent/colluding attackers, with three seeds per policy. All runs use Food101/RN50, ten clients, ten rounds, five local epochs, full participation, and . Each full/control pair uses the same honest evaluation cohort.
For each malicious update, the attacker refines its prompt using
\mathcal L_{\rm clean}(p,y),\qquad \|p-c\|_2\leq r,$$ where $c$ is the pool median, fixed during that refinement. The trigger term promotes the target label, while the equally weighted clean term encourages preservation of clean performance. We use 20 gradient steps at step size 0.01, projecting onto the radius-$r$ ball after each step; the event audit confirms completion of all requested steps. Radius 1.0 relaxes the proximity constraint relative to 0.1. Optimization uses the attacker’s own training data and fixed current mixture as a recipient surrogate, not honest clients’ data or gradients through future retrieval and training. In the colluding setting, attackers initialize from the mean of available malicious prompts and coordinate feedback: they release the maximum permitted positive rewards for malicious experts and negative rewards for honest experts. This tests coordinated prompt initialization and report manipulation under the existing authenticated identities; it does not introduce additional identities. Table 3.2: All adaptive/colluding Food101/RN50 comparisons (48 runs). F/U denote full/governance-free policies. Values are percentages, mean $\pm$ sample SD over three seeds. Each F/U pair uses the same honest cohort; the 20% and 40% contamination blocks use different honest cohorts. | Malicious % | $r$ | Collude | CA (F) | CA (U) | ASR (F) | ASR (U) | |:---|:--:|:--:|:--:|:--:|:--:|:--:| | 20 | 0.1 | No | $93.36\pm0.07$ | $93.33\pm0.10$ | $0.062\pm0.018$ | $0.066\pm0.011$ | | 20 | 0.1 | Yes | $93.27\pm0.17$ | $93.31\pm0.09$ | $0.068\pm0.017$ | $0.060\pm0.009$ | | 20 | 1.0 | No | $93.30\pm0.18$ | $93.20\pm0.18$ | $0.055\pm0.015$ | $0.076\pm0.022$ | | 20 | 1.0 | Yes | $93.28\pm0.23$ | $93.37\pm0.14$ | $0.060\pm0.013$ | $0.070\pm0.009$ | | 40 | 0.1 | No | $94.36\pm0.25$ | $94.28\pm0.14$ | $0.105\pm0.031$ | $0.073\pm0.015$ | | 40 | 0.1 | Yes | $94.32\pm0.22$ | $94.24\pm0.32$ | $0.092\pm0.040$ | $0.105\pm0.040$ | | 40 | 1.0 | No | $94.33\pm0.27$ | $94.32\pm0.16$ | $0.121\pm0.043$ | $0.124\pm0.026$ | | 40 | 1.0 | Yes | $94.37\pm0.23$ | $94.40\pm0.21$ | $0.104\pm0.024$ | $0.111\pm0.009$ | Table 3.2 shows that both policies retain high clean accuracy, with no consistent ASR advantage for full pFedREP. At 20% contamination, its ASR ranges from $0.055\%$ to $0.068\%$, versus $0.060\%$ to $0.076\%$ without governance; at 40%, the respective ranges are $0.092$–$0.121\%$ and $0.073$–$0.124\%$. These results support accuracy preservation under the specified surrogate attacks, not a general claim that governance defeats adaptive or colluding adversaries. The 20% and 40% settings evaluate different honest cohorts, so higher CA in the latter cannot be interpreted as an improvement caused by more attackers.Response to Question 2
Response. We measured scaling at 20, 50, and 100 clients on CIFAR10/RN50, comparing full pFedREP with a governance-free control at each client count. Each run uses ten rounds, 10% participation, five local epochs, and ten total expert slots. Three seeds per policy give 18 runs, all of which passed the GPU-isolation audit on an NVIDIA L20. This is a short scaling diagnostic, not the paper’s 120-round CIFAR accuracy benchmark. Table 3.3 reports the measured costs.
Table 3.3: CIFAR10/RN50 scaling: ten rounds, 10% participation, isolated L20 GPU per run. CA is final-round percentage; times are total recorded seconds divided by ten rounds. Peak is allocated GPU memory in GiB, not host memory. Means sample SD over three seeds.
| Clients | Policy | CA | Train s/round | Round s | Select s/round | Peak GiB |
|---|---|---|---|---|---|---|
| 20 | Full | |||||
| 20 | Governance-free | Not logged | ||||
| 50 | Full | |||||
| 50 | Governance-free | Not logged | ||||
| 100 | Full | |||||
| 100 | Governance-free | Not logged |
The added overhead increases with client count. Mean paired round-time overhead relative to the control is , , and at 20, 50, and 100 clients, respectively. At 100 clients, full pFedREP takes seconds per round versus for the control. Its measured selection time rises from to seconds per round between 20 and 100 clients. Because participation is fixed at 10%, the number of clients trained per round also increases; total runtime growth should therefore not be attributed entirely to pool management. The matched policy comparison at each client count measures the additional cost of governance.
Peak allocated GPU memory stays near GiB for both policies, but this does not measure the server’s host-memory cost. For available prompts of dimension , pairwise geometry requires work, and the current broadcast implementation can allocate an temporary. Directed recipient–expert history can require storage for clients. Moreover, with partial participation, the available pool can contain fewer than prompts early in training. The results therefore demonstrate execution through 100 clients under this protocol, with increasing governance overhead; they do not establish linear scaling or feasibility at thousands of clients.
Response to Weakness 3 and 4
Response to Weakness 3. Theorem 1 can be applied to a fixed prediction state at any round, but it does not certify the transition between different states. It does not require the expert pool to remain unchanged throughout training. Instead, it compares two predictions for the same image after fixing the retrieved identities, their slot ordering, and the router parameters, with only one non-local slot’s representation replaced and the local branch unchanged.
Under this comparison, prompt-independent routing keeps the slot weight ai,j(x) unchanged. With the theorem’s norm bounds, replacing that representation changes each class logit by at most 2sai,j(x)≤2sαˉ. This is the paper’s conditional influence guarantee: it bounds reliance on one retained expert, regardless of how the system arrived at that fixed state.
If a pool update changes retrieval, ordering, or trained model parameters, the resulting prediction is not the single-slot comparison covered by the theorem. An earlier margin certificate therefore cannot be carried over automatically. At the updated state, the bound and margin condition must be evaluated again using the current retrieved set, router, effective cap, and logits. The guarantee remains conditional at each such evaluation; it does not establish stability across pool updates or end-to-end robustness to a poisoned training trajectory.
Response to Weakness 4. We agree that these assumptions limit applicability. pFedREP studies malicious clients within a federation that has trusted coordination and controlled participation; it does not establish those conditions itself. The paper explicitly assumes an honest server, authenticated client identities, and an uncompromised frozen backbone. Its robustness claims should therefore not be extended to deployments where the coordinator is compromised or adversaries can create unrestricted identities.
The honest-server assumption is needed because the server maintains the expert pool and feedback records and executes screening and retrieval. A malicious coordinator could alter those records or bypass the prescribed selection procedure. pFedREP does not include a mechanism for verifying a dishonest server’s execution, so protection against server compromise is outside the stated threat model.
Authenticated identities serve a different purpose: they let the system associate repeated reports with the same client. Distinct-reporter accounting prevents one identity from increasing the reporter count merely by submitting more reports; it does not establish that the reports are truthful or that different reporters act independently. Enrolled malicious clients can still collude, and Proposition 1 separately requires a bound on their fraction within each expert’s reporter set. If an attacker can create additional accepted identities, that premise need not hold. Thus identity enrollment/authentication and coordinator integrity must be supplied by the deployment. pFedREP governs prompt sharing under these assumptions; it does not provide Sybil resistance or remove the need to trust the coordinator.