- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
The paper proposes FedOPD (Federated On-Policy Distillation). The setting is the following: a central server holds a (larger) student LLM while clients run (smaller) LLM. Clients hold data in specialized domains. The main idea is that clients evaluate the server’s own generated responses rather than generating output themselves or rather than directly disclosing model parameters. The underlying assumption is that the server already posses domain-specific (unlabeled) data, required to perform the distillation. FedOSD works in two stages: first local experts (clients’ LLM) are trained on private data, then frozen; second, the local experts evaluate the server-generated trajectories and return token-level log-probabilities, and the server distills knowledge from this feedback.
Critical strengths
- Making clients passive entities that evaluate the outputs generated by the server-held LLM is an interesting idea, and the proposed distillation protocol may have merit.
Critical weaknesses
- The paper makes an implicit assumption: the server already possesses domain-relevant, unlabeled prompts. This assumption is not clearly stated or clearly discussed in the paper. So the server’s distillation prompts are taken from the same underlying public dataset used to train clients’ expert. This means that distribution mismatch between public distillation prompts and private training data is limited by construction.
- Implementation detail of considered baselines are scarce and the paper does not seem to provide reproducible code.
Questions for authors
Q1. How does the server obtain representative prompts without accessing the private client data?
Q2. How does the method perform when the server does not know the clients’ training inputs and is forced to prompt from a different distribution?
Q3. How many public prompts are needed to achieve the reported results, and what is their ratio relative to the clients' private training data? How does FedOPD perform when the server has access to a limited number of prompts or when prompt availability is highly unbalanced across domains?
3 comments
Dear Reviewer N5ZG,
We are writing to kindly ask whether our rebuttal has addressed your concerns. We would be happy to discuss any remaining issues.
Rebuttal Assessment
W1/Q1 – Server prompts. The assumption is clarified, and privacy-filtering experiments are appreciated, although obtaining representative prompts remains a practical challenge. I consider this partially addressed.
Q2 – Distribution mismatch. The new experiments demonstrate robustness under moderate mismatch and acknowledge limitations under extreme mismatch. I consider this addressed.
Q3 – Prompt budget. The additional experiments are relevant, although the minimum prompt budget remains substantial and improvements over baselines become marginal. I consider this mainly addressed for what I asked.
W2 – Reproducibility: Implementation details are provided but (as far as I can see) reproducible code is not yet available. Reproducibility cannot be verified.
Overall, I appreciate the authors' efforts and the targeted experiments, which substantially mitigate my concerns. Some limitations remain, particularly regarding generalizability beyond the 3B fine-grained math setting for which most of the additional results are provided. Nevertheless, I have updated my score to weak accept in recognition of the improvements.
Addressing the server-prompt assumption: privacy-filtered, mismatched, and smaller prompt pools
We thank the reviewer for the precise questions. All new results below will be added to either the main text or the appendix of the final version.
W1 & Q1. The prompt assumption and how to obtain prompts privately.
This assumption is fairly standard: federated distillation with an unlabeled transfer set (FedMD, Li & Wang, 2019; FedDF, Lin et al., 2020) and FedGRPO (Zhu et al., 2026) make the same one. It is natural when the value lies in answers and expertise, not in questions, e.g., public question banks whose solutions and verifiers are proprietary, or domain queries a provider already receives. The submission already states it (Sec. 3.1, line 146; App. B.1, line 724). We will make it an explicit assumption in Sec. 3.1.
In particular, we further examine the case where the problem prompts themselves are treated as private and must be protected. Representative prompts can then be obtained in two ways. First, clients can release problem text after privacy filtering, as shown in Table R1. Second, the server can use a pool independent of client data (see our response to Q2–Q3). For the first route, we either remove the problems that contain personally identifiable information (PII) or mask only the PII spans (to be added to Appendix C):
Table R1: FedOPD with privacy-filtered server prompts (fine-grained math, 3B).
Removing all PII-bearing problems costs only 0.12 points. Masking costs 0.82, because placeholders make some problems ambiguous (e.g., "[NAME] has 3 more than [NAME]").
References
Q2. Mismatched prompt distribution. We agree that matched prompts limit mismatch by construction, so we test pools of 6,000 prompts drawn independently of the client data (to be added to Appendix C). Domain proportions are sampled from a Dirichlet distribution Dir(β). A smaller β means stronger imbalance; at β=0.1, about 90.9% of the pool consists of unrelated everyday prompts.
Table R2: FedOPD with server prompt pools that are mismatched to the client data (fine-grained math, 3B).
Under moderate mismatch (β≥1), FedOPD stays above every federated baseline that uses matched prompts. Under extreme mismatch, the gain shrinks to +0.78 over the base, because the server rarely asks about the clients' domains. We report this as a limitation.
Q3. Prompt budget and imbalance. The main runs use 2,000 prompts per domain, 6,000 in total, which is 1.0× the clients' 6,000 private problems. We vary the pool size under unbalanced domains (β=1):
Table R3: FedOPD with different server prompt-pool sizes under unbalanced domains (β=1; fine-grained math, 3B).
With half the prompts and unbalanced domains, FedOPD (51.48) still exceeds the best federated baseline (51.34). Doubling the pool offsets the imbalance (52.67 vs. 52.51 matched), mainly because the under-represented domains then have more distinct prompts.
W2. Baselines and code. Appendix. A.2 already describes each baseline and its upload. All methods share the student initialization, prompts, client experts, domain order, and step budget (hyperparameters in Appendix. A.1, experts in Appendix. B.3). Appendix A will add the shared learning-rate schedule and a per-baseline hyperparameter table. We will open-source the full code with the camera-ready version.