- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates the dynamic collaboration between a local SLM and a cloud LLM to balance privacy, efficiency, and capability. The local SLM receives the full user input and learns when to consult the LLM and how to construct requests, while the cloud LLM can either answer or request clarification. Modeling them as heterogeneous agents under information asymmetry, the local SLM is trained via VAPO with rewards covering answer quality, privacy, interaction efficiency, feedback, and format constraints. Evaluation results reveal a distinct scaling effect where stronger SLMs become more self-reliant and stronger LLMs enable fewer but more informative interactions. The learned collaboration strategies significantly outperform baselines and transfer robustly to unseen LLMs and tasks without additional training.
Critical strengths
(+) The paper addresses a concrete and important collaboration problem. A local model needs to use cloud capabilities while limiting unnecessary information disclosure and interaction. Incorporating assistance-seeking decisions and request formulation into a trainable policy has clear practical value.
(+) The experimental evaluation extends beyond a single model and the training distribution.
Critical weaknesses
(-) The evaluator-noise conclusion is overstated. Appendix H only establishes that the privacy-reward component is scaled. This does not guarantee that the overall optimization direction or optimal policy remains unchanged.
(-) The privacy guarantees lack sufficient support. Failure to detect private items does not imply that the attacker gains no inferential advantage. Overall evaluator accuracy also does not directly bound the conditional false-negative rate (Appendices F–G).
(-) Trust and deployment conditions need clarification. The trust boundaries of training and the privacy evaluator remain insufficiently specified, as do the conditions and advantages of this approach relative to trusted LLM services or alternatives such as STIP.
(-) Resource evidence for long contexts is insufficient. RTX 4090 latency tests are provided, but measurements of local generation length and peak GPU memory/KV-cache usage under long contexts, as well as latency for long reasoning trajectories, are missing.
(-) Initialization and cold start are insufficiently specified. Whether the main experiments use SFT and which parameter-update method they employ remain unclear. The paper also does not report whether a supervised cold start resolves the 0.6B model’s training failure.
(-) Reward and model-adaptation rules remain unclear. Despite weight sweeps and transfer experiments, the default reward configuration and the conditions requiring retuning or retraining after changes in tasks, devices, or models need clarification.
Questions for authors
Q1: Could you correct the conclusion about preserving the overall optimization direction?
Q2: Could you substantiate the privacy guarantees and provide the evaluator’s confusion matrix?
Q3: Which components must be trusted? Under what conditions is local collaboration preferable to directly using a trusted or protected large-model service?
Q4: How do peak memory usage and complete end-to-end latency change under long contexts and long reasoning trajectories?
Q5: What initialization and fine-tuning settings are used in the main experiments? Has cold starting weaker models been validated?
Q6: How are the default reward weights selected? Which task or deployment changes require retraining?
See weakness for more details.
4 comments
Response to Reviewer qMeN - Part (2/3)
Weakness 3 / Question 3: Local privacy protection and trust boundaries
A3. During deployment, our framework protects privacy at the source by controlling disclosure on the user’s local device. As described in Sections 3.1 and 4.3, the original query and private context are processed locally: the SLM decides whether to seek cloud assistance, formulates requests without user-specific private context, and combines the returned information with local private context to produce the final answer. The cloud sees only the submitted requests and their interaction history, with no direct access to the original query or local private context. Privacy protection is applied locally before requests reach the cloud, without relying on the cloud provider to filter private information. Our design is complementary to trusted LLM services or alternatives such as STIP: we limit what information on local device is shared, while they protect its processing in the cloud.
During training, we use examples constructed by injecting synthetic personal information into non-private, publicly available queries, as described in Section 4.3. The privacy evaluator receives the SLM’s requests and the corresponding annotated private items to compute the privacy penalty. This evaluation relies solely on synthetic private information in PrivQA and does not require access to real users’ private information at deployment.
Weakness 4 / Question 4: Resources along real collaboration trajectories
A4. We measure resource usage during inference on queries from PrivQA, PUPA, AESLC, and SAMSum, using trained Qwen3-4B with the hosted Qwen3-Max API. The local SLM runs with BF16 on NVIDIA 4090 GPU. We group trajectories by the maximum local context actually processed, including the input, accumulated local reasoning, requests, and cloud replies.
| Maximum processed context (tokens) | Local generated tokens | Peak memory (GiB) | Local latency (s) | End-to-end latency (s) | ||||
|---|---|---|---|---|---|---|---|---|
| Median | Max | GPU resident | Active KV | Median | Max | Median | Max | |
| <1,024 | 279 | 591 | 10.743 | 0.141 | 0.443 | 1.222 | 0.484 | 2.635 |
| 1,024–2,047 | 558 | 1,324 | 10.630 | 0.281 | 0.902 | 2.058 | 1.806 | 4.023 |
| 2,048-4095 | 1,354 | 2,333 | 10.743 | 0.514 | 2.195 | 5.854 | 4.395 | 10.183 |
| 4,096–8,192 | 2,862 | 7,852 | 10.743 | 1.125 | 4.216 | 11.843 | 7.032 | 16.084 |
Generated tokens include local reasoning, requests, and the final answer. GPU resident is the sampled peak GPU memory occupied by the local process, including a fixed 2.00 GiB reserved KV pool; active KV is the peak cache space allocated to active requests within that pool. Local latency measures SLM prefill and decoding time, while end-to-end latency is measured over the complete trajectory and includes hosted API generation, network round trips, and orchestration. Across the four context groups, median local latency increases from 0.443 to 4.216 seconds and peak active KV usage from 0.141 to 1.125 GiB.
Weakness 5 / Question 5: Initialization, parameter updates, and cold start
A5. Our main experiments use end-to-end RL directly from publicly released model checkpoints, without additional task-specific SFT. We update all SLM parameters while keeping the cloud LLM frozen and the SFT methods in Table 1 are separate baselines. The detailed training settings are provided in Section 4.4 and Appendix I.1.
Qwen3-0.6B uses the same end-to-end RL setup in Section 5.3, Figure 5, but reaches a low training reward with a decline in answer-quality reward. Its weak instruction-following ability makes well-formed requests rare during exploration, limiting useful cloud feedback and positive rewards. A potential remedy is a supervised cold start through trajectory distillation: a stronger model would generate successful collaboration trajectories, and the 0.6B model would learn through imitation how to formulate requests, respond to feedback, and integrate the returned information before RL. Our experiments focus on end-to-end RL and systematically analyzing how model capabilities, privacy constraints, and efficiency constraints shape the learned strategies. We leave trajectory distillation as a complementary direction for future work.
Response to Reviewer qMeN - Part (3/3)
Weakness 6 / Question 6: Default rewards and when adaptation is needed
A6. As introduced in Section 4.4, and control the privacy and efficiency penalties, respectively. We use as heuristic defaults. These weights control how strongly privacy leakage and unnecessary cloud calls are penalized during training. Figures 7 and 8 in Section 5.5 show how changing the privacy and efficiency weights affects the resulting trade-offs. We have explicitly documented the defaults and their selection rationale in Appendix I.1. Other default training settings, including learning rates and batch size, are also listed in Appendix I.1.
Whether retraining or retuning is needed depends on which part of the deployment changes.
| Change | Is retraining or reward retuning needed? | Evidence / rationale |
|---|---|---|
| Cloud LLM | No, for the demonstrated transfers within and across LLM families. | For Qwen3-8B, switching from Qwen3-235B to Qwen3-Max improves average EM from 43.11% to 46.01% without retraining in Section 5.4. Appendix I.6 also reports that the policy trained with DeepSeek-R1-Distill-Llama-70B transfers to Qwen3-235B-A22B-Instruct without retraining, increasing average EM from 37.80% to 40.10%. |
| Task | No, for the evaluated transfers. | Transfer to PUPA, AESLC, and SAMSum uses no additional training in Section 5.2. |
| Device only | No policy retraining or reward retuning, provided the model, precision, and context limits remain unchanged. | The learned policy remains unchanged; only hardware-dependent inference settings and resource measurements need updating. |
| Local SLM | Policy training or adaptation is needed for a new local model; reward retuning is not necessarily needed. | Each local model learns its own collaboration policy in Section 5.3. |
| Privacy or efficiency requirements | Only if no existing policy meets the new requirements. | Otherwise, reuse a suitable policy. If needed, adjust the corresponding reward weights and continue RL; changing coefficients only at inference does not alter the policy. Figures 7–8 illustrate the trade-offs. |
Kind Reminder of the Approaching Discussion Deadline
Dear Reviewer qMeN,
As the discussion period draws to a close, we would greatly appreciate any further feedback on our rebuttal. We have addressed your concerns point by point and would welcome the opportunity to clarify any remaining questions. If our responses have resolved your concerns, we would be grateful if you could reconsider your assessment and score in light of these clarifications.
Thank you again for your thoughtful review and for the time you have devoted to our submission.
Best regards,
Authors of Paper #25367
Response to Reviewer qMeN - Part (1/3)
We are grateful for your thorough comments. We have provided detailed responses to your concerns as follows.
A1. Only the privacy penalty relies on evaluator judgments; the other reward terms are computed by exact rules for a given trajectory. Write R=C+αrprivacy, where C collects the exactly computed reward terms and rprivacy∈−1,0. Let x denote the user query and T the complete interaction trajectory. We analyze trajectories evaluated by the privacy judge, assuming a fixed symmetric label-flip probability 0≤p<1/2, independent of the query and trajectory. Unevaluated trajectories retain their original rewards. For a fixed evaluated (x,T), the privacy penalty remains rprivacy with probability 1−p and flips to −1−rprivacy with probability p. Therefore,
=C+\alpha\big[(1-p)r_{\mathrm{privacy}}+p(-1-r_{\mathrm{privacy}})\big] =C+\alpha(1-2p)r_{\mathrm{privacy}}-\alpha p.$$ For evaluated trajectories, the effective privacy weight becomes **$\alpha(1-2p)$**, with an offset of $-\alpha p$. In the overall objective, this offset contributes in proportion to the probability of privacy evaluation, which can vary with the policy. These changes can therefore affect the combined gradient direction and optimal policy. **We have revised Section 4.4 and Appendix H to state the noise assumptions explicitly and characterize their effects on the expected reward.** **Our experiments demonstrate the effectiveness of the learned collaboration policy in balancing answer quality, privacy protection, and interaction efficiency.** With Qwen3-8B and Qwen3-235B, Table 1 reports 43.11% EM, 0.08% $\mathrm{Priv}_{\mathrm{sample}}$, and 60.91% INScore for our method, compared with 33.23%, 24.37%, and 50.00% for PAPILLON and 37.96%, 21.01%, and 55.52% for Router-R1, respectively, under the same evaluation protocol. These results support the learned policy’s effectiveness in improving answer accuracy, requesting assistance when needed, and reducing measured privacy leakage. > Weakness 2 / Question 2: Scope of privacy claims and evaluator calibration **A2.** In Appendix G, **we define privacy leakage as user-private information newly revealed to the cloud LLM through the SLM’s requests, relative to the LLM’s prior knowledge about the user.** Following the detection-based evaluation used in recent work (Li et al., NAACL 2025; Dou et al., ACL 2024), we measure leakage under this definition using an evaluator to check whether defined privacy items are explicitly disclosed or can be inferred from the posed request, covering both direct and indirect disclosure. **Our privacy objective is to reduce the disclosure of users' private information to an honest-but-curious LLM service provider, rather than to guarantee that an adversarial provider gains no inferential advantage through active attacks.** We have revised Appendix G to avoid overstating the privacy protection. We have evaluated the privacy judge on 200 samples independently labeled by three human annotators, using majority voting to obtain the reference labels. The judge achieved 98.5% agreement with these labels, as reported in Appendix F.1. We have **added the following confusion matrix and the corresponding false-negative and false-positive rates** to Appendix F.1: | Human reference | LLM Judge: leakage | LLM Judge: no leakage detected | |---|---:|---:| | Leakage | **TP: 78** | **FN: 1** | | No leakage | **FP: 2** | **TN: 119** | The judge achieves a **1.27% false-negative rate and a 1.65% false-positive rate**, with **98.73% recall and 97.50% precision**, demonstrating its reliability.