acceptodds
Under review as a conference paper at ICLR 2027

Local-First Intelligence via Proactive SLM Collaborating with Feedback-Providing LLM

Abstract

Personal intelligence requires strong capabilities under strict privacy and efficiency constraints. Unlike cloud-first intelligence, which relies on powerful but costly and privacy-vulnerable large language models (LLMs), we focus on local-first intelligence, emphasizing on-device small language models (SLMs), which enable efficient, private inference but have limited capacity. To reconcile the capability gap with privacy and efficiency, we propose a dynamic collaboration framework, where an SLM learns to proactively decide how to consult an LLM during multi-step reasoning, while the LLM provides adaptive feedback instead of acting as a passive tool, modeling them as heterogeneous agents under information asymmetry. We further systematically investigate how collaboration strategies are shaped by SLM and LLM capabilities as well as efficiency and privacy constraints. Evaluation results reveal a distinct scaling effect: stronger SLMs become more self-reliant, while stronger LLMs enable fewer and more informative interactions. In addition, the learned dynamic collaboration strategies significantly outperform various baselines and transfer robustly to unseen LLMs as well as to real-world personal tasks without additional training.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Dynamic SLM-LLM Collaboration, Personal AI, Privacy-preserving, Interaction Efficiency

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_qMeNOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the dynamic collaboration between a local SLM and a cloud LLM to balance privacy, efficiency, and capability. The local SLM receives the full user input and learns when to consult the LLM and how to construct requests, while the cloud LLM can either answer or request clarification. Modeling them as heterogeneous agents under information asymmetry, the local SLM is trained via VAPO with rewards covering answer quality, privacy, interaction efficiency, feedback, and format constraints. Evaluation results reveal a distinct scaling effect where stronger SLMs become more self-reliant and stronger LLMs enable fewer but more informative interactions. The learned collaboration strategies significantly outperform baselines and transfer robustly to unseen LLMs and tasks without additional training.

Critical strengths

(+) The paper addresses a concrete and important collaboration problem. A local model needs to use cloud capabilities while limiting unnecessary information disclosure and interaction. Incorporating assistance-seeking decisions and request formulation into a trainable policy has clear practical value.

(+) The experimental evaluation extends beyond a single model and the training distribution.

Critical weaknesses

(-) The evaluator-noise conclusion is overstated. Appendix H only establishes that the privacy-reward component is scaled. This does not guarantee that the overall optimization direction or optimal policy remains unchanged.

(-) The privacy guarantees lack sufficient support. Failure to detect private items does not imply that the attacker gains no inferential advantage. Overall evaluator accuracy also does not directly bound the conditional false-negative rate (Appendices F–G).

(-) Trust and deployment conditions need clarification. The trust boundaries of training and the privacy evaluator remain insufficiently specified, as do the conditions and advantages of this approach relative to trusted LLM services or alternatives such as STIP.

(-) Resource evidence for long contexts is insufficient. RTX 4090 latency tests are provided, but measurements of local generation length and peak GPU memory/KV-cache usage under long contexts, as well as latency for long reasoning trajectories, are missing.

(-) Initialization and cold start are insufficiently specified. Whether the main experiments use SFT and which parameter-update method they employ remain unclear. The paper also does not report whether a supervised cold start resolves the 0.6B model’s training failure.

(-) Reward and model-adaptation rules remain unclear. Despite weight sweeps and transfer experiments, the default reward configuration and the conditions requiring retuning or retraining after changes in tasks, devices, or models need clarification.

Questions for authors

Q1: Could you correct the conclusion about preserving the overall optimization direction?

Q2: Could you substantiate the privacy guarantees and provide the evaluator’s confusion matrix?

Q3: Which components must be trusted? Under what conditions is local collaboration preferable to directly using a trusted or protected large-model service?

Q4: How do peak memory usage and complete end-to-end latency change under long contexts and long reasoning trajectories?

Q5: What initialization and fine-tuning settings are used in the main experiments? Has cold starting weaker models been validated?

Q6: How are the default reward weights selected? Which task or deployment changes require retraining?

See weakness for more details.

4 comments
AuthorsOpenReview ↗

Response to Reviewer qMeN - Part (1/3)

We are grateful for your thorough comments. We have provided detailed responses to your concerns as follows.

Weakness 1 / Question 1: Effect of evaluator noise on the optimization objective

A1. Only the privacy penalty relies on evaluator judgments; the other reward terms are computed by exact rules for a given trajectory. Write R=C+αrprivacyR=C+\alpha r_{\mathrm{privacy}}, where CC collects the exactly computed reward terms and rprivacy∈−1,0r_{\mathrm{privacy}}\in\\{-1,0\\}. Let xx denote the user query and T\mathcal{T} the complete interaction trajectory. We analyze trajectories evaluated by the privacy judge, assuming a fixed symmetric label-flip probability 0≤p<1/20\le p<1/2, independent of the query and trajectory. Unevaluated trajectories retain their original rewards. For a fixed evaluated (x,T)(x,\mathcal{T}), the privacy penalty remains rprivacyr_{\mathrm{privacy}} with probability 1−p1-p and flips to −1−rprivacy-1-r_{\mathrm{privacy}} with probability pp. Therefore,

=C+\alpha\big[(1-p)r_{\mathrm{privacy}}+p(-1-r_{\mathrm{privacy}})\big] =C+\alpha(1-2p)r_{\mathrm{privacy}}-\alpha p.$$ For evaluated trajectories, the effective privacy weight becomes **$\alpha(1-2p)$**, with an offset of $-\alpha p$. In the overall objective, this offset contributes in proportion to the probability of privacy evaluation, which can vary with the policy. These changes can therefore affect the combined gradient direction and optimal policy. **We have revised Section 4.4 and Appendix H to state the noise assumptions explicitly and characterize their effects on the expected reward.** **Our experiments demonstrate the effectiveness of the learned collaboration policy in balancing answer quality, privacy protection, and interaction efficiency.** With Qwen3-8B and Qwen3-235B, Table 1 reports 43.11% EM, 0.08% $\mathrm{Priv}_{\mathrm{sample}}$, and 60.91% INScore for our method, compared with 33.23%, 24.37%, and 50.00% for PAPILLON and 37.96%, 21.01%, and 55.52% for Router-R1, respectively, under the same evaluation protocol. These results support the learned policy’s effectiveness in improving answer accuracy, requesting assistance when needed, and reducing measured privacy leakage. > Weakness 2 / Question 2: Scope of privacy claims and evaluator calibration **A2.** In Appendix G, **we define privacy leakage as user-private information newly revealed to the cloud LLM through the SLM’s requests, relative to the LLM’s prior knowledge about the user.** Following the detection-based evaluation used in recent work (Li et al., NAACL 2025; Dou et al., ACL 2024), we measure leakage under this definition using an evaluator to check whether defined privacy items are explicitly disclosed or can be inferred from the posed request, covering both direct and indirect disclosure. **Our privacy objective is to reduce the disclosure of users' private information to an honest-but-curious LLM service provider, rather than to guarantee that an adversarial provider gains no inferential advantage through active attacks.** We have revised Appendix G to avoid overstating the privacy protection. We have evaluated the privacy judge on 200 samples independently labeled by three human annotators, using majority voting to obtain the reference labels. The judge achieved 98.5% agreement with these labels, as reported in Appendix F.1. We have **added the following confusion matrix and the corresponding false-negative and false-positive rates** to Appendix F.1: | Human reference | LLM Judge: leakage | LLM Judge: no leakage detected | |---|---:|---:| | Leakage | **TP: 78** | **FN: 1** | | No leakage | **FP: 2** | **TN: 119** | The judge achieves a **1.27% false-negative rate and a 1.65% false-positive rate**, with **98.73% recall and 97.50% precision**, demonstrating its reliability.
AuthorsOpenReview ↗

Response to Reviewer qMeN - Part (2/3)

Weakness 3 / Question 3: Local privacy protection and trust boundaries

A3. During deployment, our framework protects privacy at the source by controlling disclosure on the user’s local device. As described in Sections 3.1 and 4.3, the original query and private context are processed locally: the SLM decides whether to seek cloud assistance, formulates requests without user-specific private context, and combines the returned information with local private context to produce the final answer. The cloud sees only the submitted requests and their interaction history, with no direct access to the original query or local private context. Privacy protection is applied locally before requests reach the cloud, without relying on the cloud provider to filter private information. Our design is complementary to trusted LLM services or alternatives such as STIP: we limit what information on local device is shared, while they protect its processing in the cloud.

During training, we use examples constructed by injecting synthetic personal information into non-private, publicly available queries, as described in Section 4.3. The privacy evaluator receives the SLM’s requests and the corresponding annotated private items to compute the privacy penalty. This evaluation relies solely on synthetic private information in PrivQA and does not require access to real users’ private information at deployment.

Weakness 4 / Question 4: Resources along real collaboration trajectories

A4. We measure resource usage during inference on queries from PrivQA, PUPA, AESLC, and SAMSum, using trained Qwen3-4B with the hosted Qwen3-Max API. The local SLM runs with BF16 on NVIDIA 4090 GPU. We group trajectories by the maximum local context actually processed, including the input, accumulated local reasoning, requests, and cloud replies.

Maximum processed context (tokens)Local generated tokensPeak memory (GiB)Local latency (s)End-to-end latency (s)
MedianMaxGPU residentActive KVMedianMaxMedianMax
<1,02427959110.7430.1410.4431.2220.4842.635
1,024–2,0475581,32410.6300.2810.9022.0581.8064.023
2,048-40951,3542,33310.7430.5142.1955.8544.39510.183
4,096–8,1922,8627,85210.7431.1254.21611.8437.03216.084

Generated tokens include local reasoning, requests, and the final answer. GPU resident is the sampled peak GPU memory occupied by the local process, including a fixed 2.00 GiB reserved KV pool; active KV is the peak cache space allocated to active requests within that pool. Local latency measures SLM prefill and decoding time, while end-to-end latency is measured over the complete trajectory and includes hosted API generation, network round trips, and orchestration. Across the four context groups, median local latency increases from 0.443 to 4.216 seconds and peak active KV usage from 0.141 to 1.125 GiB.

Weakness 5 / Question 5: Initialization, parameter updates, and cold start

A5. Our main experiments use end-to-end RL directly from publicly released model checkpoints, without additional task-specific SFT. We update all SLM parameters while keeping the cloud LLM frozen and the SFT methods in Table 1 are separate baselines. The detailed training settings are provided in Section 4.4 and Appendix I.1.

Qwen3-0.6B uses the same end-to-end RL setup in Section 5.3, Figure 5, but reaches a low training reward with a decline in answer-quality reward. Its weak instruction-following ability makes well-formed requests rare during exploration, limiting useful cloud feedback and positive rewards. A potential remedy is a supervised cold start through trajectory distillation: a stronger model would generate successful collaboration trajectories, and the 0.6B model would learn through imitation how to formulate requests, respond to feedback, and integrate the returned information before RL. Our experiments focus on end-to-end RL and systematically analyzing how model capabilities, privacy constraints, and efficiency constraints shape the learned strategies. We leave trajectory distillation as a complementary direction for future work.

AuthorsOpenReview ↗

Response to Reviewer qMeN - Part (3/3)

Weakness 6 / Question 6: Default rewards and when adaptation is needed

A6. As introduced in Section 4.4, α\alpha and β\beta control the privacy and efficiency penalties, respectively. We use α=β=0.3\alpha=\beta=0.3 as heuristic defaults. These weights control how strongly privacy leakage and unnecessary cloud calls are penalized during training. Figures 7 and 8 in Section 5.5 show how changing the privacy and efficiency weights affects the resulting trade-offs. We have explicitly documented the defaults and their selection rationale in Appendix I.1. Other default training settings, including learning rates and batch size, are also listed in Appendix I.1.

Whether retraining or retuning is needed depends on which part of the deployment changes.

ChangeIs retraining or reward retuning needed?Evidence / rationale
Cloud LLMNo, for the demonstrated transfers within and across LLM families.For Qwen3-8B, switching from Qwen3-235B to Qwen3-Max improves average EM from 43.11% to 46.01% without retraining in Section 5.4. Appendix I.6 also reports that the policy trained with DeepSeek-R1-Distill-Llama-70B transfers to Qwen3-235B-A22B-Instruct without retraining, increasing average EM from 37.80% to 40.10%.
TaskNo, for the evaluated transfers.Transfer to PUPA, AESLC, and SAMSum uses no additional training in Section 5.2.
Device onlyNo policy retraining or reward retuning, provided the model, precision, and context limits remain unchanged.The learned policy remains unchanged; only hardware-dependent inference settings and resource measurements need updating.
Local SLMPolicy training or adaptation is needed for a new local model; reward retuning is not necessarily needed.Each local model learns its own collaboration policy in Section 5.3.
Privacy or efficiency requirementsOnly if no existing policy meets the new requirements.Otherwise, reuse a suitable policy. If needed, adjust the corresponding reward weights and continue RL; changing coefficients only at inference does not alter the policy. Figures 7–8 illustrate the trade-offs.
AuthorsOpenReview ↗

Kind Reminder of the Approaching Discussion Deadline

Dear Reviewer qMeN,

As the discussion period draws to a close, we would greatly appreciate any further feedback on our rebuttal. We have addressed your concerns point by point and would welcome the opportunity to clarify any remaining questions. If our responses have resolved your concerns, we would be grateful if you could reconsider your assessment and score in light of these clarifications.

Thank you again for your thoughtful review and for the time you have devoted to our submission.

Best regards,

Authors of Paper #25367

Reviewer_Q4iQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes an RL-based dynamic collaboration framework between a local small language model (SLM) and a cloud-based large language model (LLM), enabling the local model to learn when to seek assistance, how to formulate queries, and how to leverage cloud feedback to achieve a better balance among task performance, invocation efficiency, and privacy protection.

Critical strengths
  1. The proposed framework is clean, intuitive, and well-motivated. The dynamic collaboration between an on-device SLM and a cloud-based LLM provides a natural way to balance task performance, efficiency, and privacy.
  2. The experiment results are promising. Table 1 demonstrates substantial improvements in both answer quality and privacy preservation over several baselines, suggesting that the learned collaboration strategy is effective.
  3. PrivQA might be a useful resource contribution to LLM privacy research.
  4. The ablation studies are comprehensive and well-designed. They systematically examine the effects of different components (SLM, LLM) and configurations (LLM feedback, efficiency penalty and privacy penalty).
Critical weaknesses
  1. The methodological novelty appears somewhat relatively limited. The framework primarily combines existing techniques, including adaptive LLM collaboration, RL-based tool use, and multi-objective reward optimization for privacy and efficiency. While their integration is meaningful, the paper could better clarify which components constitute fundamentally new contributions beyond existing collaboration frameworks.
  2. Although the paper emphasizes dynamic multi-turn collaboration as a central contribution, Table 11 shows that the average number of interaction turns is only 0.85, with 88.5% of samples requiring exactly one turn and only 0.09% involving two or more turns. This raises the question of whether the reported improvements primarily come from better single-turn request formulation and deciding when to seek LLM assistance, rather than the proposed multi-turn feedback mechanism. A more targeted evaluation on tasks that genuinely require multiple rounds of interaction would strengthen the paper's claims.
  3. Although the framework achieves strong empirical performance, the paper could provide deeper insights into what collaboration strategies the SLM actually learns. For example, does the SLM learn to identify the minimum sufficient information needed by the LLM, or does it mainly learn to avoid privacy penalties and unnecessary clarification requests?
Questions for authors

Minor Suggestions

  1. Some table captions are too brief and not well self-contained. Just an example: Table 1 would benefit from a more descriptive caption explaining the experimental setting, compared methods, and evaluation metrics.
  2. Following the weakness 3, the paper could include more qualitative examples illustrating successful and unsuccessful multi-turn collaboration, particularly cases where the SLM must balance providing enough information to obtain an answer against protecting user privacy.
5 comments
AuthorsOpenReview ↗

Response to Reviewer Q4iQ - Part (1/3)

We appreciate your careful review and constructive feedback. Below, we clarify our contributions and address all your concerns.

Weakness 1: Fundamentally new contributions beyond existing collaboration frameworks

A1. We identify that local-first personal intelligence imposes three indispensable and mutually coupled requirements: performance, privacy protection, and interaction efficiency, forming a new objective. Existing routing (Zhang et al., NeurIPS 2025) and static collaboration frameworks (Li et al., NAACL 2025; Dou et al., ACL 2024) each address at most one of these axes and cannot be trivially extended to cover the others.

At the framework level, we propose a novel dynamic collaboration framework that models the SLM and LLM as heterogeneous agents operating under information asymmetry. Only the SLM observes the original user query and private context, while the LLM contributes its stronger capabilities through selectively disclosed requests.

At the local SLM level, we introduce a proactive SLM that learns when and how to collaborate as part of its reasoning process. Beyond selecting a model or applying a fixed query-rewriting pipeline, the SLM determines which information to seek from the cloud, which user-specific reasoning to retain locally, and whether to continue interacting after receiving a response. We train this policy through end-to-end RL, as described in Sections 4.1 – 4.4.

At the cloud LLM level, we design a collaborative LLM whose feedback guides both interaction and learning. Rather than treating the LLM as a passive tool, we enable it to select between answering the SLM’s request and asking for missing context. This clarification feedback serves two connected purposes: its content guides request refinement, while the clarification action triggers a process-level penalty for an under-informative request. The resulting closed-loop mechanism enables the SLM to learn from feedback on individual requests alongside the final task outcome.

Finally, we systematically investigate how collaboration strategies are shaped by SLM and LLM capabilities as well as efficiency and privacy constraints and reveal a distinct scaling effect: stronger SLMs become more self-reliant, while stronger LLMs enable fewer and more informative interactions.

AuthorsOpenReview ↗

Response to Reviewer Q4iQ - Part (2/3)

Weakness 2: Effect of multi-turn feedback and the source of performance gains

A2. Our objective is to achieve better task performance while avoiding unnecessary cloud interactions. The difficulty-aware efficiency reward in Section 4.1 penalizes unnecessary requests, while the LLM’s clarification feedback in Section 4.2 encourages the SLM to formulate clearer and more precise requests. These training signals help the SLM learn when and how to seek assistance and whether further interaction is beneficial. The average interaction count of less than one turn and the distribution of interaction turns reported in Table 11 reflect that SLM has learned the capability of properly seeking cloud LLM assistance. To examine how SLM-LLM interaction behavior evolves during training, we analyze Qwen3-4B with Qwen3-235B on PrivQA training batches at different training steps. Clarification rate denotes the proportion of identifiable cloud LLM feedback actions that request clarification.

Training stepEMAvg TurnQueries with ≥2 turnsClarification rate
5029.30%1.1677.42%27.53%
10032.03%1.0914.69%18.71%
20035.16%0.8942.98%5.68%
30036.33%0.9653.12%5.18%
40035.94%0.9571.95%2.05%

As training progresses, the percentage of queries requiring at least two interaction turns decreases from 7.42% to 1.95%, the clarification rate drops from 27.53% to 2.05%, and the EM score increases from 29.30% to 35.94%, aligning with our training objectives.

The SLM can also use additional interaction turns when useful, subject to the efficiency objective. For Qwen3-4B with Qwen3-235B, Figure 7 shows that relaxing the efficiency weight from 0.8 to 0.0 increases average turns from 0.64 to 1.07, alongside a 3.3% increase in EM. Furthermore, interaction frequency increases as tasks become more difficult and the local–cloud model pair becomes less capable. We analyze a fixed hard HotpotQA subset defined by incorrect base Qwen3-8B CoT answers. Qwen3-8B with Qwen3-235B uses at least two turns on 11 queries (0.25%), compared with 353 queries (8.08%) for Qwen3-4B with DeepSeek-R1-Distill-Llama-70B.

To clarify the contributions of multi-turn interaction and clarification feedback, we compare two ablations with our full method in the revised Table 5. We have added RL-OneCall, an RL-trained policy that allows at most one cloud request per query during both training and inference, whereas the other ablation baseline, Ours w/o feedback, allows multiple requests. Both baselines use an answer-only LLM and share the same initialization, training data, evaluation protocol, and reward function as our method. All three use Qwen3-4B with Qwen3-235B, and results are averaged over all six PrivQA test sets.

PolicyAvg EM ↑Avg BERTScore ↑Avg Turn ↓
RL-OneCall34.58%69.460.824
Ours w/o feedback35.53%71.520.915
Ours38.34%73.510.887

With an answer-only LLM, allowing multi-turn interaction improves average EM by 0.95% and average BERTScore by 2.06% over RL-OneCall. Adding clarification feedback yields a further 2.81% improvement on average EM and 1.99% improvement on average BERTScore. Overall, our full method improves average EM by 3.76%, a 10.87% relative gain, and BERTScore by 4.05%, a 5.83% relative gain, over RL-OneCall, while requiring only 0.063 additional turns on average. These results highlight the importance of both multi-turn interaction and the feedback mechanism in our design.

We further compare different policies using Qwen3-4B with Qwen3-235B on the same hard subsets of all six PrivQA test sets, which consist of queries answered incorrectly by the base Qwen3-4B with CoT. The results are listed in the Table below.

PolicyAvg EM ↑Avg BERTScore ↑Avg Turn ↓
RL-OneCall24.13%64.970.849
Ours w/o feedback25.16%67.281.034
Ours27.03%68.590.925

Compared with RL-OneCall, our method improves average hard-set EM by 2.90% and average BERTScore by 3.62%, corresponding to relative gains of 12.02% and 5.57%, respectively. Compared with Ours w/o feedback, it improves these metrics by 1.87% and 1.31%. On the hard subset, all methods require more interaction turns on average, while the ablation results still demonstrate the effectiveness of multi-turn interaction and the feedback mechanism.

AuthorsOpenReview ↗

Response to Reviewer Q4iQ - Part (3/3)

Weakness 3: What information does the SLM learn to request

A3. Our quality reward and feedback mechanism encourage the SLM to seek task-relevant information and provide sufficient context for the LLM to respond, while privacy and efficiency penalties encourage requests to be as concise as possible while disclosing only necessary information. To examine such contextual sufficiency and minimality of the collaboration behavior, we selected 200 requests from distinct samples in each of the six PrivQA test sets, giving N=1,200N=1{,}200 requests. Let rir_i denote the assessed request, HiH_i denote its preceding cloud-visible interaction history, and JJ the evaluator, DeepSeek-V4.1-Flash.

First, contextual sufficiency. We define si=Jsuff(ri,Hi)s_i=J_{\mathrm{suff}}(r_i,H_i), where si=1s_i=1 if the evaluator judges that the supplied context is sufficient for the cloud LLM to identify and answer the intended knowledge subtask. The evaluator judges 94.75% of requests to be contextually sufficient. The remaining 5.25% include unspecified entities and ambiguous references. The feedback reward in Section 4.2 targets this property by penalizing requests that elicit clarification.

Sufficiency assessmentComputationProportion
Contextually sufficient1N∑isi\frac{1}{N}\sum_i s_i94.75%
Contextually insufficient1N∑i(1−si)\frac{1}{N}\sum_i (1-s_i)5.25%

Second, minimality. We ask the evaluator whether any substantive information can be removed from rir_i to form a shorter request ri′r_i' that addresses the same information need and preserves the cloud LLM's expected answer given HiH_i. We define di=Jdelete(ri,Hi)d_i=J_{\mathrm{delete}}(r_i,H_i), where di=1d_i=1 if the evaluator identifies such a deletion. Each positive judgment must specify the removable text and explain why the expected answer would be preserved. Removable information is identified in 0.92% of requests, indicating little evaluator-detected redundancy.

Minimality assessmentComputationProportion
Removable information identified1N∑idi\frac{1}{N}\sum_i d_i0.92%
No removable information identified1N∑i(1−di)\frac{1}{N}\sum_i (1-d_i)99.08%

These results suggest that, guided by our objective, the SLM learns to request sufficient task-relevant information with little redundancy, supporting accurate answers while limiting private disclosure and unnecessary interactions.

Minor Suggestion 1: More informative table captions

A1. Thank you for this helpful suggestion. We have revised the captions of Figures 1–4, 7, and 8 and Tables 1, 4, and 5 in the main text, as well as Tables 14–16, 18, 20, and 21 in the appendix, to make them more self-contained in the revised manuscript. For example, we have revised the caption of Table 1 as follows: Table 1. Comparison of our dynamic collaboration framework with baselines on six PrivQA test sets, including in- and out-domain datasets. Local SLMs are Qwen3-4B and Qwen3-8B, and the cloud LLM is Qwen3-235B. EM measures exact-match accuracy, Privsample\mathrm{Priv}_{\mathrm{sample}} measures sample-level privacy leakage, and INScore measures whether cloud assistance is requested when needed. Avg. denotes the mean across datasets. Bold and underlined values indicate the best and second-best results.

Minor Suggestion 2: Qualitative examples of multi-turn collaboration

A2. We have expanded the case studies in Appendix I.8 with trajectories showing what information is requested, which private associations remain local, and how subsequent requests affect the final answer. In an additional multi-turn example from the test set, the user asks where their grandmother used to live, explaining that it was the birthplace of the father of Adam Kazimierz Czartoryski, whom they are reading about. Sending the original query would expose the user’s reading history and a link to their grandmother’s former residence. The SLM instead asks, "Who was the father of Adam Kazimierz Czartoryski, and which city was he born in?" The LLM identifies the father but supplies birthplace information about the son. The SLM then asks, "What is the birthplace of August Aleksander Czartoryski?" and uses the reply to produce Warsaw, matching the reference answer. Neither request mentions the user’s grandmother or reading history. The SLM connects the public fact to the user’s private family context locally. This illustrates refining a request to resolve an ambiguous referent.

In a contrasting failure from the PrivQA test set, the user asks which university their sister attended, explaining that she attended the same university as Claudio Silva’s wife, whom the user knows personally. The SLM determines that sending a request would disclose private information and therefore relies on its own uncertain knowledge and answers "University of São Paulo," whereas the reference answer is "Stony Brook University." This case illustrates a failure to handle uncertainty locally when privacy-related concerns limit the request.

Reviewer_ZzChOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies local-first personal intelligence, where an on-device small language model (SLM) handles user requests and consults a cloud LLM when necessary. The authors propose a dynamic collaboration framework that trains the SLM with reinforcement learning to decide when and how to seek assistance, while allowing the LLM to either respond or request clarification. To support privacy-aware training, the authors introduce PrivQA, a benchmark constructed by injecting synthetic personal information into existing QA datasets. Experiments show that the proposed method improves answer accuracy and reduces privacy leakage and interaction overhead compared to static collaboration baselines.

Critical strengths

1. A practical and well-motivated approach to privacy-aware SLM–LLM collaboration.

The paper combines SLM–LLM collaboration as a learned, multi-turn decision process rather than a fixed pipeline. Through end-to-end RL, the on-device SLM learns when to request cloud assistance and how to formulate requests without exposing private user information. The framework also considers answer quality, interaction efficiency, and privacy leakage, offering a flexible alternative to existing static collaboration approaches.

2. Transferability with direct deployment value.

The learned policy transfers without retraining to unseen and stronger LLMs, across LLM families, and to real-world personal tasks. This means a collaboration policy can be trained with a cheap open LLM and deployed with a stronger cloud provider, and need not be retrained when the provider changes.

Critical weaknesses

The multi-turn collaboration is rarely used at test time, so it is unclear what actually drives the gains.

Figure 4 and 6 show that average interaction turns are below one for every SLM–LLM pairing, which shows that the local policy only answers or rewrites the query once and forwards it. This shows a limited difference with PAPILLON or Router-R1 with a learned rewriter. The comparison is also uneven, with only trained Router-R1 and SFT while othr baselines prompt-based. A single-turn rewrite-and-forward policy trained with the same data and reward would be the right baseline.

Questions for authors

Clarification requests

  1. The Self-Disclosures averages in Table 2 do not match the per-dataset values in Table 17 (averaging gives about 17.0 for Qwen3-4B and 12.4 for Qwen3-8B, but Table 2 reports 23.20 and 18.98). Please check.
  2. Are the easy/hard labels in the efficiency reward fixed with the base SLM before training, or updated as the policy improves? And is INScore at evaluation computed against the base or the trained SLM?
  3. In PrivQA, privacy-preserving reformulation is always possible by construction, as private information is added to otherwise public facts. How does the method handle cases where the required information is inherently private or cannot be removed without losing essential context?

Suggestions for future work

  1. Evaluate on queries where the needed information cannot be separated from the user, to show the privacy–quality trade-off the current benchmark hides.
  2. Test against an adversarial LLM that uses the clarify action to probe for context, since the current threat model is only honest-but-curious.
  3. Validate the real-world quality judge with human annotation, or use a judge outside the Qwen family.
4 comments
AuthorsOpenReview ↗

Response to Reviewer ZzCh - Part (1/2)

Thank you for your thoughtful review and helpful suggestions. We address your concerns below.

Weakness 1: Average interaction turns in multi-turn collaboration at test time

A1. Our objective is to achieve better task performance while avoiding unnecessary cloud interactions. The interaction efficiency reward in Section 4.1 penalizes unnecessary requests, while LLM clarification feedback in Section 4.2 identifies missing context and guides request-level learning. These training signals jointly help the SLM learn when and how to seek assistance and whether further interaction is useful, aiming to reduce unnecessary calls while retaining the benefit of cloud assistance. The average number of interaction turns below one in Figures 4 and 6 reflects that SLM has learned the capability of properly seeking cloud LLM assistance. To further examine the changes of SLM-LLM interaction turns during training, we analyze Qwen3-4B with Qwen3-235B on the PrivQA training set at different training steps. Clarification rate denotes the proportion of cloud LLM feedback actions that request clarification.

Training stepEMAvg TurnQueries with ≥2 turnsClarification rate
5029.30%1.1677.42%27.53%
10032.03%1.0914.69%18.71%
20035.16%0.8942.98%5.68%
30036.33%0.9653.12%5.18%
40035.94%0.9571.95%2.05%

As training progresses, the percentage of queries requiring at least two interaction turns decreases from 7.42% to 1.95%, the clarification rate drops from 27.53% to 2.05%, and the EM score increases from 29.30% to 35.94%, aligning with our training objectives.

The SLM can also use more interaction turns when the expected improvement in performance outweighs the interaction cost. For Qwen3-4B with Qwen3-235B, Figure 7 shows that relaxing the efficiency penalty from a weight of 0.8 to 0.0 increases average turns from 0.64 to 1.07, alongside a 3.3% improvement in EM. Furthermore, interaction frequency increases as tasks become more difficult and the local–cloud model pair becomes less capable: on a hard HotpotQA subset defined by incorrect base Qwen3-8B CoT answers, Qwen3-8B with Qwen3-235B uses at least two turns on 11 queries (0.25%), compared with 353 queries (8.08%) for Qwen3-4B with DeepSeek-R1-Distill-Llama-70B.

Following your suggestion, in addition to PAPILLON (Li et al., NAACL 2025), a fixed single-turn pipeline based on prompted query rewriting, we have added RL-OneCall, a single-turn rewrite-and-forward baseline trained with the same model initialization and data. RL-OneCall permits at most one cloud request per query during training and inference, uses an answer-only LLM, and shares our method's training data and reward. We use Qwen3-4B with Qwen3-235B and report EM and average interaction turns over all six PrivQA test sets.

PolicyAvg EM ↑Avg Turn ↓
PAPILLON32.55%1.000
RL-OneCall34.58%0.824
Ours38.34%0.887
Δ vs. PAPILLON+5.79% (+17.79%)-0.113
Δ vs. RL-OneCall+3.76% (+10.87%)+0.063

Compared with RL-OneCall, our method improves average EM by 3.76%, a 10.87% relative gain, with only 0.063 additional interaction turns on average. Compared with the untrained fixed rewriting pipeline, our method improves average EM by a further 2.03% while requiring 0.113 fewer average interaction turns.

After training, even with a single interaction turn, our method enables SLM to learn more precise interaction timing and better request formulation. We evaluate queries across the six PrivQA test sets on which all compared methods make exactly one cloud request. Using Qwen3-4B paired with Qwen3-235B, we report each method’s average EM on this shared subset. As shown below, even when all methods make a single cloud request, our method outperforms PAPILLON and RL-OneCall by 6.00% and 3.59% in average EM, respectively.

PolicyAvg EM ↑
PAPILLON30.77%
RL-OneCall33.18%
Ours36.77%
Δ vs. PAPILLON+6.00% (+19.50%)
Δ vs. RL-OneCall+3.59% (+10.82%)
AuthorsOpenReview ↗

Response to Reviewer ZzCh - Part (2/2)

Question 1: Self-Disclosures average leakage rates in Table 2 and Table 17

A1. Thank you for pointing this out. This is a typographical error. The correct average Privsample\mathrm{Priv}_{\mathrm{sample}} values for Self-Disclosures are 16.96% for Qwen3-4B and 12.36% for Qwen3-8B. We have corrected the entries in Tables 2 and 17 and updated the corresponding text in the revised manuscript.

Question 2: Difficulty labels and INScore reference

A2. We obtain difficulty labels offline from the corresponding base SLM's standalone CoT answers: correctly answered queries are labeled easy, and incorrectly answered queries are labeled hard. These labels remain fixed throughout RL training. At evaluation, INScore uses the same difficulty labels defined by the base SLM and averages two proportions: easy queries with no cloud request and hard queries with at least one cloud request.

Question 3 / Future Work 1: Privacy-quality trade-offs when essential information is inherently private or cannot be safely removed

A3. Our framework handles task-relevant private information locally in two ways. The SLM can complete the task entirely on-device, or reformulate the external information need into a user-agnostic request, obtain general knowledge or guidance from the cloud, and combine it with the private context locally. Thus, private information can remain essential to the final answer without being included in the cloud request, as described in Sections 3.1 and 4.3.

When the required private information reasoning exceeds the SLM’s capability and cannot be separated from the cloud request, a privacy–quality trade-off remains. Our objective in Section 4.4 accounts for this trade-off through the privacy penalty. For different deployment requirements, a larger training-time privacy weight places greater emphasis on limiting disclosure, potentially at the cost of answer quality, while a smaller weight places relatively more emphasis on task performance.

We have also evaluated our method on three public datasets involving real-world personal information such as names, contact details, and personal activities, including PUPA (Li et al., NAACL 2025), an email and document generation task; AESLC (Zhang et al., ACL 2019), an email subject-line generation task; and SAMSum (Gliwa et al., NewSum 2019), a dialogue summarization task. As reported in Section 5.2 of the original submission, Qwen3-8B collaborating with Qwen3-235B achieves response quality scores of 90.96, 91.68, and 93.69, with privacy leakage of 0.82%, 1.09%, and 1.12%, respectively. As a case study, a PUPA test query asks the SLM to polish an application email containing the applicant’s and recipient’s names and a list of prepared documents, including transcripts, vaccination records, a TB test, and a background check. The SLM asks the cloud only, “Are there specific formatting guidelines or tone recommendations for emails submitted as part of a PTA program application?” It then applies the returned writing guidance locally, retaining the names and document details in the final email. The cloud receives the general writing question, while the personal details needed to convey the application status remain local.

Future Work 2: Robustness to adversarial clarification

A2. Thank you for this suggestion. Our current results are based on an honest-but-curious cloud LLM, as stated in Section 4.3 and Appendix G. An adversarial LLM that uses the clarify action to probe for context represents a stronger threat. We will make this scope explicit and include testing against such adversarial LLMs as future work.

Future Work 3: Quality judgments outside the Qwen family

A3. We have added a cross-family evaluation using DeepSeek-V4.1-Flash as an additional judge, comparing its quality scores with the original Qwen scores on the same outputs. For each of PUPA, AESLC, and SAMSum, we randomly sampled 500 output records from the pooled outputs of our method and multiple baselines and re-evaluated them using the same task-specific scoring protocols. PUPA retains binary quality-preservation judgments against the reference response, while AESLC and SAMSum retain their single-response quality rubrics on a 1–100 scale. The table reports both judges’ quality scores.

DatasetQwen scoreDeepSeek scoreAgreement / Pearson r
PUPA85.6082.8092.40%
AESLC87.5684.920.794
SAMSum89.4886.420.803

For PUPA, agreement is the proportion of outputs receiving the same binary judgment from both judges. For AESLC and SAMSum, Pearson r measures the linear correlation between their paired scores across individual outputs. The two judges agree on 92.40% of PUPA outputs and show strong positive score correlations on AESLC and SAMSum, with Pearson r around 0.8 indicating that outputs rated highly by Qwen also tend to receive high scores from DeepSeek.

Reviewer_ZzChOpenReview ↗

Post-rebuttal comments

Thanks for the authors' response. I will keep my score.

AuthorsOpenReview ↗

Thank You for Your Positive Feedback and Support

Thank you very much for your positive assessment and support of our work. We sincerely appreciate your time and constructive feedback throughout the review process.

Comments on the paper

6 comments
AuthorsOpenReview ↗

Response to Reviewers’ Comments and Summary of Revisions (Final Rebuttal Update)

Dear Area Chairs and Reviewers,

We sincerely thank the Area Chairs and reviewers for their careful evaluations and constructive feedback. We have provided detailed responses to the comments and revised the manuscript accordingly. As the rebuttal period concludes, we have updated this summary to include the strengths recognized by the reviewers, our responses to their main concerns, the corresponding manuscript revisions, and the latest discussion outcomes.


Strengths Recognized by Reviewers

We appreciate the reviewers’ recognition of the following strengths:

  • Importance and practical value (ZzCh, Q4iQ, qMeN): addressing the need to balance task performance, privacy, and interaction efficiency in local-first intelligence.

  • Novel design of dynamic SLM–LLM collaboration (ZzCh, Q4iQ, qMeN): a dynamic framework in which the local SLM learns when and how to seek assistance, while the cloud LLM actively provides feedback to guide request refinement and learning, with private context retained locally.

  • Transferability without retraining (ZzCh): deploying learned policies with unseen cloud models and on real-world personal tasks.

  • Promising results and broad empirical evaluation (Q4iQ, qMeN): demonstrating improvements in answer quality and privacy, with systematic evaluation across models and configurations.

  • Potential value of PrivQA (Q4iQ): providing a benchmark for research on privacy-aware language model collaboration.


Main Concerns and Our Responses

  • Contributions beyond existing collaboration frameworks (Q4iQ, W1). We clarified our framework’s distinctive design: heterogeneous agents with asymmetric access to user context, a proactive SLM that learns when and how to collaborate, and a cloud LLM whose clarification feedback guides both interaction and request-level learning.

  • The source of performance gains (ZzCh, W1; Q4iQ, W2). We compared newly added RL-OneCall, a multi-turn answer-only variant, and our full method across the PrivQA test sets and their fixed hard subsets. These comparisons, together with training-trajectory analyses, support the contributions of both multi-turn interaction and clarification feedback.

  • Learned request behavior (Q4iQ, W3). We assessed contextual sufficiency and redundancy, finding that most requests were judged sufficient and contained little evaluator-detected redundancy. Expanded case studies illustrate successful request refinement and failures involving the balance between private context and local reasoning capabilities.

  • Privacy evaluation and claim scope (qMeN, Q1–Q3). We provided human-reference validation of the privacy evaluator, with cross-model consistency as supplementary evidence, and clarified the practical value of local disclosure minimization alongside cloud-side protection. We also corrected the evaluator-noise analysis.

  • Resources, training, and adaptation (ZzCh, Q2; qMeN, Q4–Q6). We supplied memory and latency measurements on complete collaboration trajectories across different context lengths. We also clarified full-parameter, end-to-end RL training with a frozen cloud model, fixed difficulty labels, default reward weights, and conditions for retuning or retraining.


Summary of Manuscript Revisions

The revised manuscript incorporates the following updates:

  • Section 5.5 and Appendix I.7: Added RL-OneCall and comparisons separating multi-turn interaction from clarification feedback, including per-dataset results.
  • Appendix H: Corrected the evaluator-noise analysis and its implications for the combined reward.
  • Appendices F.1 and G: Added the human-reference confusion matrix and conditional error rates, and revised the scope of the privacy claims.
  • Appendices I.1 and I.8: Specified the default privacy and efficiency weights and adaptation guidance, and expanded examples of request refinement and local use of private context.
  • Main and supplementary results tables and captions: Corrected the Self-Disclosures averages and made selected captions more self-contained.

Further measurements and analyses are documented in the detailed reviewer responses.


Discussion Outcomes

Reviewer ZzCh maintained a positive rating of 3 following our response. Reviewer qMeN confirmed that the remaining concerns had been addressed and assigned a rating of 3.


We sincerely appreciate the reviewers’ constructive engagement. We believe that our responses and revisions address the concerns raised and hope they support a favorable reassessment of our work.

Best regards,
Authors of Paper #25367

AuthorsOpenReview ↗

Grateful Reply and Further Clarifications to Reviewer qMeN - Part (1/2)

We sincerely thank you for confirming that Q1 and Q4 have been addressed and that the clarification on Q5 is helpful. We believe the responses below address your remaining concerns regarding Q2, Q3, and Q6, and we have revised the manuscript accordingly. We would appreciate your reconsideration in light of these clarifications and revisions.

Q2: The privacy evaluator's validation and the evaluation under the honest-but-curious threat model

Response: We conducted a human evaluation to validate our automated LLM-based privacy judge. The 200 samples used for human evaluation are representative and span diverse query types, including but not limited to general factual queries, scientific knowledge queries, and multi-hop queries. To obtain reliable human reference labels, three annotators independently evaluated each sample, and majority voting determined the final label. The human reference labels identify 79 samples with privacy leakage and 121 without leakage. Related work similarly uses focused multi-annotator human evaluation, balancing validation needs with annotation costs. For example, PAPILLON (Li et al., NAACL 2025) validates its privacy metric on 50 samples with five annotators, while Self-Disclosures (Dou et al., ACL 2024) evaluates effect of abstractions on 60 instances with three annotators.

Our automated LLM-based privacy evaluation closely aligns with human annotations, achieving 98.5% agreement, 98.73% recall, 97.50% precision, and a 1.27% false-negative rate. To further assess the reliability of our LLM-based privacy judge, we conducted a cross-model consistency evaluation, as reported in Appendix F.3. We selected request trajectories from six test datasets, totaling 3,989 cases, and used four models from two model families, Qwen3-235B, Qwen3-Max, DeepSeek-R1, and DeepSeek-V3, to independently evaluate the SLM requests. The table below reports their pairwise sample-level agreement.

EvaluatorQwen3-235BQwen3-MaxDeepSeek-R1DeepSeek-V3
Qwen3-235B—98.17%97.89%97.94%
Qwen3-Max98.17%—97.87%97.42%
DeepSeek-R197.89%97.87%—97.69%
DeepSeek-V397.94%97.42%97.69%—

All pairwise agreements fall within a narrow 97.42% - 98.17% band, and cross-family agreements between Qwen and DeepSeek models range from 97.42% to 97.94%. The high agreement across evaluator families further supports the reliability of our automated privacy evaluation protocol.

As for threat model, we adopt an honest-but-curious model for the LLM service provider, capturing a realistic privacy risk in commercial LLM API deployments. The provider follows the interaction protocol and performs the requested task, but may reveal private information from the requests it receives. In practice, commercial LLM providers make explicit commitments to protecting customer data and confidentiality. For example, Anthropic’s Commercial Terms (https://www.anthropic.com/legal/commercial-terms) treat customer content as confidential and require protection against unauthorized use, access, or disclosure. This threat model, honest-but-curious LLM, is also widely used in privacy-preserving inference and LLM interactions, including secure inference in Cheetah (Huang et al., USENIX Security 2022) and privacy-preserving LLM interactions in Self-Disclosures (Dou et al., ACL 2024), PAPILLON (Li et al., NAACL 2025), and Privacy-R1 (Hui et al., ACL 2026). Furthermore, building on existing work on detecting and protecting private items before requests are sent to the LLM service provider (Zhang et al., ACL 2024; Li et al., NAACL 2025), our contribution is a new SLM-LLM collaboration framework that dynamically balances privacy and task performance, rather than a new privacy protection method.

For an adversarial cloud LLM, we could extend the private-item detection module in Section 4.3 to account for adversarial attacks, drawing on methods such as DataSentinel (Liu et al., IEEE S&P 2025), while keeping the overall collaboration framework unchanged. We leave this extension to future work.

AuthorsOpenReview ↗

Grateful Reply and Further Clarifications to Reviewer qMeN - Part (2/2)

Q3: The trust assumptions and the practical advantages of the proposed approach over trusted LLM services or alternatives

Response: Our framework controls what information is disclosed before a request is sent to the cloud. Trusted LLM services and alternatives such as STIP protect information after the cloud receives the request. These approaches are therefore complementary rather than mutually exclusive, as they protect private information at different stages.

Specifically, in our design, for each user query, the SLM first reasons locally and formulates a user-agnostic request that excludes private information, retaining private context on-device. The cloud LLM answers the request, and the SLM combines the response with local context to generate the final answer. For example, in the PUPA case in Appendix I.8, the SLM requests general email-writing guidance from the cloud LLM, then incorporates locally retained names and personal document details into the final email. Compared with forwarding raw queries to the cloud, our method reduces privacy disclosure while maintaining answer accuracy. Therefore, whether the LLM service provider is honest-but-curious or adversarial, our approach limits privacy exposure at the source, offering stronger privacy protection than methods that forward raw user queries.

Q6: The rationale behind selecting the default reward weights and the conditions requiring reward retuning

Response: We choose the default penalty weights to limit privacy disclosure and interaction cost while preserving task quality. We set both weights to 0.3 because the ablations in Section 5.5 show that this setting substantially reduces measured leakage and average interaction turns, with little change in EM relative to removing the corresponding penalty. Larger weights can further reduce disclosure or interaction cost but incur additional EM loss. Thus, weights of 0.3 offer an empirical balance for our main experiments.

Different deployment requirements call for reward retuning by adjusting the penalty weights. Stricter privacy requirements favor a higher privacy penalty, while tighter interaction budgets favor a higher efficiency penalty. When task accuracy takes priority, lower penalties can be appropriate, provided that privacy requirements and interaction budgets are still met. The table summarizes these choices with illustrative deployment scenarios.

Deployment priorityExample scenarioRetuning needed?Adjustment
Balanced objectivesGeneral-purpose personal assistantNo, the defaults are suitable.Keep both weights at 0.3.
Stronger privacyDrafting emails with sensitive personal detailsOnly if no suitable policy exists.Increase the privacy penalty weight.
Lower interaction costMobile assistant with a limited API budgetOnly if no suitable policy exists.Increase the efficiency penalty weight.
Higher accuracyComplex QA with a flexible API budgetOnly if performance does not meet requirements.Lower the efficiency weight; relax the privacy weight only if permitted.

An existing policy can be reused if it achieves the desired trade-off. Otherwise, the selected penalty weights should guide RL training or policy adaptation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.