- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper looks at privacy in edge-cloud reasoning, where a small local model holds private knowledge and a cloud LLM helps with a multi-step reasoning graph. Instead of one privacy decision per request, the authors split a global DP budget across the individual steps. Their method, Swim, scores each step with local heuristics (uncertainty, importance, dependence on private evidence) and picks edge execution or a cloud call with a certain noise level. The authors also define “Operational Privacy Dependency”, prove a lower bound, and cite adaptive composition. They test on three QA datasets they built themselves and report better quality and lower attack scores than other edge-cloud methods.
Critical strengths
- The question is good. Where a limited privacy budget should be spent in multi-step reasoning is a real and under-studied problem.
- Counting the incremental privacy cost with an accountant and enforcing a hard budget is a sensible design.
Critical weaknesses
-
My main concern is that the privacy claim seems to differ from the system in the experiments. The paper says that choosing actions costs no privacy because everything is computed locally. But the scores that decide whether a step goes to the cloud, and with how much noise, are computed from private evidence. Whether the cloud is called at all is therefore visible to the cloud and depends on private data. The paper’s own appendix says such routing decisions must be counted when they depend on protected information, but the method does not do this. The same goes for the threshold that allows “privacy-free” cloud calls. That threshold is effectively an unaccounted privacy knob, and the authors’ own sensitivity analysis shows leakage rising when it is increased. The theorem only holds if each release is DP given the history, and nothing shows this holds for the real pipeline. So “better quality under the same formal privacy budget” is not established.
-
My second concern is that it is unclear how much privacy is actually being spent. The paper does not state the noise mechanism, sensitivity or noise scales. With the listed settings (64-dim state, clip norm 1, ε at most 1), the usual Gaussian mechanism would give noise much larger than the signal. The leakage numbers also hardly change when the protected releases are added to the transcript. This suggests the quality gains may mostly come from the free cloud calls and not from careful spending of privacy. If so, the main story of the paper is not supported.
Overall, the paper is hard to follow in terms of the mathematical content. The conceptual content is clearer, but it seems that the mathematical notation and explanation could be significantly improved and clarified.
Questions for authors
- What exactly is sent in a “privacy-free” action, and is it independent of unreleased private evidence? How are routing decisions and the threshold gating counted in the guarantee?
- What are the DP mechanism, sensitivity and noise scales? How much of the quality gain remains if only privacy-free actions are used?
- Can you compare against request-level, equal-split and greedy allocation using the same mechanism and accountant?
- Does the predicted gain actually rank real per-action gains correctly on held-out data?
- In the appendix you describe the "sanitization" procedure. However, there is not guarantee that an LLM will actually remove all personal and sensitive information. Can you please comment on this?
3 comments
Response to the DP specification, utility of privacy-free Actions, and accuracy of per-action gain prediction.
Thank you for requesting these clarifications. For each protected action, we encode at most private evidence units into 64-dimensional signed-hash vectors. Each vector is clipped to norm , and the aggregate uses the fixed denominator :
\qquad n\le m.$$ Under replace-one adjacency, $\Delta_2=2C/m=0.125$. We release $$\widetilde{s}=s(D)+\mathcal N(0,\sigma^2I_{64}).$$ With per-release $\delta=10^{-6}$, we calibrate the Gaussian mechanism through zCDP: $$\rho=\left(\sqrt{\log(1/\delta)+\epsilon} -\sqrt{\log(1/\delta)}\right)^2, \qquad \sigma^2=\frac{\Delta_2^2}{2\rho}.$$ For adaptive releases, the accountant composes the corresponding $\rho$ values and converts the total back to an $(\epsilon,\delta)$ guarantee. We further evaluate all actions from matched held-out pre-action states. Actual quality denotes the realized downstream quality after taking action $a$, while $\widehat g(a)$ is the gain predicted by the action selector. | Action | $\epsilon$ | Noise scale $\sigma$ | Predicted gain $\widehat g(a)$ | Actual quality | | ------------------ | ---------- | -------------------- | ------------------------------ | -------------- | | Privacy-free $r_0$ | 0 | — | 0.157 | 0.205 | | DP release | 0.2 | 3.297 | 0.166 | 0.214 | | DP release | 0.4 | 1.655 | 0.168 | 0.215 | | DP release | 0.6 | 1.107 | 0.169 | 0.219 | | DP release | 0.8 | 0.833 | 0.171 | 0.224 | | DP release | 1.0 | 0.669 | 0.175 | 0.239 | **Response to comparisons with alternative privacy-budget allocation strategies.** Thank you for this suggestion. We added three allocation baselines while keeping the DP mechanism, action space, Gaussian noise calibration, zCDP accountant, global $(\epsilon,\delta)$ budget, models, and test examples identical: \- Request-level: selects one privacy tier for the entire request; \- Equal-split: divides the global privacy budget uniformly across private candidate nodes; \- Greedy: selects the action with the largest immediate predicted gain without reserving budget for future nodes. | Allocation policy | F1 ↑ | SM ↑ | LaJ ↑ | AUC ↓ | MI ↓ | | ----------------- | ---- | ---- | ----- | ----- | ---- | | Request-level | .647 | .663 | .669 | .566 | .184 | | Equal-split | .661 | .678 | .695 | .574 | .187 | | Greedy | .672 | .681 | .723 | .607 | .193 | | Swim | .691 | .729 | .783 | .523 | .158 | Under the same formal privacy budget, Swim achieves both the highest answer quality and the lowest empirical leakage. Compared with Greedy, the strongest allocation baseline in answer quality, Swim improves F1, SM, and LaJ by 1.9%, 4.8%, and 6.0%, respectively, while reducing AUC and MI by $.084$ and $.035$. These results isolate the benefit of Swim's dependency-aware allocation: request-level and equal-split policies cannot exploit heterogeneous node-level privacy–utility returns, whereas Greedy can exhaust the budget on early nodes. Swim instead reserves and allocates privacy budget according to both the current return and downstream reasoning dependencies. **Response to the privacy guarantees of LLM-based sanitization.** Thank you for the question. Swim performs sanitization locally by replacing every field labeled private with a typed placeholder while retaining the task structure required for cloud-side decomposition. Before transmission, the sanitized representation is validated against a fixed schema that admits public task metadata, reasoning operators, dependency relations, and typed placeholders. We will add the exact sanitization prompt, the admissible output schema, and this formal statement to the appendix.Response to Authors
Thank you for the detailed response, I appreciate the effort the authors put into their rebuttal, but I fear my main concerns are confirmed rather than resolved, especially in the context of other reviewer's comments.
1. Privacy claim. You agree that routing and threshold gating depend on private evidence and are not charged to the accountant. Including them in an empirical leakage evaluation is not a formal guarantee, so "better quality under the same formal privacy budget" does not currently hold. Either account for these decisions in the guarantee or restate the claim as empirical only. The same applies to sanitization. It is described as replacing fields "labeled private," but real queries are unlabeled, so its recall remains untested, and the DP guarantee does not cover the query itself.
2. Where the gains come from. As far as I can tell, your tables show that a privacy-free cloud call already gives an actual gain of 0.205, and protected releases at the original settings add only about 0.01 to 0.03. Cutting the noise roughly 3x (ε from 0.2 to 0.6) barely changes the gain. Large gains appear only at per-release ε far above the global budget. The benefit therefore seems to come mostly from free calls, not from spending privacy.
3. Scope. I think the paper tries to do too much at once. In one paper you have a theoretical framework, a multi-component heuristic allocator, an unspecified sanitizer, three new datasets and a mobile demo, all evaluated end to end. This makes it really hard to follow and also each step requires sets of experiments and ablations etc. which cannot be effectively dealt with in one paper. I would advise the authors to maybe to split off some core ideas and write their paper around that, rather than doing too much. There are some nice ideas and implementations in this paper, but in my opinion there is too much going on. Other reviewers may disagree with this point, so it won't be my deciding factor, but this is just a point I wanted to note.
4. Presentation. Finally, I share some of the concerns about notation and exposition raised by Reviewer 7P2s. There is a lot of unclear notation with unnecessary amount of complexity. The problems seem pervasive and tied to the technical content, so I do not think they can be fixed in one revision round.
Response to privacy accounting for routing, threshold gating, and “privacy-free” actions.
Thank you—this concern reveals an ambiguity in our terminology. An edge action sends no request to the cloud. An r0 action sends only a cloud-safe payload containing the public node description and predecessor outputs already present in the cloud-visible history. Raw queries, private evidence, private slots, and unreleased predecessor outputs are excluded. Therefore, r0 has zero private-payload cost.
We agree that the invocation pattern may itself depend on private evidence. The cloud observes whether it is called and which tier is selected, while these decisions depend on the locally computed private-dependency score and threshold. In the reported implementation, the formal accountant composes the protected payload releases rℓ for ℓ≥1, but does not assign a separate ϵ to deterministic routing or threshold gating. These observable routing events are included in the metadata-only and full cloud-visible transcripts used in our empirical leakage evaluation.
Response to the DP configuration, privacy expenditure, and sources of utility gains.
Thank you for identifying the missing mechanism details. We agree that the original protected releases operate in a high-noise regime. Each of at most m=16 release units is clipped to C=1 and averaged using a fixed denominator, giving replace-one sensitivity Δ2=2C/m=0.125. We then apply a Gaussian mechanism calibrated through zCDP with atomic δ=10−6. The per-query privacy spend is obtained by composing the selected protected actions. We will add the sensitivity, calibration, noise scales, and composed privacy spend to the paper.
Table R1 confirms that, under the original configuration, the RMS noise norm of 5.35–26.38 exceeds the clean-state norm upper bound of 1. However, differential privacy limits distinguishability rather than making the release independent of the private state. For a release s~=s+z with zero-mean Gaussian noise, E[s~∣s]=s: the noise increases variance but does not remove the signal in expectation. Moreover, the RMS norm aggregates noise over all 64 dimensions, whereas task utility may depend on a smaller number of task-relevant projections. The cloud model can combine this residual information with the public node description and cloud-visible context. Consequently, the protected actions achieve actual gains of 0.214–0.239, compared with 0.205 for r0.
We also evaluate lower-noise settings. As the RMS noise decreases to 2.72 and 1.40, the maximum actual gain increases to 0.293 and 0.300, respectively. This confirms that reducing the perturbation improves the utility of protected releases, while the positive gains under the original configuration show that useful protected information remains recoverable despite the large noise.
Response to the clarity of the mathematical notation and exposition.
Thank you for this helpful feedback. We agree that the mathematical presentation can be improved. We will revise the paper to define all symbols before use, standardize notation across sections, clarify the assumptions and intuition behind the key equations, and separate the mechanism, action-selection rule, and privacy accounting more clearly. We will also add a concise overview to make the mathematical development easier to follow.