- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper investigates task intent leakage when a local small language model consults a cloud-based large language model. PriCon decomposes task intent into task context and task operation. It first constructs a domain-agnostic representation, then introduces alternative mathematical formulations according to the protection scope, and finally generates task narratives across different domains. The cloud model returns a solution blueprint, which the local model reconstructs and executes using the retained mappings. Experiments on 100 tasks show that Hit@1 is 0% when the context, operation, or both are jointly protected. Meanwhile, all model–setting combinations reported in Table 3 achieve a 100% subtask success rate.
Critical strengths
NA
Critical weaknesses
1.The key method is not established as a verifiable general procedure, and the specific reduction example is insufficient to substantiate the effectiveness of a cross-task framework.
The core of PriCon relies on a local model to construct an alternative mathematical formulation, establish a recovery mapping , and determine whether privacy is preserved and the original task remains recoverable. Section 4 explains the objectives of these steps, but the main text does not sufficiently specify how candidate formulations are constructed, how mappings are validated, or how failures are identified for general tasks. Section 4.2 requires a local reviewer to assess whether the original intent can be inferred and whether the solution can be recovered, yet it does not establish the reliability of these assessments themselves. Consequently, the critical conditions determining whether the method works are still delegated to unverified local judgments. While local trust can define a trust boundary, it does not automatically guarantee the correctness of the local model's mathematical transformations or privacy assessments.
The hierarchical DAG example in Section 4.1.2 is valid under the assumptions that different supply types are independent, integer allocations are bounded, and the objective function is additive. However, it is a dynamic programming representation tailored to a specific resource allocation problem. This example neither provides a common transformation rule applicable to all tasks nor explains how reliable mappings can be obtained for tasks lacking such a structure. Section 6 already acknowledges the difficulty of finding alternative formulations for complex operations, yet the main text still fails to clearly characterize the supported task classes and the boundaries of applicability. Reporting results across 10 task families cannot substitute for a concrete description of the transformation process and its correctness validation. Therefore, the current evidence does not establish that a locally feasible reduction example can serve as a trustworthy, general-purpose consultation method.
The technical contribution is also not adequately isolated. The Math-only and Semantic ablations merely demonstrate the changes resulting from adding an alternative narrative; they do not identify the independent value of the mathematical intermediate representation over direct, semantically rewritten formulations that remain recoverable. Since end-cloud transformation and local recovery paradigms already exist, the paper has yet to demonstrate the additional privacy benefits and gains in applicability attributable to its proposed mathematical stage.
- The privacy measurements, baseline coverage, and task correctness lack sufficient justification, leaving the experiments unable to substantiate the claimed privacy–utility advantages.
The privacy metrics are too coarse-grained and lack direct verification of sensitive information recovery. Table 2 relies primarily on Hit@1 and Hit@5 for the scenario, operation, and combined categories as evidence of privacy protection. The available materials do not provide sufficiently detailed label granularity or semantic scoring criteria for independent verification. Nor do they explain how partially recovered sensitive facts, objectives, and constraints are counted as leakage when the complete label is not retrieved. Existing studies provide more specific evidence of information recovery. InferDPT reports token recovery attack success rates, top- embedding inversion results, and n-gram leakage in cloud-generated outputs. PAPILLON calculates leakage based on annotated units of private information. EmojiPrompt compares attack-reconstructed texts with the originals using ROUGE, METEOR, and semantic similarity to examine meaning leakage that may occur even when the original wording is not recovered. The present paper reports neither entity-level leakage from the original text nor token-level recovery, and it does not establish an analysis of sensitive semantic unit recovery that covers these risks. Although intent protection requires evaluation at a higher semantic level than lexical matching, this does not eliminate the need to examine partial recovery and residual sensitive values. Failure to retrieve a coarse-grained label cannot automatically be interpreted as successful privacy protection.
The paper's own results also raise concerns about the interpretation of its metrics. Section 4.1.1 specifies that scenario protection preserves the operation, yet the operation Hit@1 scores in Table 2 drop from 98%, 95%, and 99% to 31%, 59%, and 56%, respectively. The paper must rule out the possibility that the underlying computational objective has been recovered but incorrectly scored because the original domain was not explicitly reproduced. Direct cloud consultation provides the original task to the cloud, yet its Hit@1 remains below 100%, indicating that inference failures can occur even when the information has already been disclosed. For a mechanism that deliberately constructs alternative narratives, the current evidence does not establish protection against inference by an adversary aware of the defense mechanism. Nor can zero hits be interpreted as zero leakage.
The existing comparisons do not adequately cover complete end-cloud privacy-preserving methods. The main results compare direct consultation, sensitive-value removal, and one decoy baseline. They do not report results for complete transformation–cloud processing–local recovery methods such as InferDPT, PAPILLON, and PRIV-QA, nor do they cover the decomposition-based obfuscation approach of ConfusionPrompt. Removing sensitive values once is not representative of the privacy protection and recovery capabilities of these methods. Although the native privacy objectives of different approaches would need to be harmonized before a fair comparison could be made, the current experiments do not establish such a comparison. Consequently, they are insufficient to support a broad claim of privacy–utility superiority and cannot determine whether the reported gains stem from mathematical reformulation or from the overall recovery pipeline.
The reported 100% utility lacks a reliable foundation in task correctness. The publicly available regular-expression task claims that the regular expression a*b.c+ should match aaabc, although it does not actually match that string. The two task families involving budget allocation and cross-domain composition contain 20 tasks in total that require net present value (NPV) calculations, yet neither the discount rate nor the cash-flow evaluation horizon is specified. The hospital budget allocation task does not even provide project cash flows. Lines 402–403 use direct cloud consultation as the correctness reference without establishing how errors in the reference outputs or incomplete inputs are handled. Consequently, the uniformly reported 100% utility in Table 3 cannot be verified from the public materials as evidence of functional correctness on the original tasks. It therefore cannot serve as a reliable basis for the claim that PriCon preserves privacy without sacrificing utility.
Questions for authors
1. Does the core reformulation have clearly defined applicability conditions and validation criteria? Beyond the known allocation examples that can be reduced, what establishes the correctness of the alternative representations and the mapping ? What independent evidence substantiates the local reviewer’s judgments regarding privacy and recoverability? Does the current cross-task effectiveness fundamentally depend on task-specific transformation knowledge?
2. How does Hit@k account for partial recovery of sensitive information? How does the current scoring mechanism handle residual original entity mentions, token-level recovery, synonymous semantic recovery, and the recovery of sensitive targets and constraints? Why does the hit rate decline substantially even when operations are preserved by definition? This question directly concerns whether Table 2 measures the recovery of sensitive operations or merely the reproduction of the original narrative.
3. What evidence establishes the independent advantages of mathematical reformulation over existing approaches? Neither comprehensive end-to-cloud protection nor local recovery methods are included in the main results, and the current ablation studies do not isolate the benefits of the mathematical reformulation stage relative to direct semantic rewriting. How does the existing evidence rule out the effects of the complete recovery pipeline and additional reasoning?
4. How do deficiencies in the public tasks affect the success criteria reported in Table 3? How are incorrect expected regular expressions and missing required inputs in 20 NPV tasks handled during actual evaluation? What actual inputs and scoring criteria substantiate the reported 100% functional success rate?
4 comments
Response to Reviewer MAKn (Part 2/4)
W2-1: Privacy Metrics and Task-Intent Evaluation
We clarify two key points regarding our privacy evaluation. First, Hit@k is a task-intent-level metric, measuring the attacker's ability to reconstruct the original task context and operation, rather than token-level leakage or individual sensitive-value exposure. Second, the decrease in Operation Hit@1 under context protection does not indicate a change in the underlying mathematical structure; rather, it reflects the loss of contextual clues needed to infer the original task-level operation. We elaborate on these two points below.
1. Task-intent recovery metrics. Our goal is to evaluate whether the cloud attacker can reconstruct the original task intent, rather than reproduce its exact wording. The attacker generates five ranked candidate task decompositions from the complete cloud-visible transcript. Two semantic judges compare each predicted subtask with its ground truth: the context judge checks whether the original application domain or scenario is recovered, while the operation judge checks whether the primary computation or decision objective, together with its relevant target, is recovered. Semantically equivalent expressions are accepted, but generic descriptions such as allocation or optimization alone are insufficient. Joint inference requires recovering the correct context–operation pair for the same subtask within the same candidate.
Hit@1 measures the recovery rate of the first-ranked candidate, while Hit@5 takes the highest recovery rate among the five candidates. Since tasks may contain multiple subtasks, partial recovery is reflected proportionally. We emphasize that Hit@k is a task-intent-level metric, rather than an exhaustive measure of token-level or individual sensitive-value leakage. Therefore, 0% joint Hit@1 means that the attacker fails to jointly reconstruct the original context–operation pairs under the evaluated attack, not that all residual information leakage is eliminated.
In the revised manuscript, we have clarified this metric interpretation in Section 5.1 (Privacy metrics) and provided detailed attack protocols, semantic judging criteria, and metric computation in Appendix C.
2. Why does operation Hit@1 decrease under context protection? This is an important distinction. Although context protection preserves the underlying computational structure, context and operation are not entirely independent in natural-language tasks. Domain-specific entities and application scenarios provide clues about the concrete operation and its target. Once these contextual clues are replaced, the attacker may still recognize a general computational pattern but fail to recover the original task-level operation.
For example, PriCon transforms an asteroid-orbit analysis task into a reservoir-discharge analysis task:
| Subtask | Original Subtask | Rewritten Subtask | Attacker's Actual Guess |
|---|---|---|---|
| 1 | Convert celestial coordinates into decimal degrees. | Convert reservoir crest-height readings into decimal feet. | Compute decimal crest heights from gauge readings. |
| 2 | Calculate angular displacement and angular velocity. | Compute the total change in stored reservoir volume. | Compute total volume change over 20 days. |
| 3 | Compute tangential velocity from angular velocity and observation distance. | Compute average daily volume change and related outflow. | Compute average daily volume change. |
| 4 | Estimate the orbital semi-major axis using the orbital-energy equation. | Estimate peak reservoir outflow capacity using a hydraulic equation. | Estimate peak outflow capacity using the orifice equation. |
As shown above, the attacker produces plausible reservoir-related guesses for all four subtasks, but these differ substantially from the original astronomical computations. For example, estimating reservoir volume changes or peak outflow capacity does not recover the original calculations of angular displacement, angular velocity, or orbital semi-major axis. Thus, the decrease in Operation Hit@1 does not mean that context protection changes the underlying mathematical structure; rather, it reflects the loss of contextual clues needed to infer the original task-level operation.
Thank you for pointing this out. We have added a discussion of this phenomenon in Section 5.2 (Page 8, Task-intent protection) of the revised manuscript.
Response to Reviewer MAKn (Part 3/4)
W2-2: Baseline Coverage and the Role of Mathematical Reformulation
Since task-intent protection in local–cloud LLM consultation remains relatively underexplored, we compare PriCon with representative existing protection strategies. Our results demonstrate that PriCon provides stronger task-intent protection while preserving cloud-assisted utility. Moreover, mathematical reformulation is not merely a rewriting step, but an important intermediate representation that enables broader protection capabilities and cross-domain experience reuse. We discuss these aspects below.
1. Baseline comparisons. Research on task-intent protection in local–cloud LLM consultation is still relatively limited. We therefore select representative methods from two related directions: PAPILLON for sensitive-value protection, and GS-Chaff and PrivShield for decoy-based obfuscation. These baselines are introduced in Section 5.1 (Page 8, Comparison settings). All methods are evaluated on the same 100-task benchmark, using the same local SLMs, cloud LLM, attacker, and task-intent metrics, under the same end-to-end consultation and local recovery setting.
For a concise comparison, the following table reports Hit@1 (%) with GPT-4o-mini, using the results from Table 2.
| Inference Target | PAPILLON | GS-Chaff | PrivShield-5 | PrivShield-20 | PriCon-Context | PriCon-Joint |
|---|---|---|---|---|---|---|
| Context | 94 | 13 | 10 | 6 | 0 | 0 |
| Operation | 99 | 27 | 30 | 18 | 31 | 0 |
| Joint | 94 | 12 | 7 | 3 | 0 | 0 |
PAPILLON, the sensitive-value-protection baseline, yields high Hit@1 scores of 94–99%, indicating that hiding sensitive values alone does not prevent the cloud from inferring task intent. In contrast, PriCon-Context reduces context Hit@1 to 0%, while PriCon-Joint further protects the operation and achieves 0% across all three inference targets.
Both decoy-based methods, GS-Chaff and PrivShield, reduce task-intent inference but still leave residual leakage. For example, even with 20 decoys, PrivShield retains 3% joint Hit@1 and 18% operation Hit@1, whereas PriCon-Joint reduces both to 0%. The other two local SLMs exhibit the same overall trend, with minor variations in the exact scores.
Meanwhile, PriCon maintains 100% cloud-assisted subtask success across all three local SLMs. As discussed in Section 5.2 (Page 9, Utility and privacy-overhead trade-off), PriCon-Joint achieves comparable token usage to PrivShield-5 while providing stronger task-intent protection. PrivShield-20 incurs approximately 4.9× the token usage of PriCon-Joint but still leaves residual joint inference.
2. Why is mathematical reformulation important? Semantic rewriting can directly transform an original task into a coherent story from another application domain. When only context protection is required, semantic rewriting can serve this purpose. Our motivation for retaining mathematical reformulation is not that semantic rewriting alone cannot protect task context, but that the mathematical representation provides an important intermediate step for broader protection capabilities and subsequent experience reuse.
As illustrated in Fig. 2, when operation protection is required, PriCon builds on this mathematical representation to construct an alternative formulation that changes the apparent computation while preserving the necessary mathematical structure for local recovery through the private correspondence . Furthermore, as illustrated in Fig. 4, the same representation allows the local experience bank to match tasks from different domains based on their underlying mathematical structures, rather than superficial similarities between their semantic descriptions. This enables cross-domain experience retrieval and reuse.
Table 3 compares mathematical reformulation alone (Math-only) with the complete rewriting process (Semantic). For GPT-4o-mini under joint protection, Math-only reduces context and operation Hit@1 to 3% and 1%, respectively, while Semantic further reduces both to 0%, demonstrating the additional benefit of semantic rewriting. The experience bank is disabled in Section 5.2 and therefore does not affect these privacy results; its efficiency benefits through experience reuse are evaluated separately in Section 5.3.
We thank you again for this insightful observation. It has prompted us to clarify why mathematical reformulation is retained as an important intermediate step in Section 4.1.1 (Page 5).
Response to Reviewer MAKn (Part 4/4)
W2-3: Utility Evaluation and Benchmark Refinement
Thank you for carefully examining our utility evaluation and benchmark. We clarify the utility criteria and address the benchmark concerns below.
1. Utility evaluation and direct consultation reference. PriCon does not require the cloud LLM to return an executable file. A subtask is considered successful through either of the following two paths:
- Direct solution: The cloud LLM directly provides the final answer or executable code, which can be checked or executed locally to obtain the expected result.
- Local recovery: The cloud LLM provides a solution procedure or blueprint. The local SLM maps it back to the original task through the private correspondence , incorporates locally retained values, and completes the execution.
In both cases, success is determined by verifying the locally obtained result against the corresponding task requirements and reference answers.
We use direct cloud consultation as the utility reference because the selected tasks cannot be reliably solved by local SLMs alone. It provides a natural baseline for evaluating how well PriCon preserves the utility of cloud assistance while protecting task intent.
2. Benchmark Design and Refinement. Since task-intent protection in local–cloud LLM collaboration is still an emerging research area, publicly available benchmarks remain limited. We therefore constructed this benchmark using tasks that can be solved with cloud LLM assistance but cannot be reliably completed by the three evaluated local SLMs alone, making cloud consultation necessary. As an initial effort, some task descriptions and annotations may still require refinement, and we appreciate the reviewer for helping us identify these issues.
For tasks that do not require instance-specific numerical values, utility can still be directly evaluated by checking the returned answer or executing the provided code against the task requirements, as described in the first success criterion above.
We will further review and refine the benchmark by providing complete task specifications, fixed reference answers, and corresponding verification scripts to improve its reliability and reproducibility. We hope this benchmark can serve as a useful resource for other researchers and facilitate future studies on task-intent protection.
Response to Reviewer MAKn (Part 1/4)
We sincerely appreciate your thorough review and the many detailed and constructive comments. We have carefully considered your concerns and address them point by point below.
W1 & Q1: Automatic Reformulation and Validation
We understand your concern that a single DAG example is not sufficient to demonstrate a general task transformation method. In fact, this example is intended to illustrate how a recoverable alternative formulation can be constructed, rather than provide a universal reduction rule. The key idea behind PriCon is to automatically construct and refine recoverable surrogates, allowing local SLMs to obtain cloud guidance while protecting the original task intent.
To better clarify the key ideas and technical contributions of PriCon, let us briefly explain how the framework works.
1. Automatic reformulation and review.
As illustrated in Fig. 2, PriCon uses mathematical reformulation and semantic rewriting to transform the original task into an intent-obscuring surrogate. The local SLM automatically constructs the surrogate and its recovery correspondence M, without requiring manually designed transformations for each task.
A key challenge is ensuring that the rewritten task both protects privacy and remains recoverable. To address this, we introduce a local reviewer that checks whether the original intent is hidden and whether the necessary variables, constraints, dependencies, and recovery mapping are preserved. If either check fails, PriCon automatically revises the surrogate and repeats the review. The specific review criteria, inputs, outputs, and prompts are detailed in Appendix B.2.
2. Closed-loop verification and debugging.
Passing the review does not necessarily mean that the task can already be solved successfully. After cloud consultation, PriCon uses the private correspondence M to recover the cloud guidance and verifies the result against the original task. If execution fails, it generates a privacy-preserving diagnostic request and seeks further cloud assistance.
In this way, PriCon forms an automated, closed-loop refinement process without human intervention. Rather than solving the original task independently, the local SLM focuses on protecting task intent and recovering cloud guidance. When difficulties arise, it can seek further assistance from the cloud LLM, allowing it to progressively solve complex tasks beyond its standalone reasoning capability.
The local experience bank is optional and primarily improves efficiency, as evaluated in Section 5.3. It is disabled in our main privacy evaluation (Section 5.2), so the reported privacy results do not depend on manually prepared transformation templates or retrieved experiences.
3. Experimental evidence and why it works.
As highlighted in the Abstract, PriCon reduces cloud-side task-intent inference Hit@1 to nearly 0% while preserving cloud-assisted utility. Specifically, PriCon-Joint achieves 0% joint Hit@1 across 100 tasks and three local SLMs (Table 2). Following suggestions from other reviewers, we further evaluated mechanism-aware attacks under Kerckhoffs's principle. PriCon-Joint maintains 0% joint Hit@1 and Hit@5 even when the attacker knows the rewriting mechanism (Table 6). We also added 50 tasks from 10 new task families and evaluated DeepSeek-V4-Pro and Qwen-3.5 as attacking LLMs, obtaining 0% joint Hit@1 and at most 2% joint Hit@5 across both standard and mechanism-aware settings (Appendix D.1, Table 8).
Why does it work? PriCon replaces the original task with a coherent alternative story, shifting the attacker's inference toward the surrogate rather than the original context. Under joint protection, PriCon also changes the apparent mathematical operation. Since similar mathematical structures can arise from many application domains, the attacker faces a wide range of possible original context-operation combinations, even when the rewriting mechanism is known. Meanwhile, the private correspondence M allows the local SLM to recover and execute useful cloud guidance. This explains why PriCon can achieve strong task-intent protection across diverse task families while preserving cloud-assisted utility.
We hope these explanations and additional results help clarify PriCon's design and effectiveness. The overall workflow is illustrated in Fig. 4 (Page 7) and described in Section 4.2. To make the implementation easier to follow, we have also provided detailed procedures, including module inputs, outputs, prompts, and validation criteria, in Appendix B.