- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes TASK, a prompt-sanitization framework for cloud LLM inference. It detects sensitive spans with lexicons and patterns. It scores each span’s task relevance using BGE embeddings, with a local LLM adjudicating borderline cases. Each span is then assigned a sanitization level from L0 (keep) to L4 (delete). An iterative refinement loop runs recovery and attribute-inference attacks and tightens the spans with the best risk-reduction-to-utility-loss ratio until a privacy threshold τ=0.90 is met. Experiments on NarrativeQA, InfiniteBench, and CNN/DailyMail report average gains of 23.72% in quality and 8.64% in privacy over three baselines.
Critical strengths
The paper observes that spans of the same sensitive type can play different roles depending on the task. It turns this into a concrete procedure: task-conditioned level assignment, followed by attack-guided tightening that selects actions by risk reduction per unit of task loss. This is a practical and interpretable design. Explicitly separating utility preservation from residual-risk mitigation is a reasonable way to organize the privacy–utility trade-off.
Critical weaknesses
-
Table 4 is described as a disjoint development set, yet its τ=0.90 row is identical to the main test results in Table 5 for all eight Q/S values. Table 2’s GLM-4-9B results, which use the default model, report Q=0.8916, while Tables 1 and 5 report 0.9457 for the same setting. The per-domain Q values differ by 0.03–0.07, but the S values match exactly. The ablation reports yet another TASK result (Q=0.9080, S=0.9244). The release threshold and the evaluation metric also share the same formula. In addition, the paper does not say whether prompts routed to human review are excluded from the reported results. Together, these issues undermine trust in the headline numbers.
-
Text Redaction is a 2023 Azure NER service. Prompt Obfuscation targets system-prompt protection so it is a mismatched baseline. Only IntentAnony is recent and relevant. Closely related methods that the paper cites, including PAPILLON, IncogniText, Yang et al. (2025), and Hou et al. (2025), are not compared.
Questions for authors
Why does GLM-4-9B quality in Table 2 (0.8916) differ from Table 1 (0.9457) when it is the same configuration? Why are the τ=0.90 results on the “disjoint” development set (Table 4) identical to the test results (Table 5)?
4 comments
Thank you for the response. Three key concerns remain unaddressed:
Dev/test overlap. The τ=0.90 results on the “disjoint” development set (Table 4) are identical to the test results (Table 5) for all eight values. Do the two splits overlap, and on which split was τ selected?
Source of inconsistent results. Where does 0.8916 in Table 2 come from, and why does per-domain Q differ while S is identical? Also, which configuration gives the ablation result? Please state which run the reported numbers come from.
Threshold–metric circularity. The release threshold and the evaluation metric share the same formula. How do you rule out that TASK simply optimizes the metric it is evaluated on? Results on an independent privacy measure would help.
Thank you for the reviewer’s questions. TASK uses the same definition for its release criterion and its reported privacy score so that release decisions reflect the privacy risks being evaluated. Specifically, TASK applies sensitive information recovery and attribute inference attacks to the sanitized text and computes the privacy score from the attack outcomes. The threshold determines whether an individual sample meets the release criterion, while the mean reported in the table compares the overall performance of the methods under the same attack protocol. We therefore frame the reported privacy improvement as follows. Under the attack settings evaluated in this work, TASK reduces residual leakage risk relative to the relevant baselines. The accompanying , Recaall, and F1 scores show the downstream task utility associated with these privacy gains.
Thank you for the response. The dev/test split question remains unaddressed, and it is central to interpreting the main results. I will keep my score.
Response to the Reviewer
Thank you for the reviewer’s questions. Our response is as follows:
1. Correction of Results
The relevant results in the paper have been corrected. The final result should be 0.9457.
2. Additional Baselines and Evaluation Metrics
We have further added two relevant baseline methods:
At the same time, we added Recall and F1 as two additional evaluation metrics. The overall experimental results show that TASK improves Recall by 19.04% and F1 by 26.68%.
Specifically:
Compared with the method of Pilán et al., TASK improves the quality score by 6.25%, the security score by 13.17%, Recall by 9.73%, and F1 by 16.80%;
Compared with the method of Loiseau et al., TASK improves the quality score by 0.67%, the security score by 11.08%, Recall by 2.77%, and F1 by 5.63%.
These results show that, after introducing more representative privacy–utility trade-off baselines, TASK still achieves stable advantages in task quality, security, and answer information preservation.