- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
TAPER control tool observations before they reach an LLM agent. It derive task information boundary, separates visible information from protected evidence, checks and repairs the visible view for privacy and utility and selectively release protected information when needed. Experiments on adapted AgentDAM, and AgentLeak tasks, using three backbones and two evaluators, report the highest joint privacy utility score across all settings.
Critical strengths
-
Treatment of the privacy utility tradeoff. Preserving protect evidence for later access addresses a practical limitation of irreversible minimization. Measuring unnecessary agent visible exposure alongside final output leakage also capture a important aspect of privacy that output-only evaluation miss.
-
Broad empirical support in the evaluated setting. Results span backbones and evaluators with component ablations, evaluator-agreement analysis and paired bootstrap comparisons. The analys provide meaningful support for the reported gains.
Critical weaknesses
- Task utility is insufficiently grounded in factual correctness and execution success.
The helpfulness evaluator receives the instruction, sensitive-item list and final action but no source records or reference result Appendix H.3. It cann't reliably distinguish a plausible response from one that correctly uses retrieved evidence. Moreover, the original benchmarks are converted into local ReAct tasks rather than evaluated in their original environments. Because Help directly contributes to HPM, this limitation affects the central privacy utility conclusion. Agreement between two LLM judges don't resolve the missing correctness evidence.
- The distinctive contribution is not fully isolated from adaptation and additional processing.
Several baselines undergo substantial changes including replacing MINIM’s trained scorer with prompted scoring and representing abstraction as [REDACTED]. The ablations support the usefulness of TAPER’s components, but there is no comparison against a simpler task-aware minimizer with comparable verification and inference resources. Thus, it is unclear how much improvement specifically requires protected-state recovery and selective release.
Questions for authors
-
Could you validate task success against source record, reference output and executable check and report whether the Help and HPM ranking remain consistent?
-
How does TAPER compares with a task-aware minimizer given comparable verification, and inference resources? Does this comparison preserve the advantage attributed to recoverability?
-
Since repaired views are forwarded without another check, what fraction still fails the privacy check, utility check? Figure 4 reports changes in flagged dimensions but don't distinguish these remaining failure types or their downstream consequences.
-
How does TAPER’s observation-level control relate to the reasoning based and RL contextual-integrity mitigations in [1], particularly in balancing privacy protection and task utility (tradeoff)?
[1] Contextual Integrity in LLMs via Reasoning and Reinforcement Learning. NeurIPS 2025.
2 comments
- Regarding the privacy and utility effects of Repair
Following your suggestion, we further analyzed the 429 repaired views in Fig. 4, separately examining the judgments of the Privacy and Utility Verifiers before and after Repair.
| Metric | Privacy | Utility |
|---|---|---|
| Failure rate before Repair | 44.52% | 86.25% |
| Failure rate after Repair | 41.49% | 50.58% |
| Percentage of initially failing views that pass after Repair | 51.31% | 44.05% |
| Percentage of initially passing views that fail upon re-evaluation | 35.71% | 16.95% |
The results show that Repair enables 51.31% of the views that initially failed privacy verification to pass re-evaluation and reduces the overall utility issue flagging rate by 35.66 percentage points. This indicates that Repair can correct some residual privacy exposure issues and improve the sufficiency of information required for the task. Regarding the specific impact of the remaining issues on subsequent task execution, we have not yet conducted a separate statistical analysis, and more targeted experiments may be needed in the future to investigate this impact.
- Similarities and differences with Contextual Integrity in LLMs via Reasoning and Reinforcement Learning
Both [1] and TAPER are inspired by contextual integrity (CI) theory and aim to balance privacy protection and task utility during task execution. Our Task Information Boundary similarly incorporates contextual factors such as the purpose of the task and information recipients as the basis for decisions about information access and disclosure. Similar to CI-CoT in [1], TAPER uses prompts to guide general-purpose LLMs in making privacy decisions without requiring additional training.
Regarding the privacy–utility trade-off, [1] primarily improves the model's information disclosure decisions, enabling it to make full use of task-required information while protecting sensitive information. TAPER, in contrast, focuses more on decision errors during information minimization and their recoverability. It corrects insufficient and excessive filtering through privacy and utility verification and allows the agent to recover protected information when it is genuinely needed later, thereby mitigating the loss of task utility caused by upfront privacy protection. The two methods improve the privacy–utility trade-off at different levels and are complementary.
Thank you for recognizing our work and providing constructive comments. We provide additional responses to your questions below:
Neither AgentDAM nor AgentLeak, the two datasets used in our study, provides standardized reference answers or expected actual execution outcomes for the tasks. We follow their evaluation methods and adopt LLM-as-a-judge to evaluate the outputs. Therefore, the Help evaluator indeed does not receive the original source documents, reference answers, or actual execution outcomes. We agree with your point that, without access to the original source documents or reference answers, the original Help evaluator may not be able to fully verify the factual correctness of the final outputs. To address this issue, we plan to combine LLM-based annotation with human review to establish a source-grounded reference answer for each task, specifying the key information that the final output should contain or the correct operation parameters. The evaluator will then assess task completion against the original evidence and fixed reference criteria, while checking for major factual errors or hallucinated content, to improve the credibility and reliability of task utility evaluation.
As described in Appendix A.3 and Table 3, the single-view ablation in Table 2 is a task-aware minimization method that retains the Task Boundary, Privacy and Utility Verifiers, and Repair. It uses the same budget as full TAPER, and we believe that the performance difference between this method and TAPER can demonstrate the contribution of recoverability.
Specifically, in the ablation experiment on 140 tasks, full TAPER achieves higher Help (2.19 vs. 2.16) and HPM (71.12 vs. 68.24) than Single-View Minimization. To further validate this comparison, we conducted an additional paired analysis on the full set of 336 tasks, performing 10,000 paired bootstrap resamples of the metric differences on the same tasks and calculating two-sided 95% confidence intervals.
Under the DeepSeek evaluator, compared with Single-View Minimization, full TAPER reduces the final privacy leakage rate (LR) from 2.38% to 0.60%, a relative reduction of approximately 75%, with a paired difference of −1.79 percentage points (95% CI: [−3.27, −0.60]). The paired differences in Help and HPM are −0.134 (95% CI: [−0.280, +0.012]) and −3.85 (95% CI: [−8.81, +1.11]), respectively. This indicates that full TAPER has an advantage over task-aware minimization alone in reducing final privacy leakage.
Regarding the effectiveness of recoverability that you raised, we further analyzed the execution logs of selective recovery. Among the 336 tasks, 24.40% initiated recovery requests, and 14.29% received non-empty releases. Of the 55 exact-release entries, 76.36% subsequently appeared in tool arguments or final outputs. These results indicate that the recovery mechanism can provide the evidence needed for subsequent task execution and that full TAPER has an advantage in reducing final privacy leakage. We will further verify the independent benefits of recoverability through experiments with strictly matched inference budgets.