- Rating
- 2
- Confidence
- 4(+1)
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper models privacy leakage in service-agent conversations as a mixed-motive game, introduces CI-Challenge, and tests prompt-optimized attacks across different incentives and conversation lengths on two benchmarks.
Critical strengths
I like the focus on service providers that want to complete the task while collecting extra data, and the payoff model makes those competing incentives explicit.
Critical weaknesses
-
My main concern is that collecting all required fields is counted as task completion, so the results do not establish that the attacker actually delivers the service. The comparisons also change more than the intended variable: each horizon uses a separately optimized strategy, and the utility-only baseline is not shown the protected fields, making the effects of conversation length and incentives hard to isolate.
-
The added contribution over ConVerse and earlier search-based privacy attacks seems modest without a direct comparison showing what the mixed-motive setup adds. The negotiation examples are interesting, but there are no controlled comparisons showing whether concessions and later re-requests work better than simply asking repeatedly.
Questions for authors
-
What supports treating disclosure of all required fields as successful service delivery, and do any experiments check whether the requested service is actually completed?
-
Do the main trends hold when the same attack strategy is tested across horizons and all incentive settings receive the same field information?
-
Have you compared with ConVerse-style attacks and earlier search-based approaches using the same models, tasks, and conversation budget?
-
How was the 78.1% completion rate in Section 4.3 calculated, given that the test split contains 166 tasks and Appendix B.5 specifies one episode per task?
5 comments
Re: Follow up with reviewer sriC
As the discussion period comes to an end, we would appreciate any further feedback from reviewer sriC and would be happy to address any remaining questions or concerns.
If our updated draft and rebuttal responses have addressed the reviewer's concerns, we would appreciate update of the review score.
We again thank the reviewer for their time and effort in helping us improve our work.
Re: Follow up with reviewer sriC
We kindly urge the reviewer sriC to acknowledge our rebuttal responses and updated draft, and consider updating their score if their concerns have been addressed.
As the discussion period comes to an end, we also welcome any remaining questions or concerns and would be happy to address them.
Follow up with reviewer sriC
We thank the reviewer for their time and effort in reviewing our work.
We are encouraged by the reviewer's positive reception of:
We appreciate the reviewer's feedback, which helped us improve our work in following aspects:
We further thank the reviewer for helping us improve the writing and clarity of our draft across evaluation setup and post-hoc analysis of negotiation strategies.
As the discussion period comes close to an end, we are happy to address any further questions from the reviewer. If the reviewer has no remaining concerns, we would appreciate reconsideration of the review score based on our updated draft and rebuttal responses.