- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies the temporal authorization problem in full-duplex voice agents: users may change their minds after authorizing a tool call. The authors argue that a tool call should be delayed or dropped if the user's intent changes. They propose DuplexTxn, a policy that compares dispatch-time and commit-time representations and uses a gate to determine how to handle issued tool calls. They also introduce DuplexTxn-Bench to evaluate such scenarios. The authors implement the DuplexTxn policy on DuplexOmni, and the results show significant improvements in model performance on the target scenarios.
Critical strengths
- The problem studied in this paper could arise in real-world applications of full-duplex voice agents.
- The proposed method demonstrates significant performance improvements over the baseline.
Critical weaknesses
-
Questionable Setting: I think the biggest problem with this paper lies in its setting. The paper targets scenarios in which a user authorizes a tool call but later regrets the decision and attempts to revoke it. The core assumption of this setting is that there is enough time to revoke the tool call and that the tool call itself is revocable. However, sometimes tool calls are executed so quickly that there is insufficient time to revoke them. In other cases, tool calls are irreversible. For example, once a user authorizes a payment, the transaction may be processed instantly and cannot simply be revoked. Instead, it must be refunded, which requires a completely different tool call and may take considerable time. Therefore, I personally think that such scenarios should be handled differently: if a tool call has already been executed and the user wants to cancel it, the agent could issue another tool call to reverse its effects, if possible. For sensitive operations (such as flight bookings), the agent could explicitly ask the user for confirmation a second time and wait for final authorization.
-
Unclear Tool Call Decision Time: The core of the proposed method is the commit/wait/drop decision. However, there is no clear explanation of how long a tool call should remain in the waiting state before being committed or dropped. This is not merely about reporting a specific number; it concerns the fundamental design philosophy of the method. For example, if a tool call is delayed for too long, latency increases significantly. If the waiting period is too short, the user may change their mind after the tool call has already been committed. Therefore, it is unclear how the proposed method addresses this fundamental trade-off.
-
Critical Details Missing:
- Transaction States: The authors design 41 transaction states, but do not provide a comprehensive list of them.
- Benchmark Construction: Important details about the benchmark are missing. For example, what tools are included in the benchmark? How are the benchmark cases generated? There is no clear description of the construction pipeline or algorithm. Do the authors use LLMs or TTS systems to construct the benchmark?
- Unclear Metric Definitions: The paper does not provide rigorous mathematical definitions for all evaluation metrics, leaving several ambiguities. For example, what exactly counts as a "task predicate" in the Safe Completion metric? For the Missed Effect metric, over what time window is it calculated?
Questions for authors
N/A