- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how to bind a user’s approval to one specific action. Two actions may use the same tool, target, and arguments but belong to different attempts, so approval for one should not authorize the other. The authors propose EpochSelect, which requires an action instance to be created and shown before approval. Approval can only select an existing, valid instance, and the system checks its identity, current state, and final arguments before execution. Controlled tests show that this approach accepts valid restorations and rejects incorrect bindings under the stated trust and execution assumptions.
Critical strengths
- A meaningful safety contribution. The paper explains why identical action details do not imply the same action instance. Creating the instance before approval and checking it before execution provides a clear way to prevent approval reuse.
- Useful experimental evidence. Full binding accepts all 120 valid restorations and rejects all 360 incorrect bindings. In comparison, strict object identity rejects valid restorations, while matching only business values accepts incorrect bindings. This result shows that the proposed design can preserve valid approvals while rejecting substitutions.
Critical weaknesses
-
The evaluation does not establish usefulness in realistic conversations. In Study E, the intended action is selected by looking at a later confirmed action in the recorded conversation. The system therefore does not have to identify the correct action from the conversation itself. Even in this simplified setting, the repaired approval rule accepts only 25 of 64 positive examples (39.1%). These results leave both action selection and approval coverage unresolved.
-
The last page of the main text leaves unused space, while helpful explanations and comparison tables remain in the appendix. For example, the authors could move examples of accepted and rejected approvals and the key baseline comparisons into the main text. This would help readers understand how the method works and what the results actually show. The current organization makes readers repeatedly consult the appendix to assess important parts of the paper, weakening its clarity.
Questions for authors
Can you test the system on full conversations without choosing the intended action for it in advance?
2 comments
Thank you for running the full-prefix experiment without using later assistant actions to preselect the target. Across the 12 training-set turns, the guarded path made no selections, and the separate model-plus-capability path produced 10 state-changing calls. It remains unclear how many of those calls matched the specific preview the user approved, or how many valid approvals the guarded path missed. My concern about approval handling in realistic conversations therefore remains.
My comment about placing representative approval examples and key baseline comparisons in the main text also remains unaddressed.
Thank you for recognizing the value of instance-level approval binding. Your question about full conversations focuses attention on the central issue: how an agent can preserve the meaning of a user’s approval as the interaction continues.
Consider a travel agent that presents a cancellation proposal and receives approval. The session is saved and later restored. The user expects that approval to remain attached to the proposal they saw. A cancellation proposed on a later occasion belongs to a different event, even when its tool, reservation, and arguments are identical. A faithful continuation must preserve that distinction through the agent’s internal state changes.
EpochSelect gives this expectation an explicit runtime meaning. The native preview path establishes the proposal instance before approval; approval selects that existing live instance; execution rechecks its sources and arguments. Proposition 1 explains the ambiguity of value-only selection, and Contract C specifies the binding maintained under the stated trust and synchronous-execution assumptions. The controlled restoration study makes the consequence observable: valid restorations retain approval, while the tested nonexact rebindings are rejected. The added source-aware capability comparator matches this behavior in the post-approval, in-process comparison.
Following your request, we ran an additional full-prefix development experiment on 12 multi-preview user turns from exposed STATE-Bench training trajectories. The selector received the conversation through the current user utterance and every earlier native write preview. Later assistant actions were excluded from target construction. We then examined the original-utterance guarded path and a separate model-selected persisted-preview capability path against actual native tool effects.
With multi-pending admission explicitly relaxed, the guarded path selected no instance from the original utterances. The separate model-plus-capability path produced 10 state-changing native calls. This experiment follows the input condition you requested and makes the handoff from conversational interpretation to executable approval visible. Its reported outcomes are selections and native effects on recorded training prefixes; the benchmark’s final-state goals describe task completion at the task level.
The example and comparison above express the paper’s contribution in concrete terms: preserving user control requires an executable connection between a past interaction event and a present environmental effect. The contract, reference adapter, and additional dialogue experiment give agent research a correctness criterion and a way to examine that connection in actual tool execution.