- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies policy checks when one agent delegates a task to another. If the handoff misses a needed record, the recipient cannot decide. Two histories with the same input but opposite decisions form a delegation fork.
Contributions:
- A model-independent checker that finds forks.
- A minimum field repair and a short decision encoding (DSES).
- Experiments with eight vendor-based policies and four LLMs.
Critical strengths
-
A clean diagnostic that does not depend on any model. A fork is a pair of histories that no recipient can tell apart. It separates handoff gaps from model errors.
-
The checker is well validated. Mutation testing catches 50 of 53 modified policies. A separate predicate agrees with the monitor on 5,870 histories.
Critical weaknesses
-
The formal core closely matches known results on observability and sensor selection in supervisory control (Lin and Wonham, 1988; Yoo and Lafortune, 2002). The paper does not discuss this link. The repair is shown only on two small cases that can be solved by hand.
-
The model experiments are too small and too easy. The prompts are short JSON with at most two records and no distractors. Under Policy, permitted and forbidden requests share one input, so 0/48 only shows that models fail closed.
Questions for authors
- How does handoff sufficiency relate to observability?
- How do the results change if the model sees its own earlier actions?
- Can you test LLMs with free-text records, distractors, and the repaired handoff?