Check The Task, Not Just The Rules, When One LLM Agent Hires Another
Abstract
When one LLM agent hires another, the caller’s rules can be made binding by an executor that offers the model only the moves the rules allow. But keeping the rules does not show that the provider did the task it was asked to do. In this paper, we first show that a model at such a menu often substitutes a permitted operation for the one the request needs: asked to change an existing order, it tries to place a new one in 19 of 36 episodes, while every rule holds. Asked separately which tool the request needs, it names one that does not buy in 35 of 36 episodes, yet, shown that answer at the menu, it still tries to buy. Then, we propose name-then- check: the model names one tool, and the executor removes every branch that acts if the signed contract cannot reach that tool from the current decision. We build it on PROCPACT, a procedural contract whose rules we prove bind every action, including inside a procedure the provider copies in from a third agent; the check keeps this guarantee because it only removes moves. In one procurement world with three models, attempts fall to at most 1 of 36, while at least 35 of 36 purchases go through. The check tests whether the named tool is available from the current decision, not whether the task is done, and it declines some requests it reads literally.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.