- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper aims to measure the gap between a model's plan of action and actual tool calls it invokes. Outcome based benchmarks including tau-bench miss this since they solely focus on the final state. The authors propose a method of measuring this say-do inconsistency by checking what the model said it did against the tool call history.
The paper contributes an empirical study with 7 models in 3 scenarios producing 1799 conversations. The paper reports that 40.3% of tool calls in 10.1% of all conversations are claimed by the model as having completed the action without invoking the tool itself. The paper's findings are well validated since they use human-annotators in addition to their llm-as-a-judge pipeline and the replication on tau-bench conversations is a good way of showing that this problem exists in the real world.
Critical strengths
The problem is widely observed and needs a good metric, this paper does a good job in scouting related works and positioning itself. Outcome based benchmarks miss this failure mode, this blind spot could be exploited as a reward-hacking mechanism when benchmarks that measure similar metrics are used as RL environments. Related work is thoroughly presented and Table 8 positions this paper's contribution clearly against all other works.
The findings are technically correct and novel. Replication on tau-bench is a really good addition to this paper, it tests model outputs generated by a third party and find that over-promises exist in a popular benchmark as well.
Critical weaknesses
The paper is too dense to read, tons of numbers and details are overwhelming, especially in the introduction's first paragraph where space should be given to motivate ("why") and speak about broader applications ("so what") of the work. The paragraph on "Codebook freezing" is unnecessary in the main text, the space could be much better used to provide examples of the "four-step derivation procedure"
The biggest weakness for me is the structure (or lack thereof) in rigorously establishing which questions to nail down as proxies for the say-do consistency and then answering them methodically. The paper shifts in which questions it aims to answer quite a lot throughout the paper. Specifically I observe this in the results sections where the three observations use a lot of bespoke diction that doesn't connect to the original motivation of the paper. Examples of bespoke diction: "three refinement levels," "anchored escalation," "fabrication flag," "strata," "claim substrate," "rapport decoration," "candidate-adjacent records."
Authors do a good job in reporting where their methods and findings have limitations. However, there's a lack of clarity and scientific rigor throughout the paper which is not what is expected of ICLR papers. Stemming from my first weakness, the problem is widely known but this paper doesn't do a great job in explaining how their experiments answer their question.
Questions for authors
For the inter-rater six-category agreement to be 53.8%–62.5% is fairly low. The authors' defense is that the disagreements mostly fall between "anchored escalation" and "unverifiable" not across the over-promise line, but I would like to see an elaborated explanation as to why is this agreement as low as it is.
Many scenarios were built so the user demands contact with a supervisor or salesperson but the tools offer no way to do it. To what extent is the reported over-promise rate induced by this design, and would it hold if such tools are provided?