acceptodds
Withdrawn from ICLR 2027

The No-Witness Rule: Making Fabricated Tool-Call Arguments Unreachable by Construction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Constrained decoding is the standard way to stop a tool-using agent from inventing the identifiers, paths and order numbers it passes to tools: restrict the argument to values the agent has actually seen. Every published treatment of a candidate set that cannot supply the needed value fires when that set is empty. We show the empty set is the wrong condition to test. Whether a set is ever empty is a property of how narrowly the slot is typed rather than of the trajectory, and neither end of that choice rewards testing it — under the type function we run it never arises, and under the strictest one we price it fires on 39.85% of emissions while destroying 8.39 correct values for each fabrication caught. The condition that arises regardless is a candidate set that is non-empty and does not contain the value, 10.09% of referential emissions, and nothing treats it. There the constraint requires the model to choose a wrong value, the tool fails, no witness is added, and the next step repeats the last: a livelock we reproduce outside our own system with a small model, a synthetic loop and VLLM's guided_choice, where an abstention token sat in the candidate set for all 240 constrained emissions and was chosen 0 times. The defect is the trigger, not the enforcement. Testing whether the value the model wrote is licensed cuts unwitnessed emission from 9.00% to 2.45% at 8B and 13.03% to 2.39% at 14B, constrains 8.43% of call steps, and is equivalent to the baseline on task score within this benchmark's own 7-point resolution floor; the published prompt-based method for the same failure clears neither bar. What such a rule is worth is a property of the environment rather than of the model: where referents are validated it buys back a wasted call, and where they are not it buys the difference between a wrong write and no write.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Tool-calling agents, Constrained decoding, Hallucination mitigation, Witness-based validation, Referential integrity

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_Gsx5OpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a method for preventing LLM agents from using unsupported identifiers in tool calls. The authors argue that existing constrained decoding methods can either allow unsupported values or restrict valid tool calls, leading to lower task success rates. NoWit allows agents to generate tool calls freely and then checks whether the referential arguments have appeared in previous observations, such as user messages, tool responses, or documentation. If an argument is not supported, the tool call is blocked, and the agent is guided to retrieve more information or ask for clarification. The results show that NoWit reduces unsupported tool arguments and state-changing calls compared with unconstrained generation, while maintaining comparable task performance.

Critical strengths

1.The paper addresses an important issue in tool-using agents, where models may generate unsupported identifiers that lead to incorrect tool calls.

2.The paper compares different intervention strategies and evaluates NoWit across multiple seeds, model sizes, and environments. The results show consistent reductions in unsupported tool arguments while maintaining comparable task performance.

Critical weaknesses

1.The paper mainly evaluates whether tool arguments have appeared in previous observations, but this does not necessarily mean the arguments are correct. While NoWit reduces unwitnessed emissions, task success shows little improvement. It is unclear whether the proposed method actually improves tool-call correctness and overall agent reliability.

2.The paper lacks a comparison with alternative recovery methods using the same provenance check. For example, would ordinary prompting or retrying after detecting an unsupported argument achieve similar results? Without this comparison, it is difficult to assess the additional benefit of the proposed constrained recovery mechanism.

Questions for authors

1.Would a simpler approach that uses the same detector but allows ordinary prompting or retrying achieve similar results? Such a comparison would help clarify the contribution of the proposed recovery mechanism.

2.How does NoWit handle valid arguments that have not appeared explicitly in previous observations? For example, an agent may need to derive a file path or construct an identifier based on available information. Could the witness check incorrectly block these calls, and how often does this happen in the evaluation?

Reviewer_kHTCOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper introduces NoWit, which checks whether identifiers have previously appeared in agent observations and can withhold the unwitnessed identifier calls. This prompts the agent to retrieve missing information or ask for clarification. The paper shows experiments that show fewer false identifiers while maintaining task performance within a predefined tolerance.

Critical strengths

The controlled comparison holds drafting and recovery fixed while varying the intervention trigger, providing evidence that checking proposed values offers a better balance between reducing unwitnessed identifiers and maintaining task performance. All tests have relevant statistical significance tests, so that it statistically supports the conclusions made.

Critical weaknesses

The biggest weakness is the framing and phrasing of the paper. This current version is very hard to read and understand the paper, because it contains text that reads as though it were LLM-generated. It uses many specialized terms that are not well defined. The abstract also includes too many unnecessary low-level details, such as the statistical result “paired difference −6.12 pp, [−8.44, −4.09].”

The related work section should provide a higher-level discussion of the paper’s novelty. For example, the statement “it is the reason Proposition 1 is necessary rather than decorative. Structured-output constraints can suppress tool invocation outright” refers to Proposition 1 before it has been introduced.

More detailed results should be organized into tables or figures to make the main text more readable. For example, the information in “their task differences from PERMISSIVE are −11.5 pp [−18.5, −4.5] and −24.5 pp [−31.0, −18.0]” could be presented in a table.

Figure 2(a) is unclear: what do the directions represent?

Questions for authors
  1. “equivalence uses the pre-stated ±7-point margin and a 90% interval, since absence of significance is not equivalence”: How sensitive are the candidate-based comparisons to extractor design? Could a stronger extractor or one using the same matching rules as the witness predicate reduce unnecessary interventions and narrow the performance gap? This threshold needs clarification.

  2. How broadly do the results generalize across models families? Both evaluated model sizes belong to the Qwen3 family.

  3. Have you manually evaluated cases where an observed identifier refers to the wrong entity, or where an unobserved value is legitimately derived from prior observations? Manual assessment could test how well the simple rule captures the problem of using identifiers without sufficient evidence to support their use.

Reviewer_bhXhOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work investigates referential arguments in tool-using LLM agents and proposes NOWIT, a selective provenance-checking policy that verifies drafted tool-call arguments against previously observed information before execution. When an argument lacks supporting evidence, NoWit blocks the call and invokes constrained recovery. Experiments on BFCL v4 and τ2\tau^2-bench demonstrate consistent reductions in unwitnessed referential errors while largely preserving task performance across different model sizes and settings. The authors also provide a detailed analysis of the additional execution costs and associated trade-offs.

Critical strengths
  1. Simple and effective method. The proposed NoWit method is straightforward and efficient, by checking the drafted arguments directly enables more selective interventions. It shows substantial and consistent reductions in unwitnessed referential emissions while largely preserving task performance.

  2. Comprehensive experiments supporting the main claims. The paper provides detailed experiments and analyses to support its main claims. The candidate-coverage analysis demonstrates that non-empty candidate sets do not necessarily guarantee reference coverage, while the trigger ablations show that overly strict constraints can substantially degrade task performance. Additional analyses of stopping behavior, execution costs, and robustness across different settings further support the proposed selective intervention strategy.

Critical weaknesses
  1. Circular primary metric. The primary metric (unwitnessed rate) is computed with the same W0 predicate that NoWit uses as its gate, so a large reduction is expected largely by construction. It therefore remains unclear whether the reduction translates into more reliable agent behavior. Human validation that “unwitnessed” values are actually fabricated would also help. In addition, the pre-stated +/- 7-pp equivalence margin is quite tolerant relative to a baseline task score of about 35%.

  2. Limited comparison of intervention strategies. Although the paper compares NoWit with RESTRICT and different Empty-Post configurations, these comparisons do not fully isolate the benefit of witness-based checking over candidate-membership checking. In particular, the paper does not examine a post-hoc candidate-membership strategy under the same free-drafting and recovery settings.

Questions for authors
  1. Post-hoc candidate-membership checking. The Empty-Post controls trigger recovery when a required candidate set is empty, but not when a drafted argument falls outside a non-empty candidate set. Could the authors discuss how NoWit would compare with a post-hoc candidate-membership strategy that triggers recovery when the drafted argument is missing from the candidate set, while keeping the drafting and recovery procedures unchanged?

  2. Presentation and experimental details. The submitted PDF seems to be truncated, making the referenced appendices inaccessible. Could the authors provide the complete appendix and clarify the experimental configurations? In particular, more context on RESTRICT, Empty-Post, and the PERMISSIVE/SHAPED/STRICT settings would help readers understand the differences among the compared baselines. And it will be helpful to describe it in the main text also.

Reviewer_h1NFOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

NoWit checks each eligible referential argument of a drafted tool call before execution. If a value has no verbatim source (a witness) in user turns, tool returns, documentation or schemas, NoWit withholds the draft. The model then re-decodes into a read-only lookup (RESOLVE), a clarification (ASK) or termination. On BFCL v4 (multi-turn tool use) with Qwen3-8B, the share of referential values without a witness (the unwitnessed rate) falls from 8.65% to 2.53%.

Critical strengths
  • With drafting and recovery fixed, the empty-set gate (recovery on an empty candidate list) never fires under PERMISSIVE typing. Under STRICT typing, it cuts the task score from 33.0% to 8.5%, and 2098 of its remedies fire on wholly witnessed drafts. This is a clear negative result for empty-set gates.
  • The paper uses paired bootstrap intervals and tests equivalence against a pre-stated ±7-point margin. The drop in the unwitnessed rate repeats over three seeds, a second site and 14B. Over 800 tasks, the task difference is +1.38 percentage points [−1.75, +4.50] , so the claim of no large task harm is credible.
Critical weaknesses
  • The gate and the audit use the same exact-match predicate W0, so the drop in the unwitnessed rate is largely by design. Table 2 says that the τ2-bench retail zero "follows from aligned construction and audit", so the 38-to-0 fall in refusals on unwitnessed values is automatic. Outcomes independent of W0 show no reliable gain: the BFCL task score is 35.0 against 36.0, and the retail reward is 0.04 against 0.05. The main text does not measure whether NoWit prevents wrong writes (calls that change state).
  • The paper states that a same-detector ordinary-repair arm "remains the comparator needed". The "Detect, then refuse" arm scores 67/200 against 72/200, but its unwitnessed rate is unknown because the endpoints "were not archived". The stop-and-re-verify policies of the closest prior work (Zhang et al., 2026b) are not a baseline either. So the experiments do not show what the constrained recovery adds to the detector alone.
Questions for authors
  • On BFCL, how many writes in each arm use a value absent from the reference solution?
  • In retail, how many environment refusals and infrastructure errors does each arm have in total?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.