acceptodds
Under review as a conference paper at ICLR 2027

Interface-Induced Trajectory Censoring: When Tool-Call Interfaces Distort Evaluation and RL Experience

Abstract

Tool-use evaluations typically measure calls through a serving stack, which can report zero calls even when model outputs contain call-shaped payloads. We study interface-induced trajectory censoring: a mismatch between model emissions and the interface that converts them into actions. On selected BFCL v4 subsets, changing only the serving adapter while holding model weights and other evaluation settings fixed moves scores under the official scorer from 0.00 to 0.96 on simple_python and from 0.00 to 0.19 on multi_turn_base. A template-parser factorial experiment shows strong complementarity: neither component replacement alone restores parsing from the baseline, whereas their joint replacement recovers 196/200 first-request parses. Across five Qwen2.5-Coder checkpoints, classifier-positive emissions increase with model size while the mismatched interface reports no calls; a separate matched-interface Qwen3 ladder exhibits few silent emissions through 8B. Experiments on 115 tau-bench retail tasks extend the mechanism to a stateful, non-code environment. In a preregistered, single-seed, 150-step 7B LoRA experiment, changing only the parser registered in verl increases tool executions from 10 to 16,844. However, neither a shared programmatic-feedback evaluation nor a post-hoc common native function-calling evaluation detects a held-out success gain from repair; rescue counts do not increase. Interface repair thus restores tool-mediated experience without a detected performance gain in this setting. These findings show that tool-use measurements depend on the model-interface configuration and motivate layer-wise observability and conformance checks before attributing evaluation or training outcomes to the model alone.

Primary area:
reinforcement learning
Keywords:
LLM Agents, Tool Use, Reinforcement Learning, Trajectory Censoring, Agent Evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_yeZEOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The individual Qwen incompatibility was reported previously in vLLM issue #32926. The authors' contribution is to quantify it as an evaluation-validity failure, decompose its causes, and follow it into interactive evaluation and Reinforcement Learning.

Critical strengths

The problem is easily ignored yet important: different root causes resulting in the same error really confuse humans sometimes. The motivation for solving this has real-world impact.

The experimental design focuses on single-factor interpretation, which can help rigorously identify the problems.

Critical weaknesses
  1. I am pretty sure AI largely generated this paper with little sign of human writing. (1) First of all, the core of the paper, i.e., the abstract, is completely generated by AI, as detected by GPTZero. (2) Figure 1 contains a watermark in the bottom left corner; based on my experience using vibe coding tools, I am pretty confident that AI generated the Figure without human revision. Even though people use AI to plot very often, polishing up the code to make it better is always necessary. (3) Some obscure descriptions, e.g., "Model emission", "Function Calling uses 2,048 output tokens". I am skeptical about whether the authors really put sincere effort into this paper.
Questions for authors

For BFCL, the authors said that 16,384 is the max_model_length, i.e., the input token plus the output token. But both parts are not specified; what exactly are they?

Reviewer_MagpOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper studies interface-induced trajectory censoring, where mismatches between LLM-generated tool calls and serving interfaces distort agent evaluation and RL training. Through experiments across multiple models and benchmarks, the authors show that repairing these mismatches can substantially improve measured tool-calling performance. They further investigate how these failures vary across model sizes and affect RL training, finding that parser repair increases executed training tool calls but does not yield a detectable improvement in held-out task performance. The paper advocates interface compatibility checks and layer-wise observability for more reliable agent evaluation and training.

Critical strengths
  • Important evaluation reliability issue: The paper highlights a practically relevant failure mode where interface mismatches can substantially distort measured tool-calling performance. The large performance differences observed on BFCL demonstrate why model–interface compatibility matters for reliable agent evaluation.

  • Controlled experiments and careful analysis: The paper includes useful component-level interventions, particularly the template–parser factorial experiment, and extends its investigation to RL training. The authors also distinguish restored tool execution from actual performance improvements and acknowledge the limitations of their findings, strengthening the credibility of their empirical analysis.

Critical weaknesses
  • Limited novelty and methodological contribution: The central finding that mismatched tool-call formats and parsers can distort evaluation is relatively well understood, and the main Qwen2.5-Coder incompatibility was previously documented. The proposed remedies largely follow established engineering practices, leaving the novel scientific contribution beyond characterizing known failures unclear.

  • Limited generalizability and downstream evidence: Despite multiple experiments, the strongest results focus on a specific interface mismatch in relatively old model families, with limited dataset and serving-framework diversity. The paper does not establish how prevalent these failures are in modern, properly configured agent systems. Furthermore, the RL evidence relies on a single training seed and shows no detectable downstream performance improvement despite restoring thousands of tool executions, limiting the strength and broader significance of its conclusions.

Questions for authors
  • Novelty and contribution: Given that the Qwen2.5-Coder parser incompatibility was previously documented and prior work has investigated model–harness interactions and evaluation failures, what are the key scientific or methodological contributions that distinguish this work from existing studies?

  • Relevance to newer models: Have the authors investigated whether similar interface-induced failures occur in more recent and capable tool-using or reasoning models? To what extent do the findings remain relevant to modern models with stronger tool-calling capabilities?

  • Generalizability: The strongest controlled experiments focus on Qwen2.5-Coder and a specific parser mismatch, while the scaling and RL studies primarily use single-tool coding tasks. What evidence supports generalizing these findings across different model families, more diverse tasks, multi-tool environments, and serving frameworks?

  • Practical prevalence: How frequently do these failures occur in realistic agent deployments using properly configured model-specific interfaces? Since the main Hermes baseline follows family-level rather than Coder-specific guidance, how representative is the observed mismatch of failures encountered in practice?

  • RL effectiveness: Despite restoring over 16,000 tool executions during RL training, no downstream performance improvement is detected. Can the authors provide additional analysis of whether the model actually uses the restored feedback to revise incorrect solutions, and explain why substantially increased tool interactions do not translate into measurable learning gains? How robust is this finding given the single training seed and limited training duration?

Reviewer_6EJ2OpenReview ↗
Rating
2(−1)
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies how mismatches between model emissions and tool-call interfaces distort evaluation and RL experience. It separates emission, parsing, execution, observation, and scoring through component interventions, fixed-output replay, and experiments on coding tasks, BFCL, and tau-bench. A single-seed, 150-step 7B LoRA comparison restores tool interaction after parser repair without detecting a corresponding held-out success improvement.

Critical strengths
  1. The paper provides convincing evidence that tool-use scores can conceal useful emitted actions. The BFCL template-by-parser experiment identifies component complementarity, while the separate offline replay recovers calls passing the official checker from 92/100 unchanged baseline single-turn responses (Appendix G.4). This distinguishes an evaluation failure from an improvement caused by generating different outputs. The tau-bench experiment extends the mechanism to a stateful, non-code environment. Together, these results make a useful contribution to the validity of tool-use measurement.

  2. The RL control changes only the registered parser and separates access to tool experience from task performance. The repaired arm reports 16,844 executions versus 10 in the broken arm, yet neither the shared programmatic-feedback evaluator nor the later common native function-calling evaluator detects a success gain (Section 5.5). The paper appropriately treats this as a bounded observation about one training setting. Establishing that these two outcomes can diverge adds useful evidence to the interface diagnosis.

Critical weaknesses

The quantitative scale result remains specific to one Qwen2.5-Coder/Hermes mismatch. Scale-specific classifier precision is unmeasured, and capped outputs prevent complete inspection of emissions at the largest checkpoint (Appendix F). The random-item replication supports the observed ordering but does not resolve these measurement limits. Likewise, the single-seed RL result gives limited evidence about the consequences for learning beyond the tested budget. These acknowledged limitations restrict the reach of the findings; they do not undermine the demonstrated evaluation failure or justify treating the null performance results as equivalence.

Questions for authors

Could you provide the access link for the anonymous verification package described in Appendix B? It was not located among the available submission materials. Access would allow assessment of the supplied fixed-output BFCL replay and accepted-lineage RL count checks, which are central to the measurement claims.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response

We thank all three reviewers for their time. We are glad that the reviewers found the problem practically important (Magp, yeZE), the template-by-parser factorial and single-factor design useful (Magp, yeZE), and the fixed-output replay and the separation of experience access from task performance convincing (6EJ2).

We have uploaded a revised PDF and the anonymous supplementary package. All changes are additions or clarifications; no reported number has changed. The pdfdiff shows every edit.

Anonymous verification package (6EJ2). The package described in Appendix B was omitted from the original upload. It is now attached as supplementary material. It contains the statistical code and inputs, the BFCL fixed-output replay bundle, the τ-bench trajectories, the RL accepted-lineage event checks, and the native-FC evaluation records. We checked it for identifying information before upload. New analysis: does the policy use the restored feedback during training? (Magp) Using the retained training rollouts of the repaired arm, we measured within-trajectory feedback use across all 16,844 executed calls. The model does act on feedback: after a failed first test it calls the tool again in 75.9% of cases, changes the code in 67.9% of those retries, and reaches a passing test in 14.4%. This in-training repair rate does not increase over the 150 steps (16.1%, 14.4%, 13.6%, 13.0%, 15.0% in 30-step bins), which is consistent with the absent held-out gain. The analysis is summarized in Section 5.5 and detailed in the new Appendix G.11 (Table 34). Presentation (yeZE, 6EJ2). All five figures are now vector graphics typeset in the paper's font. Figure 1 is redrawn in a larger 2×2 layout, and its explanatory footnote, which had been rendered at an unreadable size, is moved into the caption. We now gloss "emission" as "raw model output" at its first use in the abstract and introduction, and we added the exact BFCL token budgets (Appendix G.4). Scope (Magp, 6EJ2). We made the limits of the scale and RL claims more explicit in Section 6. The paper does not claim that this failure is prevalent in modern deployments, that censoring generally grows with scale, or that the null RL result shows equivalence.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.