When Is a Single Deployment-Time Pessimism Rule Too Coarse? A Post-Training Audit for Fixed-Backbone Tool-Use Controllers
Abstract
When a tool controller enters change control, its logged trajectories, representation, and value estimator can be frozen while the uncertainty coefficient used at action selection remains configurable. We identify the resolution of that coefficient—one global value or a decision-dependent field—as a post-training control variable. We hold the language model, logged data, state encoder, CQL value network, support statistics, validation choice, and evaluation protocol fixed, and change only this deployment-time rule. On a 15,000-outcome fixed-stack evaluation spanning GSM-8K-Tools, HumanEval-Exec, and NQ-RAG, StrataLCB, a monotone four-signal coefficient field, improves equal-environment mean success from to and the fixed ensemble-support Q1 success from to relative to Scalar-LCB. It also lowers the equal-environment unsafe-early-stop mean from to . A scalar-coefficient mismatch proposition characterizes why heterogeneous ideal penalties cannot in general be represented by a single coefficient. These results establish coefficient resolution as a measurable final intervention for fixed tool controllers operating over heterogeneous state–action support.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.