When Is a Single Deployment-Time Pessimism Rule Too Coarse? A Post-Training Audit for Fixed-Backbone Tool-Use Controllers
Desk rejected at ICLR 2027, so left out of the paper lists and the map.
Abstract
When a tool controller enters change control, its logged trajectories, representation, and value estimator can be frozen while the uncertainty coefficient used at action selection remains configurable. We identify the resolution of that coefficient—one global value or a decision-dependent field—as a post-training control variable. We hold the language model, logged data, state encoder, CQL value network, support statistics, validation choice, and evaluation protocol fixed, and change only this deployment-time rule. On a 15,000-outcome fixed-stack evaluation spanning GSM-8K-Tools, HumanEval-Exec, and NQ-RAG, StrataLCB, a monotone four-signal coefficient field, improves equal-environment mean success from to and the fixed ensemble-support Q1 success from to relative to Scalar-LCB. It also lowers the equal-environment unsafe-early-stop mean from to . A scalar-coefficient mismatch proposition characterizes why heterogeneous ideal penalties cannot in general be represented by a single coefficient. These results establish coefficient resolution as a measurable final intervention for fixed tool controllers operating over heterogeneous state–action support.
- Primary area:
- foundation or frontier models, including LLMs