acceptodds
Published as a conference paper at NeurIPS 2026

To Call or Not to Call: Diagnosing Intrinsic Over-Calling Bias in LLM Agents

Wei Shi, Ziheng Peng, Sihang Li, Xiting Wang, Xiang Wang, Mengnan Du, Na Zou

Abstract

LLM agents exhibit a consistent tendency to over-call, invoking tools even in situations where none is needed. On the When2Call benchmark, six models from three families show high call accuracy but much lower no-call accuracy, leaving overall accuracy in the 55%–70% range. We trace this to an Intrinsic Bias Hypothesis (**IBH**): the call/no-call decision mapping carries an activation-independent CALL offset, so the model favors CALL even at activation parity. Using Sparse Autoencoders (SAEs), we recover behavior-aligned feature bases for the CALL/NO_CALL decision, reduce them to a signed activation margin, and estimate the offset directly. Across all six models, the model is decision-neutral only when NO_CALL activation outweighs CALL activation, consistent with IBH. We then causally test IBH with Adaptive Margin-Calibrated Steering (**AMCS**), a closed-form counter-bias shift along SAE decoder directions. Cancelling the diagnosed offset mitigates over-calling and improves overall accuracy with a negligible drop in call accuracy. Our work recasts over-calling from an empirical phenomenon into a mechanistic object amenable to causal correction. The code is available at https://anonymous.4open.science/r/agent-sae-904F.

Related papers

See more related submissions when the market opens.