Procedural Exposure Bias in Skill-Augmented Language Model Agents
Abstract
External skills provide language-model agents with reusable procedural knowledge, yet how models use such knowledge after it is supplied remains poorly understood. In this paper, we identify Procedural Exposure Bias (PEB), a systematic tendency of language models to increasingly favor an action as its associated procedural pattern is repeatedly exposed in a skill, even when the interaction state and the action's relevance remain unchanged. We find that PEB is strongly dependent on how procedural knowledge is represented: presenting the same procedural content visually substantially attenuates the exposure-induced shift in action preference compared with direct textual presentation. Further analysis shows that this attenuation primarily arises from visual encoding rather than structured visual layout. Although repeated procedural signals remain recoverable from Visual representations, they exert a weaker influence on downstream action prediction, revealing a distinction between signal accessibility and its use as action evidence. This representation-dependent effect also extends to executable action selection, where increasing procedural exposure more strongly favors the corresponding action under Text than Visual. Layer-wise analyses show that exposure-dependent action evidence becomes progressively more pronounced toward the model output, with stronger evidence under Text, while activation interventions reveal architecture-specific roles of internal components in transmitting or counteracting the resulting preference shift. Finally, complete-agent evaluations show that these representation differences persist during end-to-end interaction, with visual skills yielding higher overall task success than textual skills across the evaluated models. Together, these findings establish skill representation as an important factor governing how procedural knowledge is translated into agent decisions. All code and datasets will be released on GitHub.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.