acceptodds
Under review as a conference paper at ICLR 2027

Procedural Exposure Bias in Skill-Augmented Language Model Agents

Abstract

External skills provide language-model agents with reusable procedural knowledge, yet how models use such knowledge after it is supplied remains poorly understood. In this paper, we identify Procedural Exposure Bias (PEB), a systematic tendency of language models to increasingly favor an action as its associated procedural pattern is repeatedly exposed in a skill, even when the interaction state and the action's relevance remain unchanged. We find that PEB is strongly dependent on how procedural knowledge is represented: presenting the same procedural content visually substantially attenuates the exposure-induced shift in action preference compared with direct textual presentation. Further analysis shows that this attenuation primarily arises from visual encoding rather than structured visual layout. Although repeated procedural signals remain recoverable from Visual representations, they exert a weaker influence on downstream action prediction, revealing a distinction between signal accessibility and its use as action evidence. This representation-dependent effect also extends to executable action selection, where increasing procedural exposure more strongly favors the corresponding action under Text than Visual. Layer-wise analyses show that exposure-dependent action evidence becomes progressively more pronounced toward the model output, with stronger evidence under Text, while activation interventions reveal architecture-specific roles of internal components in transmitting or counteracting the resulting preference shift. Finally, complete-agent evaluations show that these representation differences persist during end-to-end interaction, with visual skills yielding higher overall task success than textual skills across the evaluated models. Together, these findings establish skill representation as an important factor governing how procedural knowledge is translated into agent decisions. All code and datasets will be released on GitHub.

Primary area:
interpretability and explainable AI
Keywords:
language model agents, skill utilization, procedural exposure bias

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jsqMOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies Procedural Exposure Bias (PEB), where repeated procedural content biases LLM agents toward particular actions regardless of their applicability. The authors show that representing skills visually can attenuate this effect and evaluate the finding through controlled experiments, mechanistic analyses, and three agent benchmarks.

Critical strengths
  1. Extensive experimental design. The controlled states, keyword-replacement experiments, and rasterized-text comparisons provide useful evidence for the observed bias.
  2. Comprehensive evaluation. The paper investigates multiple models and benchmarks, complemented by action-level and representation-level analyses.
  3. Interesting empirical observation. The difference between text and visual skill representations offers a potentially useful perspective on skill-augmented agents.
Critical weaknesses
  1. Unclear selectivity of visual attenuation. The reduced exposure bias under visual encoding may reflect a general weakening of procedural information's influence on action selection, rather than a selective mitigation of irrelevant exposure effects. The current experiments do not sufficiently distinguish these possibilities.
  2. Limited statistical support for practical improvements. Several end-to-end improvements are relatively small, and the main end-to-end results lack confidence intervals or statistical significance tests.
Questions for authors
  1. Visual attenuation: Does visual encoding selectively suppress irrelevant exposure-induced action preferences, or does it generally weaken the influence of procedural information on action selection? Could the authors provide controls involving relevant procedural information to distinguish these possibilities?
  2. Statistical significance: Are the end-to-end improvements statistically significant under paired evaluation?
4 comments

We sincerely thank Reviewer jsqM for the thoughtful and constructive suggestions concerning access to procedural information, relevant-action exposure, and statistical support. We appreciate the attention to distinguishing the reliability of our controlled PEB findings from aggregate agent outcomes.

Weakness 1. Visual attenuation may reflect general weakening rather than resistance to irrelevant exposure.

Reply to Weakness 1. The submitted Figure 3b shows recoverability of repeated procedural signals, while Figure 1 establishes weaker distractor-preference shifts under Visual. We extend these findings with Table R1, showing accurate complete-action recognition, and Table R2, showing a larger reference-action than distractor-action preference gain under Visual in the tested verb–object condition. Together, these controls support recognizable procedural content and differentiated responses to action relevance, reinforcing the interpretation of representation-dependent resistance to irrelevant exposure. They address the alternative explanation that reduced interference necessarily means that the model cannot recover or identify the relevant procedural content.

Weakness 2. End-to-end performance differences need stronger statistical support.

Reply to Weakness 2. The submitted Table 1 reports higher Visual macro-average task success across evaluated backbones. In addition, Tables R3–R4 provide paired confidence intervals for the central action-level exposure effect, and Appendix A.3.3 reports a paired bootstrap difference in executable-action exposure slopes. These analyses strengthen the statistical evidence for representation-dependent PEB, while Table 1 provides the complementary aggregate task-success comparison.

Dear Reviewer,

Thank you again for your valuable feedback. We hope that our responses have addressed your concerns and would greatly appreciate any further thoughts you may have.

We would be happy to provide additional clarification if needed.

Thank you for your time and consideration.

Area_Chair_BBCKOpenReview ↗

Feedback

Hi, the review deadline is approaching. Please take a look at the authors' rebuttal and post your feedback (and update your rating if appropriate) at your earliest convenience. Thank you! AC

Thank you for your response. However, it only partially addresses my concerns, and the experimental evaluation remains insufficient to convincingly support the main claims. Therefore, I will keep my rating unchanged.

Reviewer_WgAQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how language-model agents use procedural knowledge supplied in external skill documents. It asks whether repeatedly mentioning an action-related pattern in a skill shifts action preference regardless of whether the action applies in the current state, a property authors dub Procedural Exposure Bias (PEB). They also compare skills given as text with the same skills rendered as images and processed by the model’s vision encoder to verify the effect of modality on PEB.

The controlled evaluation focuses on successful trajectories from ALFWorld, an interactive household-task environment. A task-relevant but inadmissible distractor action is added to the candidate set, and an object noun tied to it is inserted between 0 and 8 times into fixed slots of the skill, with word count held constant. Preference is then measured by normalizing length-normalized action logprobs over the fixed candidate action set.

For text inputs, distractor preference rises with exposure to the repeated keyword across all tested models. Visual inputs are instead shown to attenuate the effect of repeated mentions. Additional ablations highlight how this affects the model internal processing via logit-lens readouts and activation patching. Evaluations spanning four models and three benchmarks also report higher average task success with visual skills.

Critical strengths
  • The controlled experiments proposed by the authors convincingly show that LLMs suffer from action bias from repeated exposure under controlled conditions. The proposed approach of varying object repetition while keeping the situation and candidate actions fixed is also well thought, since it helps disambiguate the effect of repetition from other factors that could influence the agent's choices.

  • The comparison of ordinary text with a plain image of that text is useful to understand model sensitivity to such biases across modalities. The rasterized-text condition is also a well-chosen control, since the main visual condition tests changes both the encoding pathway and the skill layout, so attributing the attenuated effect to one of these properties requires separating the two.

Critical weaknesses
  • The bias presented in this work is not very surprising. Models are already known to favor words they have seen repeatedly in their context (see e.g. [1]). Since the repeated word (e.g. “microwave”) also appears in the distractor action (“open microwave”), PEB looks like a version of this known effect. The more novel claim concerns the Visual input results, where the model still “sees” the repetitions but uses them less when choosing actions. In practice, though, the repetition signal is much weaker under Visual because the modalities differ, which could explain the weaker effect on actions. The pattern looks more convincing for Gemma-4-31B-it, but the paper does not compare the two measures on a common scale. In practice, current experiments do not show whether the model relies less on rendered text overall, so all skill content has less influence, not just repetitions.

  • The agent-level results are too small and too noisy to support the claim that visual skills work better because of reduced PEB. Visual beats Text by only 0.9-3.4 points on avg, with no repeated runs or error bars. The paper also never connects agent results back to PEB, lacking concrete examples of how often real skills repeat words tied to wrong actions, or whether Text failures come from such actions.

[1] Llama See, Llama Do: A Mechanistic Perspective on Contextual Entrainment and Distraction in LLMs (Niu et al., ACL 2025)

Questions for authors

Could you measure how strongly relevant skill content drives the correct action under each representation? If Visual input weakens skill influence overall, the result might indicate reduced sensitivity to rendered-text conditioning.

3 comments

We sincerely thank Reviewer WgAQ for the insightful comments on contextual entrainment, procedural accessibility, and the behavioral implications of PEB. We appreciate the opportunity to clarify how our state-conditioned skill setting differs from broader observations of repetition effects.

Weakness 1. PEB resembles contextual entrainment; Visual may merely weaken skill information overall.

Reply to Weakness 1. Llama See, Llama Do: A Mechanistic Perspective on Contextual Entrainment and Distraction in LLMs provides relevant context for repetition-dependent behavior. Our study examines this effect specifically in state-conditioned action selection with supplied procedural skills, holding the state, action applicability, and candidate set fixed.

The additional verb and matched-word-bag controls (Tables R3–R4) extend the effect to action-related expressions and confirm weaker distractor influence under Visual. Meanwhile, Tables R1–R2, together with the submitted Figure 3a–b, show that visual procedural content remains recognizable and that relevant versus irrelevant exposure can produce differentiated preferences. This combination clarifies how representation changes the use of accessible procedural information, rather than simply whether it can be read.

Weakness 2. Agent-level gains are variable, and PEB should be linked more directly to behavior.

Reply to Weakness 2. The submitted Figure 3c and Appendix A.3.3 demonstrate a representation-dependent exposure response in freely generated admissible actions. Appendix A.3.4 assesses subsequent execution consequences through paired continuations, while Table 1 shows higher Visual macro-average task success under default skills across all evaluated backbones. These complementary evaluations connect controlled susceptibility, executable action selection, and aggregate agent outcomes without conflating their metrics.

Dear Reviewer,

Thank you again for your valuable feedback. We hope that our responses have addressed your concerns and would greatly appreciate any further thoughts you may have.

We would be happy to provide additional clarification if needed.

Thank you for your time and consideration.

Area_Chair_BBCKOpenReview ↗

Feedback

Hi, the review deadline is approaching. Please take a look at the authors' rebuttal and post your feedback (and update your rating if appropriate) at your earliest convenience. Thank you! AC

Reviewer_SP5gOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper identifies that language models tend to favor repeated actions exposed in skills and defines it as Procedural Exposure Bias (PEB). The paper then studies if PEB affects action prediction independent of current decision in both text and image representation of skills. To further study why PEB has less effect in visual representation than text representation and examine where in a model PEB develops.

Critical strengths
  • Analyses show that LLMs seem to favor distractor actions when they are repeated exposed in skills and representing it in text form is more severe.

  • The format of image representation of distractor actions is not the main cause of less PEB when using images.

Critical weaknesses
  • Target keyword only includes the noun in an action, this design choice is not properly justified. It could happen that the noun in a distractor action also exist in a correct action, the distractor action probability can then contain noise. Analyses using verb and noun for counting should be reported.

  • PEB is attenuated when skills are represented as images, this may be the reason that images are more ambiguous than text, a model may 'understand' the distractor action as other actions. An analysis on how accurate models identify the actions represented by images will be helpful. Moreover, it will also be helpful to see the behaviour if repeated distractor actions are replaced with correct actions.

  • I'd assume the results in Table 1 are without distractor actions in skills. Although the author provide extensive analyses, they are all with controlled setting. How PEB affect in the real task setting is unclear. Results with distractor action on different exposure level in both text skill and image skill on all dataset is needed to see the PEB effect in real tasks.

  • It is also not clear if PEB can be recovered by finetuing language models with sft and/or RL finetuning.

Questions for authors
  • What if assigning confidence scores to the actions, i.e. the correct actions have high confidence and the distractor actions have low confidence? How will it affect PEB?
4 comments

We sincerely thank Reviewer SP5g for the careful and constructive suggestions on linguistic controls, action recognition, and executable agent behavior. We appreciate the emphasis on directly testing action-level procedural patterns and their consequences. The additional experiments extend our analysis along these dimensions.

Weakness 1. Noun-only exposure may be ambiguous; verb–object exposure should be examined.

Reply to Weakness 1. We extend the original probe to verbs and contiguous action expressions (General Response, Weakness 2). Table R3 shows smaller Visual distractor-preference increases under verb repetition, while Table R4 demonstrates an action-expression effect beyond unigram counts, with weaker sensitivity under Visual. Thus, the finding is not limited to repeated object nouns.

Weakness 2. Visual attenuation may reflect ambiguous action recognition; correct-action exposure should also be tested.

Reply to Weakness 2. Table R1 demonstrates accurate recognition of complete actions from the tested visual procedure cards. Table R2 further shows that, under verb–object exposure, the reference action gains more preference than the state-inapplicable distractor within Visual. These complementary controls support action accessibility alongside relevance-dependent exposure responses, strengthening the distinction between reading a skill and using repeated content as action evidence.

Weakness 3. The impact of exposure levels on complete-agent behavior remains unclear.

Reply to Weakness 3. The submitted Figure 3c and Appendix A.3.3 directly test exposure-dependent selection of freely generated admissible actions, including a paired slope comparison. Appendix A.3.4 studies downstream effects through paired continuations, and Table 1 evaluates complete-agent performance under default skills. Together these analyses connect exposure-sensitive decisions to execution while appropriately distinguishing action-level exposure responses from complete-episode success.

Weakness 4. Could SFT or RL mitigate PEB?

Reply to Weakness 4. SFT and RL offer complementary directions. Our current experiments keep model parameters fixed to isolate the effect of skill representation. Training-based interventions could build on this design to study how learned policies regulate repeated procedural evidence without conflating training changes with representation changes.

Weakness 5. Could action-confidence annotations influence PEB?

Reply to Weakness 5. Confidence annotations could provide additional reliability cues. A controlled comparison of confidence aligned or misaligned with current-state applicability would help characterize their interaction with repeated procedural guidance, complementing our representation-focused analysis.

Dear Reviewer,

Thank you again for your valuable feedback. We hope that our responses have addressed your concerns and would greatly appreciate any further thoughts you may have.

We would be happy to provide additional clarification if needed.

Thank you for your time and consideration.

Area_Chair_BBCKOpenReview ↗

Feedback

Hi, the review deadline is approaching. Please take a look at the authors' rebuttal and post your feedback (and update your rating if appropriate) at your earliest convenience. Thank you! AC

Thanks for the response. Some comments are addressed but evaluation on real task setting, can sft/rl-based finetuning recover PEB and how confidence scores affect PEB are not addressed. I will keep the original score.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Response-1/3

We sincerely thank all reviewers for their thoughtful and constructive feedback. We appreciate their careful assessment of our controlled design and the distinction between procedural knowledge accessibility and its use in agent decisions. In response, we have conducted additional controls examining linguistic exposure, complete-action recognition, and the effects of relevant versus irrelevant actions.

The new results reinforce our central finding: Visual exhibits smaller exposure-induced distractor-preference shifts under noun and verb repetition; matched-word-bag controls show that action-expression organization matters beyond word frequency; and complete actions remain recognizable from visual procedure cards. A correct-action control further reveals differentiated responses to relevant and irrelevant exposure in the evaluated verb–object setting. Together with the submitted experiments, these results support the conclusion that visual skill representation attenuates the influence of repeated irrelevant procedural information on action selection while retaining access to procedural content.

We first address shared concerns, followed by reviewer-specific responses.

General Response

Weakness 1. Visual attenuation may reflect reduced procedural-information accessibility rather than greater resistance to irrelevant exposure.

The submitted Figure 1 establishes a smaller exposure-induced distractor-preference shift under Visual. Figure 3a shows that substantial attenuation already appears with plain text rasterization, indicating an important role for visual encoding beyond structured layout; Figure 3b shows that repeated target-word information remains recoverable.

To extend this evidence to full actions, we introduce a Full-Action Recognition Control. The model transcribes the complete action expressions and their order from an isolated procedure-example card.

RepresentationStatesEvaluationsExact Action Transcription
Text78390100%
Visual78390100%

Table R1. Exact transcription of complete actions from isolated procedure-example cards across matched exposure conditions. Correct verbs, objects, and order are required.

Table R1 shows that complete actions remain accurately recognizable under Visual, extending the original word-level readout to action expressions. Together with Figure 3b, this supports a distinction between accessing procedural information and the strength of its influence on action prediction. The recognition probe tests isolated cards, while the original controlled PEB experiment tests the influence of supplied skills on a state-conditioned decision; their complementary objectives help separate readability from downstream use.

We further examine action relevance with a Correct-Action Exposure Control. Under matched decision states and candidate sets, we compare the exposure-induced preference gains for the reference action and a state-inapplicable distractor.

Exposure UnitStatesText: Reference − Distractor (pp)Visual: Reference − Distractor (pp)Visual 95% CI (pp)
NOUN16−0.63−0.26—
VERB16−0.02+1.78—
Verb–Object16−0.43+1.53[0.79, 2.26]

Table R2. Reference-minus-distractor differences in normalized candidate-preference changes from zero to high exposure. Positive values favor the reference action. The interval tests the contrast within Visual, not a cross-representation interaction; dashes indicate unavailable intervals.

For verb–object exposure, Visual yields a larger preference gain for the reference action than for the distractor (Table R2). This targeted result shows that relevant and irrelevant procedural exposure can influence action preference differently. The reported confidence interval concerns this within-Visual contrast, rather than a cross-representation difference in selectivity. Combined with the recognition control, the result strengthens the interpretation that visual attenuation is not simply a failure to read the procedural actions.

AuthorsOpenReview ↗

General Response-2/3

Weakness 2. Noun repetition may introduce lexical confounds; PEB should be examined at the action-expression level.

The submitted probe holds the environment state and action applicability fixed while manipulating an object-related keyword. We extend it with part-of-speech and matched-word-bag controls, preserving the state and candidate actions within each comparison.

Exposure UnitStatesText Change (pp)Visual Change (pp)Text − Visual (pp)Paired 95% CI (pp)
NOUN74+0.96+0.53+0.43[0.28, 0.58]
VERB78+0.91+0.41+0.50[0.37, 0.63]

Table R3. Distractor-preference changes between zero and high exposure over fixed candidate-action sets, in percentage points (pp). Eligible states are selected separately for each exposure unit.

Table R3 confirms smaller exposure-induced distractor-preference increases under Visual for both nouns and verbs, with paired confidence intervals supporting the differences.

We also compare a contiguous distractor-action expression with a reordered word-bag control containing identical case-normalized words and frequencies. This tests whether unigram counts alone explain the effect.

Condition or ContrastStatesText (pp; 95% CI where available)Visual (pp; 95% CI where available)Text − Visual (pp; Paired 95% CI)
Contiguous Action Target85+3.67+2.92+0.75 [0.24, 1.26]
Reordered Word-Bag Control85+0.67+0.55—
Target − Control85+3.00 [2.51, 3.54]+2.37 [2.00, 2.78]—

Table R4. Matched-word-bag action-expression control on eligible ALFWorld decision states. Values are exposure-induced changes or paired contrasts in percentage points; brackets give paired confidence intervals. Reordering changes phrase continuity and word order while preserving unigram frequencies.

As shown in Table R4, retaining the contiguous action expression produces a larger preference shift than reordering the same words, indicating that unigram frequency alone does not explain the effect. Importantly, the action-target condition still exhibits weaker distractor influence under Visual, with paired statistical support. The control changes both word order and phrase contiguity; accordingly, we interpret it as evidence for the contribution of action-expression organization rather than an isolated syntactic mechanism. These controls extend PEB beyond isolated object words to verbs and action-related expressions.

This action-conditioned setting also clarifies the relationship to contextual entrainment: our focus is how repeated procedural patterns affect the choice of a state-inapplicable action when state and applicability remain unchanged, and how that effect depends on skill representation.

AuthorsOpenReview ↗

General Response-3/3

Weakness 3. Controlled PEB findings should be connected to executable decisions and complete-agent behavior.

The submitted paper already provides this connection at complementary levels. Figure 3c evaluates freely generated actions, replacing the inapplicable scoring distractor with an admissible competing action. Exposure increases its selection more strongly under Text than Visual; Appendix A.3.3 reports the protocol and a paired bootstrap comparison of exposure-response slopes.

Appendix A.3.4, Tables 7–8 further examine paired continuations after executing either the reference or competing action. Among pairs in which both branches succeed, the competing-action branch generally requires more steps, illustrating potential downstream costs of alternative executable choices.

Finally, the submitted Table 1 reports default-skill performance on ALFWorld, ScienceWorld, and WebShop: Visual achieves higher macro-average task success than Text for each evaluated backbone, with Text and Visual using the same ranked skill content. Together, controlled preference, executable generation, paired continuations, and complete-task success offer a coherent progression from exposure sensitivity to practical agent behavior. The paired continuations assess the consequences of alternative initial actions, not the causal effect of exposure on full-task success. Likewise, the default-skill success rates and controlled exposure responses are complementary rather than interchangeable outcomes.

Weakness 4. The statistical reliability of exposure effects and end-to-end performance should be clarified.

The additional paired comparisons in Tables R3–R4 support smaller exposure-induced distractor-preference shifts under Visual, with confidence intervals excluding zero in the reported cross-representation comparisons. The submitted Appendix A.3.3 likewise provides a paired bootstrap comparison for exposure-response slopes in freely generated executable actions. These results statistically support the representation dependence of PEB at the controlled action level.

The submitted Table 1 independently reports higher Visual macro-average task success across backbones under matched default-skill content and ranking. We present this as the observed aggregate agent-performance result, without treating action-level confidence intervals as significance tests of complete-task success.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.