acceptodds
Under review as a conference paper at ICLR 2027

Names That Steer: Identifier Bias Across Text and Vision in Code Understanding

Abstract

Code is increasingly presented to multimodal models as rendered images, yet little is known about whether visual presentation changes how symbolic cues shape program understanding. We trace identifier bias across textual and visual code using a semantics-preserving source–donor construction that holds a source implementation fixed while replacing its identifiers with names from a functionally different donor. Across eight vision-capable model instances, donor-aligned names shift both functional choices and open-ended summaries toward the donor under Text and Readable Image. The Readable Image choice effect is positive in every instance (4.2–22.8 percentage points) and exceeds its Text counterpart in all eight, although carrier differences vary by task and readout. Readback and constrained visual controls make gross transcription failure insufficient to explain this persistence. In an open-weight focal model, we further examine carrier-conditioned output likelihood, cross-carrier representation geometry, and activation patching. Text and Image yield strongly corresponding representations, and identifier-induced changes align more closely for the same program than across programs. A matched Text direction steers Image judgments more than a mismatched direction, while a same-carrier Image direction remains stronger. Similar generated summaries nevertheless retain a self-carrier likelihood advantage. Together, these findings are consistent with a partial-anchoring account in which visual-code representations share text-compatible, program-specific components while retaining carrier-conditioned structure. Identifier bias therefore persists beyond textual input, and text-derived identifier directions can functionally steer image judgments.

Primary area:
interpretability and explainable AI
Keywords:
vision-language models, cross-modal representations, identifier bias, visual code understanding

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_oMtKOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies the difference of code understanding between text format and image format. The results are "misleading identifiers persists under readable visual presentation".

Critical strengths

None

Critical weaknesses
  1. The presentation makes the paper hard to read. The authors just list experimental numbers everywhere without decent analysis. The figures are also bad.

  2. The scope is small for a top-tier conference. Understanding code as rendered images is a very small user case in modern coding agent. I won't say no one does that but this is not an important research question. Even if one wants to study this, the current one-shot judgment setting is still not realistic. Vision-in-the-loop is a good research problem now, but that is long-horizon tasks like webdev or computer use tasks like Blenders, not code as rendered images.

Questions for authors

None

2 comments

We have revised the motivation and presentation to make the research question and evidence easier to follow.

The motivating question is how symbolic information is processed across text and visual carriers. Visual text and code inputs arise in screenshots, documents, and interfaces; deliberate rendering is also studied for context coverage and input efficiency, as discussed in the related work. Code offers a useful controlled setting because eligible identifiers can be changed while the implementation remains fixed. This makes a symbolic intervention traceable across carriers and into internal states.

Long-horizon agent tasks measure end-to-end interaction. Our paired function-level design complements that setting by holding implementation evidence fixed while changing one family of cues. The contribution includes program-representation geometry and Text-to-Image activation steering, alongside the behavioral persistence result. The revised paper separates geometric content correspondence, functional logit steering, and retained carrier dependence in the same-carrier intervention and realized-output likelihood. Partial anchoring organizes these observations as a working interpretation.

We have redrawn Figure 1 to show the controlled intervention and behavioral readouts, and clarified Figure 2's caption on pre-generation readouts and the illustrated Text-to-Image forward patch. Figures 3–4 use consistent typography and clearer panel roles. The revised presentation distinguishes geometric correspondence, cross-carrier logit steering, and carrier retention. We reduced repeated numerical narration, connected the readouts to their interpretation, and streamlined the appendix around construction, configurations, complete results, and reproducibility information.

Dear Reviewer oMtK,

Thank you for your time and thoughtful reviews. Please be reminded that the author-reviewer discussion period ends on November 18.

Since the authors have procvided their feedback, please review the feedback and actively participate in the discussion. If you have any remaining questions or concerns, please raise them before the deadline so the authors have sufficient time to respond. Thanks.

Best regards,

AC

Reviewer_PPv5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies whether misleading names affect how vision-language models understand code shown as an image instead of text. It keeps the code of a Python function fixed and replaces its function, parameter, and local variable names with names from a different function (the donor). On 500 functions and eight models, the misleading names make models pick the donor's description more often in a two-option task, by 4.2 to 22.8 percentage points for image input. The increase is larger for image input than for text input in all eight models, and the models can still copy the names correctly from the image. In Qwen3-VL-8B, the hidden states for the text and image versions of the same program are highly similar. Adding the change that renaming causes in the text input to the image input also moves the answer toward the donor, and more so when the change comes from the same program than from a different one. The paper calls this "partial anchoring with carrier retention."

Critical strengths

S1. The experiments are carefully designed. Each name is changed at every place it is used, and the renamed code passes syntax and execution checks. A third condition with placeholder names such as var0 separates the effect of removing the original names from the effect of adding misleading ones. Confidence intervals account for clustering by repository, and the layer used for patching is chosen on a separate development set. The main result holds in all eight models.

S2. The main finding is useful in practice. For several strong models, misleading names barely change the answer for text input but change it often for image input. With misleading names, Claude Sonnet 4.5 picks the donor's description 0.8% of the time for text and 16.8% for images, and GLM-5V-Turbo 2.4% and 14.2% (Table 7). Code is more and more often given to models as screenshots, and tests on text alone would miss this problem.

Critical weaknesses

W1. The effect may already be close to zero for current frontier models, which would limit the significance of the finding. The task is easy: when names are not misleading, every model almost always picks the correct description (Table 7). GPT-5 already shows only 4.2 percentage points under images. The newest OpenAI and Anthropic models tested, GPT-5 and Claude Sonnet 4.5, were both released in 2025. The paper should evaluate current frontier models, such as GPT-6.x and Claude Opus 5.x or Fable 5.x. If the effect disappears for them, the test needs harder tasks, for example longer code or donors that differ only slightly from the original function. (§4.1, Table 7)

W2. The mechanistic evidence is weak. In the patching experiment, a direction from a different program already gives +1.93 of the +2.71 effect, so the part specific to the same program is only +0.78, on 64 programs in one model at one layer (§4.3). The final account is, by the paper's own description, not identifiable, and it does not predict when the image effect is larger (§4.4).

Questions for authors

Q1 (clarification). Appendix A.4 says thinking was turned off "where supported," and GPT-5 used minimal reasoning. Which models ran without reasoning? For the models with the largest gap (Qwen3-VL-Plus, Claude Sonnet 4.5, GLM-5V-Turbo), does the gap remain when reasoning is turned on?

Q2 (additional result). How does the gap between image and text input change with code length or with the number of rendered pages? If the gap grows with length, longer inputs could keep the test informative for stronger models (W1).

2 comments

Thank you for recognizing the controlled construction and the Text–Image differences. We have sharpened the research purpose and revised the mechanism interpretation.

W1: Model recency and task difficulty

The source–donor construction is a probe of symbolic information use across carriers. Its fixed implementation and protected symbols continue to support the source; near-ceiling Original performance occurs when these cues and the original names agree. The paired C−O contrast asks whether changing names redirects judgments. This provides a known direction for examining representation changes and intervention, without requiring a difficulty-scaled benchmark.

A newer model may reduce the final behavioral effect, which would extend its measured boundary. That result alone would not determine whether its pre-generation representations are carrier-conditioned. Later generated reasoning can revisit implementation evidence and could compensate for an initial tendency. Our internal analyses examine states after processing the input and task scaffold, before generated reasoning or answers. Accessible states and controlled execution motivate the open-weight focal model; its internal results are reported at that scope. The revised Introduction, setup, and discussion make this information-processing question explicit.

W2: What the mechanistic evidence establishes

Patching supports functional steering across carriers: a Text-derived identifier-change direction shifts Image answer logits (+2.709), with near-zero Random and stronger Same-carrier Image effects. Mismatched Text (+1.926) has interpretive limitations inherent to this reference condition. Both Text directions are measured after the choice context, so a cross-program vector may itself encode a strong answer-option preference alongside name-related information. Sender–receiver source-option positions are also incompletely matched (30/64 coincide). We retain the Matched−Mismatched estimate, +.783 [.215, 1.352], with its original prespecified status, and treat Mismatched Text as a cross-program reference. The revised interpretation removes the claim that this contrast isolates program-specific causal alignment.

Independent geometric evidence continues to support program-specific correspondence. In the 96-program summary experiment, the prompt contains no A/B descriptions, and each program's identifier-change direction is compared with those of the other 95 programs. This evidence complements cross-carrier logit steering and own-carrier likelihood support for similar realized summaries. Together, these findings motivate partial anchoring with carrier retention as a working account. We do not assign the mismatched effect to a pure shared semantic component or claim that the account predicts visual amplification. The figures, results, and integrated discussion now reflect these distinct evidential roles.

Q1: Reasoning configurations and reasoning-on results

Appendix B now gives the recorded settings for each instance.

Default-off records and absent visible reasoning do not establish the same guarantee as an explicit control or reported token count. GPT-5 minimal also does not universally imply zero reasoning: 35 summary responses report 128 reasoning tokens and are truncated, as documented in Appendix B.

We have not performed a formal reasoning-on comparison for the three highlighted models. Such a comparison would measure how additional reasoning changes final behavioral expression. “Before generated reasoning” describes our internal measurement site; it does not mean no inference occurs during prefill, or that changing reasoning settings would leave prefill states unchanged.

Q2: Code length and rendered pages

Length-conditioned results could help characterize the probe's behavioral sensitivity for stronger models, as suggested in W1. Our fixed source–donor conflict provides a controlled setting for tracing identifier information across carriers through behavior, pre-generation geometry, and intervention. A smaller final behavioral gap alone would not establish that Text and Image form identical pre-generation representations.

The frozen 500-program set has 277 medium, 156 long, and 67 very-long samples under its construction strata. Readable inputs contain one page for 480 programs, two pages for 19, and four pages for one. We have not estimated a length-conditioned Text–Image interaction and therefore cannot claim that longer inputs enlarge the gap. The small multi-page group also limits a separate page-count inference. This analysis would extend the measured behavioral scope of the probe.

Dear Reviewer PPv5,

Thank you for your time and thoughtful reviews. Please be reminded that the author-reviewer discussion period ends on November 18.

Since the authors have procvided their feedback, please review the feedback and actively participate in the discussion. If you have any remaining questions or concerns, please raise them before the deadline so the authors have sufficient time to respond. Thanks.

Best regards,

AC

Reviewer_XgaiOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how misleading identifier names affect code understanding when code is presented as text or images. Using semantics-preserving renaming, the authors evaluate eight VLM instances and find consistent identifier bias across both input formats. Further representation analysis and activation patching on Qwen3-VL-8B suggest that text and image inputs share some program-specific representations, while retaining carrier-dependent differences.

Critical strengths
  • Solid empirical evaluation. The controlled renaming design and experiments across eight models provide convincing evidence that identifier bias persists in visual code inputs.

  • Interesting mechanistic analysis. The representation alignment and matched activation patching experiments go beyond behavioral observations and provide useful evidence of cross-carrier transfer.

Critical weaknesses
  • Limited evaluation of actual functional understanding. The choice task depends on source–donor descriptions, while the summary evaluation relies on reference similarity. A shift toward donor descriptions does not necessarily imply an incorrect understanding of the code.

  • Mechanistic evidence is not fully conclusive. The mismatched text direction already produces substantial steering, while the additional matched effect is relatively modest and heterogeneous across samples. The proposed partial-anchoring interpretation therefore remains somewhat speculative.

Questions for authors
  • Were docstrings and comments excluded from the code inputs? How was the correctness of source–donor functional descriptions verified?

  • Could the authors provide a few qualitative examples or manual evaluations showing whether donor-directed summary shifts reflect actual functional errors?

  • How often does matched activation patching change the final predicted choice, rather than only shifting logits?

2 comments

Thank you for recognizing the controlled behavioral evaluation and the value of the internal analyses. We have clarified the meaning of the functional readouts and narrowed the intervention claim.

Functional choice, input cleaning, and reference validation

Functional choice measures a directed cue conflict. The implementation always supports the source description, and attributes, methods, external APIs, and other protected symbols remain available. Only eligible function, parameter, and local-variable bindings change. The evidence for the two descriptions is therefore asymmetric, although their A/B positions are balanced. The paired Original-to-Confusing change measures whether conflicting names redirect judgments despite retained source evidence.

Comments, function docstrings, and type comments were removed before arm construction. We rechecked all 2,000 variants for the 500 formal programs and found none remaining. References originate from CodeSearchNet documentation and underwent one completed human review pass with recorded description corrections. Review checked fidelity to visible code, snippet suitability, donor-description quality, and functional distinctness. All 500 formal samples have accepted review records. Appendix A now documents these procedures and the renaming scope.

Do summary shifts reflect functional errors?

Reference-similarity movement alone does not estimate functional-error prevalence. However, existing paired outputs contain clear functional-attribution errors. A tree-counting function retains tree-related keys and returns a total. Claude Sonnet 4.5 correctly summarizes counting under Original; after firmware-related renaming, its Readable Image summary begins “This function updates firmware,” while still describing extraction and counting. Another function loads .p0 and .nll0 model-parameter files; after renaming, the summary describes Google image-search parameters and assigns unsupported URL and soup interpretations to the values.

Appendix C now describes these retrospectively selected examples. They show that correct local operations can coexist with an incorrect overall functional attribution; they are not presented as a systematic manual error-rate estimate.

Geometric correspondence, intervention, and final choices

The geometric and intervention findings have different roles. Independent evidence for program-specific correspondence comes from the 96-program summary experiment: same-program Text–Image identifier-change directions exceed the cross-program baseline by +.552 [.529, .574] for C−O and +.337 [.315, .360] for A−O; both gaps are positive in 96/96 programs. The prompt contains no A/B descriptions, and the geometric baseline averages the other 95 programs.

Patching tests functional steering of image-conditioned answer logits. Matched Text produces a signed bidirectional effect of +2.709 [2.118, 3.301], while equal-norm Random is near zero; Same-carrier Image remains stronger. Mismatched Text has interpretive limitations inherent to this reference condition. Both Text interventions use C−O vectors measured after the choice context, so a cross-program vector may itself encode a strong answer-option preference alongside name-related information. Sender–receiver source-option positions are also incompletely matched (30/64 coincide). These properties prevent the Matched−Mismatched contrast from isolating program-specific causal alignment. We retain its prespecified estimate, +.783 [.215, 1.352], and treat Mismatched Text as a cross-program reference. We have removed the stronger specificity claim from the patching interpretation; the separate geometric evidence for program-specific correspondence remains. Sections 3.2 and 4.3 and Appendix E now clarify these distinct evidential roles.

For the requested threshold diagnostic, Matched Text flips none of the 64 Original receivers under forward patching. Restoration flips 4/64 Confusing receivers: two donor-to-source and two source-to-donor, leaving the marginal donor count unchanged. These are preferences between the two next-option-token logits; complete patched answers were not generated. We now report them alongside the prespecified continuous endpoint. The intervention supports logit steering across carriers; program-specific correspondence is supported independently by the geometry analyses.

Dear Reviewer Xgai,

Thank you for your time and thoughtful reviews. Please be reminded that the author-reviewer discussion period ends on November 18.

Since the authors have procvided their feedback, please review the feedback and actively participate in the discussion. If you have any remaining questions or concerns, please raise them before the deadline so the authors have sufficient time to respond. Thanks.

Best regards,

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.