acceptodds
Under review as a conference paper at ICLR 2027

Do LMs Know What Variables Do? Interpreting Code Representations via Variable Roles

Abstract

Variable roles describe recurring behaviors such as accumulation, traversal, and retaining the best candidate encountered. We use Sajaniemi's independently defined taxonomy to study which role distinctions can be recovered from large language models, how sparse autoencoders (SAEs) retain those distinctions, and whether associated directions support control of generation. We introduce static annotation rules for all eleven roles and a corpus of 126,926 natural-code files across five languages. Across five model–SAE systems, residual probes outperform individual SAE features and tested lexical baselines, with modest losses after neutral target renaming. Combining SAE features closes much of the single-feature gap, but probes on SAE reconstructions are less sensitive to role-changing edits reviewed by LLMs. High paired ranking accuracy coexists with frequent false positives at fixed thresholds. During generation, interventions often change role annotations at the cost of correctness; changes accompanied by passing tests show no clear aggregate advantage over random controls. The benchmark separates role-related recoverability, SAE fidelity, and correctness-preserving control through matched prediction, source edits, and generation interventions.

Primary area:
interpretability and explainable AI
Keywords:
Sparse autoencoder, Mechanistic interpretability, Program semantics, Variable roles, Code representation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_V879OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates whether large language models encode semantic information about variable roles in their internal representations. Based on Sajaniemi's taxonomy of eleven variable roles, the authors develop an automatic annotation pipeline and construct a dataset containing 126,926 source-code files and approximately 1.74 million role-annotated variables across five programming languages. The paper then conducts representation probing experiments across five language models and their corresponding sparse autoencoders (SAEs). The results suggest that variable-role annotations can be recovered from residual-stream representations more effectively than from individual SAE features or the tested lexical baselines. Finally, the authors investigate whether SAE-associated directions can steer code generation toward or away from particular variable roles.

Critical strengths
  • The paper investigates a potentially meaningful question concerning how language models represent programming concepts internally.
  • The authors make a substantial engineering effort to construct a multilingual variable-role dataset and evaluate multiple models and representation types.
  • The experiments include several useful controls, including variable renaming, counterfactual code edits, and random intervention directions.
Critical weaknesses
  • As the authors already discussed, the core research question and the main methodological components have already been substantially explored in prior literature[1-6]. Against this background, the present paper mainly combines established techniques and applies them to a larger collection of predefined variable roles. Although the resulting dataset and empirical comparisons provide some incremental value, the paper does not introduce a substantially new representation analysis method, nor does it establish a sufficiently distinctive mechanistic insight beyond these previous studies.
  • While investigating the internal representations of programming concepts is potentially valuable, the paper does not convincingly demonstrate how its findings advance our understanding of code language models beyond existing research.
  • A major concern is the reliability of the automatically generated variable-role annotations. The proposed dataset relies on author-defined static analysis rules over CPG graphs. However, the authors explicitly acknowledge that corpus-wide annotation precision and recall remain unmeasured. Furthermore, variables without a target-role annotation are used as candidate negative examples, even though the authors acknowledge that the automatic extractor may fail to identify valid instances of a role. Consequently, both positive-label errors and false negatives may affect the reported results.
  • The causal relation between decodability and semantic representation is weak. The decodability of an automatically generated label is not sufficient to show that the model represents the corresponding semantic concept in a generalizable way.
  • The paper attempts to address several substantial problems: large-scale semantic annotation, representation recoverability, SAE fidelity, counterfactual sensitivity, and causal control of code generation. Although these components are conceptually related, their connections are not developed into a sufficiently clear and coherent scientific argument. The manuscript frequently moves between different experimental settings, predictors, evaluation populations, and metrics, making it difficult to identify the principal hypothesis and understand how each experiment contributes to validating it.

[1] A Study of Variable-Role-based Feature Enrichment in Neural Models of Code Hussain et al. · InteNSE 2023

[2] Emergent Representations of Program Semantics in Language Models Trained on Programs Jin & Rinard · ICML 2024

[3] Do Large Code Models Understand Programming Concepts? Counterfactual Analysis for Code Predicates Hooda et al. · ICML 2024

[4] A Critical Study of What Code-LLMs (Do Not) Learn Anand et al. · Findings of ACL 2024

[5] Analyzing Latent Concepts in Code Language Models Sharma et al.

[6] Neuron-Guided Interpretation of Code LLMs: Where, Why, and How? Yin et al. · FSE 2026

Questions for authors

See the "Weakness" part.

1 comment

We thank the reviewer for the critical assessment. The original paper insufficiently distinguished methodological breadth from scientific contribution and decodability from semantic representation. We revised the narrative and added targeted experiments to clarify our findings and limitations.

1. Novelty and scientific contribution

Our contribution is empirical: using Sajaniemi's independently defined variable roles, we study which variable-role distinctions are recoverable from code-LM activations, how far this information generalizes, and whether associated directions support selective control during generation..

Hussain et al. [1] use stepper and walker labels to enrich Code2Seq; we study all eleven roles as representation targets. Jin and Rinard [2], Hooda et al. [3], and Anand et al. [4] examine execution states, predicates, and learned code properties; Sharma et al. [5] and Yin et al. [6] investigate latent concepts and neurons. We connect variable-level residual and SAE readouts, controlled source comparisons, and generation interventions.

Our three main findings are:

  1. Recoverability: Variable-role information is decodable from LM activations. Residual probes outperform tested lexical baselines and individual SAE features across the evaluated roles and models.
  2. Partial generalization: This information survives several lexical changes and transfers across some languages and implementations, but performance varies by role, surface pattern, and calibration.
  3. Limited causal influence: Role-associated directions sometimes affect code generation or continuation likelihoods, but do not enable reliable, selective steering toward a target role while preserving correctness.

The contribution is not a new interpretability method or identified role-specific circuits, but an empirical account of the gap between decodability, generalization, and controllability.

2. Reliability of automatic annotations

We added a blinded human evaluation of 660 stratified variables (Appendix D), covering extractor positives, role-suggestive negatives, and variables without extracted roles. Micro-precision/recall/F1 is 0.855/0.678/0.756. Precision is high for most roles, while recall varies substantially (0.467–1.000), revealing both missed roles and false positives.

The stratified sample does not support corpus-wide performance estimates, and inter-annotator agreement remains unmeasured. We acknowledge these limitations and treat unannotated variables as candidate rather than verified negatives.

3. Generalization and causal evidence

We agree that label decodability alone cannot establish semantic abstraction. Section 4 and Appendices H.2–H.4 test stronger lexical obfuscation, held-out languages and implementations, and function-matched programs.

Removing additional identifier and comment cues has a modest average impact. Most-wanted holder, gatherer, and one-way flag transfer relatively well, whereas stepper and walker show greater implementation dependence. On function-matched programs, residual probes reach within-task AUROC 0.921 but only 67.8% balanced accuracy at natural-code thresholds. Output agreement on finite tests does not prove full equivalence. Thus, the evidence supports partial generalization beyond tested surface cues, not implementation-invariant representations.

Appendix H.1 shows that fitted SAE role scores combine multiple reused features. SAE reconstructions preserve much matched-label discrimination but show reduced sensitivity to role-changing edits. These are predictive, not mechanistic, results.

Section 5 and Appendices Q–R test functional influence. Global interventions occasionally change annotations or code structure, but correctness-preserving target changes remain rare, without a clear aggregate advantage over random controls. Localized norm-matched patches barely change joint correctness and target outcomes (172/216 untreated versus 173/216 under either patch). Fixed-dose tests show small, role-dependent likelihood shifts without reversing preferred continuations. We therefore observe limited effects on generation, not precise, reliable role-specific control.

4. Coherence and presentation

The paper now follows three linked questions: (i) What role information is recoverable? (ii) How does it generalize across code variations? (iii) Can associated directions selectively influence generated roles?

Section 4 links decoding to lexical robustness, implementation transfer, and behavioral sensitivity, distinguishing ranking from classification. Section 5 assesses correctness, collateral changes, and random controls; appendices document protocols and limitations.

Our conclusion is deliberately narrower: variable-role information is decodable and partially generalizable; associated directions show limited effects on generation but do not support reliable, correctness-preserving control of target roles.

Reviewer_yWzzOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether language models represent variable roles in code through probing, SAE analysis, and steering experiments. The authors find that variable-role annotations are recoverable from hidden states, but SAE features capture these distinctions less effectively, and steering rarely changes variable roles while preserving correctness.

Critical strengths
  1. Well-motivated abstraction and large-scale annotation effort. The authors use an established taxonomy of variable roles from Sajaniemi (2002), which provides a principled target for representation analysis. The authors develop an automatic annotation pipeline covering eleven variable roles across 126,926 files and five programming languages.
  2. Thoughtful evaluation controls. Matched negatives, role-changing edits, and variable renaming provide stronger evidence than standard probing accuracy alone.
Critical weaknesses
  1. Insufficient evidence of behavioral abstraction and annotation reliability. High probing accuracy may reflect syntactic patterns rather than variable behavior (e.g., code-prefix baseline achieves 0.90 AP for MWH vs. 0.95 for residual probes). A good experiment to add to would be evaluating across structurally different but semantically equivalent implementations or holding out implementation patterns during training. In addition, the paper doesn't systematically validate the precision or recall of its automatic role annotations, and variables without extracted roles may be incorrectly treated as negatives.

  2. Limited mechanistic insight and connection between experiments. The SAE experiments primarily establish differences in decodability without explaining whether variable roles are distributed across features or entangled with other programming concepts. It would be interesting to analyze feature decomposition and see whether specific feature subsets selectively mediate role information. In addition, more causal evidence is needed, such as targeted activation patching or localized interventions on controlled program pairs. The paper establishes recoverability, SAE information loss, and weak steering effectiveness. But the paper lacks presenting a direct connection between the representations identified through probing and their functional contributions, thus the evidence feels scattered and the scientific narrative feels thin.

Questions for authors
  1. The paper reports substantial differences in role recoverability across programming languages and implementations (Section 4). Do the authors have an explanation for these differences, and to what extent might they reflect differences in annotation coverage rather than model representations?

  2. For the role-changing counterfactuals, are your edits required to preserve the program's functional behavior or only to remove the target variable role? If functionality can change, how should we interpret the probe's sensitivity to these edits as evidence of role specific information?

1 comment

We thank the reviewer for the constructive suggestions. We have added human annotation validation (Appendix D), stronger robustness and SAE analyses (Appendix H), and localized causal experiments (Appendix R), with the main findings integrated into Sections 3–5.

1. Behavioral abstraction and annotation reliability

We agree that predicting automatically extracted labels does not, by itself, establish semantic abstraction. A blinded human evaluation of 660 stratified variables yields micro-averaged precision/recall/F1 of 0.855/0.678/0.756 (Section 3; Appendix D). The extractor achieves high precision for most roles, with more heterogeneous recall. Some annotation errors and missed roles remain, with performance varying across roles. As the sample is stratified, these are not corpus-wide estimates, and inter-annotator agreement remains unmeasured.

To test reliance on surface cues, we added stronger lexical obfuscation, implementation holdouts, and function-matched programs (Section 4; Appendices H.2–H.4). Residual probes retain substantial discrimination after obfuscation and reach 0.921 within-task AUROC on neutral-name, function-matched programs, versus 0.892 for the code-prefix baseline. However, transfer is uneven and predictions remain sensitive to some lexical cues. We therefore claim partial generalization, not syntax-independent semantic representations.

2. Differences across languages and implementations

Appendix H.3 now evaluates probes trained without particular languages or implementation patterns on the same held-out examples. Most-wanted holder, one-way flag, and gatherer transfer relatively well, whereas stepper and walker are more implementation-dependent. For example, withholding Python for-loop steppers reduces paired ranking accuracy by 22.9 points; walker implementation and language are strongly confounded.

These results identify differences in probe transfer, but cannot fully separate effects of annotation coverage, language-specific conventions, and training-distribution shift.

3. SAE feature structure

Following the suggestion to examine feature subsets, Appendix H.1 analyzes standardized feature contributions, dominant-feature removal, and feature budgets. Across 220 Top-64 probes, features are extensively reused between roles, and removing the strongest selected feature preserves most matched discrimination. For one-way flag, multi-feature probes also retain strong discrimination after controlling for explicit Boolean cues.

Thus, fitted readouts combine signal from multiple, often shared SAE features rather than a single role-specific latent. These analyses establish predictive contributions, not causal mediation of the model's computations.

4. Localized causal evidence

We added controlled Python continuation experiments for gatherer, most-wanted holder, and one-way flag using localized probe- and SAE-aligned interventions and norm-matched random controls (Section 5; Appendix R). Directional patches barely change joint correctness and target-invariant success (172/216 untreated versus 173/216 with either patch). Fixed-dose interventions shift the likelihood of competing updates slightly but never reverse the preferred continuation.

Together with the MBPP results, these findings provide little evidence that decoded directions support selective, correctness-preserving control of variable-role behavior.

5. Role-changing counterfactuals

The role-changing edits are not required to preserve program functionality; they locally modify behavior defining the target role, with LLM review of the original role and edit. We agree that probe sensitivity may therefore reflect correlated syntactic or functional changes, rather than role removal alone.

To address this limitation, Appendix H.4 adds function-matched implementations that use different target-variable behaviors while agreeing on tested outputs. This reduces the output-function confound, although finite tests cannot establish full equivalence, and structural differences remain. We now distinguish clearly what each protocol can support.

Reviewer_iQMVOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This is an empirical paper. It asks whether LMs represent what individual variables do, using Sajaniemi's 11 variable roles, a labeling scheme defined outside the models being studied. The authors annotate a large multilingual corpus with static CPG queries, then build a benchmark with matched pairs, LLM-reviewed counterfactual edits, and renaming that keeps the role unchanged. They compare residual probes, SAE reconstruction probes, top-64 features, and single SAE features across several model-SAE pairs, and finally test two interventions on MBP—adding feature directions during generation and removing them from the weights. On recoverability, the role distinctions can be read out from the residual stream and are largely preserved by the SAE, with single features alone being much weaker. On interventions, steering can change role annotations, but doing so without hurting correctness is hard.

Critical strengths
  1. The paper gives an annotation pipeline and benchmark for variable roles. Pulling all 11 roles out of multilingual code with static CPG queries is a real piece of work, and the corpus is far larger than anything in the prior role literature. The benchmark is something other people can actually reuse.
  2. The paper investigates variable-role representation in LMs from multiple perspectives, and the experiments are quite thorough. It covers recoverability, SAE fidelity, and controllability, and reports negative results such as no clear advantage over random steering directions.
Critical weaknesses
  1. The causal-control claim is not supported by reliable correctness-preserving improvements. Interventions often change extracted role labels while degrading program correctness (Secs. 5–7). For removal, only 2/91, 4/109, and 15/187 target–task outcomes both changed the annotation and passed supplied tests across the three reported Gemma models, and the differences from random-direction controls have bootstrap intervals spanning zero. Moreover, correctness-constrained calibration selects zero intervention in ten of eleven completed model–role cells. These results support an informative negative finding, but substantially limit the practical and mechanistic conclusions that can be drawn from steering.

  2. The evaluation may overstate behavioral understanding through shortcut cues. The target-name renaming control changes only the selected identifier while retaining other identifiers, comments, APIs, and structural patterns (Sec. 4). The code-prefix baseline is already competitive with single SAE features for some roles (e.g., most-wanted holder). Thus, good probing performance is not sufficient to establish that models encode variable behavior independently of surface regularities; more comprehensive semantics-preserving transformations and cross-implementation tests are needed.

  3. Counterfactual evaluation and binary classification disagree substantially. Although residual probes rank original examples above edited counterparts at 0.813–0.877 paired accuracy, they still label 68–71% of role-removed examples as positive at validation thresholds (Sec. 4). This weakens claims that the recovered directions reliably track the presence or absence of roles in practical classification settings. The paper would benefit from a clearer characterization of threshold calibration and distribution shift between matched negatives and source edits.

Questions for authors
  1. . How do probe and SAE results change under semantics-preserving transformations beyond target renaming, such as renaming related variables, removing comments, changing loop constructs, or refactoring equivalent implementations?
  2. Why do residual probes classify 68–71% of edited negatives as positive despite much better paired rankings? Could thresholds calibrated on role-changing edits, rather than matched negatives, materially improve balanced accuracy?
  3. Can the authors measure whether steering a target role also changes unrelated role annotations, output length, parse validity, and generic code quality relative to norm-matched random controls? This would help distinguish role-specific effects from broad disruption.
  4. Given that the confidence intervals for correctness-preserving changes overlap the random controls, what evidence supports a role-specific causal mechanism rather than direction-dependent generation perturbation? Would the authors narrow the corresponding claims? 5.Why do residual probes classify 68–71% of edited negatives as positive despite much better paired rankings? Could thresholds calibrated on role-changing edits, rather than matched negatives, materially improve balanced accuracy?
2 comments

We thank the reviewer for their constructive feedback and recognition of the benchmark's value. We have revised Sections 4–5 and added analyses in Appendices H, Q, and R to distinguish recoverability, robustness, and role-specific causal control.

1. Robustness beyond target renaming

We extended the original renaming control in three ways:

  • Stronger obfuscation (H.2): Renaming safely resolved variables and masking comments reduces residual-probe AP only modestly (89.84% to 88.98%).
  • Language and implementation holdouts (H.3): MWH, gatherer, and one-way flag transfer relatively well, while stepper and walker lose up to 28 points in paired ranking accuracy.
  • Function-matched comparisons (H.4): Across 15 Python tasks, different implementations agree on tested outputs. Residual probes achieve within-task AUROC of 0.921 versus 0.892 for the code-prefix baseline, but still respond to role-indicative names and irrelevant initializations.

Thus, role-related information survives the tested lexical changes, but robustness varies by implementation; we do not claim syntax-independent representations.

2. Counterfactual rankings versus binary classification

These results are not contradictory: a role-changing edit can lower a probe score while leaving it above a threshold calibrated on matched negatives. To test this distribution mismatch, we recalibrated thresholds on counterfactual scores using five source-group-held-out folds (Appendix H). Balanced accuracy rises from 0.610–0.637 to 0.700–0.727, at the cost of lower positive recall.

This exploratory analysis reuses the existing evaluation pool rather than an independent test set. Calibration therefore explains part, but not all, of the gap: substantial overlap remains between original and edited scores.

3. Collateral effects and code quality

Appendix Q (Table 17) now reports target and unrelated role changes, output length, AST validity, parsing, and test success.

Global direction removal changes target annotations in 45.1% and 66.1% of eligible outputs for Gemma 2 2B and Gemma 3 4B, but also changes unrelated roles in 48.4% and 56.9%, with marked correctness and parsing degradation. Only 2.2–5.5% of removal outcomes both change the target role and pass the supplied tests. Positive steering has smaller target effects but still affects unrelated roles.

These results indicate broad generation disruption rather than selective control. Historical SAE-addition controls were layer-matched but not explicitly norm-matched, limiting causal attribution. Parsing and test success are also imperfect proxies for general code quality.

4. Evidence for a role-specific causal mechanism

We agree that the original interventions do not establish such a mechanism. Across the three Gemma models, correctness-preserving target changes exceed random controls by only 1.3–3.7 percentage points for removal and 0.9–1.9 for addition; all six bootstrap confidence intervals include zero. Correctness-constrained calibration selects no intervention in ten of eleven completed settings.

To test whether global perturbations concealed localized effects, Appendix R introduces token-level interventions with norm-matched random controls. Targeted directional patches leave joint correctness and target-invariant success essentially unchanged (172/216 untreated versus 173/216 patched). Fixed-dose interventions yield small role-dependent likelihood shifts but do not reverse preferred continuations.

We have consequently narrowed our claim: role information can be decoded from activations without the tested directions providing reliable, selective, correctness-preserving control over the corresponding behavior.

Thank you for the detailed response and additional analyses. I appreciate the authors' efforts to address my concerns, and I find the clarifications reasonable. However, after considering the points raised by other reviewers, I remain unconvinced that the manuscript offers sufficiently substantial and generalizable takeaways for the broader research community. I therefore maintain my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.