acceptodds
Withdrawn from ICLR 2027

CAUSAL ABSTRACTION OF MAGNITUDE IN LAN- GUAGE MODELS

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Language models can distinguish “5 degrees Celsius” from “35 degrees Celsius,” but whether such quantitative concepts are reusable latent variables, rather than context-specific surface features, is unclear. We test this via cross-form causal interchangeability (transplanting a magnitude direction across linguistic surface forms) and multi-consequence coherence (whether one intervention coherently shifts many independent consequences, not just one), against random, output-specific, and novel-context controls. On Qwen2.5-7B-Instruct, temperature is strongly decodable (), and a magnitude direction transfers almost as well across forms as within one: mean coherence over 15 consequences is same-form and cross-form, against a random-direction null of – a gap of about pooled standard deviations, confirmed by a non-overlapping bootstrap 95% CI, and equally strong for a novel context category and a continuous dose-response. Reaching this required catching two of our own bugs: an all-position intervention let a random direction mimic the real effect, and 3 of 15 consequences were authored with the wrong sign, caught only by the dose-response check itself. Extending to a second concept and two more models gives a mixed picture: distance replicates far more weakly (1–2 pooled SD), Phi-3.5-mini-instruct shows a moderate effect (2 SD), and Mistral-7B-Instruct-v0.3 shows none (0 SD) despite decoding temperature more accurately than Qwen. The central claim rests on this evidence chain, not on an output-specific-steering control, which stayed unreliable after two fix attempts; a small held-out set and an unresolved context-magnitude compositionality test are the main remaining gaps. Probing alone can overstate conceptual representations: decodability is not equivalent to causal reusability, and causal abstraction – cross-form transfer plus multi-consequence coherence – is a stronger criterion for studying whether representations support flexible computation.

Primary area:
foundation or frontier models, including LLMs
Keywords:
temperature representation, causal abstraction, concept representation, causal reusability, cross-form transfer, multi-consequence coherence, latent variables, language models, interpretability, representation learning, concept steering, causal interventions, activation steering, model comparison, probing vs causality.

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_SHCNOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

Please don't waste my time. Appendices E, F, G, H, and I simply copied and pasted the original "Research Proposal" written by humans. It contains numerous statements like "We will perform SAE analysis (Appendix G)" and "We will perform subspace dimension ablation (Appendix H.2)," but in reality, it never ran, or it was simply marked "not run."

Critical strengths

good Claudecode used skill

Critical weaknesses

bad human experience

Questions for authors

none

Reviewer_eNKZOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies whether LLMs use a shared representation of magnitude (specifically for temperature and distance) across different prompt formats and descriptions. The authors estimate an activation direction from low-high sample contrasts and test whether applying it changes several downstream judgments across surface forms and contexts. Experimental results show that such transfer is stronger for Qwen than for other models in temperature, though important alternative explanations and tests remain unresolved.

Critical strengths
  1. The paper turns a focused question into concrete experiments. Checking whether an activation patch transfers across contexts and various controls is a useful step beyond measuring probe accuracy.

  2. The authors honestly disclose negative results and incomplete tests.

Critical weaknesses
  1. Narrow Scope and Limited Generalization: The main study only covers temperature and distance on small synthetic datasets. They provide little evidence that the conclusions extend to other concepts or realistic LLM applications.

  2. Unjustified Oversimplified Models: The additive magnitude/context model in Eq. 1 leaves the residual and interactions with wording largely unspecified. The authors also assume the same meanings across phrasing, although just above freezing (e.g., different people may have different senses of temperature, so context cannot be that easily marginalized) and 5 degrees Celsius need not describe the same temperature. The presented coherence tests assign each consequence one expected direction regardless of context.

  3. Weak Experiment Results: Moving ice and coat judgments together does not necessarily establish that an activation patch changes how the model represents temperature. Therefore, the "causal coherence test" may overstate the evidence for a shared temperature variable. The paper's control for steering individual answers is also inconclusive, and the context-dependent test was not run. The mixed results also do not justify the proposed hierarchy or broader conclusions about magnitude use. (Sections 5.3-6; Appendices C.6 and C.12.)

  4. Unclear Formulation and Presentation: Key details about controls and which analyses actually ran are left to the appendix. This makes the main text not self-contained. The results often read as a record of implementation changes, with little explanation of what each finding teaches us. The paper also does not clearly establish its advance over earlier activation steering and causal-abstraction work. A clearer comparison should also explain how the study addresses known pitfalls in activation-patching evaluation [1, 2, 3].

References:

[1] Geiger, Atticus, et al. "Causal abstraction: A theoretical foundation for mechanistic interpretability." Journal of Machine Learning Research 26.83 (2025): 1-64.

[2] Rimsky, Nina, et al. "Steering llama 2 via contrastive activation addition." Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

[3] Zhang, Fred, and Neel Nanda. "Towards best practices of activation patching in language models: Metrics and methods." International Conference on Learning Representations. Vol. 2024. 2024.

Questions for authors
  1. How do the experiments presented in the main study impact general natural language processing tasks/applications? Please demonstrate clear generalization evidence to justify the narrow scope.

  2. The ϵ\epsilon in Eq. 1 is extremely misleading, as it reads like a minor term, but there are no clear experiments to justify its magnitude or impact on activations. The additive form also seems overly simplified. Please provide justification.

  3. What would be the main insights from each of the experiments? Please consider rewriting the whole experiment section with careful pacing. Currently, it is almost a dry read of the experimental results, and some conclusions go beyond the reported evidence.

Reviewer_QNoUOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper tries to study if language models represent quantitative concepts, e.g. temperature and distance, as reusable internal variables. To do so, the authors extract steering vectors representing concepts from one linguistic representation and test transferring them to different phrasings and contexts. Experiments are done on three language models and two quantitative concepts. Experimental results show strong evidence of cross-form and cross-context transfer for the concept of temperature in Qwen2.5-7B-Instruct model. Results for other models and concepts are substantially weaker and mixed.

Critical strengths
  • The studied question of whether language models represent quantitative concepts as reusable internal variables is meaningful
  • The idea of extracting steering vectors representing concepts from one linguistic representation and testing transferring them to different phrasings and contexts is meaningful as one of the ways to assessing concept representations.
Critical weaknesses

Methodology contains serious flaws. Sec. 5.1. clearly writes about finding an error in the evaluation setup that affected the initial results, and Sec. C.8 reports correcting the expected signs for three judgments post-hoc and recalculating the results. Table 5, 6 have unfilled placeholders. Next, Sec. 5.4. describes an experiment that was not actually run. Sec. 5.2. discusses "early version" of the experiment and where it failed. Section D in the Appendix acknowledges experiments as pilot. These parts of the text referring to earlier experiments, pilot, post-hoc corrections do not allow for reliable assessment of the final methodology.

Overall, the paper is extremely hard to read. It also contains unfilled placeholders in references, and references to a codebase that is not attached.

Questions for authors

I do not have any meaningful questions as I am not able to reliable assess the paper

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.