acceptodds
Withdrawn from ICLR 2027

When Representation Effects Depend on the Measurement Condition

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Representation analysis frequently treats an observed quantity as an intrinsic property of the representation being analyzed, despite the fact that it is generated by a particular measurement procedure. We make this dependency explicit by writing , where denotes the representation being measured and denotes the measurement condition. This leads to a concrete invariance question: with held constant, does the measured size of a representational contrast stay the same as varies? To examine this empirically, we design a controlled protocol that keeps the representation fed into the measurement instrument bitwise identical while changing only the measurement condition. For vision representations, we find that two trained conditions yield very similar pairwise structure, yet they can assign markedly different magnitudes to identical representational contrasts. Therefore, matching relational structure does not guarantee that magnitudes transfer across conditions. Operating-point normalization accounts for a substantial portion of the observed scale gap, but does not eliminate it, and simpler probe families show the same qualitative behavior. Relative ordering and spatial concordance are far more stable than absolute magnitude, although they are not strictly invariant and may become effectively unmeasurable under a degenerate measurement condition. These findings support a measurement-conditioned view of representation analysis: a measured representational quantity should be attributed to alone only when invariance across the relevant measurement conditions is demonstrated; otherwise, the condition should remain part of the claim's scope, .

Primary area:
interpretability and explainable AI
Keywords:
Neural Representation Analysis, Measurement Invariance, Measurement Dependence, Probing, Representation Similarity, Relational Structure

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_TxNBOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The authors want to study how different trained measurement instruments (here probes) assign different values to differences between representations.

Critical strengths
  • I appreciate contributions that are outside of methodological advances, such as improving our understanding of the reliability of measurements or critiquing protocols.
Critical weaknesses
  • W1. For me, the motivation of the paper remains unclear. The paper spends considerable space explaining that measurements depend on how they are obtained, but I still do not understand which concrete problem the experiments address. The main problem is that the authors try to write the paper, especially the earlier parts, very vaguely, without concrete examples of what it actually means in practice. Can the authors actually give an example of a representational claim that would be wrong because of the effect studied here, and explain what their result adds to this existing concern?
  • W1.b The paper is difficult to follow because the authors decided to go with an abstract formulation that comes before a clear explanation of the experiment. Even after a reader has read through the front pages and the introduction, it is still unclear what the instrument is, what its states mean, and what the authors want to infer from its output. Also the appendix then introduces many experiment identifiers, local paths, and ledgers, while essential details remain scattered across sections.
  • W2. It is also unclear what the state entropy measures about the representation. As far as I can understand, it is the entropy of an internal distribution in a learned module, but the paper does not establish which property of the representation it captures. For me, it was surprising that the authors list this as a limitation despite it being essential to the paper. Especially because without that connection, I am not convinced that a difference in this statistic tells us much about the reliability of representational claims.
  • W3. For me, it is also unclear why the main comparison changes the objective from segmentation to reconstruction, especially since tasks reward different information. So why should the resulting instruments assign the same magnitude to differences between representations? Keeping the architecture and input fixed does not mean that the instruments measure the same property.
  • W4. I would have found an experiment with conventional probing more convincing. Especially because the main setup uses a custom module with millions of parameters and measures its internal entropy. What was the reason why the authors didn't just keep the probing task fixed and vary training choices among well-performing probes, including a standard linear classification probe?
Questions for authors

See weaknesses.

Reviewer_9ZCFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the measurement context C’s influence on a measurement of a representational difference, denoted R=M(F;C)R=M(F;C), using trained neural networks as instruments. Specifically, the authors compare two instruments that differ in training objective (segmentation vs. reconstruction) and measurement context (linear vs. centered kernel CKA), applied to 45 pairs of representations from 10 DeepLabV3-ResNet50 networks trained on PASCAL VOC 2012. Despite highly correlated differences across conditions (r=0.975), average magnitudes differ by a factor of 5.93. The paper includes ablations on probe architecture and random seeds, as well as analyses on other datasets/backbones. When rescaling by the remaining entropy headroom, this ratio drops to 1.42. The authors argue for reporting measurement details and results under multiple instruments/conditions.

Critical strengths
  1. The paper tackles an important question about the invariance of representational measurements to changes in context. It convincingly demonstrates via thorough experiments that two contexts can yield highly correlated measurements that nonetheless differ systematically in scale.

  2. The experiments are thoughtfully designed with strong controls to rule out alternative explanations: the backbone and BatchNorm parameters/statistics are frozen (verified via bitwise-identical activations, §3.1/Appendix A), host checkpoint dependence is addressed via resampling (important given correlated differences among the 45 pairs), and instrument stability is supported by multiple retrained instruments. Additional instrument variants/architectures are included to improve generality.

Critical weaknesses
  1. It remains unclear how much this context dependence should influence scientists’ interpretation of representational measurements.

    • While correlated measurements can differ in scale, this does not necessarily undermine representational analysis broadly: the paper’s headline numbers rely on particular instruments (segmentation vs. reconstruction probes whose internal-state entropies serve as the observable, which is acknowledged as operational rather than intrinsically validated, §7.3).
    • Moreover, Appendix D.3 contrasts an effective segmentation probe with a near-saturated reconstruction probe, so measurement-context effects there partly reflect one degenerate instrument.
    • Although Hewitt and Liang (2019) previously showed conclusions can be probe-dependent using control tasks, this paper would be strengthened by a concrete case where, after reasonable normalization/calibration (e.g., accounting for entropy headroom/operating point, which already shrinks the 5.93× gap to 1.42×), context still reverses or materially changes a scientific conclusion, rather than mainly rescaling effect sizes.
    • As it stands, the practical “so-what” beyond documenting context/instrument dependence (especially for effect magnitudes) is not fully established.
  2. The finding that measurement contexts can yield highly correlated yet differently scaled representational differences is itself not especially surprising, and part of the discrepancy is attributable to the instruments’ operating points.

    • Accordingly, the paper would benefit from a sharper articulation of which aspects go beyond prior expectations (e.g., Hewitt and Liang (2019), https://aclanthology.org/D19-1275/).
    • It would also benefit from experiments isolating context effects between two well-performing, non-degenerate instruments, since the alternative-probe results in Appendix D.3 do not directly replicate the normalized 1.42× residual due to the near-saturated reconstruction probe.
Questions for authors
  1. Could the authors provide examples of scientifically meaningful representational conclusions that change, not merely effect-size rescaling, after accounting for measurement context, even given the high (0.975) probe-agreement correlation?

  2. Beyond headroom normalization, do simpler bias/correction procedures also largely reconcile effect magnitudes across measurement contexts, and if so, what does that imply about the novelty/strength of the reported context effects relative to prior findings (e.g., Hewitt and Liang (2019))?

  3. Appendix D.3’s alternative-probe results involve a near-saturated reconstruction probe; did the authors test context effects between two effective, non-degenerate instruments, and if not, could such a control be added to strengthen the paper?

  4. ConvNeXt-T Experiment: In the ConvNeXt-T experiment, since the untrained baseline is used instead of the reconstruction baseline, and a different definition of ratio is used, it’s not an exact replica of the original experiment. Did the authors also conduct a control experiment using the reconstruction baseline?

  5. Transform Families: The host bootstrap CIs all include 1 for the ratio across the five transform families (Table 6). What’s the motivation for including this experiment? If the goal is to show that the effect size varies across contexts, it seems like this experiment actually weakens that claim since the effect sizes don’t vary significantly in this case.

Reviewer_t4t2OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies whether probing representations is scale-invariant under different measurement conditions/probing parameters. This is empirically tested by training different probes on fixed representations. When keeping the representation fixed but altering the measurement mechanism, the paper finds that relational structure can be preserved while absolute magnitudes differ (up to 5.9x in the presented setting). As a conclusion, the authors argue that findings from representational analysis about the backbone should be clearly stated with respect to under which measurement condition C the result was obtained.

Critical strengths
  • Many papers use representation analysis including probing of representations for claiming whether a certain attribute is encoded and sometimes also how strongly it is encoded. Evaluating whether the conclusions change under a different measurement regime is an important research question that the authors try to tackle in this paper.
Critical weaknesses
  • The paper reads very LLM generated and Figure 1 looks also like it was directly exported from a LLM tool. Within the current writing, the paper significantly lacks the required clarity for being able to interpret the experimental setup, results and conclusions properly as a reader. From my view, the paper needs a major rewrite to be considered as a serious paper for ICLR.

  • A number of really odd word choices are made in the paper. For example section 3.1 just describes regular well-established linear probing as far as I can tell but through terms like “instrument”, “measurement” “attachment time”, and “host”, it is quite hard to understand what the authors mean here. Aligning these terms to established terms of the representation learning community (e.g. “linear probing”, “weight initilization”) would help to understand what the authors want to communicate.

  • The paper draws conclusions with quite generality, although experiments are only performed on a specific vision model setting (which is not clearly described), and the conclusions are based on empirical evidence.

  • Usually, scientifically rigorous papers that compare different representations clearly state how the measurement was done (e.g. how the probe was trained). So I’m not convinced that the paper is pointing out a real problem (as this is already acknowledged in current literature).

Questions for authors
  • In section 3.2, the training details are super vague. How are the different probes exactly trained (e.g. which segmentation objective is optimized)? What was the reasoning for using Pascal VOC as training data? Why do you think the conclusions of this specific setup transfer to other setups?

  • Why would we assume that two total different training objectives (MSE, segmentation training) would lead to the same output magnitude? Isn’t the conclusion somehow trivial?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.