acceptodds
Under review as a conference paper at ICLR 2027

CKA's Blind Spot: Why Representation Analysis Misses Policy Change in RLVR, and How to Fix It

Abstract

Mechanistic interpretability asks whether representation-level tools can explain what RLVR does to a model. We find a sharp answer: a core such tool cannot. First, we show that the phenomenon motivating much of this line of work—RLVR "capability narrowing" (pass@k decreases for large k)—is a measurement artifact. Under a faithful multi-seed protocol it vanishes (Δpass@64 ≥ 0 across 4 seeds), and tracing it exposes three methodological pitfalls (evaluation protocol drift, fixed-dictionary SAE confound, spectral estimator noise) that plausibly affect 26–34 of 87 surveyed papers. As a frame for why policy-level changes barely move geometry, we prove a Strategy-Representation Gap theorem: policy-level (L1/L2) perturbations produce only bounded changes in representation geometry, with the bound vanishing under KL regularization; we validate it across scales, algorithms, and tasks (144× CKA discrimination; GRPO/DPO/Zero-RL mean CKA = 1.0000). Our central result is the CKA Blind Spot. Models whose representations CKA judges identical (CKA = 1.000000 to six decimals) produce measurably different outputs (KL = 0.143; 3–16% of top-1 positions disagree). Linear probing resolves the paradox: the strategy is present and linearly decodable (60–100% accuracy), so the gap is metric-specific, not information-theoretic. We prove CKA deviation is O(ε²) while probe sensitivity is O(ε)—a  1,000× gap at RLVR's ε   10⁻³ perturbation scale—and confirm it via weight interpolation (CKA flat at >0.9999 along the entire path while outputs phase-transition) and spectral decomposition (updates  2,500× smaller than base weights, yielding 1−CKA = Θ(ε²)   10⁻⁷, consistent with the saturated float32 reading). We fix the blind spot with NSARS, combining CKA with linear probing, which detects the gap at 5/8 layers vs. CKA's 1/8; causal mediation localizes strategy to early layers 0–6 (90.02% of divergence), living in a  5-dimensional subspace. A cross-paradigm boundary pins the mechanism: full-parameter RLVR leaves representations invariant (CKA = 1.0), while high-rank LoRA (7B, r=256) alters them dramatically (CKA = 0.059, erank −81%)—spectral smallness, not optimization, drives the blind spot. CKA should not be used alone: every CKA-based interpretability study should report linear-probe accuracy, and early layers deserve the attention now reserved for middle layers.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, CKA, representation analysis, RLVR, linear probing, evaluation methodology

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_kquaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates the limitations of representation-level metrics in analyzing policy changes induced by Reinforcement Learning with Verifiable Rewards (RLVR). The authors first revisit the previously reported capability-narrowing phenomenon and argue that it may stem from inconsistencies in evaluation protocols. They then examine the discrepancy between representation similarity and model behavior, showing that models trained with different post-training objectives can exhibit nearly identical CKA scores despite measurable differences in their output distributions. To explain this phenomenon, the authors introduce the Strategy-Representation Gap and CKA Blind Spot theorems, arguing that CKA is relatively insensitive to small representation perturbations compared with linear probing. They further propose NSARS, a metric combining CKA with linear probing accuracy, to detect representation differences that CKA may overlook. Additional experiments involving weight interpolation, layer-wise interventions, spectral analysis, and comparisons between full-parameter training and LoRA are presented to investigate the location and characteristics of these changes.

Critical strengths

The paper raises a potentially important concern about representation similarity metrics in RLVR analysis. The observation that different post-trained models can exhibit nearly identical CKA scores while producing measurably different output distributions is interesting. It highlights the need to distinguish global representation similarity from functionally relevant changes in model behavior.

Critical weaknesses

I'm sorry to say, this paper suffers from substantial presentation issues that significantly reduce its readability. For example, Table 2 has a cluttered layout that makes the results difficult to interpret, while several equations suffer from inconsistent notation and formatting. The writing also contains excessive parenthetical expressions, enumerations, and densely packed claims. Moreover, the logical connections between different experiments, theoretical analyses, and central conclusions are not always clearly established. These issues make the paper unnecessarily difficult to follow and weaken its overall clarity.

In addition, this paper makes several strong claims that are not fully supported by the presented evidence. For example, the authors argue that the previously reported RLVR capability-narrowing phenomenon is a measurement artifact. However, the main re-evaluation supporting this conclusion is conducted on a single model with four training seeds and a relatively limited evaluation set. Although the results suggest that the phenomenon may be sensitive to evaluation protocols, they are insufficient to establish that capability narrowing is generally an artifact of incorrect evaluation. Similarly, some of the paper's broader conclusions regarding representation invariance and the mechanisms underlying RLVR appear to extend beyond the scope of the experiments.

Furthermore, the theoretical analysis relies on several strong assumptions whose validity and general applicability are not sufficiently justified. For instance, the Strategy-Representation Gap theorem assumes a particular form of gradient attenuation under KL regularization, which appears closely related to the conclusion the theorem aims to establish. The conditions required for some of the asymptotic claims in the CKA Blind Spot theorem also deserve further clarification. Overall, both the experimental evidence and theoretical arguments appear insufficient to support the strength and generality of the paper's central claims.

Questions for authors
  1. could the authors improve the formatting of figures, tables, and equations, reduce excessive parenthetical expressions and enumerations, and clarify the connections between the main findings?

  2. could the authors provide stronger evidence that RLVR capability narrowing is caused by evaluation errors? How generalizable are the results from a single-model, four-seed evaluation to other models and tasks?

Reviewer_CncGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This submission aims to show that Centred Kernel Alignment (CKA) can make two models’ internal representations look almost identical even when their learned behaviours are different. This shows a inherent limitation ("blind spot") of CKA. Starting from a practical case study of Reinforcement Learning with Verifiable Rewards (RLVR), the paper shows that models trained using different methods can have near-perfect CKA scores while a simple classifier can still distinguish their internal representations, suggesting that CKA misses some meaningful changes.

The submission has two main theoretical results. Theorem 2 argues that by changing a model’s internal representations by ϵ\epsilon (in relative terms), CKA changes as Θ(ϵ2)\Theta(\epsilon^2), meaning CKA will miss small (relative) changes. Theorem 1 argues that changes to the model’s output behaviour can be much more noticeable than changes in the effective dimensionality of its internal representations when training assumes certain regularisations constraints (KL-regularisation).

Critical strengths
  • I enjoyed the fact that precise mathematical statements were made (even if they can be a little awkward, cf. theorem 1, but this cannot be helped given the complexity of the models involved).

  • The main empirical case study is convincing (but its exact setup is very hard to follow for non-experts).

Critical weaknesses
  • I think this submission has ideas that are worthy of publications, but I don't think it has landed yet. The main idea - to investigate the limits of CKA - goes way beyond what is presented here and the focus on RLVR applied to quite specific LLM tasks is a little odd given the huge potential field of applications. Basically, you're looking at all the applications of CKA where relative differences are sufficiently small for their square to be undetectable in 32bit precision. I would encourage the authors to think big. This feels like a good idea turned into a very incremental piece of research.

  • I would rearrange the logic of the paper to put CKA's theoretical limits first, followed by a series of applications, with policy changes in RLVR being one of them. For the moment it feels like it needs to be read in reverse.

  • The writing could be improved substantially by using more plain English, more maths, less jargon and better, explicitly defined notation.

    • your abstract is too long and contains too much micro-level information. Focus on the macro picture.
    • section 2 is very hard to read.
    • table 1: what is z? line 156: what is L? l166-170: choosing H1,H2,H3H_1, H_2, H_3 and HlH_l where ll can presumably take values l=1,2,3l=1,2,3 is a bad idea. What exactly is p^i\hat{p}_i? What exactly is HlH_l? This is important because a lot of what follows depend on HlH_l and I'm incapable of finding out exactly what it refers to. Etc.
    • On Figure 1 and in the text, what exactly is being CKA'ed on each layer? What is the methodology here?
  • I think the statement of Theorem 2 is incorrect. It is correct is you use O(ϵ2)\mathcal{O}(\epsilon^2), but not Θ(ϵ2)\Theta(\epsilon^2). A counterexample is provided by X′=(1+ϵ)XX'=(1+\epsilon)X (the relative error is bounded by ϵ\epsilon) in which case CKA(X,X′)=1CKA(X,X')=1.

  • I don't see the point of NSARS as a similarity metric because it is parametric in the choice of a (linear) probe. How are we supposed to choose this probe in general? This seems like a very ad-hoc similarity metric, designed specifically for the problem at hand, but of little general value.

Questions for authors

Can you think of other situations where CKA is used in practice and might be missing important changes due to your theorem 2?

Reviewer_Q8FkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates why CKA remains close to one despite behavioral changes following RLVR. It attributes this discrepancy to CKA’s second-order sensitivity to small perturbations, proposes NSARS by combining CKA with linear-probe accuracy, and uses weight interpolation and layer swapping to investigate where changes occur. It also argues that previously reported capability narrowing is an evaluation artifact.

Critical strengths
  1. The paper addresses a meaningful methodological concern: high representational similarity does not necessarily imply functional equivalence.
  2. The head-swapping, weight-interpolation, and layer-swapping experiments provide complementary evidence beyond aggregate similarity scores.
Critical weaknesses
  1. The central theorem is overstated. Theorem 2 claims (1−CKA(X,X+ΔX)=Θ(ϵ2))(1-\mathrm{CKA}(X,X+\Delta X)=\Theta(\epsilon^2)) without excluding invariant perturbations. However, for (ΔX=ϵX)(\Delta X=\epsilon X), CKA is exactly one because of its invariance to isotropic scaling. Cancellation of the first-order term establishes neither a nonzero second-order coefficient nor a matching lower bound. The theorem requires additional nondegeneracy assumptions or a weaker statement. Moreover, first-order sensitivity of a probe’s score does not directly establish an equivalent advantage in classification accuracy or statistical detection.
  2. Model identification is conflated with strategy decoding. The probes in Section 5.2 predict which checkpoint produced a hidden state, rather than an independently defined reasoning strategy or behavior. High accuracy therefore demonstrates checkpoint distinguishability, but does not establish that the distinguishing directions encode the strategy responsible for output changes. Layer swapping does not directly connect these probe directions to behavior. NSARS inherits this limitation: incorporating checkpoint-classification accuracy makes it sensitive to differences, without establishing that those differences are functionally meaningful.
  3. The evidence does not justify dismissing capability narrowing as a measurement artifact. The main re-evaluation uses 100 MATH problems, one model configuration, and four training seeds. All reported confidence intervals for the pass@64 differences include negative values. These results support a narrower conclusion: narrowing was not reliably observed under this particular setup, and the authors’ original pipeline contained protocol inconsistencies. They do not establish that the broader findings of prior work are artifacts.
  4. The full-parameter versus LoRA comparison is confounded. Full-parameter experiments use 0.5B–1.5B models, whereas the LoRA experiment uses a 7B model. Consequently, the substantial CKA difference cannot be attributed specifically to the adaptation mechanism. A matched comparison using the same base model, data, and comparable training conditions is necessary to support the proposed cross-paradigm explanation.
Questions for authors
  1. Can Theorem 2 be revised to specify when the quadratic coefficient is nonzero and distinguish probe-score sensitivity from detection performance?
  2. Can the authors demonstrate that probe-derived directions causally affect a defined behavior, beyond identifying checkpoints?
  3. Can the capability-narrowing and LoRA conclusions be supported by broader replication and matched controls?
Reviewer_qMfPOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper begins by running a series of RLVR benchmarks on the Qwen model family, and calls into question the results of a previous paper (Yue 2025) with regards to a phenomenon called "capability narrowing". The paper then exposes the paradox that model updates that leave CKA unchanged can still change model outputs. Finally, the paper proposes a new measure, called NSARS, that combines CKA with linear probing.

Critical strengths
  1. The paper calls into question findings of a previously published influential paper. If true, this could be relevant for the community.
  2. The observation that changes small enough to keep CKA = 1 can still change model behaviour is interesting.
Critical weaknesses
  1. The paper is heavily AI-written with little attention to detail. As an example, line 563 says "The claim of a universal constant 2 in earlier drafts is therefore withdrawn", which no human author would leave after a sensible proofreading. As other examples, RLVR is first defined at the bottom of page 3, after 2.5 pages of talking about RLVR; and the paper keeps referring to "87 surveyed papers" but doesn't say anything about how the survey was conducted, which papers were included, or what it has to do with the rest of the paper. The paper is overall extremely difficult to follow, fails to provide adequate context, and virtually every section is full of undefined symbols.
  2. NSARS is defined arbitrarily. Multiplying CKA and probe accuracy is prone to all sorts of parameter sensitivities like dimensionality, regularisation, probe cross-validation, etc. This is defined without any theoretical backing and is not tested in sufficient depth.
  3. The "blind spot" theorem is obvious. CKA is a smooth function and is maximised at 1, so by basic calculus its first derivative is zero and any variation is O(ε2)\mathcal{O}(\varepsilon^2). I don't think this can be reasonably called a "blind spot" and it applies to every bounded similarity measure. The main result seems to be not fundamental and more a statement of the limitations of float32 and the behaviour of softmax.
  4. Evidence is not strong enough to overcome Yue 2026. This paper only uses two relatively small models from the same family in a much smaller task setting, so for this finding to be taken seriously would require stronger evidence.
Questions for authors

N/A

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.