acceptodds
Under review as a conference paper at ICLR 2027

What Does Reconstruction Fidelity Measure under Distribution Shift?

Abstract

Reconstruction fidelity, reported as variance explained or fraction of variance unexplained, is used to judge learned representations, including when a model fitted on one dataset reconstructs another. Under such distribution shift, one score mixes three quantities: credit from the reference it is measured against, error from the reconstruction's mean being in the wrong place, and the structure around the target's own mean that the reconstruction captures. Reference dependence and the split of mean-referenced squared error into mean bias and centered error are classical; for reconstructions confined to a convex set, as in archetypal analysis, we show that the score is at most the fit available after one free translation (the translated fit) minus the squared distance from the target mean to the set, so geometry alone can force a negative score. In cross-corpus transfer of archetypal dictionaries (K = 32) over text embeddings, 37 of 48 settings score below the target mean, yet their fit around that mean exceeds that of randomly rotated hulls, though it nearly matches hulls of randomly chosen source points; transferred dictionaries reach 32–77% of the target-informed translated fit of dictionaries fitted on the target corpus, and location is the larger share of the transfer excess in all 48 settings, at each of K = 8, 16, 32 and 64. Representing the same records by their original text makes every cross-corpus score negative. For the Gemma Scope sparse autoencoders we evaluate, the same split shows that on natural data these SAEs sit far from the zero-score boundary, and that their decoder bias supplies substantial location correction when the codes are held fixed. Reporting the three quantities separately keeps a negative score from being read as absence of target-centered fit.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
reconstruction fidelity, variance explained, distribution shift, evaluation methodology, archetypal analysis, sparse autoencoders, text embeddings, interpretability

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_9gFqOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies how reconstruction fidelity should be interpreted when a representation learned on a source distribution is evaluated on a different target distribution. The authors argue that a single reconstruction score, such as explained variance or fraction of variance unexplained, is insufficient under distribution shift because it conflates three distinct factors: the reference predictor used in the metric, the mismatch between the reconstructed and target means, and the ability to capture variation within the target distribution.

The paper provides theoretical results for convex reconstruction sets, together with a more extensive empirical study. In particular, it argues that a negative mean-referenced reconstruction score does not necessarily imply an absence of target-centered structure: reconstructions may capture some relative variation in the target while being globally misplaced in the representation space.

Critical strengths
  • The paper is generally easy to follow. The explicit discussion of “what is classical and what is new” in the introduction is particularly helpful for positioning the contribution. The motivating issue of negative reconstruction scores is also introduced clearly and compellingly.
  • The running example is effective: it makes the successive theoretical quantities and empirical observations easier to interpret.
  • The paper raises a relevant and well-motivated issue for reconstruction-fidelity metrics under distribution shift. Its recommended reporting practices could be useful for future work on dictionary learning and sparse autoencoders.
  • The experimental protocol appears careful. In particular, the use of group-disjoint splits, matched held-out-self baselines with equal capacity, numerical verification of signs, group-level bootstrap intervals, and structural controls strengthens the empirical study.
  • I appreciate that the paper does not overstate the novelty of the central decomposition, and clearly acknowledges its relationship to classical error decompositions.
Critical weaknesses
  • The central decomposition is explicitly classical. The main novelty therefore lies in its framing for reconstruction under distribution shift, the convex-set bound, and the empirical analysis. This is not necessarily a problem, especially since the paper is transparent about it, but it limits the conceptual novelty relative to a paper introducing an entirely new theoretical decomposition.
  • The strongest theoretical results apply to convex reconstruction sets and do not directly cover standard sparse autoencoders.
  • The SAE study is limited to Gemma 2 2B, Gemma Scope SAEs, three layers, and five text distributions whose genuinely out-of-distribution character is not established. As a result, this part currently reads more as an illustrative application of the fixed-map decomposition than as a sufficiently broad investigation of SAEs under distribution shift.

Special Note: I did not verify the mathematical proofs, as this is outside my area of expertise. My assessment focuses on the novelty, methodological design, and empirical evaluation. I would therefore encourage the Area Chair to ensure that at least one reviewer has carefully assessed the correctness of the theoretical results.

Questions for authors
  • Can an analogue of the proposed geometric certificate be developed for standard SAEs or other non-convex reconstruction maps?
  • To what extent do the conclusions change after centering, whitening, or otherwise reducing embedding anisotropy? In particular, how much of the observed importance of the location term is specific to L2-normalized text embeddings with high centroid mass?
3 comments
AuthorsOpenReview ↗

Non-convex SAE bounds, shifted SAEs, and whitening

Thank you for the review and the note to the AC (our General Response gives a four-step proof of Theorem 1 for that purpose). Short answers first:

  • SAE certificate: Only the location half of Theorem 1 carries over via the output envelope and decoder cone (new Proposition 3; up to 0.28σ0.28\sigma in Table J.3, below the 1σ1\sigma needed to force a negative score alone).
  • Explicit OOD SAE shifts: Keeping the 2B scope limitation in Section 10, we added Gemma 2 9B Base-to-IT (187,860 IT tokens; centered error dominates) and pooled CLIP SAEs evaluated on six benchmarks outside their training pool (output recentering moves out-of-pool steering from −1.71 to +1.59 on ViT-B/16 and −2.98 to +0.98 on ViT-B/32; Table J.7).
  • Centering, whitening, normalization: Translation invariance makes Rcen2R^2_{\mathrm{cen}}, RST2R^2_{\mathrm{ST}}, and ρH\rho_H independent of centroid mass mm; whitening cuts ρH2\rho_H^2 by up to 56.2% while rescuing 0 of 37 negative cells (Table J.1), and unnormalized CLIP tokens give 144 of 144 negative cross cells (Table J.2).

The decomposition is "explicitly classical"

Agreed—and keeping that boundary explicit is central to the paper. Beyond the classical split, we contribute the weight-free convex-set bound and sign certificate, Proposition 2's four-term hull split, the fixed-shape translation control, and cross-regime empirical evidence.

The theory does "not directly cover standard sparse autoencoders"; can an analogue of the certificate be developed?

A map's mean reconstruction lies in the closed convex envelope of its outputs; for non-negative codes, that envelope sits inside bdecb_{\mathrm{dec}} plus the cone of decoder directions, so the normalized distance ρ‾(∞)\underline\rho(\infty) to that cone bounds ρ^\hat\rho without target codes (Proposition 3, Appendix J.4). Translated fit RST2R^2_{\mathrm{ST}} does not carry over because re-optimizing in the envelope changes the encoder (Section 10). Across dictionaries (Table J.3):

  • Gemma Scope 2B, 16k: positive in 0 of 75 evaluations (the cone contains μ\mu).
  • Gemma Scope 9B, Base SAEs on IT activations: positive in 6 of 15, at most 0.035σ0.035\sigma.
  • Canonical CLIP ReLU top-kk SAEs: positive in 18 of 27 evaluations in three of four suites, up to 0.28σ0.28\sigma (median observed ρ^\hat\rho: 0.27σ0.27\sigma to 0.31σ0.31\sigma).

The SAE study is narrow, and the "genuinely out-of-distribution character is not established"

  • Gemma 2 9B Base-to-IT (layers 9, 20, 31; 15 L0-matched pairs; Appendix J.5): Layer-median Base–IT FVU gap is 0.02–0.04; in the 12 pairs with gap ≥0.02\ge 0.02, 79–88% (layer medians) is centered error and 12–21% is Δρ^2\Delta\hat\rho^2. Removing the offset lowers FVU, as Proposition 1 requires, yet worsens downstream loss at layers 20 and 31 (input-plus-output shifting is mixed; Table J.5).
  • CLIP top-kk SAEs on six unseen benchmarks outside a three-dataset pool (Appendix J.6): On the six OOD benchmarks, output recentering on unlabeled target images moves ViT-B/16 steering from −1.71 to +1.59 (nine-dataset: −1.59 to +1.36; Table J.7, canonical ReLU suites in Table J.9) and ViT-B/32 from −2.98 to +0.98, where EuroSAT (+7.03) carries the mean and 4 of 6 stay slightly below zero-shot (−0.08 to −0.86). Matching Chatzoudis et al.'s prompt text reproduces every published zero-shot score on the three in-pool datasets plus Food-101 on B/32 and gives the same centering pattern at α=1.05\alpha=1.05 (General Response; Table J.10).

Centering, whitening and anisotropy; L2 normalization and centroid mass

Every arm applies one shared transform, fitted on the source or on fourth corpus PFG, to both sides (Table J.1; centering each corpus on its own mean uses the target mean and removes location directly, Appendix B.1).

  • Centroid mass: Plain source centering lowers median mm from 0.801 to 0.390 while leaving translation-invariant Rcen2R^2_{\mathrm{cen}}, RST2R^2_{\mathrm{ST}}, ρH\rho_H, and the 37/48 negative count unchanged.
  • Anisotropy: All-but-the-top (r=10r=10) cuts ρH2\rho_H^2 by 23.4% and external PFG ZCA cuts it by 56.2%, but translated fit falls too (0.175 to 0.149 and 0.085), leaving excess location share at 90.0% and 62.0% (source ZCA collapses K=32K=32 translated fit to 0.024 even on held-out self). Across all 12 transforms, 0 of 37 negative cells turn non-negative (37–48 negative).
  • L2 normalization: Unnormalized 768-d CLIP CLS tokens give Rcen2<0R^2_{\mathrm{cen}}<0 in 144/144 cross cells, with 74.8% median excess location share (Table J.2).

Does this answer your second question as you meant it?

Ok thanks for your reply, that's seems good for me, I keep my accept.

Thank you for reading our response and for confirming your assessment. We are glad it addressed your questions. If anything still holds the paper back in your view, we would be glad to address it before the discussion closes.

Reviewer_VUxTOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that a reconstruction-fidelity score under distribution shift conflates three things: the reference it is measured against (origin vs target mean), the offset between the reconstruction mean and the target mean, and the target-centered structure actually captured.

Critical strengths

The accounting is correct, the proofs are clean, and the paper is scrupulous about what is classical (Murphy-style bias/centered split, centered vs uncentered R²) versus new (translated fit, centroid-to-set distance, the certificate, the four-term split). Controlled translation part is convincing, and nice to have empirical rigor.

Critical weaknesses

The new theory is a short step past the classical result. The question is whether the empirical payoff carries it, and the payoff is mixed.

The problem is largely absent in the regime the community cares about (the Gemma Scope SAEs). The demonstrated harm is therefore confined to cross-corpus archetypal-dictionary transfer, a niche practice.

Questions for authors

Could you address weakness above. specifically, Can you point to one or more published results where a reported fidelity number under shift was over- or under-read because location was invisible?

2 comments
AuthorsOpenReview ↗

Where location is small vs. decisive in SAEs

Thank you for the review. Short answers first:

  • Where location matters in SAEs: Full Gemma Scope reconstructions have tiny offset (ρ^2≤0.0016\hat\rho^2\le0.0016), but location becomes decisive when the vector applied downstream is not the full reconstruction: on Chatzoudis et al.'s matched CUB-200 baseline (55.16 vs. 55.06; Table J.10), full reconstructions fit well (Rcen2=0.916R^2_{\mathrm{cen}}=0.916), yet bias-stripped steering drops accuracy by 6.54 points (theirs: −5.84), centering at bpreb_{\mathrm{pre}} (the SAE's learned center, initialized at its training-pool mean) with no SAE drops it by 6.82, and recentering the push on unlabeled target images removes the drop (−0.17; scope in General Response).
  • Published numbers: O'Neill et al. (2025, Fig. 2) is a clean null (omitted ρ^2\hat\rho^2 under 0.8 percentage points; Table J.6); in Chatzoudis et al., switching between their two FVU references shifts Flowers-102 gated coverage from 49.6% to 0.1% in our suite (Table J.8).
  • Short theoretical step: Theorem 1's simplicity is what makes it a clean diagnostic—certifying negative signs from the model set and target alone (33/37 negative cells) and pairing with a rigid-shift control that turns all 2,400 evaluations positive.

"a short step past the classical result"

Agreed that Proposition 1 plus two one-line relaxations is elementary. Using only the model set and target keeps the sign bound independent of solver weights, while rigid translation isolates location at fixed translated fit across the cross-corpus evaluations (Appendix B.1).

"largely absent in the regime the community cares about"

For full Gemma Scope residual-stream outputs, our 9B Base-to-IT check agrees with you (Appendix J.5): the layer-median FVU gap is 0.02–0.04, Δρ^2\Delta\hat\rho^2 is 12–21% of it, and output shifting worsens downstream loss at layers 20 and 31 (input-plus-output is mixed; Table J.5).

Location is larger in three SAE settings:

  • Fixed-code bias removal at 2B: dropping bdecb_{\mathrm{dec}} lowers Rcen2R^2_{\mathrm{cen}} by 0.33–0.71 (Section 7).

  • Canonical ReLU top-kk CLIP SAEs: median full-output ρ^\hat\rho is 0.17σ0.17\sigma to 0.31σ0.31\sigma (ρ^2\hat\rho^2 of 0.03–0.10; Table J.3).

  • Bias-stripped CLIP steering (Section 7, Appendix J.6): omitting bpreb_{\mathrm{pre}} gives B/16 CUB-200 push FVU 1.55 = 0.08 centered + 1.47 mean offset (adding constant push α(μD−bpre)\alpha(\mu_D-b_{\mathrm{pre}}) to their Proposition 3.1). Across nine benchmarks under our standard prompt (Table J.7), centering at bpreb_{\mathrm{pre}} with no SAE accounts for 98% of pooled steering's shortfall below target-mean steering on B/16 and 81% on B/32; output recentering beats pooled steering on all 9/9 benchmarks, moving means from −1.59 to +1.36 on B/16 (0/9 below zero-shot, matching target-trained +1.30) and −2.29 to +0.86 on B/32 (5/9 below zero-shot, vs. +1.41 target-trained; EuroSAT-carried).

    Four canonical ReLU suites likewise move from [−2.42, −1.34] to [+0.25, +1.21]. Without EuroSAT, recentering still beats pooled steering by +1.62 to +2.23 points in all six suites; what remains vs. zero-shot (+0.92 down to −0.32) follows validation FVU in 6/6 inverse order (Table J.9). Matching their prompt text matches all nine published zero-shot scores on the three in-pool datasets (six single-prompt, three ensemble) plus Food-101 on B/32, tracks their Figure 3 α\alpha-curve on CIFAR-100 (0.10 MAD) and CUB-200's small-α\alpha peak shape while diverging on Tiny-ImageNet (9.16 MAD), and at α=1.05\alpha=1.05 gives the same wrong-center cost and recentering gain in all 8 matched cells (Table J.10). Across α∈[0.01,6.00]\alpha\in[0.01,6.00], wrong-center cost is negative at ≥251/255\ge 251/255 values in 7 of 8 cells and largest on CUB-200 throughout (General Response).

A published number "over- or under-read because location was invisible"

Not a confirmed misreading of released checkpoints:

  • O'Neill et al. (2025, Eq. 13): omits ρ^2\hat\rho^2 as written, measuring 0.09–0.76 pp across all 21 Gemma Scope points (Table J.6).
  • Two FVU references in one paper: Chatzoudis et al. use the training mean for per-sample FVU (Section 3.2) and the batch mean for batch FVU (Appendix C), shifting B/32 Flowers-102 (q=0.95q=0.95) gated coverage from 49.6% to 0.1% here (Table J.8); without their checkpoints, we do not diagnose their 1.93 CUB-200 FVU.

"a niche practice" and presentation

Archetypal dictionaries give a convex model system where the sign bound and translation control are exact; plain-named Table 1 (Section 3.3, top of page 6) puts the related quantities and their scope side by side. We left almost all reviewed main-text prose untouched during discussion so the PDF diff stays easy to check, and will tighten dense passages across the main text for the camera-ready; if a specific passage drove your presentation score or any concern remains, please let us know.

Dear Reviewer VUxT, a brief note before the discussion closes on Nov 18. On whether location matters in the SAE regime: full Gemma Scope reconstructions have tiny offset, as you noted. It becomes decisive when the vector applied downstream is not the full reconstruction. On Chatzoudis et al.'s CUB-200 setting, with our zero-shot baseline matching theirs (55.16 vs. 55.06), reconstructions fit well (R2_cen = 0.916), yet bias-stripped steering lowers accuracy by 6.54 points (theirs: 5.84). Recentering the push on unlabeled target images removes the drop (-0.17; Table J.10). Does this address your concern? We are glad to clarify anything in the time left.

Reviewer_UPJmOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks a well-motivated measurement question about the failure mode of a commonly used metric for reconstruction fidelity, namely the R2R^2 score. A poor reconstruction score can still fit the variation in data well even if the reconstructions are offset by a wrong mean. The paper studies this on 3 conversational datasets by fitting a representation model on one dataset and evaluating on another.

The first part of the paper gives a classical decomposition of the R2R^2 score into a recentered fit score and the mismatch between the means between the target and reconstruction. The new theoretical result in the paper extends this to closed convex reconstruction sets. The authors validate this analysis using Principal Convex Hull Analysis (PCHA) dictionaries, a method similar in spirit to classical PCA. They also apply it to Sparse Autoencoders (SAEs). The distinction between the classical results and the new theoretical results are clear, however closely related quantities made the presentation hard to follow than the underlying hardness of the idea.

I did not clearly understand though if the main convex set theory developed also applies to the SAE case. Overall, writing is understandable but there is room for making it sharper. I did not check each and every step of the math derivations but they seem correct from a glance and seem to be a nice extension of the classical result.

Critical strengths
  • S1: The paper starts with classical result and extends to a closed convex reconstruction set. It decomposes the score into two components: variation in the target data and the error caused by placing reconstructions around the wrong mean. For the convex reconstruction model, the latter component can result in a negative overall score even when some variation in target data is captured.
  • S2: The PCHA experiments contain several good controls for a measurement paper such as held out self-dictionaries, random , rotated, isotropic hulls. The authors also propose and test a translation intervention which directly tests the proposed theoretical explanation.
Critical weaknesses
  • W1: The paper shows that R2R^2 score can be misleading. But it does not show how correcting for this interpretation can lead to a change in a real decision. A usecase or an example on how a practitioner could use this interpretation to inform model selection or predict some downstream behavior would make the case for practical utility.

  • W2: SAE are a common interpretability tool for modern LLMs but the sparse autoencoding application (SAE) does not have a strong overlap with the PCHA theory. The geometry of SAE is somewhat different which does not fit the theory nicely. The paper would be stronger with some extension or discussion on how it could be extended to nonlinear sparse encoders.

  • W3: The motivation for using these negotiation style conversational datasets is also not very clear. The PCHA experiments use LLM-generated blueprints from three conversational datasets. It is unclear why this setting is representative of reconstruction fidelity problems more broadly.

Questions for authors
  1. Can the authors show a case where the decomposition changes model selection
  2. What evidence supports extending the PCHA-based conclusions to nonlinear SAEs and to representation settings beyond conversational blueprint embeddings?
2 comments
AuthorsOpenReview ↗

Downstream steering, SAE scope, and new domains

Thank you for the review. Short answers first:

  • W1/Q1 (Model selection & downstream payoff): Yes. In Appendix E.6, 88% of PCHA's lead over random source points is mean placement, changing whether fitting PCHA pays off once recentered. Downstream, decomposing the applied vector separates a wrong-center drop in CLIP steering from centered error in Gemma 2 9B.
  • W2/Q2 (Convex theory vs. SAEs): Proposition 1 applies unchanged; of the convex-set theory, only the location half carries over via the output envelope and decoder cone (new Proposition 3; Table 1).
  • W3 (Beyond negotiation text): We do not claim negotiation text is representative; negative cross-domain Rcen2R^2_{\mathrm{cen}} recurs in 144/144 unnormalized (143/144 normalized) CLIP image cells and 54/60 expository, code, and news text cells (Table J.2).

W1 and Q1: a case where the split changes model selection

Already in the submission (Appendix E.6, Table E.8). On the mean-referenced score, PCHA archetypes beat hulls of 32 random source points in 48/48 cells. On recentered fit, PCHA leads in 33/48 by a 0.005 median: 88% of its edge is a better-placed mean, so whether fitting PCHA pays off depends on whether outputs can be recentered. Extreme Support's edge over PCHA is almost all recentered fit, RST2R^2_{\mathrm{ST}} swaps their winner in 6/60 cells, and all five summaries score below zero in 46/60—so a sign rule would discard every candidate (Appendix C.3).

Downstream CLIP steering (Section 7, Appendix J.6). With one pooled signed top-kk SAE, steering drops CUB-200 accuracy by 5.66 points under our standard prompt (6.54 on Chatzoudis et al.'s matched baseline, vs. their −5.84; Table J.10), though full reconstructions fit well (Rcen2=0.916R^2_{\mathrm{cen}}=0.916, ρ^2<10−4\hat\rho^2<10^{-4} on ViT-B/16). Scoring f(x)Wdecf(x)W_{\mathrm{dec}} against x−μDx-\mu_D shows why: omitting bpreb_{\mathrm{pre}} (the SAE's learned center, initialized at the pool mean) gives FVU 1.55 = 0.08 centered + 1.47 offset. Nine-dataset steering means (Δ\Delta pp, α=1.05\alpha=1.05; Table J.7):

ArmB/16B/32
Pooled SAE−1.59−2.29
Same center (bpreb_{\mathrm{pre}}), no SAE−1.54−1.58
Pooled SAE, output-centered+1.36+0.86
Pooled SAE, input-centered+1.38+1.11
Target-trained SAE+1.30+1.41
Target mean, no SAE+1.42+1.36

Against no-SAE target-mean steering, centering at bpreb_{\mathrm{pre}} accounts for 98% of pooled steering's shortfall on B/16 and 81% on B/32, where the dictionary adds harm of its own. Output recentering on unlabeled target images beats pooled steering on 9/9 datasets on both backbones, matching target-trained SAEs on B/16 (0/9 below zero-shot) and leaving 5/9 below zero-shot on B/32 (EuroSAT at +7.03 carries +0.86; 0/9 at α=0.50\alpha=0.50), with 64 unlabeled images within 0.35 points. Limits: CUB-200 is 3.8% of the SAE pool, no-SAE target-mean steering matches the best SAE arm, and our SAEs are not their unreleased checkpoints (General Response). Contrast & boundary cases: On Gemma 2 9B Base-to-IT (Appendix J.5), offset is 12–21% of the gap and output shifting hurts downstream loss at layers 20 and 31; in Appendix J.3, K=32K=32 CLIP hull reconstructions have 0.04 median recentered fit (3.85% accuracy vs. 60.52% raw CLIP), and mean alignment lowers accuracy to 2.97%.

W2 and Q2: overlap between the convex theory and SAEs

Table 1 (Section 3.3, top of page 6) maps what applies where:

  • Unchanged: Proposition 1, for any fixed operator.
  • Partial (Proposition 3): The location half of Theorem 1 via the closed convex envelope. For ReLU top-kk CLIP SAEs, the cone bound is positive in 18/27 evaluations in three of four suites (up to 0.28σ0.28\sigma), vs. 0/75 at Gemma Scope 2B and ≤0.035σ\le 0.035\sigma at 9B (Table J.3; Appendix J.4 uses a mean-L1L_1 polytope up to 0.28σ0.28\sigma for signed top-kk).
  • Does not carry over: RST2R^2_{\mathrm{ST}}, Corollary 1, and Proposition 2's four-term split.

W3: why negotiation corpora

We do not claim negotiation text is representative. Blueprints strip topical vocabulary to give transfer its best chance (Section 4), yet 37/48 cells are negative (60/60 on raw text; Section 6). Repeating K=32K=32 PCHA gives Rcen2<0R^2_{\mathrm{cen}}<0 in 144/144 unnormalized (143/144 L2-normalized) CLIP vision cells and 54/60 expository, code, and news cells (Table J.2).

Presentation

Table 1 (top of page 6) puts the closely related quantities side by side in plain words, including an "Applies to" column showing which results hold for any map and which need a convex set. We left Section 1 and almost all reviewed main-text prose untouched during discussion so the PDF diff stays easy to check; for the camera-ready, we will state the SAE scope upfront in Section 1 and tighten dense passages across the main text. Please let us know if a specific section felt hard to follow or if any concern remains.

Dear Reviewer UPJm, a brief note before the discussion closes on Nov 18. On your main question, whether the split changes model selection: PCHA beats random source hulls in all 48 cells on the usual score, but loses its lead in 15 of 48 once location is removed. The CLIP steering example shows the same thing downstream. Does this answer your question? We are glad to clarify anything in the time left.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General response: summary of revisions, caveats, and short proof of Theorem 1

We thank the three reviewers.

Main takeaway from the revision. Separating mean offset from centered error on the deployed operator (the vector applied downstream, not the full reconstruction) shows why a score moved and when recentering helps:

  • Large offset share (CLIP steering push, Section 7 & Appendix J.6). On CUB-200 (ViT-B/16), full reconstructions fit well (Rcen2=0.916R^2_{\mathrm{cen}}=0.916), yet the bias-stripped push omits bpreb_{\mathrm{pre}} (the SAE's learned center, initialized at its training-pool mean) and has FVU 1.55, of which 1.47 is mean offset (0.08 centered). Output recentering on unlabeled target images beats pooled steering on all 9/9 benchmarks on both backbones, moving means from −1.59 to +1.36 on ViT-B/16 (0/9 below zero-shot) and −2.29 to +0.86 on ViT-B/32, with 5/9 still below zero-shot (Table J.7).
  • Small offset share (Gemma 2 9B Base-to-IT, Appendix J.5). Offset is 12–21% of a 0.02–0.04 layer-median FVU gap; output shifting lowers FVU slightly yet worsens downstream loss at layers 20 and 31 (input-plus-output shifting is mixed; Table J.5).

These are two worked cases, not a decision rule (Appendix J.3 adds a boundary case with 0.04 centered fit where recentering does not help).

Where we agree on scope—and what the simple bound buys

  1. As Sections 1–2 state, the base split is classical and Theorem 1 is a short convex-set step beyond it (9gFq, VUxT). That simplicity is what makes it a clean diagnostic: from the model set and target alone, it certifies negative signs regardless of weights (33/37 negative cells) and pairs with rigid translation to isolate location at fixed translated fit (all 2,400 evaluations positive).
  2. Full reconstructions of Gemma Scope residual-stream SAEs sit in the centered-mismatch regime at both 2B and 9B (VUxT), in contrast to bias-stripped CLIP steering, where offset dominates.

What changed in the PDF

Unchanged: Abstract, Section 1, Section 4, Section 5 prose, original Section 6, Section 7's natural-data/bias paragraphs, Figures 1–2. No original claim depends on Appendix J. Changed:

  • Section 3: Plain-named Table 1 (introduced in Section 3.3, top of page 6; former Tables 1–2 become 2–3), adding reference rescaling and Proposition 3; Section 3.1 now points to Proposition 3.
  • Sections 6–10: Whitening and vision/text PCHA (Section 6); 7.1×7.1\times width note, Proposition 3 (replacing two "untested" sentences), 9B Base-to-IT, and CLIP steering (Section 7); deployed-operator wording and Appendix J scope, keeping all original limitations (Sections 8–10).
  • Appendices: New Appendix J (J.2 whitening, J.3 vision/text PCHA, J.4 Proposition 3, J.5 Gemma 2 9B & O'Neill Figure 2, J.6 CLIP steering & matched-prompt/α\alpha-curves); two counts in C.3 (6/60, 46/60); pointer/scope updates in A, D.4, G, I, and page 11.

Caveats

  1. CLIP steering scope (Tables J.7–J.10). Chatzoudis et al. (arXiv:2506.01247v3) do not release checkpoints. Our CLIP pipeline matches Radford et al. (2021, Table 11) on CIFAR-100, Food-101, and SUN397; dropping the period in "a photo of a {label}" with dataset label strings matches all nine published zero-shot scores on the three in-pool datasets (six single-prompt, three ensemble; 0.00–0.10 pp) and Food-101 on B/32 (0.01 pp; five OOD splits stay open). On ViT-B/32, our target SAE tracks their Figure 3 α\alpha-curve on CIFAR-100 (0.10 MAD) and CUB-200's small-α\alpha peak shape while diverging on Tiny-ImageNet (9.16 MAD). At α=1.05\alpha=1.05, wrong-center cost and recentering gain hold in all 8 matched cells (Table J.10), while across α∈[0.01,6.00]\alpha\in[0.01,6.00] wrong-center cost is negative at ≥251/255\ge 251/255 values in 7 of 8 cells and largest on CUB-200 throughout. Because CUB-200 FVU, gate coverage, and Food-101 B/32 sign (−2.15 vs. +0.34) still differ, we do not diagnose their checkpoints.
  2. O'Neill et al. (2025) Figure 2 audit (Table J.6). Null result: omitted ρ^2\hat\rho^2 across 21 Gemma Scope points is 0.09–0.76 percentage points.
  3. Proposition 3 (Table J.3). Bounds location only (at most 0.28σ0.28\sigma here, below 1σ1\sigma).

For the AC, on 9gFq's note about the proofs

Reviewer VUxT noted that "The accounting is correct, the proofs are clean", and Theorem 1 is four short steps (μ^\hat\mu is the mean of the reconstructed rows):

  1. Proposition 1: X−X^X-\hat X splits orthogonally into μ−μ^\mu-\hat\mu and its centered part; dividing by TSS gives Rcen2=Rshift2−ρ^2R^2_{\mathrm{cen}}=R^2_{\mathrm{shift}}-\hat\rho^2.
  2. Lemma 2: fixed X^\hat X with shift μ−μ^\mu-\hat\mu is feasible in the free-translation problem, so Rshift2≤RST2R^2_{\mathrm{shift}}\le R^2_{\mathrm{ST}}.
  3. Lemma 3: μ^\hat\mu averages points of convex HH, so μ^∈H\hat\mu\in H and ρ^≥ρH\hat\rho\ge\rho_H.
  4. Theorem 1 combines 1–3: Rcen2≤RST2−ρH2R^2_{\mathrm{cen}}\le R^2_{\mathrm{ST}}-\rho_H^2. Proposition 3 reuses step 3 on the closed convex envelope of a map's outputs (Appendices A.7, E.3, J.4).
AuthorsOpenReview ↗

Second revision uploaded: clarifications only

We uploaded a second revision. No table value or result changed. (1) Section 7 and Appendix J.5 now say that the 9B downstream-loss result is for output-only shifting, and report the input-plus-output arm of Table J.5, which is mixed: better at layers 9 and 31, worse at layer 20. Our replies are edited to match. (2) Section 7 states that CUB-200 is 3.8% of the pooled SAE's training data. (3) The new Section 6 paragraph is shorter and points to Tables J.1–J.2, and Table 1 is in larger type. (4) Appendix J.4 states the envelope used for signed top-kk codes (a symmetric mean-L1L_1 polytope, up to 0.28σ0.28\sigma), and the captions of Tables J.3–J.5 define every column and count.

AuthorsOpenReview ↗

Third Revision

We uploaded a third revision and edited our four replies in place so each leads with short answers. No existing number in the main text or Tables J.7–J.9 changed, and the main text still ends on page 10. Three points changed in substance:

  1. Prompt text explained the in-pool zero-shot gap (Table J.10). We had written that our CLIP pipeline did not match Chatzoudis et al. (2026) on zero-shot baselines. Checking our pipeline against Radford et al. (2021, Table 11) gives 65.08, 84.01, and 63.53 vs. 65.1, 84.4, and 63.2 on CIFAR-100, Food-101, and SUN397, and dropping the period in "a photo of a {label}" with dataset label strings matches all nine published zero-shot scores on the three in-pool datasets (six single-prompt, three ensemble) within 0.00–0.10 points and Food-101 on ViT-B/32 within 0.01. At α=1.05\alpha=1.05, CUB-200 pooled harm (−6.54, −4.44), no-SAE bpreb_{\mathrm{pre}} harm (−6.82, −4.05), and output-recentering recovery (−0.17, −0.82; positive gain in all 8/8 matched cells) all hold. Across 255 values of α∈[0.01,6.00]\alpha\in[0.01,6.00], our ViT-B/32 target SAE tracks their Figure 3 curve on CIFAR-100 (0.10 MAD) and CUB-200's small-α\alpha peak (+0.94 at α=0.25\alpha=0.25 vs. −1.28 at α=1.05\alpha=1.05) while diverging on Tiny-ImageNet (9.16 MAD), and the wrong-center cost is negative at ≥251/255\ge 251/255 values in 7 of 8 cells and largest on CUB-200 throughout.
  2. Exact arm formulas and wrong-center share (Section 7, Appendix J.6). Revision 2's prose called arm (ii) source-mean centering and wrote xˉD,tr\bar{x} _ {D,\mathrm{tr}} in arm (iv); the code centers arm (ii) at the SAE's trained pre-bias bpreb_{\mathrm{pre}} (x+α(x−bpre)x+\alpha(x-b_{\mathrm{pre}}); 3-seed SDs 0.01–0.53 pp noted in Table J.7) and subtracts the target-train mean reconstruction x^ˉD,tr\bar{\hat{x}} _ {D,\mathrm{tr}} in arm (iv) (x+α(x^(x)−x^ˉD,tr)x+\alpha(\hat{x}(x)-\bar{\hat{x}} _ {D,\mathrm{tr}})), with zero number changes. We also state the wrong-center share against no-SAE target-mean steering (98% on ViT-B/16, 81% on ViT-B/32) rather than against zero-shot ("nearly all" and "about two-thirds" in earlier replies).
  3. Table J.9 without EuroSAT. A final column reports 8-dataset output-centered means excluding EuroSAT (+0.92 down to −0.32): recentering still beats pooled steering by +1.62 to +2.23 points in all six suites, while the remaining change vs. zero-shot follows validation FVU (0.0629 to 0.2158) in 6/6 inverse order.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.