- Rating
- 1(−1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1(−1)
- Soundness
- 2
Read review
Paper summary
The paper investigates whether activation responses observed in one sparse autoencoder remain recoverable across independently fitted dictionaries, including before and after model adaptation. It compares predictors using small groups of target latents against an oracle that selects and affinely fits the best individual target latent directly on evaluation data. Experiments on TinyLlama and Qwen3 show that compact groups recover some responses beyond this singleton ceiling. The authors compare unbalanced optimal transport (UOT), descriptor-cost selection, and activation-correlation selection; simpler selectors achieve higher recovery than UOT.
Critical strengths
S1. The affine-singleton oracle provides a strong comparator. Because it searches the entire target dictionary and optimizes on evaluation inputs, successful group predictors demonstrate a limitation of affine single-latent prediction on those inputs that cannot be explained by an unsuccessful matching algorithm.
S2. Independent SAE fits, unchanged-model refits, and separately calibrated group predictors support the central empirical finding. The authors also transparently report that simpler selectors outperform UOT and carefully distinguish response recovery from semantic or causal equivalence.
Critical weaknesses
W1. Limited novelty. SAE feature matching across random seeds has already been studied extensively in [1], which compares several matching approaches, including optimal transport, and evaluates them through functional tests. Moreover, [2] already shows that unstable individual SAE features can span reproducible subspaces. The present work extends these ideas to fine-tuning and compact activation reconstruction, but the overall contribution appears incremental.
W2. Limited value of the proposed method. Despite the substantial development of unbalanced optimal transport, simple descriptor-cost and activation-correlation baselines consistently outperform it. The paper therefore provides little evidence that its more sophisticated matching procedure is necessary or beneficial.
W3. Limited functional validation. The main result shows that combining several target features improves activation reconstruction. However, improved reconstruction does not necessarily imply semantic or causal equivalence. Compared with the intervention-based evaluation in [1], the paper provides limited evidence that these recovered groups represent functionally corresponding features.
[1] Benchmarking cross-seed feature correspondence in sparse autoencoders. 2026. https://openreview.net/forum?id=5cy6WtSC8f
[2] Gerasimov et al. Unstable features, reproducible subspaces: Understanding seed dependence in sparse autoencoders. 2026. https://arxiv.org/abs/2606.12138
Questions for authors
Q1. How does the proposed approach advance beyond the matching methods already studied in [1] and the reproducible-subspace findings in [2]? Could the authors clarify what fundamentally new insight is established by compact response recovery?
Q2. Given that simple descriptor-cost and activation-correlation matching consistently outperform UOT, what practical advantage does the proposed transport formulation provide? Are there settings where UOT performs better?
Q3. How does the proposed method compare with simple greedy forward selection or sparse regression using the same four-feature budget? Could these baselines explain most of the observed improvement over singleton matching?
Q4. Do the recovered feature groups preserve causal effects under activation substitution or ablation, as evaluated in [1]? How can we establish that successful activation reconstruction corresponds to meaningful functional equivalence?
7 comments
Reply to 3GLL — Part 2/5
W1 & Q1 — continued
Two researchers testing whether a source activation response stays reproducible on the same fine-tuned checkpoint can reach opposite verdicts from an exhaustive 1-to-1 adequacy test solely because they trained different target SAE seeds. Among the responses rescued across all five target fits by four-feature cost groups where the singleton oracle fails in at least one seed (2,975 on TinyLlama, 14.5% vs. 0.5% in reverse; 2,203 on Qwen3, 11.0% vs. 0.8% in reverse), 89.4% and 90.2% pass as a single feature in at least one target seed yet fail the singleton oracle in another seed where a calibrated four-feature group succeeds.
Per-seed singleton pass rates are nearly flat across (36.9%–37.4% on TinyLlama; 28.1%–29.2% on Qwen3), so no tested seed consistently eliminates this dictionary dependence. Where a singleton passes (as in in Figure 1), no group is needed; recording each dictionary's fitted predictor and held-out error (such as two latents in ) retains responses that a singleton-only criterion would miss, without a canonical vocabulary (Abstract; Section 6). Because calibration fits the source response, a successful predictor shows that a few target features can estimate that response on the tested distribution. It does not tell us whether native activation amplitudes changed or whether causal circuits were preserved.
Using the same four target features, fitting weights to activations improves held-out response recovery over the tested decoder-projection weighting rule (Reply to HrVG, Table R1). On the four nearest target decoder directions, static projection (calibrated scalar gain and offset) recovers 26.0% on TinyLlama and 15.9% on Qwen3 (barely above Qwen3's 15.0% exhaustive singleton oracle); calibrating those four coefficients on activations raises five-dictionary recovery to 32.8% and 24.0%. Decoder geometry identifies where to look; activation calibration sets the relative weights that carry to held-out text. On recovered source–target pairs, four-step nonnegative OMP places over 72% of its fitted weight inside each source's eight-nearest descriptor-cost set ; excluding drops OMP recovery to 4.4% and 3.6%, while prevalence-matched random groups and positively correlated near-orthogonal targets recover little.
Same-checkpoint, threshold, and split controls: Holding the checkpoint fixed and varying only the SAE seed (Section 5.3, p. 7; Appendices A.2–A.4, e.g., 25.7% singleton vs. 39.0%–42.2% four-feature recovery across documents) produces the same separation, isolating dictionary re-estimation from fine-tuning. Requiring singleton error in one target fit and in another still flags 22.2% of TinyLlama and 16.0% of Qwen3 records (over 31% recovered across all five fits by frozen cost groups); evaluating descriptor-free predictors (decoder cosine, correlation, OMP, Lasso) on the unused 1,998 descriptor documents reproduces the separation alongside our token (Table 1) and 0.15–0.35 cutoff (Figure 2) checks.
W3 & Q4: Cohort-level causal interventions (activation substitution and ablation)
Although fitted only to activations rather than downstream logits, four-feature groups selected by decoder cosine or descriptor cost cut held-out substitution squared logit error by 36%–39% and weighted-ablation shift error by 30.5%–34.7% relative to matched singletons across all four target seeds on our class-selective cohort. To isolate dictionary correspondence from downstream fine-tuning, we compare reseeded SAEs on a fixed TinyLlama-1.1B checkpoint (, 256 evaluation documents), following [1]'s fixed-model setting. Our primary cohort comprises the 30 most class-selective features (tie-averaged calibration AUROC) among the 2,987 active in calibration documents whose best calibration singleton fails in at least one target seed (0.188 zero-ablation margin MAE).
Reply to 3GLL — Part 3/5
W3 & Q4 — continued
Candidate selection uses splits disjoint from the 256 evaluation documents (unit-decoder cosine for closest decoder, 1,998 descriptor docs for UOT/cost, 998 calibration docs for correlation); every and arm freezes 998-doc calibration weights before evaluation and never fits downstream logits. Document-calibrated slopes apply tokenwise without an intercept, so inactive targets contribute zero. Because the patched vector sums all 30 source-to-group terms, these tests evaluate cohort-level readout fidelity rather than per-feature causality. For Layer-11 substitution (Table R6), we subtract , add , and run the remaining layers:
scoring source-normalized residual vector error, zero-ablation-normalized four-class logit error (= 1.000), and true-minus-best-other margin MAE:
Table R6: Layer-11 activation substitution on class-selective cohort ( to –, mean ± SD across 4 target seeds)
| Substitution arm | Sq. Vector Error | Sq. Logit Error | Margin MAE |
|---|---|---|---|
| Zero ablation (remove source cohort, insert 0; reference) | 1.000 ± 0.000 | 1.000 ± 0.000 | 0.188 ± 0.000 |
| Positive calibration correlation top-1 (, NNLS) | 0.982 ± 0.144 | 0.429 ± 0.079 | 0.110 ± 0.006 |
| Descriptor-cost top-1 (, NNLS) | 0.772 ± 0.042 | 0.358 ± 0.092 | 0.103 ± 0.009 |
| Closest decoder direction top-1 (, NNLS) | 0.768 ± 0.039 | 0.339 ± 0.068 | 0.098 ± 0.003 |
| Closest decoder directions top-4 (, NNLS) | 0.663 ± 0.023 | 0.217 ± 0.034 | 0.079 ± 0.003 |
| Descriptor-cost top-4 (, NNLS) | 0.661 ± 0.026 | 0.218 ± 0.019 | 0.081 ± 0.004 |
| UOT top-4 (, NNLS) | 0.713 ± 0.022 | 0.252 ± 0.020 | 0.086 ± 0.002 |
| Positive calibration correlation top-4 (, NNLS) | 0.768 ± 0.041 | 0.294 ± 0.052 | 0.092 ± 0.011 |
| Orthogonal positively correlated top-4 (abs(cos) < 0.10, NNLS) | 1.586 ± 0.213 | 0.491 ± 0.037 | 0.122 ± 0.004 |
On , decoder-selected singletons outperform correlation singletons on substitution, consistent with target decoder direction mattering in residual-stream interventions. Expanding from to within each local selector lowers squared logit error by 36% (closest decoder) and 39% (descriptor cost), improving all four seeds, whereas correlated near-orthogonal targets () perform worse than .
Rebuttal to 3GLL — Part 1/5
Reply to 3GLL — Part 1/5
Thank you for highlighting the strength of our evaluation-fitted singleton comparator (S1) and the distinction between response recovery and causal equivalence (S2). The new capability is to reuse a specified activation measurement across existing, independently learned dictionaries even when no affine single-feature predictor can recover it adequately. Prior findings of feature instability and shared decoder subspaces leave open whether the same input-dependent response remains accessible through a few target activations on held-out inputs. Establishing that it does gives researchers a way to carry numerical observations across SAE fits and model adaptation without requiring a shared feature vocabulary or jointly retraining the dictionaries.
Across five base and five adapted SAEs per model, descriptor-cost predictors using at most four target features raise recovery in all five target dictionaries from 19.3% to 33.3% on TinyLlama and 15.0% to 25.2% on Qwen3 at NSE≤0.25. Requiring an adequate individual feature therefore excludes a substantial set of measurements that remain recoverable across all five target dictionaries. For these measurements, reproducibility does not require the same individual-feature representation to recur. These predictors use existing dictionaries, are fitted on separate calibration data, and are fixed before evaluation; the singleton comparator is fitted on evaluation data. Among responses that both methods recover in a common target fit, the groups also nearly halve the failure rate in the other target fits.
W1 & Q1: Retaining reproducible response measurements beyond [1] and [2]
With the target transformer held fixed, an affine single-feature predictor can meet the error threshold in one SAE fit and fail in another. Calibrated groups recover additional responses across the target fits. Standard sparse regression reproduces this gain (OMP and Lasso, Q3). Fitting activation weights also improves recovery over the tested decoder-projection weights on the same four target features. [1] benchmarks pairwise matching through functional interventions and visualizes split/merge plans; [2] identifies reproducible decoder subspaces under basis ambiguity. We measure whether a specified source response remains accurately predictable across independently learned target dictionaries (Section 2, p. 2).
We test this on existing SAEs across mutually disjoint 1,998-document descriptor, 998-document calibration, and 7,600-document evaluation splits. Holding each base-SAE response and the fine-tuned model fixed across five target SAE seeds, the singleton comparator searches all 4,096 targets and fits signed scale and offset directly on evaluation text. On those evaluation responses, this oracle upper-bounds every 1-to-1 matcher restricted to predicting from a single target latent with an affine map, however it selects that partner. Where a calibration-frozen group (≤4 latents, fitted on the 998 calibration documents) passes NSE≤0.25 and this oracle fails, at most four target coordinates recover the response where no affine singleton does (Section 3.1):
Pooled across five base SAEs (each target fit uses a separately calibrated, frozen predictor). Rows 1–3 use all eligible records; row 4 conditions on row 1 (UOT: 32.2%, 30.9%). Row 5 shows that on the same 37,577 TinyLlama and 28,074 Qwen3 (source, Br) positive anchors, four-feature recovery nearly halves failure across the four other seeds Bs=r (150,308 and 112,296 ordered comparisons; using each method's own Br passes gives 23.6% → 15.8% and 24.1% → 16.9%). On the fixed set passing ≥1 singleton seed, five-seed mixed outcomes fall from 65.0% to 41.9% and 65.4% to 43.4% (5/5 rising 35.0% → 57.7% and 34.6% → 55.7%, with only 0.5% and 1.0% falling to 0/5), while unconditional mixed rates are 35.8% → 32.4% and 28.4% → 27.1% as 0/5 singleton failures enter partial recovery.