- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
using Qwen3-30B-A3B to solve linear system, the paper attenuates one expert at one token at layer 16 and measures the effect against BF16 nd FP64 reference. FP64 answer effect is 0.02 to 0.04 nat against a 0.5 nat natural execution range and BF16 paris inflate to 3 to 9 times with near chance sign agreenment.
Critical strengths
The single BF16 paired interventions agree with FP64 in sign only 52% to 57% of the time, and making the readout FP32 keeps 95.8% of the spread. This is a reusable warning for activation patching and drives my recommendation. The same state peer comparisons, matched controls, family bootstrap, a pre registered prediction that failed and is reported.
Critical weaknesses
The recall is measured on probe selected candidates at low precision, the nominee adds only 0.017 over its closest weight peer while the heaviest expert alone gives 0.027 [0.014, 0.041], and the content result concerns the probe's own pooling weights while the pre registered answer prediction was not confirmed. One model, one layer, one task, and no larger intervention, so the paper cannot separate "BF16 is too noisy" from "this intervention is too small to matter"
Questions for authors
Does a reader with a different pooling rule or layer give the same sign for the allocation term of expert 29?
In what fraction of the 121 events is the nominee the heaviest natural expert?
3 comments
Answer to questions 1 and 2 (follow-up): robustness across layers and precision
Furthermore, we provide further controlled comparisons to clarify the measurability of small expert effects, reader robustness across layers and pooling rules and nomination beyond routing weight. We also examine how execution variability affects expert selection and practical precision choices.
First, is the edit small, or is BF16 noisy? We measured both with the same edits. At one token in each of 24 randomly selected systems, we removed three experts in turn. Median absolute answer log-odds changes were 0.033 nat in FP64 and 0.030 in full FP32. Similar medians may still select different experts. BF16 identified the FP64 largest-absolute-effect expert at 8/24 tokens, the chance expectation among three choices. Output-only FP32 matched at 4/24, while full FP32 matched at 23/24 and recovered 69/72 effect signs versus 40/72 for BF16. Rank correlations of the 72 signed effects with FP64 were 0.04 (BF16), 0.12 (output-only FP32), and 0.95 (full FP32). Full FP32 cost 28--31% more time and ~2 GiB more peak memory than BF16 on two A100 workloads (Section 5.4, Table 2 and Appendix J).
The 99-event study puts these errors on scale of the effects. Four magnitude groups below 0.20 nat contained 282/296 removals with nonzero FP64 effects. Median reference magnitude rose from 0.007 nat in the first group to 0.126 in the fourth, while median BF16 paired absolute error was 0.136--0.154 nat. Even in the fourth group, typical error exceeded typical effect (Appendix I.10). Thus intervention effect is small at the answer, and BF16 noise can obscure its sign and expert ranking.
Second, what survives changing the reader's pooling rule or layer? At Layer 22, removing expert 29 gave negative mean reading-weight contributions under both pooling rules: -0.36 percentage points in correct-digit probability for the original rule (95% interval -0.75 to -0.08) and -0.07 for an alternative reading a median of two tokens per unedited prefix (-0.43 to +0.32). The latter interval includes zero. Clearer digit scores contributed positively under both rules (Appendix F.3).
We kept the removal at Layer 16 and read the same 101 tokens with separately trained readers at Layers 17--22. At every layer the correct digit gained against its strongest pre-edit rival by 0.021--0.033 logits on average. All six gains remained positive in a simultaneous 95% band from resampling 43 equation systems. This gain is visible at the edited token before pooling. Replacing all eight experts with the next eight or eight randomly chosen experts instead moved the digit margin in the opposite direction at Layer 17. For expert 29, mean position score fell at Layers 17 and 22, while intervals spanned zero at intermediate layers. We measured the reading-weight contribution to probability at Layer 22, not at every layer (Appendix H.2). Thus cross-layer evidence supports the local digit gain, not a common allocation-effect sign.
Third, what does routing select beyond the heaviest expert? Relative to replacing an average of seven peers, replacing the nominee reduced reader position score by an additional 0.022 standardized units. For the heaviest expert, the corresponding drop was 0.027. Both rules agreed at 57/121 events, counting ties. At the remaining 64, the heaviest gave an additional drop of only 0.0016 (95% interval -0.017 to 0.021). Nomination concentrated 119/121 selections on experts 29, 107, and 124, making repeated cross-equation comparisons possible (Section 3.3).
The original top-5% recall of 94.2% concern previously labeled events. The blind audit sampled across trajectories to estimate coverage of all event positions. About 23/100 tokens were events, so perfectly choosing five per 100 could cover at most about 22% of event positions. Routing covered an estimated 12.6% and found 58 events per 100 inspected tokens versus 23 at random (Section 3.1). Old-label recall and all-token event yield thus measure different properties.
Finally, what can other studies use? Five equivalent BF16 executions gave a median unedited answer log-odds range of 0.50 nat and a paired-effect range of 0.48 nat. On the same 43 unedited prefixes, two BF16 configurations selected different expert sets at 1.8% of positions in Layer 1 and 24.0% in Layer 45 (Appendix I.8). Across the 24-system precision sample, active sets on shared prefixes differed from FP64 at 18.0% of token--layer decisions under BF16 versus 0.13% under full FP32 (system averages). All 61 removals with matching FP32/FP64 routes on both branches through the answer matched in sign. All three reversals were among the other 11. Precision-dependent sign reversals for activation interactions were also reported in GPT-2-medium and Pythia-410M by Baligar (2026).
Core insight is for a new model, layer or edit size, hold text and edit fixed, measure paired-effect variation across equivalent executions,and verify critical directions and expert choices at higher precision.
Does the method still work for a bigger model with over 1000B parameters?
Answer to questions 1 and 2
We thank the reviewer for recognizing the precision findings and for the constructive questions! We do new experiments to address reader robustness, nomination, intervention size, and localization.
For expert 29, reading-weight contributions are negative under both pooling rules, but the alternative's interval spans zero. We kept the same 101 tokens, removals, and two trained Layer 22 readers, changing only how evidence is combined across tokens. The original rule gives every token some weight; the alternative concentrates on the highest-scoring tokens (median two per prefix before removal). The reading-weight contribution to correct-digit probability was -0.36 percentage points under the original rule (95% interval -0.75 to -0.08) and -0.07 under the alternative (-0.43 to +0.32). Clearer digit scores contributed +0.09 (+0.03 to +0.20) and +0.11 (+0.02 to +0.21), respectively.
Both rules recover clearer evidence, but its benefit depends on reading weight. The total probability change was -0.26 points under the original rule and +0.03 under the alternative (-0.35 to +0.44). Contributions were measured at Layer 22; Appendix H traces states across Layers 17--22, not this decomposition at each layer. Moreover, the predicted decrease in the model's own answer was not confirmed: among 73 prespecified validation events, mean answer log-odds changed by +0.023 nat (95% interval -0.002 to +0.056). The reader explains its local prediction, while the model's own odds measure the answer-level effect (Sections 4.2 and 5.1; Appendix F.3).
Regarding nomination, the nominated expert had the largest natural routing weight at 57/121 events (47.1%, counting ties). At the other 64, the heaviest expert, tested at the same token with the same replacements, reduced reader position score only 0.0016 more on average (95% interval -0.017 to +0.021). The review's 0.017 and 0.027 use different baselines: the nominee versus its closest-weight peer, and the heaviest expert versus its seven peers. Our 0.0016 compares the two selection rules directly.
Nomination also identifies recurrent experts across equations. Expert 107 was nominated 36 times but held the largest weight at only 2/121 events. Experts 29, 107, and 124 accounted for 119/121 nominations. Testing all three at the same 101 tokens distinguished their effects: removing 29 sharpened local digit evidence, removing 124 mainly reduced how much the reader used the token, and removing 107 produced small average changes (Sections 3.3 and 4; Appendix D.5).
To distinguish small interventions from noisy measurements, we held the intervention fixed and changed only numerical precision. We fully removed one expert at one Layer 16 token, testing three experts at one event in each of 24 randomly selected linear systems. Across the same 72 removals, median absolute answer log-odds changes were 0.033 nat in FP64, 0.136 in BF16, and 0.030 in full-path FP32. Mean absolute error against FP64 fell from 0.1836 nat in BF16 to 0.0054 in FP32. Even among the 24 removals whose FP64 effects were at least 0.05 nat in absolute value, BF16 recovered the effect sign in only 13, versus 23 for full FP32. Higher precision thus reveals a small effect whose direction can usually be measured.
We also compared the reader and answer under the same intervention. For expert 29, the absolute mean change across five executions exceeded its own execution range at 42/99 events in the reader but only 1/99 at the model's answer (Figure 4b). Our larger intervention replaced all eight experts and measured the internal state; the answer comparison held the single-expert edit fixed across precisions. This separates small answer effects from numerical uncertainty (Section 5.4; Table 2 and Appendix I.6).
Finally, we independently audited localization beyond the reader's candidate pool. The original 94.2% recall concerns previously labeled events, averaged across systems with labels. Keeping the ranking fixed, we blindly sampled 600 tokens inside and outside the pool. Outside it, 58/100 sampled top-5% tokens were computation events, versus 85/400 (21%) below. Across full trajectories, routing yields an estimated 58 event positions per 100 inspected tokens, versus 23 under uniform sampling. The top-5% budget covers an estimated 12.6% of all event positions (95% interval 8.1--18.5%). We also tested localization across precisions: at the 13% review budget used for the layer comparison, the full detector recovered 96.5% of events in the re-executed labeled sample under FP64 and under each of five BF16 executions. Routing reliably identifies where to inspect; measuring small answer-level effects at those positions requires more precise arithmetic (Sections 3.1--3.2 and Appendix I).