acceptodds
Under review as a conference paper at ICLR 2027

Routing-Guided Single-Expert Interventions Measured Against a Model’s Own Execution Variability

Abstract

How large is the effect of changing one expert at one token in a sparse mixture-of-experts model, and how precisely can it be measured? We answer both questions by comparing the model with itself: five mathematically equivalent BF16 executions of the same tokens establish how much each measured quantity varies with execution, and an end-to-end FP64 reference measures the intervention at higher arithmetic precision. We apply this approach to Qwen3-30B-A3B solving linear systems. A linear detector on executed gate weights retrieves 94% of annotated computation events within a 5% review budget on each held-out trajectory. Its ranking is highly stable as expert membership changes across executions. A fixed rule on the same weights nominates the expert to remove; in same-state comparisons, the average drop in a frozen numerical probe's position score grows with the removed expert's natural weight. Attenuating the most frequently nominated expert improves local digit evidence while reducing that position's share of the probe's readout. These opposing contributions retain their average directions across executions. At the model's answer, median high-precision effect magnitudes are 0.02–0.04 nat against a median natural execution range of 0.50 nat. Single BF16 pairs report several-fold larger median magnitudes and carry little directional information. Raising only the precision of the final readout preserves most of that spread, which therefore forms before the readout. Small gaps between competing router or output scores identify the choices most likely to change. Routing guides where to look and which expert to test, interventions reveal what changes, and execution comparisons make numerical precision part of experimental design.

Primary area:
interpretability and explainable AI
Keywords:
expert routing, causal intervention, activation patching, numerical precision, mixture-of-experts, inference nondeterminism, linear probes, reasoning traces

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_NFE9OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

using Qwen3-30B-A3B to solve linear system, the paper attenuates one expert at one token at layer 16 and measures the effect against BF16 nd FP64 reference. FP64 answer effect is 0.02 to 0.04 nat against a 0.5 nat natural execution range and BF16 paris inflate to 3 to 9 times with near chance sign agreenment.

Critical strengths

The single BF16 paired interventions agree with FP64 in sign only 52% to 57% of the time, and making the readout FP32 keeps 95.8% of the spread. This is a reusable warning for activation patching and drives my recommendation. The same state peer comparisons, matched controls, family bootstrap, a pre registered prediction that failed and is reported.

Critical weaknesses

The recall is measured on probe selected candidates at low precision, the nominee adds only 0.017 over its closest weight peer while the heaviest expert alone gives 0.027 [0.014, 0.041], and the content result concerns the probe's own pooling weights while the pre registered answer prediction was not confirmed. One model, one layer, one task, and no larger intervention, so the paper cannot separate "BF16 is too noisy" from "this intervention is too small to matter"

Questions for authors

Does a reader with a different pooling rule or layer give the same sign for the allocation term of expert 29?

In what fraction of the 121 events is the nominee the heaviest natural expert?

3 comments
AuthorsOpenReview ↗

Answer to questions 1 and 2

We thank the reviewer for recognizing the precision findings and for the constructive questions! We do new experiments to address reader robustness, nomination, intervention size, and localization.

For expert 29, reading-weight contributions are negative under both pooling rules, but the alternative's interval spans zero. We kept the same 101 tokens, removals, and two trained Layer 22 readers, changing only how evidence is combined across tokens. The original rule gives every token some weight; the alternative concentrates on the highest-scoring tokens (median two per prefix before removal). The reading-weight contribution to correct-digit probability was -0.36 percentage points under the original rule (95% interval -0.75 to -0.08) and -0.07 under the alternative (-0.43 to +0.32). Clearer digit scores contributed +0.09 (+0.03 to +0.20) and +0.11 (+0.02 to +0.21), respectively.

Both rules recover clearer evidence, but its benefit depends on reading weight. The total probability change was -0.26 points under the original rule and +0.03 under the alternative (-0.35 to +0.44). Contributions were measured at Layer 22; Appendix H traces states across Layers 17--22, not this decomposition at each layer. Moreover, the predicted decrease in the model's own answer was not confirmed: among 73 prespecified validation events, mean answer log-odds changed by +0.023 nat (95% interval -0.002 to +0.056). The reader explains its local prediction, while the model's own odds measure the answer-level effect (Sections 4.2 and 5.1; Appendix F.3).

Regarding nomination, the nominated expert had the largest natural routing weight at 57/121 events (47.1%, counting ties). At the other 64, the heaviest expert, tested at the same token with the same replacements, reduced reader position score only 0.0016 more on average (95% interval -0.017 to +0.021). The review's 0.017 and 0.027 use different baselines: the nominee versus its closest-weight peer, and the heaviest expert versus its seven peers. Our 0.0016 compares the two selection rules directly.

Nomination also identifies recurrent experts across equations. Expert 107 was nominated 36 times but held the largest weight at only 2/121 events. Experts 29, 107, and 124 accounted for 119/121 nominations. Testing all three at the same 101 tokens distinguished their effects: removing 29 sharpened local digit evidence, removing 124 mainly reduced how much the reader used the token, and removing 107 produced small average changes (Sections 3.3 and 4; Appendix D.5).

To distinguish small interventions from noisy measurements, we held the intervention fixed and changed only numerical precision. We fully removed one expert at one Layer 16 token, testing three experts at one event in each of 24 randomly selected linear systems. Across the same 72 removals, median absolute answer log-odds changes were 0.033 nat in FP64, 0.136 in BF16, and 0.030 in full-path FP32. Mean absolute error against FP64 fell from 0.1836 nat in BF16 to 0.0054 in FP32. Even among the 24 removals whose FP64 effects were at least 0.05 nat in absolute value, BF16 recovered the effect sign in only 13, versus 23 for full FP32. Higher precision thus reveals a small effect whose direction can usually be measured.

We also compared the reader and answer under the same intervention. For expert 29, the absolute mean change across five executions exceeded its own execution range at 42/99 events in the reader but only 1/99 at the model's answer (Figure 4b). Our larger intervention replaced all eight experts and measured the internal state; the answer comparison held the single-expert edit fixed across precisions. This separates small answer effects from numerical uncertainty (Section 5.4; Table 2 and Appendix I.6).

Finally, we independently audited localization beyond the reader's candidate pool. The original 94.2% recall concerns previously labeled events, averaged across systems with labels. Keeping the ranking fixed, we blindly sampled 600 tokens inside and outside the pool. Outside it, 58/100 sampled top-5% tokens were computation events, versus 85/400 (21%) below. Across full trajectories, routing yields an estimated 58 event positions per 100 inspected tokens, versus 23 under uniform sampling. The top-5% budget covers an estimated 12.6% of all event positions (95% interval 8.1--18.5%). We also tested localization across precisions: at the 13% review budget used for the layer comparison, the full detector recovered 96.5% of events in the re-executed labeled sample under FP64 and under each of five BF16 executions. Routing reliably identifies where to inspect; measuring small answer-level effects at those positions requires more precise arithmetic (Sections 3.1--3.2 and Appendix I).

AuthorsOpenReview ↗

Answer to questions 1 and 2 (follow-up): robustness across layers and precision

Furthermore, we provide further controlled comparisons to clarify the measurability of small expert effects, reader robustness across layers and pooling rules and nomination beyond routing weight. We also examine how execution variability affects expert selection and practical precision choices.

First, is the edit small, or is BF16 noisy? We measured both with the same edits. At one token in each of 24 randomly selected systems, we removed three experts in turn. Median absolute answer log-odds changes were 0.033 nat in FP64 and 0.030 in full FP32. Similar medians may still select different experts. BF16 identified the FP64 largest-absolute-effect expert at 8/24 tokens, the chance expectation among three choices. Output-only FP32 matched at 4/24, while full FP32 matched at 23/24 and recovered 69/72 effect signs versus 40/72 for BF16. Rank correlations of the 72 signed effects with FP64 were 0.04 (BF16), 0.12 (output-only FP32), and 0.95 (full FP32). Full FP32 cost 28--31% more time and ~2 GiB more peak memory than BF16 on two A100 workloads (Section 5.4, Table 2 and Appendix J).

The 99-event study puts these errors on scale of the effects. Four magnitude groups below 0.20 nat contained 282/296 removals with nonzero FP64 effects. Median reference magnitude rose from 0.007 nat in the first group to 0.126 in the fourth, while median BF16 paired absolute error was 0.136--0.154 nat. Even in the fourth group, typical error exceeded typical effect (Appendix I.10). Thus intervention effect is small at the answer, and BF16 noise can obscure its sign and expert ranking.

Second, what survives changing the reader's pooling rule or layer? At Layer 22, removing expert 29 gave negative mean reading-weight contributions under both pooling rules: -0.36 percentage points in correct-digit probability for the original rule (95% interval -0.75 to -0.08) and -0.07 for an alternative reading a median of two tokens per unedited prefix (-0.43 to +0.32). The latter interval includes zero. Clearer digit scores contributed positively under both rules (Appendix F.3).

We kept the removal at Layer 16 and read the same 101 tokens with separately trained readers at Layers 17--22. At every layer the correct digit gained against its strongest pre-edit rival by 0.021--0.033 logits on average. All six gains remained positive in a simultaneous 95% band from resampling 43 equation systems. This gain is visible at the edited token before pooling. Replacing all eight experts with the next eight or eight randomly chosen experts instead moved the digit margin in the opposite direction at Layer 17. For expert 29, mean position score fell at Layers 17 and 22, while intervals spanned zero at intermediate layers. We measured the reading-weight contribution to probability at Layer 22, not at every layer (Appendix H.2). Thus cross-layer evidence supports the local digit gain, not a common allocation-effect sign.

Third, what does routing select beyond the heaviest expert? Relative to replacing an average of seven peers, replacing the nominee reduced reader position score by an additional 0.022 standardized units. For the heaviest expert, the corresponding drop was 0.027. Both rules agreed at 57/121 events, counting ties. At the remaining 64, the heaviest gave an additional drop of only 0.0016 (95% interval -0.017 to 0.021). Nomination concentrated 119/121 selections on experts 29, 107, and 124, making repeated cross-equation comparisons possible (Section 3.3).

The original top-5% recall of 94.2% concern previously labeled events. The blind audit sampled across trajectories to estimate coverage of all event positions. About 23/100 tokens were events, so perfectly choosing five per 100 could cover at most about 22% of event positions. Routing covered an estimated 12.6% and found 58 events per 100 inspected tokens versus 23 at random (Section 3.1). Old-label recall and all-token event yield thus measure different properties.

Finally, what can other studies use? Five equivalent BF16 executions gave a median unedited answer log-odds range of 0.50 nat and a paired-effect range of 0.48 nat. On the same 43 unedited prefixes, two BF16 configurations selected different expert sets at 1.8% of positions in Layer 1 and 24.0% in Layer 45 (Appendix I.8). Across the 24-system precision sample, active sets on shared prefixes differed from FP64 at 18.0% of token--layer decisions under BF16 versus 0.13% under full FP32 (system averages). All 61 removals with matching FP32/FP64 routes on both branches through the answer matched in sign. All three reversals were among the other 11. Precision-dependent sign reversals for activation interactions were also reported in GPT-2-medium and Pythia-410M by Baligar (2026).

Core insight is for a new model, layer or edit size, hold text and edit fixed, measure paired-effect variation across equivalent executions,and verify critical directions and expert choices at higher precision.

Does the method still work for a bigger model with over 1000B parameters?

Reviewer_yxJfOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper investigates the effects of changing a single expert at a single token in MoE models and examines the reliability of measuring these effects. The paper demonstrates that routing patterns can be used to identify computational events and analyzes the effects of single-expert interventions. The results reveal that the model's own execution variability can have a substantially greater impact on the measured outcomes than the expert interventions themselves.

Critical strengths
  1. The paper focuses on a widely overlooked aspect of MoE research, suggesting that model’s own execution variability should not be ignored when analyzing its behavior. This is particularly important because figuring out the underlying causes of behavior changes is essential for understanding model mechanisms.
Critical weaknesses
  1. The presentation of this paper is unclear and appears to lack logic. The paper uses many conceptions before defining them, making it hard to follow.

  2. The paper appears to solve two separate questions with no relationship. The main idea is unclear.

Questions for authors
  1. Could author provide experiments results on other models? In the paper, the experiments are conducted exclusively on Qwen3-30B-A3B solving three-variable linear equation systems, with expert interventions primarily performed at Layer 16.

  2. The paper shows that intervention effects are strongly associated with the experts' original gate weights, so whether the observed effects reflect the specific computational roles of experts or simply the larger amount of mixture weight redistributed during intervention?

  3. The numerical reader is independently trained to predict target digits, but its responses to expert interventions do not necessarily reflect the original model's behavior. Given the negative signed correlation (−0.15) between reader effects and answer effects reported in Section 5.1, how should we interpret the content–allocation analysis as an explanation of the model's actual computation?

3 comments
AuthorsOpenReview ↗

Paper narrative, numerical precision, and cross-model evidence

We thank the reviewer for highlighting execution variability and for explaining where the presentation was difficult to follow. We have reorganized the introduction around one question: when an expert is removed at one token, how much does it contribute to the model's later answer, and can that contribution be measured reliably? The experiments form one sequence: routing identifies a calculation step and an expert (Section 3), a separate digit reader diagnoses the intermediate state change (Section 4) and the model's own answer reveals the downstream effect (Section 5). Numerical precision is essential because it can change both the measured direction and which expert appears most important.

To make this connection concrete, the revised introduction follows −497y=0⇒y=0-497y=0\Rightarrow y=0. At the first zero, one division determines yy. Five tokens later, the model writes the second zero as its value. The detector ranks the first zero eighth among 2,175 tokens. In a separate blind audit, the top 5% of the ranking contained an estimated 58 computation-event positions per 100 inspected tokens, versus 23 under uniform inspection (Section 3.1). At the example token, our rule nominates expert 124 by selecting the eligible replacement that most reduces detector score. We remove that expert and redistribute its weight proportionally among the seven remaining active experts. Section 2 now introduces both kinds of weight before using them: the router weights experts at a token, while the independently trained reader weights tokens when combining digit evidence. Table 1 follows the same intervention through both measurements.

The precision experiment shows why these steps belong together. Holding the text, parameters and removal fixed, FP64 measures a -0.052 nat change in the log-odds of the second zero; five mathematically equivalent BF16 executions yield changes from -0.290 to +0.145 nat (Figure 1a). The executions differ only in how the same computation is organized, for example processing the prefix in one pass or chunks. Pairing edited and unedited runs therefore does not eliminate the error. Over 99 events, the median range of unedited answer log-odds was 0.50 nat and the median range of paired effects was still 0.48 nat. Across 297 removals, the median absolute FP64 effect was only 7% of the corresponding unedited execution range (Section 5.2). Recomputing only final normalization and output projection in FP32 retained 95.8% of the BF16 spread (system-weighted median of within-event ratios): most variability was already present in upstream states (Section 5.3).

This affects the next scientific decision: which expert to study. We randomly sampled 24 systems and tested three removals at one event per system. BF16, output-only FP32 and full-path FP32 matched the FP64 effect sign in 40/72, 42/72 and 69/72 removals, respectively. They identified the same largest-absolute-effect expert in 8/24, 4/24 and 23/24 events; 8/24 is the chance baseline for three candidates. In 61/72 removals, full FP32 and FP64 also agreed on expert routes along both edited and unedited branches through the answer, and all 61 agreed in effect sign. The three remaining sign reversals were among the other 11 removals (Section 5.4 and Table 2).

Thhough our controlled experiments use Qwen3-30B-A3B at Layer 16 on linear systems, related work provides independent evidence for the mechanisms involved. In OLMoE, Engmann et al. found a larger next-token loss increase from removing the highest- than lowest-weight expert; Gu linked part of the loss under cache quantization to changes in expert selection. In GPT-2-medium and Pythia-410M, Baligar found precision-dependent sign reversals in activation interactions. Together, these studies motivate examining weight redistribution, routing changes and numerical sensitivity. Our focused experiment establishes how they interact in one controlled setting. The methodological test transfers directly: fix tokens and removals, compare precisions with a high-precision reference, and ask both which direction the effect takes and which expert it identifies.

AuthorsOpenReview ↗

Expert contributions, equal-weight controls, and explanatory value of the reader

We thank the reviewer for raising distinction between routing weight, expert-specific contribution, and reader's relation to actual model behavior. To address these, we do matched-weight interventions and separate measurements of local state and final answer.

First, expert weight is not the same as expert contribution. When an active expert is removed, its weight is redistributed to the other seven experts. At the edited layer, the change in their weighted output is the amount transferred times the difference between the peers' weighted mean output and the removed expert's output. We therefore compare removing equal fractions of natural weight with removing equal absolute amounts. Across 121 events, the detector nominee's advantage over the average peer, measured as area between reader position-score-drop curves, was 0.010 under equal fractions but 0.0026 under equal amounts (95% interval -0.0002 to 0.0054 for the latter). Natural mixture weight explains part of the advantage. Choosing the naturally heaviest expert also works well: the two rules agreed at 57/121 events, and at the other 64, the heaviest-expert rule produced only 0.0016 more score reduction on average (95% interval -0.017 to 0.021). Importantly, nomination identifies recurring candidates before intervention: experts 29, 107 and 124 account for 119/121 selections (Section 3.3).

That consistency enabled a stronger same-token, same-weight comparison. At 101 positions where all three experts were active, we removed the same amount from each in turn, at four levels up to the smallest active weight. At the largest common amount, the Layer-22 score governing how much the reader uses the token fell by 0.0058 standardized units for expert 29 and 0.0071 for 124, but rose by 0.0023 for 107. Expert 29 caused a larger mean drop than 107 at every level; the area difference was 0.0039 (95% interval 0.0002 to 0.0076 and resampling 43 systems with adjustment for all three expert pairs). Thus the effects are not solely explained by the quantity of redistributed weight: at the same token and weight, different experts change this downstream state feature differently. This does not, by itself, assign an algebraic operation to any expert.

Second, the reader makes a precise but deliberately limited claim about the model. It is independently trained to extract digit evidence from Layer-22 hidden states and to combine that evidence across tokens; the model itself generates its answer through its own subsequent computation and output head. At 90/101 events, before intervention, the token selected for editing favored the correct digit over its nearest local rival more strongly than all preceding tokens combined. Removing expert 29 made the digit evidence clearer at that token, yet the reader's correct-digit probability fell. A four-way comparison, independently combining digit scores and reading weights from before and after the edit, explains the contrast: improved digit evidence contributed +0.09 percentage points, while reduced weight on this informative token contributed -0.36 points (Sections 4.1-4.2).

We also tested whether this explanation depended on how tokens were pooled. With the same positions and removals, an alternative rule concentrating on the highest-scoring tokens again showed clearer digit evidence (+0.11 points, 95% interval 0.02 to 0.21), while the reading-weight contribution was -0.07 points (95% interval -0.43 to +0.32). Thus the improvement in digit evidence persists across pooling rules, probability loss depends on how the reader allocates attention to that evidence.

Finally, we independently tested the model's own answer rather than treating reader probability as a proxy for it. Before validation, we predicted that removing expert 29 would decrease answer odds along with the reader's probability. Among 73 prespecified eligible events, the mean FP64 answer log-odds change instead pointed upward, +0.023 nat (95% interval -0.002 to +0.056). The cited -0.15 is the rank correlation between signed reader and answer changes pooled over 99 events and three experts (95% interval -0.27 to -0.02), so the reader's signed response does not reliably predict the answer's direction. The distinction is the result: the reader identifies what the intervention changes in local digit evidence and its use. But most importantly, direct FP64 answer measurements determine whether the model ultimately favors the answer more or less (Section 5.1).

AuthorsOpenReview ↗

reading guide of this paper(from the first zero to the second)

To make the connection between routing, expert interventions, and numerical precision concrete, we follow one example through the paper.

Consider one line the model writes: −497y=0⇒y=0-497y = 0 \Rightarrow y = 0. At the first zero, one division determines the value of yy. Five tokens later, the model writes that value as the second zero. This gives us a precise question: How much does one expert at the first zero change the model's odds of writing the second?

Section 2 introduces two kinds of weight. At a single token, the router weights the outputs of eight experts. A separate digit predictor, which we call the reader, weights the tokens it has seen when combining their evidence for a digit. Table 1 follows this example through the paper.

Section 3    Find the token and choose an expert to test.
             A detector ranks the first zero 8th among 2,175 tokens.
             At that token, we ask which eligible expert swap would
             lower the detector score most. This selects expert 124.

             Now make one edit at Layer 16. Remove expert 124 and
             give its weight to the other seven active experts
             in proportion to their original weights.
             |
             +-- Section 4. Read the state at the FIRST zero.
             |   What evidence does this token carry for the digit 0?
             |   How much weight does the reader give that evidence?
             |
             +-- Section 5. Score the SECOND zero with the model itself.
                 Keep the intervening text fixed. How much does removal
                 change the odds that the model writes 0 here?

These two measurements reveal different consequences of the same intervention.

At the first zero, removing expert 124 lowers the reader's probability for the correct digit by 0.72 percentage points. Of that drop, 0.65 points come from the reader giving this token less weight. This distinction also reveals an important mechanism elsewhere in the paper. Across the 101 events studied in Section 4, removing expert 29 actually makes the digit clearer at the edited token on average. Yet the reader's overall probability still falls because it draws less on that useful token. Thus, improving local evidence and increasing its use are distinct effects.

We then follow the intervention to the model's own answer. At the second zero, removing expert 124 lowers the model's answer log-odds by 0.052 nat in double precision. Five mathematically equivalent executions at the model's usual 16-bit precision give changes ranging from -0.290 to +0.145 nat for the same removal and text (Figure 1a). The same intervention can therefore appear to help or hurt depending on numerical execution.

This makes precision essential to identifying which expert matters, rather than a separate numerical concern. At 24 separately sampled events, we tested which of three expert removals had the largest absolute effect on the answer. The usual 16-bit calculation matched the double-precision choice at 8 events. Reading the same states with a more precise output calculation gave 4 matches. Computing the states in single precision from the first token gave 23 matches (Table 2).

The choice of arithmetic therefore changes which expert the experiment identifies as most influential. Together, Sections 3--5 connect where computation is localized, what an expert changes in the intermediate state, and whether its effect on the eventual answer can be measured reliably.

These findings suggest a practical workflow for future MoE intervention studies: use BF16 for efficient routing-based localization, full-path FP32 to measure causal effects on the model's answers, and FP64 to verify decisive interventions. The key distinction is that locating where computation occurs and reliably measuring what an intervention changes are different tasks, with different precision requirements.

Reviewer_2P9zOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates routing-guided, single-token expert interventions in a frozen Qwen3-30B-A3B model solving integer linear systems. A linear detector using executed gate weights retrieves 94.23% of annotated computation events at a 5% review budget. Interventions at Layer 16 reveal opposing content and allocation effects within a frozen diagnostic reader. The main methodological finding is that answer-level intervention effects measured using an end-to-end FP64 reference are substantially smaller than the variation across five mathematically equivalent BF16 execution configurations. Individual BF16 paired measurements frequently overestimate effect magnitudes and agree with the reference direction only slightly above chance. The paper argues that numerical precision should be an explicit component of mechanistic intervention design.

Critical strengths
  1. The comparison between multiple BF16 execution configurations and a numerically validated FP64 reference directly tests whether small intervention effects can be measured reliably. The final-normalization and output-head controls further demonstrate that most execution variation is already present upstream. This provides a useful methodological warning: pairing natural and intervened runs does not necessarily cancel numerical errors.
  2. The study distinguishes routing-based localization, same-state expert sensitivity, diagnostic-reader responses, and model-answer effects. Whole-system held-out evaluation, family-level uncertainty estimates, and the exact content–allocation decomposition strengthen the analysis. The authors also report unsupported directional hypotheses and weight-matched controls, helping readers avoid interpreting probe responses as direct evidence of answer-generation mechanisms.
Critical weaknesses
  1. Computation events are annotated from reader-guided candidates rather than an exhaustive or randomly sampled set of reasoning positions. Consequently, the reported 94.23% recall demonstrates retrieval of the selected annotations but does not establish coverage of computation events outside that candidate pool. Holding out equation systems addresses train–test leakage, but does not remove this selection dependence. An independent annotation audit would strengthen the localization claim.
  2. The experiments establish discrepancies between BF16 measurements and the FP64 reference, but provide limited evidence about the least expensive configuration that measures these effects accurately. Without a systematic comparison of measurement accuracy and computational cost, the practical guidance for reproducing this methodology remains incomplete.
Questions for authors
  1. How sensitive is the localization result to reader-guided candidate selection? Could a small blinded audit of randomly sampled positions from the existing trajectories estimate how often computation events fall outside the candidate pool?
  2. On a representative subset of events, how closely does end-to-end FP32 reproduce the FP64 intervention effects, including their signs and routing decisions? What are the corresponding runtime and peak-memory costs relative to BF16 and FP64?
  3. What incremental benefit does detector-based expert nomination provide over selecting the largest-gate-weight expert? Given the equal-weight and heaviest-expert controls in Appendix D, which conclusions about nomination remain supported beyond sensitivity associated with natural mixture weight?
2 comments

We thank the reviewer for thought ful questions. We ran experiments to address concerns: a blind audit, an FP32/FP64 accuracy--cost comparison, and nomination controls.

First, we examine how sensitive is localization to the candidate pool. Recall is selection-dependent, but routing also finds computation outside the pool. An estimated 99.5% of event positions lie outside it (95% interval 99.0--99.8%). Outside the pool, 58/100 sampled top-5% tokens were events versus 85/400 below; among digits, 51/92 versus 28/99; among non-digits, estimated rates were 50% in the top fifth versus 8% in the bottom 30%. Routing thus identifies more than digits.

The reader required high scores and states predicting the correct digit; annotators followed the algebra. A position qualified when at most a short calculation remained and progress toward the value outweighed visible digit clues. After −2z=4-2z=4, both marked the equals sign in z =: one division gives zz. That sign lay outside both the pool and the top 5%. One calculation spans multiple positions: ~61,000 event positions, 23% of tokens. The pool held 924/264,166 tokens and at most 1.5% of event positions.

The top 5% recovered 94.2% of labeled events (mean across 80 systems; 210/234 pooled). Blind-audit recall was 83.2% inside the pool and 12.2% outside, giving 12.6% overall (95% interval 8.1--18.5%). At this density, a perfect 5% budget covers at most 22% of events. Routing finds 58 events per 100 tokens read versus 23 at random; reading the top 20% or 40% covers an estimated 41% or 65%.

With ranks fixed, we sampled 600 tokens in four rank/pool groups (inside: 22/40 in the top 5%, 7/60 below; outside: 58/100 and 85/400). Two human annotators independently rated all three variables using text through each token and resolved 34 disagreements blind to ranks, pool membership, and old labels. A token counted once if any variable qualified. Estimates weight groups by their sizes across 120 held-out trajectories (Section 3.1; Appendix B.1).

Second, we assess which precision achieves FP64-level accuracy at the lowest cost. We randomly drew 24 of 43 systems and one event per system. Three removals per event gave 72 FP64 comparisons on identical text and stored parameters.

PrecisionSign /72Top expert /24Mean error (nat)Time (s)Memory (GiB)
BF164080.1836125 / 21.357.1 / 59.8
Output-only FP324240.1628same BF16 states--
Full FP3269230.0054160 / 28.059.3 / 61.6
FP64------172 / 30.661.3 / 66.4

Full FP32 cut mean absolute error by 97% and put 63/72 effects within 0.005 nat of FP64, versus 3/72 for BF16. Output-only FP32 reads the same BF16 states and cannot repair upstream errors. Remaining discrepancies occur where routes differ. On shared prefixes, active expert sets differed from FP64 at 18.0% of token--layer decisions in BF16 versus 0.13% in full FP32 (system averages). After editing, FP32 and FP64 routes matched through the answer on both branches for 61 removals: all matched in sign, with error below 0.0005 nat. All three sign reversals occurred among the other 11. Time/memory pairs represent 599-token/33-step and 2,915-token/5-step prefixes, with four continuations per prefix (steps per continuation). Times are medians and memory peaks over three runs on one 80-GB A100. Full FP32 cost 28--31% more time and about 2 GiB more memory than BF16; FP64 took 8--9% longer than full FP32 (Section 5.4; Table 2; Appendix J).

Finally, we examine what expert nomination adds beyond routing weight. Neither rule clearly wins on reader score, but their choices differ. Nomination combines token-level expert weight with how strongly expert use marks computation: the four experts with the largest detector coefficients carry 53.6% of Layer 16 weight at labeled events versus 22.5% elsewhere. Replacing our nominee reduced the reader's position score by 0.0255 standardized units versus 0.0036 for an average peer (difference 0.022; 95% interval 0.007--0.037). The largest-weight rule agreed at 57/121 events. At the other 64, it selected experts with higher average weight (20.9% versus 15.4%) but produced only 0.0016 more reduction (95% interval -0.017 to 0.021). Equal-weight removal narrowed the gap to peers, so some sensitivity reflects natural mixture weight.

Two contributions remain beyond weight. First, experts 29, 107, and 124 account for 119/121 nominations across equations. The largest-weight rule selects eight different experts and picks expert 107 only twice, versus 36 nominations, enabling us to test all three at 101 shared positions. Second, their effects per unit of removed weight differ: position-score reductions are 0.36 (0.22--0.50), 0.21 (0.10--0.31), and -0.02 (-0.09 to 0.04) for experts 124, 29, and 107, respectively; only removing 29 sharpens local digit evidence (Sections 3.3 and 4; Appendix D.5).

AuthorsOpenReview ↗

Three comparisons help clarify what the new experiments establish

We provide three additional comparisons that clarify what our experiments establish: annotation agreement versus event coverage, expert identity versus redistributed weight, and precision needed for localization versus that needed for causal measurement.

First, we examined why the original labeled-event recall was 94.2%, while the blind audit estimated 12.6% recall over all event positions. The difference primarily concerns the population being evaluated, rather than inconsistent event judgments. Among 100 randomly sampled positions inside the original candidate pool, the blind audit agreed with the original event judgment in 97 cases when comparing the same variable at the same token. All audit labels were assigned independently by two human annotators, who resolved disagreements while blind to routing ranks, pool membership, and original labels.

The original pool selected positions where the digit reader could already identify the correct digit, whereas algebraic computation may determine a variable's value at operators, equals signs, or other positions the reader never proposed. We estimate that 99.5% of event positions lie outside this small pool (95% interval 99.0–99.8%). Under the same blind judgments, the routing top 5% covered 83.2% of event positions inside the pool but 12.2% outside. Since nearly all events lie outside, the latter determines overall recall. Thus, 97/100 agreement supports the consistency of the original annotations, but does not imply that the candidate pool was exhaustive. Importantly, routing still finds 58 event positions per 100 tokens inspected, versus 23 under uniform sampling: 2.5 times the yield at the same reading budget (Section 3.1 and Appendix B.1).

Second, we isolated expert-specific effects from the amount of routing weight redistributed. At the same 101 tokens where experts 29, 107, and 124 were all active, we removed identical amounts from each expert in turn, using the same redistribution rule. We tested four removal amounts, up to the smallest active weight at each token. At every level, removing expert 29 produced a larger average decrease than removing expert 107 in the position score governing how strongly the reader uses that token. The difference in area under their score-drop curves was 0.0039 (95% interval 0.0002–0.0076, resampling 43 systems and adjusting for all three expert-pair comparisons).

This distinction matters because the detector's nomination rule and the largest-weight rule produced similar average effects in their direct comparison: among the 64 events where they disagreed, choosing the largest-weight expert gave only 0.0016 more score reduction (95% interval −0.017 to 0.021). We therefore do not claim that nomination consistently outperforms largest-weight selection at a known token. Its additional value is identifying recurring experts across equations: experts 29, 107, and 124 accounted for 119/121 nominations, enabling controlled comparisons at shared positions. These comparisons show that even with token position, removed weight, and redistribution rule fixed, expert identity still matters for the measured downstream reader feature. Natural mixture weight explains part of the sensitivity, but not all of it (Section 3.3 and Appendix D.5).

Third, the precision experiments distinguish locating a computation event from reliably measuring an intervention's effect on the answer. For this model, BF16 is effective for routing-based localization: at the 13% review budget used in our layer comparison, the detector recovered 96.5% of events in the re-executed labeled sample under FP64 and each of five BF16 executions. However, small answer-level effects require greater precision. Recomputing only the output in FP32 leaves the upstream BF16 states unchanged. Computing both edited and unedited trajectories in FP32 from the first token recovered the FP64 effect direction in 69/72 removals and identified the same largest-absolute-effect expert at 23/24 events. Mean absolute error against FP64 fell from 0.1836 nat under BF16 to 0.0054 nat under full FP32.

These results suggest a practical measurement strategy for this model: use BF16 to identify promising intervention positions, full-path FP32 to compare answer-level effects, and targeted FP64 calculations when a mechanistic conclusion depends on the direction of a few individual removals. Full FP32 required 28–31% more time than BF16, while FP64 required only another 8–9% over FP32 on our two matched workloads, making selective high-precision verification very practical (Section 5.4, Table 2 and Appendix J).

Together, the three comparisons establish distinct roles for the components of our method: 1) blind auditing determines what routing actually retrieves beyond the original labels 2)matched-weight interventions isolate expert-dependent effects on the reader 3)high-precision execution determines whether the measured answer-level effects are numerically reliable.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.