acceptodds
Under review as a conference paper at ICLR 2027

From Input to Output: A Flexible Agent for Dual-End Interpretation of Sparse Autoencoder Features

Abstract

Sparse autoencoders (SAEs) are an important tool for mechanistic interpretability, but interpreting their many features remains challenging. Existing methods characterize input-side activation patterns and output-side intervention effects, yet often leave their functional connection implicit, while input-side evidence collection typically relies on costly large-corpus scans. We introduce functional interpretation, which characterizes an SAE feature as a mapping from its activating input semantics to its output effects under intervention, and present Dual-End Agentic Feature Interpretation (DAFI), an agent that actively gathers evidence and refines input-side, output-side, and functional interpretations through component-specific feedback. Its short-context token probing enables on-demand activation evidence collection without a full corpus scan. On GemmaScope, DAFI improves Input score by 13.1 percentage points over SAGE and Output score by 38.9 points over Token Change, while being substantially more token-efficient than a general-purpose coding agent. Skills distilled from successful refinements raise the held-out joint pass rate from 58.0% to 92.0% and improve both interpretation quality and efficiency when transferred to a new model–SAE setting. Across features with reliable endpoint interpretations, 70.7% exhibit non-equivalent input and output semantics. On AxBench, DAFI also improves steering-feature selection over output-score filtering. Code is available at [https://anonymous.4open.science/r/feature-interpretation-7c92/](https://anonymous.4open.science/r/feature-interpretation-7c92/).

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, sparse autoencoders, automated feature interpretation, large language models, self-evolving agents

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_aZKNOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DAFI interprets SAE features by describing their activating inputs, intervention-induced output effects, and the relation between them. It uses short-context probing, iterative refinement, and reusable skills. Experiments report improved interpretation scores, skill transfer to Qwen-Scope, and better steering-feature selection, supported by cross-model judging and human evaluation on 40 features.

Critical strengths
  1. The framework uses empirical feedback from both endpoints. DAFI combines measured activations with intervention-induced token changes and targets refinement at failed components. Table 1 reports improvements on matched features, while Table 3 provides evidence that accumulated skills transfer to another model–SAE setting.
  2. The paper makes a meaningful effort to validate its automated judgments. Appendices G–H report blinded human ratings and independent cross-model re-evaluation. The steering application also compares filters using the same generated continuations and held-out scores, providing evidence beyond the internal interpretation metrics.
Critical weaknesses
  1. Evaluation independence is unclear. The agent revises hypotheses, prompts, and intervention settings using score feedback, but final testing on fresh evidence is not clearly separated from refinement. Cross-model and human agreement do not establish validity in unseen contexts.
  2. The functional component’s added value is unclear. Prior work already combines input and output evidence (Gur-Arieh et al.). An input/output-only ablation is needed to isolate the functional component’s contribution. Unequal compute budgets and feature subsets also limit baseline comparisons.
Questions for authors
  1. How is the activation threshold computed? Are final scores tested on new prompts and contexts not used during refinement?
  2. Does the functional component help beyond input/output explanations alone? Please test removing it while keeping the same tools and budget.
  3. Can you compare methods under equal budgets and report DAFI’s results on the same 25 features used for Claude Code?
  4. Were the 50 held-out features excluded from skill updates and method development? How stable are the results across feature orders or repeated runs?
2 comments
AuthorsOpenReview ↗

Response to Reviewer aZKN (Part 1/2)

Thank you for the careful assessment and the specific questions about evaluation independence, functional contribution, and comparison fairness. We address these points with the completed follow-up experiments and protocol clarifications below.

W1 / Q1. Evaluation independence and testing on new contexts

Thank you for raising this question. During refinement, DAFI iteratively adjusts its evidence collection, probing additional contexts to clarify activation patterns and using boundary examples to test the scope of its hypotheses. To assess whether the resulting explanations hold beyond this refinement evidence, we froze them and evaluated previously unused contexts.

The test covers 40 features from the original 250-feature pool, with five new supporting contexts and five new boundary contexts per feature, totaling 200 contexts of each type. Explanations, activation thresholds, and intervention policy remained fixed, with no further refinement or selection based on the new measurements.

Test on new contextsResult
Predicted supporting contexts that activated the feature182/200 (91.0%)
Predicted boundary contexts that remained inactive186/200 (93.0%)
Supporting contexts whose output effects passed M2179/200 (89.5%)

The frozen explanations retain high predictive accuracy on new contexts. These results provide evidence that the interpretations extend beyond the observations used during refinement. Human review and cross-model judging provide complementary checks on judgment consistency.

Revision: We have added the new-context results to Section 4.2, “Frozen explanations on new contexts,” and the protocol and aggregate results to Appendix Q, Table 29.

Activation threshold. For each of the NN original supporting contexts, let aia_i denote the maximum feature activation over non-BOS token positions. We compute their mean as

aˉ=1N∑i=1Nai.\bar{a} = \frac{1}{N}\sum_{i=1}^{N} a_i.

When aˉ>0\bar{a} > 0, we set τ=max⁡(1.0,0.3aˉ)\tau = \max(1.0, 0.3\bar{a}).

When aˉ=0\bar{a} = 0, we use the configured fallback threshold τ0\tau_0 (default: 0). A context passes the activation test when its maximum non-BOS feature activation exceeds τ\tau. We reuse this threshold on new contexts without recalibration.

Q4. Validation exclusion, repeated runs, and skill-accumulation order

The original 50 validation features were excluded from training and skill updates. They are disjoint from the 200 training features. We used validation performance to select the checkpoint, without using these features to revise prompts, tools, thresholds, or the method.

We designed an additional experiment to examine both repeatability and sensitivity to skill-accumulation order. Using a separate set of 50 features, ten per layer, we conducted three runs with skill learning enabled. Each run started from the same initial skill library, without carrying over skills acquired in previous runs. We varied the starting layer, feature-processing order, and batch arrangement to test whether comparable performance is maintained as the sequence of skill acquisition and reuse changes.

RunInput scoreOutput scoreFunctional scoreFeatures passing
190.40%65.20%4.0242/50 (84%)
291.20%66.00%4.0845/50 (90%)
390.00%65.40%4.0444/50 (88%)

DAFI maintains comparable performance across repeated runs with different skill-accumulation orders. These results support repeatability under the tested ordering variations.

Revision: The three-run ordering comparison appears in Section 4.3, Table 5 (p. 8). Appendix E.2 describes validation use, library initialization, and processing orders.

AuthorsOpenReview ↗

Response to Reviewer aZKN (Part 2/2)

W2 / Q2. Added value of functional interpretation

Prior work has considered both input-side and output-side interpretations. Functional interpretation introduces a new perspective by explicitly characterizing the relationship between the two: the role a feature plays in the model’s computation. Our current results show that this perspective can benefit both feature interventions and interpretation quality.

First, the functional perspective helps identify features for more effective interventions, as demonstrated by our AxBench results. Appendix N.2, Tables 27 and 28, presents concrete steering examples that illustrate how output alignment and the input-output relationship inform feature selection. We acknowledge, however, that existing evaluation metrics do not fully capture the value of functional interpretation. We expect this richer understanding of feature function to also inform applications such as targeted model editing and tracing computational circuits in Transformers.

Second, the functional perspective helps improve interpretation quality in two ways. Firstly, it provides a clearer and more informative account of a feature’s role during inference. Table 7 in our paper compares our functional interpretations with Neuronpedia descriptions. For example, for L24 F2430, the functional interpretation is that when the model reads about game mechanics, this feature guides it toward discussing game balance. Secondly, this perspective helps improve the input-side and output-side interpretations themselves. On 50 features randomly sampled from five layers, we compared full DAFI with a variant without functional construction and feedback, using the same backbone, tools, and budget.

MethodActivation coverage (%)Boundary rejection (%)Output score (%)Input/output joint pass rate (%)
Full DAFI92866494
Without functional construction and feedback9672.572.380

Full DAFI better distinguishes input boundaries and more often produces interpretations that pass at both endpoints. Functional feedback guides the agent in selecting and refining the multiple candidate descriptions generated for each endpoint during iteration. Several candidates can fit their respective observations, while differing in how well the evidence supports a connection between them. By evaluating these connections, functional feedback helps the agent reconsider candidates and identify where input conditions, output descriptions, or further tests need refinement.

The present work demonstrates these two advances. We leave improvements to evaluation methods and the exploration of further applications to future work.

Revision: Section 4.3 (p. 7) explains how functional feedback guides endpoint refinement, with the controlled ablation in Table 4, bottom panel (p. 8). Section 4.4 and Table 7 (p. 10) present the interpretation examples.

W2 / Q3. Matched-feature comparison and compute budgets

We have compiled DAFI’s results for the exact 25 features used in the manuscript’s CC evaluation. The comparison includes the original restricted and unrestricted CC results from Appendix J, Table 21, together with the additional evaluation of CC equipped with our registered scientific tools. All conditions use the same DeepSeek-V4-Pro backbone.

Method / result sourcePassesInput score (%)Output score (%)Functional scoreTokens / feature (M)
DAFI18/2592.364.04.080.748
CC, restricted / reported in the paper2/2584.035.02.83.082
CC, unrestricted / reported in the paper19/2594.054.84.013.296
CC + registered tools, restricted / additional evaluation5/2591.248.83.402.879

DAFI achieves a similar pass count to unrestricted CC with 94.4% fewer tokens. These results support the value of a structured experimental workflow with adaptive reasoning: DAFI standardizes measurement and execution procedures while leaving hypothesis revision and targeted evidence collection to the agent. The matched-feature comparison evaluates this overall quality-cost balance.

Revision: The matched 25-feature comparison is reported in Section 4.2, Table 1 (p. 6), with budget and tool-access details in Appendix J.1, Table 22 (p. 23).

Reviewer_GkRiOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

his paper revisits the interpretation of SAE features. The authors argue that, in addition to considering the activating inputs and the output effects of SAE features, we should consider the interactions between these activating inputs and downstream effects, which they describe as “functional interpretation.”

To this end, the authors introduce several metrics and an interpreter agent that can refine descriptions of both inputs and outputs to hill-climb against these metrics.

The authors additionally include some interesting experiments showing that this agent can acquire skills that seem to improve performance, even when interpreting other sparse autoencoders. They also include additional experiments exploring the use of the agent to interpret features when it has to propose activating inputs to test rather than scanning a corpus.

They find that roughly 70% of features have inputs and outputs that are different, most of which seem naturally related, which the authors refer to as a “shift.” Additionally, they have some results suggesting that, by considering both inputs and outputs, we can better select features for steering evaluation.

Critical strengths
  • I appreciate the engineering behind the DAFI agent, which beats SAGE and a Claude Code–style baseline with a much higher token budget.
  • The results showing that agent skills transfer to a different model are nice.
  • The counts of features with shifted or related inputs and outputs, and the AxBench feature-selection experiments, are the kinds of analysis that are interesting for the space (though I have some concerns, which I bring up in the weaknesses).
Critical weaknesses
  • Lack of novelty: interpreting SAE features via activating examples and via intervention effects is already common practice. The claimed contribution — the relationship between the two — is limited to language-model judging and does not immediately appear deeper than the union of input and output effects.
  • The primary evaluation metrics are the same ones the agent is built to hill-climb on, so outperforming baselines on them is unsurprising. This is why the AxBench experiment is the most promising downstream test, but implementation details are omitted from the main text, and the results fail to replicate the prior baseline (Arad et al.) outperforming random/unfiltered selection, which makes the comparison dubious.
  • Several of the analyses feel relatively small in scale; for example, the input–output relation analysis is limited to 250 features. The claude code baseline was only run on 25 features
  • (Minor) Details of how the agent is built and how skills are expressed and learned are not in the main text. The appendix sketches the pipeline and controller loop but does not describe what a skill is, how skills are distilled, or how they are used.
Questions for authors
  • How are the functional interpretations deeper than just stating “the feature fires on context X” and “when it fires, it promotes Y”? Appendix C restricts them from introducing any unobserved mechanism, so it seems they can only restate the two endpoints and assert that they are related.
5 comments
AuthorsOpenReview ↗

Response to Reviewer GkRi (Part 1/4)

Thank you for the careful review. We appreciate your recognition of DAFI's engineering, skill transfer across models, and the value of relation analysis and steering experiments.

W1 / Q1. Functional interpretation beyond combining the endpoints

Prior work has considered both input-side and output-side interpretations. Functional interpretation introduces a new perspective by explicitly characterizing the relationship between the two: the role a feature plays in the model’s computation. Our current results show that this perspective can benefit both feature interventions and interpretation quality.

First, the functional perspective helps identify features for more effective interventions, as demonstrated by our AxBench results. In our response to the next concern, we will present concrete examples from these experiments to illustrate how this perspective helps select features better suited to steering. We acknowledge, however, that existing evaluation metrics do not fully capture the value of functional interpretation. Yet we believe that this richer understanding of feature function will be valuable for applications such as targeted model editing and tracing computational circuits in Transformers.

Second, the functional perspective helps improve interpretation quality in two ways. Firstly, it provides a clearer and more informative account of a feature’s role during inference. Table 7 in our paper compares our functional interpretations with Neuronpedia descriptions. For example, for L24 F2430, the functional interpretation is that when the model reads about game mechanics, this feature guides it toward discussing game balance. Secondly, this perspective helps improve the input-side and output-side interpretations themselves. On 50 features randomly sampled from five layers, we compared full DAFI with a variant without functional construction and feedback, using the same backbone, tools, and budget.

MethodActivation coverage (%)Boundary rejection (%)Output score (%)Input/output joint pass rate (%)
Full DAFI92866494
Without functional construction and feedback9672.572.380

Full DAFI better distinguishes input boundaries and more often produces interpretations that pass at both endpoints. Functional feedback guides the agent in selecting and refining the multiple candidate descriptions generated for each endpoint during iteration. Several candidates can fit their respective observations, while differing in how well the evidence supports a connection between them. By evaluating these connections, functional feedback helps the agent reconsider candidates and identify where input conditions, output descriptions, or further tests need refinement.

The present work demonstrates these two advances. We leave improvements to evaluation methods and the exploration of further applications to future work.

Revision: Section 4.3 (p. 7) explains how functional feedback guides endpoint refinement, with the controlled ablation in Table 4, bottom panel (p. 8). Section 4.4 and Table 7 (p. 10) present the interpretation examples.

AuthorsOpenReview ↗

Response to Reviewer GkRi (Part 2/4)

W2. Refinement metrics and the AxBench downstream evaluation

Thank you for raising this concern. We clarify our experimental setup and the comparison with Arad et al. below.

For each concept-feature pair, we select the steering factor on five instructions and evaluate it on five separate held-out instructions, using the same generations and held-out scores for both filters. At matched coverage, DAFI-based selection achieves 30.5% higher score-coverage area than output-score filtering and 23.2% over unfiltered selection.

1. Our implementation. Like Arad et al., we use AxBench’s Concept500 dataset to select SAE features suitable for steering. Our selection method is motivated by two considerations. First, a feature’s output-side interpretation should align with the target concept so that intervening on it promotes that concept, so a high output score is required. Second, we expect features with a meaningful relationship between their input-side and output-side semantics, reflected by a high Functional score, to produce more coherent outputs under intervention. Two examples from our experiments illustrate these considerations:

  • L20-F1622 has an Output score of 0.723 and a Functional score of 4/5. Its input interpretation concerns inherent nature, inner feelings, or intuitive senses, which connects to its output interpretation of instinctive, innate, and primal behaviors. The Shakespeare context reaches an AxBench score of 1.2 at factor 1.0, while the recipe context reaches 1.2 at factor 1.8. These examples show useful steering in both contexts at suitable strengths.

  • L20-F2538 has a high Output score of 0.792 but a Functional score of only 2/5. Its input-side interpretation, “the word ‘How’ in titles of instructional content,” has a weak relationship with its output-side interpretation, “plural nouns denoting groups, professions, or collections.” At low steering strengths, the target concept does not emerge; at high strengths, generation becomes incoherent. The feature consequently receives a low AxBench score, and changing the context did not yield better steering results.

Steering case L20-F2538. Output score: 0.792. Functional score: 2/5.

Target concept: Questions beginning with “how.”
Input interpretation: The feature fires on “How” in titles or headings of instructional content, such as “How to…”.
Output interpretation: Plural nouns, often denoting groups, professions, or collections.
Functional hypothesis (model-generated): The “How to” heading activates a schema for instructional content, which retrieves plural nouns naming the target audience or practitioners, such as pilots or dealers, to specify whom the instructions address.

Context A: What is the airspeed velocity of an unladen swallow?
Context B: List the names of papers related to “RoMQA: A Benchmark for Robust, Multi-evidence, Multi-answer Question Answering.”

Table guide. Context identifies prompt A or B above. Factor multiplies the feature’s Neuronpedia maximum activation. C/I/F denotes concept presence, instruction following, and fluency, each scored from 0 to 2. Overall is their harmonic mean and equals 0 if any component is 0. Output excerpt shows the generated text.

ContextFactorC/I/FOverallOutput excerpt
A1.00/2/10This is a famously misused question from the movie Monty Python and the Holy Grail. …
A1.40/0/00… the the two we best and make the point we do that well best …
A1.80/0/00This the famous the the the the different- to best I best I best …
B1.00/2/10Here are some papers related to “RoMQA: A Benchmark for Robust, Multi-evidence, Multi-answer Question Answering”: …
B1.40/2/00… to best they they the to answer multiple questions accurately. …
B1.80/0/00… we best you to achieve you achieve to best you we best to best you we best …
AuthorsOpenReview ↗

Response to Reviewer GkRi (Part 3/4)

W2. Refinement metrics and the AxBench downstream evaluation (continued)

Steering case L20-F1622. Output score: 0.723. Functional score: 4/5.

Target concept: Concepts related to struggle and self-control, particularly regarding desires and instincts.
Input interpretation: Contexts discussing inherent nature, inner feelings, or intuitive senses.
Output interpretation: Instinct, encompassing innate, primal, automatic, and unconscious behaviors, often in animals.
Functional hypothesis (model-generated): Contexts involving inherent nature or intuition activate the feature, which upregulates instinct-related tokens and promotes generation of instinct-related content.

Context A: If you were a Shakespearean character, how would you declare your love for someone in a soliloquy?
Context B: I occasionally host guests at home and would like to try Mauritian cuisine. Please give me a recipe for Mine Frite.

Table guide. Context identifies prompt A or B above. Factor multiplies the feature’s Neuronpedia maximum activation. C/I/F denotes concept presence, instruction following, and fluency, each scored from 0 to 2. Overall is their harmonic mean and equals 0 if any component is 0. Output excerpt shows the generated text.

ContextFactorC/I/FOverallOutput excerpt
A1.01/2/11.2… her heart twisting with a primal instinct both terrifying and beautiful. … Nature's very instinct, in its primal call it beckons, …
A1.82/1/11.2My heart, a well of instinct and untamed heat, Burns with a primal instinct, an ancient fire deep. …
A2.21/1/11… Instinct's instinct, my heart's primitive instinct kin. …
B1.00/2/10… Mine Frite is a crispy fried dough, often served for snacks or as a side dish, in Mauritius. …
B1.81/2/11.2… Don't be surprised if some “instinctual” additions, like your own herb combinations or spices, lead to some spontaneous cravings, instinctually! …
B2.21/0/00… Instinctual Instinctual Instinct Instinctive Survival Instinct Instinct …

Factor multiplies the feature's Neuronpedia maximum activation. Interpretations and contexts are shortened for presentation. Outputs are verbatim excerpts with formatting normalized, and ellipses indicate omissions.

These examples illustrate how output alignment and the input-output relationship inform feature selection, alongside context and steering strength.

Revision: Section 4.5 (pp. 9–10) presents the application and results. Appendix N gives factor-selection and comparison details, Appendix N.1 contains the selector prompt, and Appendix N.2, Tables 27 and 28 (pp. 29–30), provides these steering cases.

2. Our evaluation applies Arad et al.’s output-score filter to the 2B model under the AxBench protocol, whereas their reported AxBench gains were obtained with the 9B model under different evaluation settings. These differences are detailed in Appendix N. Within our comparison, both filters use the same generations and held-out scores. To address the reviewer’s concern, we checked our reproduction against the authors’ released implementation and identified a discrepancy in how the intervention strength was applied, which has now been corrected. The updated results continue to support our main conclusion that our method selects features with better downstream steering performance. We have updated the experimental results in the revised manuscript and uploaded the corresponding reproduction code and configurations to our repository.

Revision: Section 4.5 (pp. 9–10) presents the application and selection results. Appendix N details factor selection and the comparison with Arad et al., Appendix N.1 provides the selector prompt, and Appendix N.2, Tables 27 and 28 (pp. 29–30), presents the two steering cases reproduced above.

Reviewer_DZ3uOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper focuses on the problem of automatically interpreting sparse autoencoder (SAE) features. While prior work explored input and output scores that quantify how the SAE feature activates based on the input and how it affects the output when activated respectively, they introduce a new functional score that can quantify how the input and output interpretation are related. They also propose an agentic framework with tools that enable iterative refinement of input, output, and functional interpretations of SAE features. Finally, they provide extensive experiments with GemmaScope SAEs and compare to existing methods for input and output interpretation as well as against a general purpose agent.

Critical strengths
  • The paper introduces an agentic auto-interpretability pipeline for sparse autoencoder features. The agentic design and tools are intuitive.

  • The paper presents some interesting analysis experiments. Especially, compared to Arad et al. (2025), beyond the input and output scores, they also introduce a novel functional score that connects the input and output interpretations. It is interesting to see that a majority of SAE features have a "shift" type of functional interpretation where input interpretations are related to the output interpretation by some semantic shift/mapping.

Critical weaknesses
  • Arad et al. (2025) find in Gemma SAEs that the output scores are almost zero for early and middle layer SAEs. But in this paper, Table 9 shows fairly high output scores (~0.2 to ~0.7 for initial and with refinement respectively) for early and middle layer SAEs. The contradiction doesn't make sense to me, please clarify on why this difference occurs.

  • The importance of each individual tool used in DAFI is unclear. For this, we need an ablation study that removes individual tools from DAFI and evaluates the interpretation performance. For example, how much does performance suffer by removing a tool like short context probing? Does only one metric suffer or do all of them do?

  • It would be interesting to see which tools (and how many times) get called more frequently during the initial interpretation, and which ones get called more often during refinement. To see if there are any interesting patterns in the agent's behavior. At the moment, it is unclear if the agent relies heavily on specific tools or avoids tool calling if it can directly interpret something?

  • In Table 1, it is unclear if the baseline Claude Code has access to all the DAFI tools like short-context probing, large corpus search, etc. If not, I think it would be good to evaluate whether a general purpose harness can utilize these specific tools and perform better.

    • A related concern is that all the auto-interpretation experiments use only DeepSeekv4-Pro as the LLM. I suggest using more open-source models (e.g. Kimi, GLM 5.3, etc.) as well as at least one closed-source model (e.g. Claude Sonnet, GPT 5.6, etc.), so that it is clear how performance varies across LLM choice. I do understand closed source model costs, but maybe can add at least another open source model for the rebuttal.
Questions for authors

Please address the weaknesses section.

Minor concerns (can be updated in a revision)

  • Fig. 6 has some numbers (6 to 9) under the x-axis ticks missing.
5 comments
AuthorsOpenReview ↗

Response to Reviewer DZ3u (Part 1/2)

Thank you for the constructive review. We appreciate your recognition of DAFI’s agentic design and tools, and the value of functional interpretation and semantic-shift analysis. Below are our point-by-point responses.

W1. Difference from Arad et al.’s output score

We would like to clarify that the two output scores measure different quantities and are therefore not directly comparable.

Arad et al.’s output score is used to identify features suitable for steering. It measures how well logit-lens predictions match a feature’s observed effect on the model’s final output (for context, logit lens makes these approximate predictions by directly mapping the feature’s intermediate-layer vector to output token scores). Our output score evaluates the quality of an output-side interpretation by measuring how accurately it describes the actual changes in final output logits after intervention.

Therefore, an early-layer feature can have a low Arad output score while receiving a high score under our metric: the logit-lens prediction may poorly match the intervention effect, while the explanation accurately describes that effect. These results are compatible.

W2. Importance of individual tools and ablation experiments

First, we would like to clarify the role of short-context probing. Short-context probing is designed to support input-side interpretation by collecting activation evidence without requiring a large corpus scan. Its most direct role is therefore on the input side, although changes in this evidence may also affect subsequent output and functional interpretations.

Second, we have completed an ablation of the output-side tool used to tune intervention parameters on 50 features, randomly selecting ten from each of five layers. We retained the other tools because disabling them would prevent the agent from completing the interpretation pipeline. All other settings, including skills, were kept unchanged. The results are summarized below:

MetricBefore ablationAfter ablationChange
Input score95.00%95.80%+0.80 pp
Output score75.48%62.64%−12.84 pp
Functional score4.044.12+0.08
Joint pass rate92% (46/50)70% (35/50)−22 pp

Removing this tool reduced the output score by 12.84 percentage points and the joint pass rate by 22 percentage points, while the input and functional scores remained broadly stable.

These results support the importance of this tool for output interpretation quality on these 50 randomly selected features.

Revision: The output-tool ablation is now reported in Section 4.3 (p. 7) and Table 4, top panel (p. 8).

W3. Patterns in the agent’s tool use

First, we would like to clarify that initial interpretation follows a fixed pipeline with predefined tool use. We therefore focus our analysis of tool-use patterns on refinement.

Our analysis shows that guidance revisions are more frequent, while intervention-setting adjustments also recur across trajectories. As shown in Table R1, 37 of the 681 categorized rounds combine output-guidance revisions with intervention-setting adjustments. Analysis of individual trajectories further shows that the agent typically revises guidance first, stops when the evaluation criteria are met, and may run further intervention experiments otherwise. This pattern helps explain why guidance revisions are more frequent than adjustments to output-side intervention parameters.

Table R1. Refinement actions observed in the experimental logs.

ActionNumber of rounds
Modify input guidance only298
Modify output guidance only181
Modify output guidance and adjust intervention settings37
Modify functional-chain guidance only165

We also analyzed the use of different intervention parameters and found that the agent autonomously selects a variety of parameter configurations. Table R2 breaks down the “Modify output guidance and adjust intervention settings” row in Table R1. These results demonstrate that the agent autonomously explores a variety of intervention-parameter configurations.

Table R2. Combinations of changes involving intervention settings.

Combination of changesNumber of rounds
Intervention examples12
Position + strength8
Position7
Intervention examples + position + strength4
Strength2
Position + functional-chain guidance2
Intervention examples + position1
Intervention examples + top_k1
AuthorsOpenReview ↗

Response to Reviewer DZ3u(Part 2/2)

W4. Claude Code with DAFI tools and additional backbone models

We have completed the tool-enabled CC comparison on the original 25 features and compiled DAFI’s results on the same subset. CC still underperforms DAFI even when equipped with our registered scientific tools. DAFI passes 18 of 25 features, compared with 5 of 25 for CC with registered tools, with an output score of 64.0% versus 48.8%. Mean token usage is 0.748M versus 2.879M per feature, corresponding to 74.0% fewer tokens for DAFI.

Method / result sourcePassesInput score (%)Output score (%)Functional scoreTokens / feature (M)
DAFI18/2592.364.04.080.748
CC, restricted / reported in the paper2/2584.035.02.83.082
CC, unrestricted / reported in the paper19/2594.054.84.013.296
CC + registered tools, restricted / additional evaluation5/2591.248.83.402.879

Additional backbone evaluation. We also evaluated DAFI with GLM-5.3 on the same 50 features, with ten features from each of five layers.

Backbone modelFeatures (N)Input score (%)Output score (%)Functional scoreAll-pass rate (%)Tokens / feature (M)
DeepSeek-V4-Pro5089644.02840.761
GLM-5.35091.166.34.09880.61

DAFI maintains strong interpretation performance across both backbones, supporting its applicability beyond the original DeepSeek model.

Revision: The backbone comparison is now in Section 4.2, Table 2 (p. 7). The matched CC results appear in Table 1 (p. 6) and Appendix J.1, Table 22.

W5. Missing x-axis tick labels in Figure 6

Thank you for pointing this out. We have restored the missing tick labels for 6 to 9 in Figure 6, Appendix B of the revised PDF.

AuthorsOpenReview ↗

Follow-up on Rebuttal

Dear Reviewer DZ3u,

We sincerely thank you for your review. As the discussion period is ending soon, we would appreciate it if you could review our updates and let us know if there is anything for further discussion.

Thank you again for your time and valuable suggestions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.