- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
DAFI interprets SAE features by describing their activating inputs, intervention-induced output effects, and the relation between them. It uses short-context probing, iterative refinement, and reusable skills. Experiments report improved interpretation scores, skill transfer to Qwen-Scope, and better steering-feature selection, supported by cross-model judging and human evaluation on 40 features.
Critical strengths
- The framework uses empirical feedback from both endpoints. DAFI combines measured activations with intervention-induced token changes and targets refinement at failed components. Table 1 reports improvements on matched features, while Table 3 provides evidence that accumulated skills transfer to another model–SAE setting.
- The paper makes a meaningful effort to validate its automated judgments. Appendices G–H report blinded human ratings and independent cross-model re-evaluation. The steering application also compares filters using the same generated continuations and held-out scores, providing evidence beyond the internal interpretation metrics.
Critical weaknesses
- Evaluation independence is unclear. The agent revises hypotheses, prompts, and intervention settings using score feedback, but final testing on fresh evidence is not clearly separated from refinement. Cross-model and human agreement do not establish validity in unseen contexts.
- The functional component’s added value is unclear. Prior work already combines input and output evidence (Gur-Arieh et al.). An input/output-only ablation is needed to isolate the functional component’s contribution. Unequal compute budgets and feature subsets also limit baseline comparisons.
Questions for authors
- How is the activation threshold computed? Are final scores tested on new prompts and contexts not used during refinement?
- Does the functional component help beyond input/output explanations alone? Please test removing it while keeping the same tools and budget.
- Can you compare methods under equal budgets and report DAFI’s results on the same 25 features used for Claude Code?
- Were the 50 held-out features excluded from skill updates and method development? How stable are the results across feature orders or repeated runs?
2 comments
Response to Reviewer aZKN (Part 2/2)
W2 / Q2. Added value of functional interpretation
Prior work has considered both input-side and output-side interpretations. Functional interpretation introduces a new perspective by explicitly characterizing the relationship between the two: the role a feature plays in the model’s computation. Our current results show that this perspective can benefit both feature interventions and interpretation quality.
First, the functional perspective helps identify features for more effective interventions, as demonstrated by our AxBench results. Appendix N.2, Tables 27 and 28, presents concrete steering examples that illustrate how output alignment and the input-output relationship inform feature selection. We acknowledge, however, that existing evaluation metrics do not fully capture the value of functional interpretation. We expect this richer understanding of feature function to also inform applications such as targeted model editing and tracing computational circuits in Transformers.
Second, the functional perspective helps improve interpretation quality in two ways. Firstly, it provides a clearer and more informative account of a feature’s role during inference. Table 7 in our paper compares our functional interpretations with Neuronpedia descriptions. For example, for L24 F2430, the functional interpretation is that when the model reads about game mechanics, this feature guides it toward discussing game balance. Secondly, this perspective helps improve the input-side and output-side interpretations themselves. On 50 features randomly sampled from five layers, we compared full DAFI with a variant without functional construction and feedback, using the same backbone, tools, and budget.
| Method | Activation coverage (%) | Boundary rejection (%) | Output score (%) | Input/output joint pass rate (%) |
|---|---|---|---|---|
| Full DAFI | 92 | 86 | 64 | 94 |
| Without functional construction and feedback | 96 | 72.5 | 72.3 | 80 |
Full DAFI better distinguishes input boundaries and more often produces interpretations that pass at both endpoints. Functional feedback guides the agent in selecting and refining the multiple candidate descriptions generated for each endpoint during iteration. Several candidates can fit their respective observations, while differing in how well the evidence supports a connection between them. By evaluating these connections, functional feedback helps the agent reconsider candidates and identify where input conditions, output descriptions, or further tests need refinement.
The present work demonstrates these two advances. We leave improvements to evaluation methods and the exploration of further applications to future work.
Revision: Section 4.3 (p. 7) explains how functional feedback guides endpoint refinement, with the controlled ablation in Table 4, bottom panel (p. 8). Section 4.4 and Table 7 (p. 10) present the interpretation examples.
W2 / Q3. Matched-feature comparison and compute budgets
We have compiled DAFI’s results for the exact 25 features used in the manuscript’s CC evaluation. The comparison includes the original restricted and unrestricted CC results from Appendix J, Table 21, together with the additional evaluation of CC equipped with our registered scientific tools. All conditions use the same DeepSeek-V4-Pro backbone.
| Method / result source | Passes | Input score (%) | Output score (%) | Functional score | Tokens / feature (M) |
|---|---|---|---|---|---|
| DAFI | 18/25 | 92.3 | 64.0 | 4.08 | 0.748 |
| CC, restricted / reported in the paper | 2/25 | 84.0 | 35.0 | 2.8 | 3.082 |
| CC, unrestricted / reported in the paper | 19/25 | 94.0 | 54.8 | 4.0 | 13.296 |
| CC + registered tools, restricted / additional evaluation | 5/25 | 91.2 | 48.8 | 3.40 | 2.879 |
DAFI achieves a similar pass count to unrestricted CC with 94.4% fewer tokens. These results support the value of a structured experimental workflow with adaptive reasoning: DAFI standardizes measurement and execution procedures while leaving hypothesis revision and targeted evidence collection to the agent. The matched-feature comparison evaluates this overall quality-cost balance.
Revision: The matched 25-feature comparison is reported in Section 4.2, Table 1 (p. 6), with budget and tool-access details in Appendix J.1, Table 22 (p. 23).
Response to Reviewer aZKN (Part 1/2)
Thank you for the careful assessment and the specific questions about evaluation independence, functional contribution, and comparison fairness. We address these points with the completed follow-up experiments and protocol clarifications below.
W1 / Q1. Evaluation independence and testing on new contexts
Thank you for raising this question. During refinement, DAFI iteratively adjusts its evidence collection, probing additional contexts to clarify activation patterns and using boundary examples to test the scope of its hypotheses. To assess whether the resulting explanations hold beyond this refinement evidence, we froze them and evaluated previously unused contexts.
The test covers 40 features from the original 250-feature pool, with five new supporting contexts and five new boundary contexts per feature, totaling 200 contexts of each type. Explanations, activation thresholds, and intervention policy remained fixed, with no further refinement or selection based on the new measurements.
The frozen explanations retain high predictive accuracy on new contexts. These results provide evidence that the interpretations extend beyond the observations used during refinement. Human review and cross-model judging provide complementary checks on judgment consistency.
Revision: We have added the new-context results to Section 4.2, “Frozen explanations on new contexts,” and the protocol and aggregate results to Appendix Q, Table 29.
Activation threshold. For each of the N original supporting contexts, let ai denote the maximum feature activation over non-BOS token positions. We compute their mean as
aˉ=N1∑i=1Nai.
When aˉ>0, we set τ=max(1.0,0.3aˉ).
When aˉ=0, we use the configured fallback threshold τ0 (default: 0). A context passes the activation test when its maximum non-BOS feature activation exceeds τ. We reuse this threshold on new contexts without recalibration.
Q4. Validation exclusion, repeated runs, and skill-accumulation order
The original 50 validation features were excluded from training and skill updates. They are disjoint from the 200 training features. We used validation performance to select the checkpoint, without using these features to revise prompts, tools, thresholds, or the method.
We designed an additional experiment to examine both repeatability and sensitivity to skill-accumulation order. Using a separate set of 50 features, ten per layer, we conducted three runs with skill learning enabled. Each run started from the same initial skill library, without carrying over skills acquired in previous runs. We varied the starting layer, feature-processing order, and batch arrangement to test whether comparable performance is maintained as the sequence of skill acquisition and reuse changes.
DAFI maintains comparable performance across repeated runs with different skill-accumulation orders. These results support repeatability under the tested ordering variations.
Revision: The three-run ordering comparison appears in Section 4.3, Table 5 (p. 8). Appendix E.2 describes validation use, library initialization, and processing orders.