- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
SAEScientist-Bench evaluates whether agents can discover features for specified concepts in fixed SAE dictionaries. It contains 20 tasks across two layers of Gemma 2 9B IT, with hidden text and generation tests measuring dictionary prominence, activation selectivity, and steering. Across ten agent configurations, selected features often distinguish relevant texts well but provide substantially weaker generative control than curated references.
Critical strengths
I appreciate the separation of discovery quality into distinct dimensions. RANK measures prominence, ACTIVATION tests selectivity against confusable controls, and STEERING measures an actual generative effect. This is more informative than asking whether highly activating examples merely appear consistent with a concept, and it makes the distinction between input detection and output control explicit.
Critical weaknesses
-
STEERING does not yet establish usable generative control. Equation (7) rewards increased target relevance without including task preservation or degeneration in the final score. The coefficient calibration does impose a nondegeneration constraint, but Table 16 reports instruction preservation of approximately 0.95 to 1.12 out of 4 and degeneration rates of roughly 21% to 32% for alternative selections across the 13 analysed tasks. Thus, calibration does not fully resolve quality failures at evaluation. A joint measure requiring both concept induction and preserved task completion would show whether the rankings reflect reliable control.
-
The benchmark lacks a simple causal selection baseline that calibrates its difficulty. How well would activation based candidate retrieval followed by a small number of steering tests perform under a comparable query budget? Given prior evidence that input selectivity and output effects differ, this is an important reference point. Without it, I cannot tell how much of the agent gap reflects difficult experimental reasoning and how much reflects omission of an established validation step.
Questions for authors
-
Although your paper describes a process for discovering features—resembling the pipeline researchers typically follow—I still fail to see how this serves as a valid metric for evaluating an agent. In other words, I am not convinced that this benchmark adds any real value to the assessment of an agent’s capabilities; it is difficult for me to interpret what a high or low score actually signifies. Could you provide a reliable, well-founded reference to support this approach?
-
Could you provide the relevant original task instructions? Were agents explicitly told that the objective included generative steering, rather than only identifying a feature that detects the concept? Section 4.1 states that steering tests were available, but I could not determine how this capability was exposed or how often it was used. Please include the relevant tool instructions and the fraction of investigations that performed steering tests. These details materially affect how I interpret the reported neglect of causal validation.
6 comments
Author Response to Reviewer 2tnT (2/n)
Response to Weakness #2
Given prior evidence that input selectivity and output effects differ, this is an important reference point. Without it, I cannot tell how much of the agent gap reflects difficult experimental reasoning and how much reflects omission of an established validation step.
We appreciate the reviewer’s suggestion to include a baseline that tests steering before selecting a feature. We added four non-agent baselines and compared them with the Expert reference and all nine agents on the expanded evaluation described in our response to Weakness #1. For each task, the activation-based baselines use a common sample of positive and negative texts drawn from the probes submitted by agents.
- Activation Top-1 selects the feature with the largest activation difference between positive and negative probes.
- Activation Top-3 takes the three features with the largest activation differences, tests each on the steering calibration prompts, and selects the feature with the highest calibration Steering score. We evaluate the selected feature on separate held-out prompts.
- FeatureFinder applies its feature selection procedure to the same sampled probes.
- Random Feature samples a feature uniformly from the task’s SAE.
| Model / Method | Rank | Activation | Steering |
|---|---|---|---|
| Expert | 53.56 | 88.79 | 50.13 |
| Activation Top-1 | 67.20 | 78.20 | 30.92 |
| Activation Top-3 | 62.26 | 80.77 | 33.31 |
| FeatureFinder | 36.77 | 72.76 | 10.63 |
| Random Feature | 5.74 | 11.09 | 0.44 |
| Claude Opus 5 | 61.65 | 92.06 | 33.98 |
| Gemini 3.8 Flash | 57.75 | 89.98 | 37.97 |
| GLM-5.2 | 60.21 | 89.09 | 31.81 |
| Claude Sonnet 5 | 58.71 | 86.75 | 33.98 |
| GPT-5.6 Sol | 58.21 | 90.14 | 31.95 |
| Grok 4.6 | 58.91 | 89.62 | 31.56 |
| Kimi K3 | 58.38 | 88.32 | 33.36 |
| GPT-5.5 | 58.89 | 87.95 | 32.98 |
| GPT-5.6 Luna | 56.95 | 85.58 | 29.55 |
Comparison within Non-Agent Baselines. Activation Top-3 improved held-out Steering from 30.92 to 33.31 and Activation from 78.20 to 80.77 compared with Activation Top-1, while its Rank fell from 67.20 to 62.26. This trade-off shows how testing the three candidates during calibration can favor a feature that steers better on held-out prompts. FeatureFinder used the same probe sample and scored below both activation baselines on all three metrics, with a Steering score of 10.63.
Comparison of Non-Agent Baselines and Agents. All nine agents exceeded Activation Top-3 on Activation, and four also exceeded it on held-out Steering. These four agents found features through exploration that separated positive and negative examples more clearly and steered generation more effectively than the heuristic. The other five scored below Activation Top-3 on Steering, showing that selecting features that perform well on both metrics remains challenging. We will include all four baselines in the main results table to make these differences across agents visible.
Author Response to Reviewer 2tnT (3/n)
Response to Question #1
Q1. What capability does the benchmark measure, and how should its scores be interpreted?
We appreciate the reviewer’s question about how the benchmark measures an agent’s ability to conduct interpretability research.
What capability the benchmark measures. Each task asks the agent to find an SAE feature for a given concept. The agent decides how to design probes, which candidates to compare, and which feature to submit. We evaluate the submitted feature on held-out data. This follows the outcome-based approach used in RE-Bench, which scores the solutions agents produce for ML research engineering tasks [1], and MLE-bench, which scores agents’ Kaggle submissions on held-out test sets [2]. Here, the outcome measures how well the agent’s investigation identifies a feature for the target concept.
Why the scores are valid measures of a concept feature. AxBench evaluates concept representations by their ability to detect a concept in inputs and to steer generation toward it [3]. In our benchmark, Rank measures how prominently the submitted feature activates on concept-relevant inputs relative to other features in the SAE, and Activation measures how well it separates positive and negative examples. Steering tests whether intervening on the feature increases expression of the concept in generated responses. Arad et al. show that features with strong input selectivity can have weak effects on generation and that selecting features by their output effects improves steering [4]. Together, these metrics assess what the feature detects and what it can change.
What a high or low score means. We interpret the scores alongside the Random Feature and Expert results on the same tasks. RE-Bench similarly uses starting and expert reference solutions to contextualize agent performance [1]. In our expanded evaluation, agent Activation scores ranged from 85.58 to 92.06, compared with 88.79 for the Expert. Agent Steering scores ranged from 29.55 to 37.97, compared with 0.44 for Random Feature and 50.13 for the Expert. As discussed in Weakness #2, four of the nine agents outperformed Activation Top-3 on both Activation and Steering. This shows that stronger agents can use exploration to find more selective features that also steer generation more effectively than a heuristic that already tests steering.
Hence, the benchmark provides a verifiable way to compare agents on a task researchers already perform. Agents that can reliably find and verify these features could reduce the manual work involved in SAE analysis.
[1] Wijk H, Lin T R, Becker J, et al. RE-Bench: Evaluating Frontier AI R&D Capabilities of Language Model Agents against Human Experts. Proceedings of ICML, 2025.
[2] Chan J S, Chowdhury N, Jaffe O, et al. MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering. Proceedings of ICLR, 2025.
[3] Wu Z, Arora A, Geiger A, et al. AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders. Proceedings of ICML, 2025.
[4] Arad D, Mueller A, Belinkov Y. SAEs Are Good for Steering – If You Select the Right Features. Proceedings of EMNLP, 2025.
Response to Question #2
Q2. Could you provide the original task and tool instructions, clarify whether agents were explicitly asked to test generative steering, and report how often they did so?
We appreciate the reviewer’s question about how steering was presented to agents. We provide the task instructions for the expanded evaluation in Figure 12 (Appendix C.5). Agents were asked to identify the SAE feature that best matched the concept description. We provided two tools. probe_sae measures feature activations on input texts, and steer_sae tests a candidate’s effect on generation. The tool takes a feature ID, an intervention strength, and test prompts, and returns the generated text and stop reason. To keep the comparison fair and assess how agents design their investigations, we withheld the evaluation prompts and scoring criteria, including the use of Steering as a final metric.
All reported investigations in the expanded evaluation successfully used steer_sae at least once. Under the same instructions and tool access, Gemini 3.8 Flash averaged 15.03 steering calls per investigation, compared with 5.34 for Grok 4.6 High and 2.94 for Claude Sonnet 5 (Figure 11). The differences in call frequency show how much emphasis each agent placed on testing generative effects before selecting a feature. Gemini 3.8 Flash tested steering most extensively and achieved the highest held-out Steering score among agents (37.97).
Agents need to interpret the target concept, decide which tests will distinguish promising candidates, and use the results from both tools to support their final choice. The tools give agents a practical way to carry out that analysis, and the common task setup lets us compare how effectively they do it.
Author Response to Reviewer 2tnT (1/n)
We appreciate your careful reading of our manuscript and your constructive feedback. Your comments helped guide our revisions, which we describe point by point below.
Response to Weakness #1
We appreciate the reviewer’s suggestion to assess whether steering can induce the target concept while preserving task completion and response quality. Some agent-selected features produced degenerate responses on held-out prompts despite the non-degeneration constraint used during calibration. We added Joint Steering to account for concept induction, task preservation, and degeneration in the final score.
For each response, we take the harmonic mean of the target relevance and task preservation ratings and rescale it to 0–100. Degenerate responses receive a score of zero. Within each task, we average the response scores separately for feature steering, no intervention, and random-direction steering. Following Eq. (7), we compute
Joint Steering = max(0, Q_feature − max(Q_unsteered, Q_random)), where each Q is the mean response score for the corresponding condition. The final score is the average across tasks.Expanded evaluation. We apologize for the limited coverage in the original submission, as limited API resources constrained the range of tasks we could evaluate. During the rebuttal period, we added 100 tasks, bringing the evaluation to 120 tasks across two Gemma-2 models and five SAE configurations.
We compared Steering and Joint Steering on the expanded evaluation.
The Expert remains highest overall, and Gemini 3.8 Flash remains the top agent. Accounting for response quality changes several other positions. Claude Sonnet 5 falls from tied second to eighth, while Kimi K3 rises from fourth to second and Grok 4.6 rises from eighth to fourth.
We further analyzed how agents used steering during exploration. Claude Sonnet 5 made the fewest steering calls, averaging 2.94 per task, while Gemini 3.8 Flash made the most, averaging 15.03 per task (Figure 11). More extensive steering tests may help agents identify features that induce the target concept while preserving response quality. These examples illustrate how agents differ in the extent of their causal testing and in the quality of their final selections.
We appreciate your suggestion to evaluate concept induction and response quality jointly. We will include this comparison in the main ablation analysis.