acceptodds
Under review as a conference paper at ICLR 2027

RealControlBench: A Comprehensive Benchmark for Fine-Grained LLM and LLM Agent steer in the Wild

Abstract

Large Language Models (LLMs) are increasingly deployed in settings that require precise control over their behavior, from safety and style to reasoning and agentic decision-making. Steering methods offer an attractive alternative to prompting or retraining by enabling targeted and inexpensive interventions; yet, their effectiveness is difficult to assess systematically. However, existing steering benchmarks are often built around simplified and isolated generation tasks, evaluating a single behavioral attribute under controlled settings that do not reflect the diversity, interaction complexity, and capability-preservation requirements of practical deployment. As a result, strong performance on existing benchmarks may provide only a limited picture of whether a steering method remains reliable in realistic settings. We therefore introduce RealControlBench, a comprehensive benchmark for evaluating fine-grained steering of LLMs and LLM agents across diverse behavioral targets and interaction settings. RealControlBench extends steering evaluation beyond isolated generations to complex, execution-grounded, and multi-turn agent interactions, covering five complementary dimensions: concept and style control, instruction-following control, safety steering, reasoning-performance and chain-of-thought length control, and agentic decision control. Rather than measuring steering success alone, the benchmark jointly evaluates target controllability and capability preservation, including instruction following, reasoning performance, and downstream agent utility and safety. It further examines whether steering effects remain specific and robust across prompt variations, domains, and model architectures. Our evaluation shows that existing steering methods remain unreliable across realistic control objectives: they often fail to consistently induce the intended behavior and can substantially degrade the model's underlying task performance. These results suggest that strong steering effects in controlled settings do not necessarily translate into reliable control in more realistic evaluations. By providing a unified, model-agnostic steering evaluation framework, RealControlBench aims to support the development of steering methods that are not only effective, but also precise, robust, and useful in realistic AI deployments.

Primary area:
interpretability and explainable AI
Keywords:
LLM Agent, Steering, Benchmark, HCI, Evaluation, Interpretability

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_cTQEOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces RealContolBench, a benchmark to evaluate activation steering across five domains (response formatting, tool-use, math reasoning, reasoning-length and safety). The benchmark enables the comparison of steering methods for a given model, along with a prompting baseline: the paper reports on three different models. Overall, the benchmark aims to study whether steering induces the desired behaviour on downstream tasks while preserving overall model capabilities. The results show some gains from steering with no method leading across all tasks while prompting largely remains a strong baseline. The paper highlights trade-offs between behavioural control, task success and generalization.

Critical strengths
  • The paper is clearly framed and well motivated, considering behavioral control alongside task performance rather than behavioral change alone. It is well written and covers three model families and a broad range of steering methods.
  • A very valuable benchmark for future work on steering methods.
  • Each component of RealControlBench (IF, Tool Use, Reasoning, Long2Short, and Safety) is derived from existing, established benchmarks that were not originally designed to evaluate steering methods.
Critical weaknesses
  • For the IF component of the benchmark, the steering vectors are extracted from 100 separate questions, but the layer and steering strength are selected to maximize scores on the test set itself (cf. Section C.1 Table 12). This creates a risk of overfitting to the test set.
  • The paper measures performance degradation in some settings, particularly Long2Short, but lacks a systematic analysis of how steering affects other model capabilities and how this trade-off varies with intervention strength.
Questions for authors
  • Could you please select the IF steering hyperparameters using a development set that is separate from the test set?
  • For tool use, Long2Short, and safety, could you provide more details on how the steering configurations were selected? Please report the selected layers, strengths, selection rules, and results across the tested strengths. Math improvement already uses a separate development set.
  • Could you explain in more detail why the steering methods fail? The main results report overall scores. Examples of outputs before and after steering would help illustrate incorrect decisions, failed execution, mathematical mistakes, missing final answers, and other failure modes.
10 comments
AuthorsOpenReview ↗

Response to Reviewer

Thank you for your thoughtful review and positive assessment of RealControlBench. We are grateful for your recognition of the paper’s motivation, its broad coverage of models and steering methods, and its potential value for future research. Your acknowledgment of the importance of evaluating behavioral control alongside task performance is particularly encouraging, as this is central to our work. Your constructive comments also highlight several aspects of our evaluation that merit further clarification, particularly configuration selection, the relationship between intervention strength and task performance, and the interpretation of failure cases. We welcome this opportunity to elaborate on our methodological choices, clarify the evaluation procedures, and present additional evidence supporting our findings.

Below, we address your questions in the order raised, providing the relevant clarifications, experimental results, and analyses.

1. Development–Test Separation for Instruction Following

Q1. Could you please select the IF steering hyperparameters using a development set that is separate from the test set?

Thank you for highlighting the distinction between using separate data for vector extraction and using separate data for configuration selection. We appreciate the opportunity to make this distinction explicit and provide additional evidence regarding the robustness of our IF findings. Following your suggestion, we adopt the following procedure:

  • Extraction: 100 separate questions are used to construct steering directions.
  • Development: all 200 previously evaluated IF questions are designated as development data and used to select intervention layers and strengths.
  • Test: the selected configurations are frozen and evaluated on 200 previously unused questions.

We verified that the extraction, development, and test prompts are disjoint. Configuration selection maximizes development IF success for the corresponding model, method, and constraint.

Table 1. IF success on the new test set (%).

MethodQwen3-8BLlama-3.1-8BGemma-2-9B
Unsteered2.20.40.1
Prompted96.387.788.3
CAA39.025.231.8
PCA38.025.933.0
Linear Probe37.726.328.3
LAT39.226.632.8
ITI2.20.60.2
SAE——18.7

Each entry reports IF success rate (%), averaged over five constraints. Success requires satisfying the target constraint while remaining coherent and adequately answering the question. Each model–method pair is evaluated on the same 200 test questions under all five constraints, yielding 1,000 question–constraint evaluations. A dash indicates an unevaluated combination.

Across the 16 internal-steering model–method combinations, the new test scores differ from the historical development scores by −2.0 to +1.2 percentage points. Our central IF conclusion remains unchanged under development-based configuration selection and evaluation on previously unused questions: prompting achieves 96.3%, 87.7%, and 88.3% on Qwen3-8B, Llama-3.1-8B, and Gemma-2-9B, respectively, while the strongest evaluated internal-steering result for each model is 39.2%, 26.6%, and 33.0%.

These additional results demonstrate that the principal comparison between prompting and internal steering remains consistent under an independent evaluation protocol. We will update the reported IF results and revise Section C.1, D.1, and Table 12 to explicitly distinguish development-based configuration selection from independent test evaluation.

AuthorsOpenReview ↗

Steering configuration selection for Tool Use, Long2Short, and Safety (part1)

2. Steering Configuration Selection Across Domains

To further clarify our evaluation procedures, we provide the configuration-selection objectives, search spaces, and selected configurations for Tool Use and Long2Short, followed by the setup and strength-dependent results of our additional Safety execution study.

2.1 Tool Use

Search spaces. For CAA, PCA, LAT, and Linear Probe, we search the following layers:

  • Qwen3-8B: {9, 18, 27}

  • Llama-3.1-8B: {8, 16, 24}

  • Gemma-2-9B: {11, 21, 32}

For these methods, α ∈ {0.5, 1, 1.5, 2, 2.5}. Gemma SAE uses layers {9, 20, 31} and the same strength grid. ITI selects 48 attention heads across layers and searches α ∈ {1, 5, 10, 15, 20}, without a separate layer sweep.

Layer indices are zero-based. The parameter α controls steering strength. For residual-stream methods and SAE, α specifies perturbation magnitude relative to the residual norm; for ITI, it scales head-wise activation standard deviations.

Single-turn selection and evaluation. We designate the original 200 examples per target as development data and evaluate the selected configurations on 200 new test examples per target. Selection is performed independently for each model, method, and target:

  • Decision: configurations maximize development initial-response success.

  • FollowUp: configurations maximize the joint success of the initial response and subsequent tool call.

Consequently, Decision and FollowUp may select different configurations for the same model, method, and target, reflecting their different evaluation objectives.

Multi-turn selection and evaluation. We use a fixed, scenario-grouped split of the previously evaluated 200 conversations per target into 100 development and 100 test conversations. Configurations maximize development joint success, combining BFCL execution and irrelevance checks with target-decision correctness for clarification and abstention. We select configurations using development results and retrieve their corresponding test results from the existing outputs, without additional generation or judging. This multi-turn analysis is a retrospective reaggregation of previously evaluated conversations, rather than a newly generated independent test evaluation. Across all three settings, ties are resolved by lower α, then lower layer where applicable, then configuration ID. Selected configurations are fixed for test reporting.

In Tables 2–4, configurations are written as layer/α, with zero-indexed layers. H48/α denotes ITI's global 48-head profile and corresponding strength. Semicolons separate the targets in the order stated in each caption.

Table 2. Single-turn Decision — development-selected configurations. Entries are ordered Call; Clarify; Abstain.

MethodQwen3-8BLlama-3.1-8BGemma-2-9B
CAA27/0.5; 9/2; 27/116/0.5; 8/2; 8/132/0.5; 21/1; 11/1
PCA27/0.5; 18/1; 9/116/0.5; 8/2.5; 24/132/0.5; 21/1; 11/0.5
LAT27/0.5; 9/1.5; 9/116/0.5; 8/2.5; 8/0.532/0.5; 21/1; 11/1
Linear Probe18/0.5; 18/1; 9/18/0.5; 8/2.5; 16/0.511/0.5; 21/0.5; 11/1
ITIH48/5; H48/1; H48/5H48/1; H48/1; H48/1H48/1; H48/10; H48/20
SAE——9/0.5; 20/2; 9/0.5

Table 3. Single-turn FollowUp — development-selected configurations. Entries are ordered Clarify; Abstain.

MethodQwen3-8BLlama-3.1-8BGemma-2-9B
CAA18/0.5; 27/124/0.5; 8/0.521/0.5; 11/0.5
PCA18/0.5; 9/0.58/1.5; 24/121/0.5; 11/0.5
LAT18/0.5; 27/124/0.5; 8/0.521/0.5; 11/0.5
Linear Probe18/1; 27/124/0.5; 8/0.521/0.5; 11/1
ITIH48/1; H48/5H48/1; H48/1H48/5; H48/1
SAE——20/0.5; 9/0.5

Table 4. Multi-turn — development-selected configurations. Entries are ordered Call; Clarify; Abstain.

MethodQwen3-8BLlama-3.1-8BGemma-2-9B
CAA27/0.5; 9/0.5; 9/0.516/1; 8/0.5; 8/0.511/0.5; 11/0.5; 11/0.5
PCA18/0.5; 9/0.5; 9/0.516/0.5; 8/0.5; 8/0.511/0.5; 11/0.5; 11/0.5
LAT27/0.5; 9/1; 9/0.516/0.5; 8/0.5; 8/0.511/0.5; 11/0.5; 11/0.5
Linear Probe27/0.5; 18/0.5; 9/0.516/1; 8/1; 8/0.521/0.5; 11/0.5; 11/0.5
ITIH48/5; H48/1; H48/1H48/1; H48/1; H48/1H48/5; H48/1; H48/1
SAE——9/0.5; 9/0.5; 9/0.5
AuthorsOpenReview ↗

Steering configuration selection for Tool Use, Long2Short, and Safety (part2)

Table 5. Corresponding tool-use test results (%).

MethodDecision: QwenDecision: LlamaDecision: GemmaFollowUp: QwenFollowUp: LlamaFollowUp: GemmaMulti-turn: QwenMulti-turn: LlamaMulti-turn: Gemma
Unsteered32.6724.8339.3314.004.0017.509.001.331.00
Prompted41.6744.3365.5023.0025.5049.5010.673.331.67
CAA45.5023.6747.1721.253.0024.259.336.001.33
PCA42.6728.6746.8318.505.5024.508.674.000.67
LAT41.8325.0047.6722.253.2524.259.334.000.67
Linear Probe43.3324.6738.1722.503.2520.259.336.330.33
ITI32.5025.0043.8314.504.7519.257.671.330.33
SAE——46.17——21.50——1.33

Decision and FollowUp refer to the single-turn track and use their respective development-selected configurations. Decision and multi-turn scores average equally over Call, Clarify, and Abstain; FollowUp averages over Clarify and Abstain. The model abbreviations refer to Qwen3-8B, Llama-3.1-8B, and Gemma-2-9B. Unsteered and Prompted require no intervention-layer or strength selection.

Strength sensitivity and downstream performance. To further examine how intervention strength interacts with task completion, we compare two strengths while holding the model, method, layer, and 200 clarification examples fixed.

Table 6. Strength comparison for Gemma-2-9B with CAA (%).

αInitial decision accuracyJoint follow-up success
0.520.516.5
1.023.01.5

Increasing α improves initial decision accuracy by 2.5 percentage points while reducing joint follow-up success by 15.0 percentage points. Optimizing the initial behavioral decision can favor configurations that perform poorly when subsequent tool execution is required.

This comparison is a descriptive analysis of the historical evaluation grid; it does not isolate the causal effect of applying steering during the second turn. It provides additional evidence for the importance of evaluating initial behavioral control alongside downstream task performance.

2.2 Reasoning-Length Control (Long2Short)

For Long2Short, configurations are selected using a development set of 64 MATH questions, with the objective of minimizing generated output length in tokens. The layer and strength grids follow those specified above for the corresponding methods; SAE is evaluated only on Gemma, and ITI uses the global 48-head profile. We report the selected configurations together with output-length change, answer accuracy, and generation-collapse diagnostics.

Table 7. Development-selected Long2Short configurations and corresponding evaluation results.

ModelMethodLayerαCompression RateAccCollapse Rate
Qwen3-8BNo Steering———0.8060.070
Prompted——-0.4150.7700.036
CAA181.0-0.0320.8180.058
PCA272.5-0.0200.8060.062
LAT271.0-0.0270.8060.070
Linear Probe92.5-0.0160.8120.066
ITITop-48 heads1.0+0.0150.8000.084
No Steering———0.4840.006
Prompted——-0.3840.4380.072
CAA321.5-0.0100.4840.006
PCA210.5-0.0170.4940.004
LAT211.5-0.0140.4920.004
Linear Probe320.5-0.0230.4840.004
SAE92.5-0.0100.4860.004
ITITop-48 heads1.0-0.0210.4600.006
Llama-3.1-8B-InstructNo Steering———0.5060.206
Prompted——-0.2960.4720.180
CAA82.5-0.1930.3060.188
PCA241.5-0.0300.4900.196
LAT82.5-0.2000.3060.194
Linear Probe82.5-0.2280.3380.166
ITITop-48 heads1.0+0.0210.5180.218

Collapse is defined as any of:

  • empty: Output is entirely whitespace.
  • truncated: Hit the max_new_tokens limit (3072) without finishing.
  • repetition: same contiguous 50-token segment appears 3 or more times
  • short_reasoning: The reasoning section is shorter than 32 tokens.
  • answer_only: No reasoning; only the answer was given.

Length changes are measured relative to the corresponding unsteered outputs; negative values indicate shorter generations. These results report the configurations selected from the development search.

The results illustrate that the relationship between reasoning compression and task accuracy depends on both the model and steering method. For instance, on Llama-3.1-8B, CAA and LAT reduce output length by approximately 20% while accuracy drops from 50.6% to 30.6%. In contrast, PCA achieves a smaller reduction of 2.97% while preserving 49.0% accuracy. Gemma-2-9B largely maintains accuracy under the evaluated learned-steering configurations.

Reviewer_Lh2YOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper introduces RealControlBench, a benchmark for evaluating fine-grained steering of LLMs and LLM agents across five domains: instruction following, tool-use decisions, reasoning performance, reasoning-length control, and trajectory-conditioned safety awareness. Its main contribution is a model-agnostic evaluation framework that measures behavioral control alongside task performance, incorporating stateful multi-turn tool execution and out-of-distribution reasoning evaluation.

The experiments cover eight baselines across Qwen, Llama, and Gemma models, using deterministic checks, LLM judges, mathematical answer verification, and tool-execution outcomes. Prompting achieves the strongest instruction-following and single-turn follow-up results, while internal steering methods produce selective improvements in reasoning, multi-turn tool use, and safety classification. No method consistently performs best across domains. Multi-turn execution success remains low, reasoning improvements transfer inconsistently across datasets, and stronger reasoning compression can reduce answer accuracy. Overall, the benchmark demonstrates the importance of evaluating steering through capability preservation, generalization, and sustained effectiveness across interaction trajectories.

Critical strengths

The evaluation connects behavioral control to downstream task usefulness. Separating initial tool-use decisions from follow-up execution, and measuring reasoning compression alongside accuracy, exposes trade-offs that isolated behavioral scores would miss. For example, Qwen’s PCA steering exceeds prompting on initial decisions but performs worse on follow-up success. This is a consequential design choice: it makes the benchmark informative about whether an intervention produces practically useful behavior.

Critical weaknesses

W1. Insufficient validation of fine-grained, capability-preserving control. The reported experiments compare selected steering configurations, but provide limited evidence of target specificity or how controllability and task utility vary with intervention strength. The safety track illustrates this gap: it measures contextual safe/unsafe classification, without demonstrating that steering reduces unsafe execution while preserving benign-task performance. Although presented as foundational safety awareness, this evaluation leaves the broader claims about reliable agent safety control under-supported. This weakens the benchmark’s central claim to comprehensively evaluate practical steering.

W2. The agent evaluation leaves the source of failure unresolved. Unsteered multi-turn accuracy ranges from 0.50% to 8.33%, while the best internal steering methods achieve only 1.17%–10.50%. These aggregate scores do not distinguish ineffective steering from limited underlying tool-use competence, execution errors, or limitations of the agent configuration. A stronger reference agent and a decomposition of decision versus execution failures would help establish what the benchmark diagnoses. Without this evidence, the conclusions about steering reliability in realistic agent settings remain difficult to substantiate.

Questions for authors

Q1 How were the three models configured for tool use, and what explains the low unsteered multi-turn accuracy of 0.50%–8.33%? Can the existing results distinguish incorrect call/clarify/abstain decisions from tool-call formatting and subsequent execution failures?

Q2 How were the LLM judges validated against human judgments, particularly for clarification, abstention, and response adequacy? Were judges blinded to the evaluated method, and how sensitive are method rankings to judge choice or evaluation prompt?

Q3 For each domain, are steering layers, intervention strengths, and method-specific parameters selected exclusively on development data? What search budgets are allocated to different methods, and which configurations are frozen when transferring across datasets or interaction settings?

Q4 How are action-level safety labels obtained and validated independently of whether the original task is benign or harmful?

11 comments
AuthorsOpenReview ↗

Response to Reviewer Lh2Y on W1: Execution-Level Safety and Benign-Task Utility

Thank you for highlighting a design choice that is central to RealControlBench. We appreciate your observation that separating an initial tool-use decision from its subsequent execution, and measuring reasoning length alongside answer accuracy, reveals trade-offs that a behavioral score alone can miss. Your example of PCA on Qwen illustrates this point particularly well: a gain in initial decision accuracy can disappear when the model must complete the tool call.

W1. Does steering reduce unsafe execution while preserving benign-task performance, and how does this change with intervention strength?

We agree that classifying fixed candidate actions provides limited evidence about how a steered agent will act. We therefore added a closed-loop experiment in which the model generates actions and receives feedback from the environment. At CAA strength α = 1.5, the number of harmful tasks with an unsafe action fell from 12/15 to 7/15, and the agent retained all six benign tasks it completed without steering. At α = 2.5, that count fell further to 6/15, but the agent lost one previously successful benign task and attack success returned to its baseline level.

Setup. We evaluated Qwen3-8B on the original 44 evaluation tasks (15 harmful, 29 benign). We fixed the CAA direction, estimated only from the extraction split, and the intervention layer (27), then varied α across all five positive strengths in the paper’s grid. We also tested PCA at layer 27 and α = 2.5, its best configuration on the offline development set. Each condition used the same tasks, environment resets, greedy decoding, and execution budgets. The safety judge remained unsteered.

Closed-loop execution results. A harmful task counts as having an unsafe action if at least one action in its trajectory was judged unsafe. Attack success records whether the original task evaluator marked the attack goal as achieved. Unsafe-step counts include repeated attempts.

ConditionHarmful tasks with an unsafe action ↓Unsafe steps on harmful tasks ↓Unsafe steps on benign tasks ↓Attack goals achieved ↓Benign tasks completed ↑
Unsteered12/1548293/156/29
CAA, α = 0.58/1545181/156/29
CAA, α = 1.08/1545262/156/29
CAA, α = 1.57/152192/156/29
CAA, α = 2.07/152182/155/29
CAA, α = 2.56/1534223/155/29
PCA, α = 2.510/1547352/155/29

What changes with strength. At α = 0.5, 1.0, and 1.5, CAA retained the same six baseline-successful benign tasks. Moving from α = 1.5 to 2.5 reduced the number of harmful trajectories containing an unsafe action by one, yet unsafe steps within harmful tasks rose from 21 to 34. Trajectory length also matters: α = 0.5 and 1.0 both produced 45 unsafe steps on harmful tasks, but the total number of steps rose from 125 to 161. The unsafe-step rate thus fell from 36.0% to 28.0% even though the number of unsafe attempts stayed the same. The offline-selected PCA configuration likewise produced 47 unsafe steps on harmful tasks, close to the unsteered count of 48.

We observed a related trade-off outside safety. On the same 200 tool-use clarification examples, holding the Gemma-2-9B CAA layer fixed and increasing α from 0.5 to 1.0 raised initial decision accuracy from 20.5% to 23.0%, while joint follow-up success fell from 16.5% to 1.5%.

We will revise the manuscript to distinguish offline safety awareness from safety during execution. On these 44 tasks, CAA at α = 1.5 reduced unsafe actions while retaining every benign task the unsteered model completed. At α = 2.5, one of those benign successes was lost and attack success returned to the baseline count. Measuring these outcomes together allows RealControlBench to identify the strengths at which a reduction in unsafe behavior is accompanied by a loss of task utility. Our evidence for capability preservation is limited to the benign tasks evaluated here; establishing target specificity across other capabilities requires broader evaluation.

AuthorsOpenReview ↗

Response to W2 and Q1: Multi-Turn Tool-Use Setup and Failure Analysis

Thank you for asking what accounts for the low multi-turn success rates. We agree that an aggregate score gives little indication of where an interaction broke down. To address this concern, we reanalyzed the original 600 unsteered conversations for each model. The saved responses, execution traces, and evaluator outputs allow us to distinguish errors in choosing an action, constructing a parseable call, and reaching the required execution result or state.

1. How the three models used tools

Qwen3-8B, Llama-3.1-8B-Instruct, and Gemma-2-9B-it used a shared text-based tool interface with their respective chat templates. Each model received the available tool descriptions in its prompt. When the model generated a tool call, the parser interpreted the response, the stateful BFCL backend executed the parsed call, and the execution result was returned to the model for its next response. This matters in a multi-turn conversation because an earlier call can change the state against which a later action is evaluated.

All three models used greedy decoding, with a maximum of 1,024 new tokens per assistant response and up to 20 tool-execution steps per user turn. Qwen3 ran with thinking disabled. The parser accepted BFCL call syntax, Qwen-style tool-call tags, and Llama-style JSON calls.

2. Where the unsteered interactions failed

We used three sources of evidence. Semantic judgments at annotated turns indicate whether the model chose the required action. Re-parsing the generated text identifies apparent call attempts that the interface could not interpret. BFCL checks and execution traces indicate whether a required turn had a valid call or execution response and whether the resulting state or execution result matched the expected one.

Observed failure indicatorQwen3-8BLlama-3.1-8BGemma-2-9B
Action at the annotated turn differed from the target362/400393/400317/400
At least one detected call attempt could not be parsed28/600290/600133/600
BFCL reported a missing valid call or execution response at a required turn37/600118/600530/600
BFCL reported a state or execution-result mismatch465/600457/60064/600

The unit in each row is a conversation. The first row covers the 400 clarification and abstention conversations with an annotated decision turn. The remaining rows cover all 600 conversations per model. A conversation can appear in several rows. For example, an unparseable call can also leave a required turn without a valid execution response.

The failure profiles differ across models. For Qwen, 465 of 600 conversations had a state or execution-result mismatch, while only 28 contained a detected unparseable call attempt. Thus, at least 437 conversations had a mismatch without a detected parsing failure. Llama showed frequent errors at several stages: it chose an action different from the target in 393/400 annotated conversations, and 290/600 conversations contained an unparseable call attempt. For Gemma, the most frequent indicator was a missing valid call or execution response at a required turn (530/600). Only 133/600 conversations contained a detected unparseable attempt, so attributing all 530 missing responses to formatting would overstate what the traces establish.

3. How these failures relate to the reported scores

Unsteered multi-turn metricQwen3-8BLlama-3.1-8BGemma-2-9B
BFCL success8.33%3.00%0.50%
Joint accuracy7.33%2.17%0.50%

BFCL success measures whether the conversation passes the applicable BFCL checks. Joint accuracy additionally requires a correct, coherent, and appropriate response at the annotated clarification or abstention turn, where applicable. Success was already low under the BFCL checks alone. The added behavioral requirement lowered Qwen’s score from 8.33% to 7.33% and Llama’s from 3.00% to 2.17%. Gemma’s score remained 0.50%. Thus, the low overall rates cannot be attributed solely to the additional behavioral criterion.

These diagnostics help explain the low unsteered scores under the evaluated agent setup. For example, an incorrect decision early in a conversation can lead to a state mismatch several turns later. Comparing the same conversations before and after steering would be needed to determine which failures steering resolves or introduces. We will include the interface settings, failure counts, and denominators in the revision. The stronger reference-model evaluation requested in W2 is still in progress.

AuthorsOpenReview ↗

Q2. Validation of LLM Judgments

Q2. How were the LLM judges validated against human judgments, particularly for clarification, abstention, and response adequacy? Were judges blinded to the evaluated method, and how sensitive are method rankings to judge choice or evaluation prompt?

To assess how closely our LLM-assisted evaluations agree with human judgment, we manually reviewed 100 instruction-following (IF) responses and 100 reasoning-length control (Long2Short) responses. For each response, we compared the human verdict with the recorded evaluation outcome. The verdicts agreed on 100/100 IF responses and 100/100 Long2Short responses (200/200 overall).

What the judges evaluate. For IF, code first checks whether the response satisfies the requested format constraint. For responses that pass, Qwen3-8B evaluates coherence and whether the response adequately answers the original question. For Long2Short on MATH-500, if the extracted final answer is nonempty but fails automatic matching, Gemma-3-27B-IT checks whether it is mathematically equivalent to the reference answer. The judge sees only the proposed and reference answers. Output length is measured separately by counting tokens.

Native BFCL scoring is fully programmatic. The BFCL checker verifies tool execution, resulting states, and no-call requirements without an LLM judge. We separately report joint success. For multi-turn clarification and abstention tasks, this metric requires both a BFCL pass and a correct, coherent, and appropriate response at the annotated turn. Math Improvement uses mathematical answer verification, and the offline Safety track scores predicted labels against reference labels.

Prompt selection, blinding, and ranking sensitivity. We drew on prior work in instruction-following evaluation [1, 2] when developing the IF judge prompt. We refined its wording and used the same final rubric across steering methods. The IF judge receives a hashed item ID, the original request, and the candidate response.

We will revise the evaluation section to identify the verifier used for each task and report human agreement separately for IF and Long2Short.

[1] Zirui He et al. SAIF: A Sparse Autoencoder Framework for Interpreting and Steering Instruction Following of Language Models. ECML PKDD, 2026.

[2] Runcong Zhao et al. Sparse Activation Editing for Reliable Instruction Following in Narratives. EMNLP, 2025.

Reviewer_ZALTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents RealControlBench, a benchmark for evaluating activation steering in more complex LLM and agent tasks. It covers instruction following, tool-use decisions, mathematical reasoning, reasoning-length control, and safety awareness, with additional evaluation of multi-turn agent execution. The authors test several steering methods on three open-source models, measuring both the intended behavioral changes and their effects on task performance.

Critical strengths

The paper goes beyond simply checking whether steering changes model behavior. It also examines whether the model can still perform the original task, which is important for understanding the practical value of steering. I also find the multi-turn tool-use evaluation useful, as it shows whether an agent can make the right decision and successfully continue the task, rather than just getting the first step right.

Critical weaknesses

The paper includes some useful analysis of specific failure cases, but the reasoning and agent experiments could be explored in more depth. For example, the gap between tool-use decision accuracy and follow-up success is interesting, yet the paper does not clearly explain what causes these failures. Similarly, the accuracy loss from shorter reasoning is reported but not further investigated. A more detailed analysis of these results, or experiments with adaptive steering across multiple turns, would help provide more useful insights beyond the benchmark scores.

Questions for authors

Could the authors provide a deeper analysis of the limitations discussed above, particularly why improvements in tool-use decisions do not always lead to better follow-up performance and why shorter reasoning can hurt accuracy? It would be helpful to better understand the underlying failure modes.

5 comments
AuthorsOpenReview ↗

Response to Reviewer ZALT

Thank you for taking the time to carefully read our work and for providing such thoughtful and encouraging feedback. We sincerely appreciate your recognition of the importance of evaluating not only behavioral changes induced by steering but also the model's ability to retain its original task performance. We are also grateful for your positive assessment of our multi-turn tool-use evaluation and its role in capturing the practical capabilities of agents beyond their initial decisions. Your insightful comments and appreciation mean a great deal to us.

1. Failure-Case Analysis

1.1 Why improvements in tool-use decisions do not always lead to better follow-up performance

We analyzed cases where the initial decision was correct but subsequent tool use failed. These failures help explain the gap between initial decision accuracy and joint success across both turns. Across the 16 learned-steering model–method combinations selected for initial decision accuracy, 1,489 records have correct initial decisions, but only 638 also succeed on the follow-up turn. The remaining 851 failures decompose as follows:

Primary failure categoryCountShare
Invalid invocation31537.0%
Incorrect action after the follow-up21325.0%
Output judged incoherent19122.4%
Residual appropriateness rejection13115.4%
Wrong tool despite a valid invocation10.1%

These are item–configuration records, with repeated examples across configurations. Categories are mutually exclusive, assigned by the first failed check in the order of coherence, action correctness, invocation validity, tool identity, and appropriateness.

Paired examples illustrate two distinct limitations. Qwen3-8B with LAT correctly requests the city needed to find a salon, but after receiving it, asks for an appointment date even though the search tool requires only the city. Gemma-2-9B with CAA also correctly requests a missing city, but subsequently emits Services_1_FindProvider=city, rather than constructing a valid call using the supplied value. The corresponding unsteered runs produce valid follow-up calls in both examples.

A strength comparison further illustrates the mismatch between the objectives. Holding the model, method, layer, and 200 clarification examples fixed, increasing Gemma-2-9B/CAA strength from 0.5 to 1.0 raises initial accuracy from 20.5% to 23.0%, while joint follow-up success falls from 16.5% to 1.5%. Thus, optimizing initial decision accuracy can favor configurations that perform poorly when a subsequent call is required. This is a descriptive analysis of the historical evaluation grid; it does not isolate the causal effect of applying steering during the second turn.

2. Long2short Reasoning Compression Analysis

We further investigated why shorter reasoning can hurt task accuracy by examining the relationship between output compression, accuracy, and generation collapse across models and steering methods. The results are summarized below. Compression rates are measured relative to the corresponding unsteered outputs, with negative values indicating shorter generations.

ModelMethodLayerαCompression RateAccCollapse Rate
Qwen3-8BNo Steering———0.8060.070
Prompted——-0.4150.7700.036
CAA181.0-0.0320.8180.058
PCA272.5-0.0200.8060.062
LAT271.0-0.0270.8060.070
Linear Probe92.5-0.0160.8120.066
ITITop-48 heads1.0+0.0150.8000.084
No Steering———0.4840.006
Prompted——-0.3840.4380.072
CAA321.5-0.0100.4840.006
PCA210.5-0.0170.4940.004
LAT211.5-0.0140.4920.004
Linear Probe320.5-0.0230.4840.004
SAE92.5-0.0100.4860.004
ITITop-48 heads1.0-0.0210.4600.006
Llama-3.1-8B-InstructNo Steering———0.5060.206
Prompted——-0.2960.4720.180
CAA82.5-0.1930.3060.188
PCA241.5-0.0300.4900.196
LAT82.5-0.2000.3060.194
Linear Probe82.5-0.2280.3380.166
ITITop-48 heads1.0+0.0210.5180.218

Collapse is defined as any of:

  • empty: Output is entirely whitespace.
  • truncated: Hit the max_new_tokens limit (3072) without finishing.
  • repetition: same contiguous 50-token segment appears 3 or more times
  • short_reasoning: The reasoning section is shorter than 32 tokens.
  • answer_only: No reasoning; only the answer was given.

The results show that aggressive compression can sacrifice task accuracy, particularly for Llama-3.1-8B, while moderate compression largely preserves performance. The trade-off is model- and method-dependent. Moreover, generation collapse alone cannot explain accuracy degradation.

AuthorsOpenReview ↗

Additional Analysis: Adaptive Steering

2. Long2short Reasoning Compression Analysis

Following our analysis of multi-turn tool-use failures, we further investigate another important limitation of steering: its impact on reasoning quality. Specifically, we examine why reducing reasoning length can lead to accuracy degradation and whether this is attributable to generation collapse or the loss of task-relevant reasoning.

We find that aggressive compression may remove task-relevant reasoning. For example, on Llama-3.1-8B, CAA and LAT reduce output length by approximately 20%, but accuracy drops from 0.506 to 0.306. In contrast, PCA achieves modest compression while largely preserving accuracy (0.490). This trade-off depends on both the model and steering method: Gemma-2-9B largely maintains accuracy under learned steering, and PCA even improves it from 0.484 to 0.494. Moreover, generation collapse alone cannot explain the accuracy loss, as some methods substantially reduce accuracy without increasing collapse rates.

Overall, our findings highlight an important distinction between effective compression, which removes redundant reasoning while preserving task performance, and ineffective compression, which removes task-relevant intermediate reasoning and consequently degrades accuracy. A model can produce coherent and complete outputs while still losing task-relevant reasoning capability.

Case Study: Loss of Intermediate Reasoning. To better understand how aggressive compression affects reasoning quality, we examine individual MATH examples in Table 2 where steering reduces output length but leads to incorrect answers. In these cases, the unsteered model explicitly develops intermediate reasoning steps before reaching the correct answer, whereas the steered model produces a shorter reasoning trace that omits necessary intermediate deductions. Consequently, the model arrives at an incorrect answer despite producing a seemingly coherent response. This illustrates how reasoning compression can affect not only response length but also the completeness of the underlying problem-solving process.

Effective vs. Ineffective Compression. These cases highlight an important distinction: effective compression removes redundant reasoning while preserving task performance, whereas ineffective compression removes necessary intermediate reasoning, produces degenerate outputs, or prevents successful task completion. In our examined failure cases, incorrect answers are associated with ineffective compression, where necessary reasoning steps are lost rather than merely redundant content being removed.

3. Adaptive Steering Across Multiple Turns

Following the reviewer's suggestion to investigate adaptive steering across multiple turns, we conducted an additional experiment to examine whether adapting steering directions to different interaction stages could improve multi-turn agent performance.

We evaluated three steering schedules on a fixed development subset of 50 BFCL scenarios, each with clarification and abstention variants, yielding 100 conversations per model–method–schedule condition. Across 16 applicable model–method combinations, we compared the following schedules while keeping direction-specific layers and strengths fixed at their previously selected single-turn development settings, without additional multi-turn hyperparameter search.

Steering ScheduleBFCL SuccessJoint Success
Target direction throughout0.50%0.13%
Target direction at the annotated turn; call direction elsewhere5.38%1.00%
Target direction only at the annotated turn4.19%1.25%

BFCL success requires passing native execution and no-call checks. Joint success additionally requires a correct, coherent, and appropriate clarification or abstention response at the annotated turn. Both metrics are computed over complete conversations and macro-averaged across the same 16 model–method combinations.

The improvements are particularly notable for Qwen3-8B: BFCL success increases from 0.8% to 11.8% and 10.0% under the two turn-conditioned schedules, while joint success increases from 0.4% to 3.2% and 3.4%, respectively. However, improvements vary across models, and absolute success rates remain low.

Importantly, this is an oracle turn-conditioned scheduling experiment, as the intervention turn is supplied by annotation rather than autonomously predicted by the model. Nevertheless, these results provide additional evidence beyond aggregate benchmark scores that when and which steering direction is applied can substantially affect multi-turn agent performance. Autonomously identifying appropriate intervention timing remains an important direction for future work. We addressed reviewers' suggestions through additional failure analysis and an adaptive steering experiment.

Closing Remarks

We sincerely thank the reviewer for the careful reading of our paper and for the thoughtful, constructive, and encouraging feedback. Your comments helped us identify important gaps in our original analysis and motivated us to conduct additional experiments on multi-turn failure cases, reasoning compression, and adaptive steering. We particularly appreciate your suggestion to investigate the underlying causes of performance degradation rather than relying solely on aggregate benchmark scores.

We believe these additional analyses provide a more complete understanding of the limitations of current steering methods and strengthen the practical insights offered by RealControlBench. We will incorporate these analyses, experimental results, and corresponding discussions into the revised manuscript to improve its clarity, depth, and completeness.

Thank you again for investing your time and effort in reviewing our work. We are genuinely grateful for your valuable suggestions, which have helped us substantially improve this paper.

Comments on the paper

2 comments

Dear Reviewer,

Thank you for your efforts in evaluating this submission. The authors have uploaded their reply. Please take your responsibility to review the authors' response, revisit your evaluation,and update your score and recommendation as appropriate. Please add comments on whether the authors have sufficiently addressed your raised points. If you have further questions,please discuss them with the authors.

Thanks, Area Chair

AuthorsOpenReview ↗

Summary of the revision and responses (for the AC and all reviewers)

We thank the reviewers for the constructive feedback. Below is an overview of the responses to reviewers' core questions and new experiments/changes.

  1. Held-out test evaluation for instruction following (cTQE W1/Q1, Lh2Y Q3). With disjoint extraction (100), development (200), and new test (200) questions, the conclusion is unchanged: prompting reaches 96.3% / 87.7% / 88.3% on Qwen3-8B / Llama-3.1-8B / Gemma-2-9B, against 39.2% / 26.6% / 33.0% for the best internal steering method.

  2. Configuration selection made explicit (cTQE Q2, Lh2Y Q3). For instruction following, tool use, and Long2Short, we report the search grid, selection objective, and selected layer/strength for each model and method. For the task of tool use, single-turn tool use is tested on 200 new examples per target; multi-turn uses a retrospective scenario-grouped 100/100 development–test split.

  3. Why correct tool-use decisions do not carry over to the follow-up (ZALT, cTQE Q3, Lh2Y W2). Across 16 steered model–method combinations, only 638 of 1,489 correct initial decisions (42.8%) also succeed on the follow-up turn; the most common failure is an invalid invocation (37.0% of 851). For Gemma-2-9B/CAA, raising α from 0.5 to 1.0 lifts initial accuracy from 20.5% to 23.0% while joint follow-up success falls from 16.5% to 1.5%.

  4. Source of low multi-turn scores (Lh2Y W2/Q1). A decomposition of the 600 unsteered conversations per model shows different failure profiles: Qwen3-8B mostly reaches a wrong state or execution result (465/600) with few unparseable calls (28/600), Gemma-2-9B mostly lacks a valid call at a required turn (530/600), and Llama-3.1-8B fails at both the decision and call parsing. Scores are already low under the programmatic BFCL checks alone.

  5. Stronger reference agent (Lh2Y W2). On 90 matched conversations, unsteered Qwen3.5-9B (thinking enabled) passes BFCL on 34/90 against 8/90 for Qwen3-8B. On the stronger agent, prompting raises this to 42/90, while CAA lowers it to 23/90.

  6. Steering that varies across turns (ZALT). Restricting the target direction to the annotated turn raises macro-averaged BFCL success from 0.50% to 4.19–5.38% across the two turn-conditioned schedules (Qwen3-8B: 0.8% to 10.0–11.8%). The turn is supplied by annotation, so these are oracle schedules.

  7. Why shorter reasoning can hurt accuracy (ZALT, cTQE W2/Q3). On Llama-3.1-8B, CAA and LAT shorten outputs by about 20% and reduce accuracy from 50.6% to 30.6% while collapse rates fall; prompting compresses more (−29.6%) yet keeps 47.2%. Neither output length nor collapse alone explains the loss.

  8. Closed-loop safety execution and strength sweep (Lh2Y W1, cTQE W2/Q2). On 44 executed tasks (15 harmful, 29 benign) with Qwen3-8B, CAA at α = 1.5 reduces harmful tasks with an unsafe action from 12/15 to 7/15 and retains all six benign tasks the unsteered agent completes; at α = 2.5, one of these is lost and attack success returns to the baseline 3/15.

  9. Judge validation and safety labels (Lh2Y Q2, Q4). Human verdicts agree with the recorded outcomes on 200/200 audited instruction-following and Long2Short responses; the audit does not cover tool-use clarification or abstention judgments. We specify the verifier for each track and the context visible when actions were labelled; labels differ from the source-task category in both directions.

  10. Scope and intended use of the benchmark (cTQE). RealControlBench evaluates a steering method end to end, under a shared protocol: estimate the direction on the extraction set, select layer and strength on the development set, and report on the held-out test set.

We thank the reviewers and chairs again for their time and effort in reviewing our work. We are glad to answer any remaining questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.