acceptodds
Under review as a conference paper at ICLR 2027

Think or Act? State-Conditioned Representation Steering for Strategic Tool Use in Language Agents

Abstract

Large language model agents increasingly rely on external tools to access information beyond internal reasoning, yet tool use is not uniformly beneficial: a tool call can correct failed internal reasoning, but it can also introduce noise, increase cost, or corrupt an otherwise correct answer. We study this problem as a Think-or-Act decision: whether an agent should rely on internal reasoning or invoke an external tool for a given input. To evaluate this decision, we introduce ToA-Eval, a model-specific protocol that contrasts no-tool and forced-tool executions to identify when tool use helps or harms a particular agent. To control the decision, we propose ToA-Steer, a dynamic cluster-routed steering controller that extracts a shared THINK/ACT direction from contrastive completions and dynamically selects input-specific steering strengths in latent space. Across four reasoning benchmarks and four open-weight LLMs, ToA-Steer raises mean accuracy from 0.585 to 0.723, gains +16pp accuracy on ACT-needed instances and +9pp on THINK-preferred instances simultaneously, and is the only method we evaluate that moves the tool-call rate in opposite directions on these two classes, outperforming prompt-based, training-based, and static-steering baselines. Further analysis shows that THINK/ACT signals concentrate in specific layers and that optimal steering is cluster-dependent, supporting instance-conditioned rather than globally fixed tool-use control.

Primary area:
foundation or frontier models, including LLMs
Keywords:
representation steering, language agent, strategic tool calling, agent decision making

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_f7rDOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies when agents should use external tools and when they should rely on their own reasoning. It introduces ToA-Eval, an evaluation protocol that compares each model’s answers without tools and with forced tool use, identifying cases where tools help or harm answer correctness. The proposed method, ToA-Steer, extracts a tool-use direction from 50 pairs of contrasting responses. It then groups inputs by their internal representations and uses a development set to choose how strongly to encourage or discourage tool use for each group, without updating the model’s parameters. Across four models and four reasoning benchmarks, the authors report the highest accuracy in all settings.

Critical strengths

A simple control method. ToA-Steer uses the model’s internal representations to encourage or discourage tool use for different groups of inputs, without updating the model’s weights. The paper reports the highest accuracy across all model–benchmark settings.

Critical weaknesses
  1. Limited evidence of robustness to new settings. The controller assigns new inputs to existing clusters and reuses their calibrated steering strengths. This assumes that similar internal representations imply similar tool-use needs, which may not hold when question types, difficulty, or tool reliability change. An unfamiliar input is still assigned to the nearest cluster, even if it is far from every cluster, and may receive harmful steering. The paper provides no explicit fallback for this situation. Although the web-search transfer experiment is encouraging, it covers only one model and one target environment. This limits confidence in the method’s reliability beyond its calibration settings.
  2. Unclear data construction. The paper does not sufficiently explain the sources of the contrastive queries or report the size of each data split, particularly for AIME. This makes the reported sample counts difficult to reconcile and limits reproducibility in the evaluation.
Questions for authors

Could you clarify where the 50 contrastive pairs come from and how many examples are in each data split?

5 comments
AuthorsOpenReview ↗

Response to Reviewer f7rD Part A

What happens to inputs outside the calibrated regions (W1)

We thank the reviewer for this concern. Inputs far from every cluster receive no harmful steering, and setting m = 0 for them gives an explicit fallback.

We call a test input “outside” if its distance to the nearest centroid is larger than that of 95% of the calibration data, and compare Base and ToA-Steer on the two groups.

Test inputsShareBase AccToA-Steer AccΔ
Inside the calibrated regions88.7%0.588 ± 0.0120.742 ± 0.014+0.154
Outside (far from every cluster)11.3%0.561 ± 0.0410.578 ± 0.045+0.017
All100%0.5850.723+0.138

Table F1: Qwen3-8B, ToA-Eval test split, four benchmarks pooled, mean ± SD over five runs.

Key findings.

  • No harmful steering on out-of-region inputs: the outside group stays within one SD of Base (+0.017). The selected magnitudes are moderate (|m| of 20–80, Appendix K), so a misrouted input receives a small push, not an off-manifold one.
  • A hard guarantee is available at no cost: setting m = 0 for the outside group reproduces the Base column exactly and changes overall accuracy by less than 0.01.

Overall, inputs the controller has not seen the like of receive little benefit but no damage.

Revision. Add Table F1 and the m = 0 option to Section 3.2.

AuthorsOpenReview ↗

Response to Reviewer f7rD Part B

How the controller behaves when the setting changes (W1)

We thank the reviewer for naming the three kinds of change. Without re-calibration, ToA-Steer stays above Base under all three shifts, and the web-search transfer now holds on a second model. ToA-Steer is calibrated per setting by design. To see what happens without re-calibration, we calibrate on one setting and evaluate on another for each of the three changes. “Transferred” applies the source calibration unchanged; “reference” is the controller calibrated on the target itself and equals Table 2 where applicable.

Table F2a: Change in question type (MATH / Python → GPQA / Wikipedia)

ModelBase AccToA-Steer Acc (transferred)ΔReference
Qwen3-8B0.58 ± 0.050.64 ± 0.06+0.060.74 ± 0.05
Qwen3.5-27B0.62 ± 0.070.64 ± 0.08+0.020.72 ± 0.07
Gemma3-12B0.55 ± 0.050.60 ± 0.06+0.050.69 ± 0.05
Gemma4-31B0.65 ± 0.070.67 ± 0.07+0.020.73 ± 0.06

Table F2b: Change in difficulty (MATH → AIME)

ModelBase AccToA-Steer Acc (transferred)ΔReference
Qwen3-8B0.38 ± 0.090.50 ± 0.11+0.120.62 ± 0.10
Qwen3.5-27B0.45 ± 0.150.48 ± 0.16+0.030.59 ± 0.14
Gemma3-12B0.42 ± 0.090.51 ± 0.10+0.090.62 ± 0.09
Gemma4-31B0.48 ± 0.180.52 ± 0.19+0.040.60 ± 0.17

Table F2c: Change in tool reliability (GPQA / Wikipedia → BFCL V4 Web Search, 50% of page fetches fail)

ModelBaseGRPOToA-Steer (transferred)ToA − Base
Qwen3.5-27B (paper)31.0 ± 3.034.9 ± 3.136.0 ± 2.7+5.0
Qwen3-8B (new)22.4 ± 3.426.1 ± 3.626.9 ± 3.2+4.5

Tables F2a–b: accuracy on the ToA-Eval test split of the target, mean ± SD over five runs.

Key findings.

  • Transfer without re-calibration stays above Base under all three changes: +0.05 to +0.12 for the 8B and 12B models, +0.02 to +0.04 for the 27B and 31B models whose in-domain gains are also smaller.
  • The live-environment result holds on a second model: under 50% fetch failure, Qwen3-8B gains +4.5 over Base and matches GRPO within one SD, the same pattern as Qwen3.5-27B.
  • Per-setting calibration is still preferable: the target-calibrated reference is higher in every case.

Overall, the controller degrades gracefully rather than harmfully when the setting moves away from its calibration.

Revision. Add Tables F2a–c to Appendix O with performed analysis.

AuthorsOpenReview ↗

Response to Reviewer f7rD Part C

Where the contrastive queries come from, and the size of each split (Q1, W2)

We thank the reviewer for this request. The contrastive queries are unlabeled questions outside Test, so they never overlap the labeled pool, and the counts reconcile even on AIME. The full split sizes are below.

1. Contrastive pairs and the router

Each pair is one question with two fixed template completions: “Use [tools] to address this query.” (ACT) and “Answer directly from your own knowledge.” (THINK). We use 50 questions per benchmark, taken from the source questions that are not in Test. The same kind of unlabeled questions form the Router-train set on which PCA and K-means are fit. Only the development set and Test are drawn from the labeled pool. For AIME, the labeled pool uses AIME 2024–2026 (90 problems); the 50 pair queries and the N > 50 points in Fig. 8 use AIME 2000–2023.

Small pools use 5-fold cross-validation: where the labeled pool has at least 50 instances, Dev is 25 ACT + 25 THINK and Test is the rest. Where it has fewer, a single split would leave a test set of a few questions, so we calibrate with 5-fold cross-validation: each fold is held out in turn, the other four form Dev, and the held-out predictions are pooled. Every labeled instance is then a test instance exactly once.

ModelBenchmarkLabeled poolVector-trainRouter-trainDevTest
Qwen3-8BMATH309505050259
Qwen3-8BAIME30505024*30*
Qwen3-8BGPQA8450505034
Qwen3-8BMMLU-Pro627505050577
Qwen3.5-27BMATH415505050365
Qwen3.5-27BAIME14505011*14*
Qwen3.5-27BGPQA42505034*42*
Qwen3.5-27BMMLU-Pro555505050505
Gemma3-12BMATH315505050265
Gemma3-12BAIME34505027*34*
Gemma3-12BGPQA9550505045
Gemma3-12BMMLU-Pro650505050600
Gemma4-31BMATH365505050315
Gemma4-31BAIME12505010*12*
Gemma4-31BGPQA45505036*45*
Gemma4-31BMMLU-Pro623505050573

Table F3: Unique question counts. Vector-train and Router-train are unlabeled source questions outside Test. *5-fold cross-validated calibration: Dev is the size of the four calibration folds, Test is the whole labeled pool with each instance held out once.

Revision. Add Table F3 and the cross-validation protocol to Appendix G; rewrite the split description there (two unlabeled splits from source questions, Dev and Test from the labeled pool); correct the AIME year ranges in Section 2 and Appendix E; state the benchmark versions used; mark the cross-validated cells in Table 2.

Reviewer_M4YEOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper surrounds an interesting topic in current agentic tool calling area: should a model rely on internal thinking or calling external tool. The authors first made a investigation one without any tool call, and another run with mandatory tool call for every question in some reasoning datasets (MATH, AIME, etc.). After execution, they retain two groups of examples: those where tool use corrects an otherwise incorrect answer, and those where tool use causes an otherwise correct answer to become incorrect.

Based on this observation, the authors propose a method to control whether a model should use tools. Specifically, they construct 50 contrastive pairs by providing the same question with two manually designed completions, one encouraging tool use and the other encouraging internal reasoning. They extract the corresponding hidden representations from a frozen LLM and average the differences to obtain a shared steering direction. Since different questions may require different tool-use decisions, the authors further apply clustering to input representations and select a steering strength for each cluster based on validation accuracy.

In summary, the proposed method is training-free, the only concern for its generalization might be the manual construction of the 50 contrastive pairs. Experiments across four backbones and four reasoning benchmarks show that the proposed method consistently outperforms prompting, static steering, SFT, and GRPO in the authors' evaluation setting.

Critical strengths
  1. The paper investigates an important question in agentic tool use: when should a model rely on internal reasoning, and when should it call an external tool?
  2. The proposed method achieves consistent improvements across four models and four reasoning benchmarks, outperforming prompting, SFT, GRPO, and static steering.
  3. The authors also provide useful ablations on steering directions, clustering, and intervention timing, which help explain why the proposed approach works.
Critical weaknesses
  1. It is unclear how much of the improvement comes from activation steering itself.
  2. Too much conditions to achieve, the method requires an additional clustering and calibration process, as well as a manual construction of the 50 contrastive pairs. All make the method less practical to generalize.
Questions for authors
  1. How do the authors distinguish cases where tool use is genuinely harmful from cases where the model simply makes an incorrect tool call or misinterprets the tool output?
  2. Would it be possible for the authors compare the proposed method against a lightweight router trained with the same data and calibration budget, but directly deciding whether to use tools instead of performing activation steering? This could help answer whether an automatic trained planner shared with same information could work better.
6 comments
AuthorsOpenReview ↗

Response to Reviewer M4YE Part A

Steering vs. a router that directly decides tool use (Q2, W1)

Thank you for raising this comparison. Under the same data and budget, a direct router is clearly weaker than ToA-Steer (0.55 / 0.64 vs. 0.72).

1. Compare ToA-Steer with a direct router trained with the same data and budget

We trained two logistic-regression routers on the same splits as ToA-Steer. The question router uses a sentence embedding of the question. The hidden-state router uses the same decision-token feature that ToA-Steer clusters.

We tested each router in two ways: 1) Direct control in the output level: when the router predicts THINK, we ask the model to predict without external tools; when it predicts ACT, we force it to call tools. 2) Steering: we apply one signed magnitude per predicted class, chosen on the development set. The results are as follows:

MethodRouting Acc ↑ACT call ↑THINK call ↓ToA-Eval Acc ↑
Base—0.5750.6850.585
Question router0.31
+ direct control0.480.650.44
+ steering0.610.550.57
Hidden-state router0.49
+ direct control0.560.610.55
+ steering0.630.520.64
Ours
ToA-Steer0.620.750.450.72
Oracle label + direct control1.000.980.000.957

Key findings.

  • Steering beats direct control for both routers: 0.57 vs. 0.44 for the question router and 0.64 vs. 0.55 for the hidden-state router.
  • ToA-Steer beats every router: +8 points over the best router with steering and +17 over the best direct control.
  • Oracle labels with direct control reach 0.957: this is an upper bound that needs gold labels, which are unavailable at test time.

2. Ablation on the effect of steering control in ToA-Steer

The gain comes from the learned direction and is realized at the decision point. As shown in the table above, default steering alone can not produce matched performance as ToA-Steer, we hereby demonstrate where does the extra performance gain come from:

  • Table 10 and Appendix L show that the learned direction does most of the work. We replaced the learned direction with a random one of the same norm and kept everything else unchanged. We can see the accuracy barely moved (0.585 → 0.591), while ToA-Steer reaches 0.723. So almost all of the 13.8-point gain depends on the learned direction. Routing accounts for the rest: with one shared magnitude for all clusters, accuracy falls to 0.635, and with questions assigned to clusters at random, it falls to 0.674.
  • Table 5 in Section 5 shows that most of the effect happens at the moment the model decides. We stopped steering as soon as the model took its first action. This variant still reaches 76.8 on average, compared with 75.5 for Static-best and 77.6 for full steering. This demonstrates that most of the gain comes from controlling the tool decision itself, not from changing how the model reasons afterwards.

3. What ToA-Steer offers but a external router can not?

  • Soft intervention vs. a hard override: Fig. 5 in the original paper shows that the decision-token representation already separates instances that need opposite interventions, but the base model acts on this only about 43% of the time. We show steering changes how strongly the model leans toward a tool call without taking the decision away from it: under ToA-Steer the tool-call rate is 0.75 on ACT-needed and 0.45 on THINK-preferred instances (Table 2), so the model still decides in each case. A direct router does not allow this: when it predicts THINK the tool schema is removed, when it predicts ACT a call is forced, so a wrong prediction cannot be corrected later.
  • Causal evidence vs. a readout: A router only shows that ACT/THINK can be decoded from the latent representation, steering shows that the model actually uses this direction. In Table 10, a random steering direction with the same ToA-Steer process gives no gain over baseline. Later in Appendix I-J, we show that the intervention has no effect in early layers, controls tool use in the middle-late layers, and becomes noisy in the final layers, with the tool-use information reaching the peak at 0.71–0.72L for all four models. These results tell us where the decision is formed, which a classifier on top of the hidden state cannot.

Revision. We will add the updated Table and the analysis to Section 5.

AuthorsOpenReview ↗

Response to Reviewer M4YE Part B

How much setup ToA-Steer actually needs (W2)

None of the setup steps is manual or costly: the pairs are generated from templates, clustering is unsupervised, and calibration needs 450 rollouts with no training.

1. The 50 pairs are auto-generated from two fixed templates

The contrastive pairs are auto-generated. For each unlabeled question in the Vector-train split, we append one of two fixed assistant replies (Section 3.1): - ACT: “Use [tools] to address this query.” - THINK: “Answer directly from your own knowledge.”

We can also use less contrastive pairs to show the think-or-act decision control at the minor cost of performance (Fig. 8). Here we show the generalizability of different prompt template

Template wordingCosine to default direction ↑Avg. Acc ↑
Default (paper)1.000.723
Paraphrase A0.910.698
Paraphrase B0.940.720
Paraphrase C0.880.718

The direction is robust to template wording: cosine ≥ 0.88 and accuracy within 3 points.

2. Clustering is a one-off step and needs no tuning

  • PCA and K-means are off-the-shelf, unsupervised steps with nothing to train. They are fit once on the small dev set in seconds on CPU (Table 9). At inference they add one projection and one centroid lookup per question and no extra LLM call. A new model or task only needs the same two routines run again on new hidden states.
  • The layer does not need excessive searching. The signal peaks at 0.71–0.72L in all four models (Appendix I).
  • Each step contributes to the final performance. Removing PCA costs 5.0 points, replacing K-means with simple PC1 bins costs 8.0, and sharing one magnitude across clusters costs 8.8 (Table 10).
  • The whole pipeline runs end-to-end with no manual step. Contrastive pair completion → steering vector extraction → routing and calibration using PCA + K-means → Inference using assigned magnitude. Each stage takes the previous one's output as input, so applying ToA-Steer to a new model or task is a single script run without excessive steps.

3. No need to recalibrate for every new setting

A calibration transfers without re-calibration: from GPQA/Wikipedia to BFCL V4 Web Search, overall 43.8 → 48.6, and 31.0 → 36.0 with 50% failed fetches. We calibrated ToA-Steer on GPQA with the Wikipedia tool and then used it as-is on BFCL V4 Web Search, a live web-search setting (Appendix O, Qwen3.5-27B). It had the best score in all four settings. This experiment covers only one model and one environment. Calibration is per model on purpose, because ToA-Eval labels are model-specific.

4. The full cost compared with other methods.

We further provide the computing resources needed for running ToA-Steer. For a new benchmark, ToA-Steer needs three things: 100 prefill passes to get the direction, a PCA + K-means fit that takes seconds on CPU, and 450 rollouts (9 magnitudes × 50 development questions) to pick each cluster's magnitude. We compare ToA-Steer with other methods as follows:

Method# Rollouts# Trainable parametersExtra inference latency per questionToA-Eval Acc ↑
Base0000.585
SFT10043.6M00.630
GRPO90043.6M00.688
Static-best4500< 1 ms0.645
Direct router4504,097≈ 30 ms0.67
ToA-Steer (Ours)4500< 1 ms0.723

Table 2: Cost and accuracy per benchmark (Qwen3-8B, one H100). Rollouts are counted as needed for a new task: two per labeled question, nine per development question, eight per prompt per epoch for GRPO. Answering one question takes about 8 s, so neither inference overhead is noticeable in practice; ToA-Steer adds one projection per question and one vector addition per token, the direct router adds one extra prefill of the prompt to read latent representation. SFT and GRPO merge the LoRA weights, so they add nothing.

Key findings. ToA-Steer trains nothing, adds under a millisecond at inference, and reaches the highest accuracy. Its 450 rollouts sit in the same range as Static-best and the direct router.

We also checked how small the Dev set can be:

Dev questions per benchmark10203050 (paper)
Generations90180270450
Avg. Acc ↑0.6680.6970.7140.723

Table 3: Qwen3-8B, ToA-Eval test split. Base = 0.585.

20 Dev questions already give 81% of the full gain (180 generations).

Revision. We will make the template construction explicit in Section 3.1, replace “cheap to calibrate” in Section 5 with the cost tables above (added to Appendix G), and add the template and Dev-size results to the appendix.

AuthorsOpenReview ↗

Response to Reviewer M4YE Part C

Distinguishing harmful tool use from tool misuse (Q1)

We thank the reviewer for this question. ToA-Eval does not separate the two cases, and this is by design. A THINK label records whether calling a tool turns a correct answer into a wrong one for each particular model. Whether the cause is misleading tool content, a malformed call, or a misread output, the model is better if answering directly, and that is exactly the decision we want the controller to make . Section 2 therefore defines the labels as model-specific outcome contrasts rather than as a property of the tool.

1. The contrast comes from what the tool returns, not from the act of calling it

Appendix B (Table 8) isolates the calling step. When the tool is called but its output is withheld from the model, accuracy moves by at most 3.2 points relative to the no-tool pass; when the actual output is shown, it moves by +86.8 points on ACT-needed and −87.7 points on THINK-preferred instances. A THINK label is therefore about how the model handles the tool's output, which covers both bad content and misuse, and not about the format or interruption of the call.

2. What goes wrong in THINK cases

To show how the two cases are distributed, we annotated 100 randomly sampled THINK-preferred trajectories on Qwen3-8B.

Cause of the wrong with-tool answerShare
Irrelevant or misleading tool content55%
Correct tool output, misread or misused by the model15%
Wrong tool or malformed arguments10%
Budget exhausted after the tool call20%

Table 4: Causes of THINK outcomes (LLM-annotated and further checked by human).

Key findings. Misleading content is the largest cause (55%), but 15% of cases are misuse of correct output, so THINK covers both.

Revision. State in Section 2 that THINK covers both harmful tool output and tool misuse, and add Table R3-C1 to Appendix B.

Reviewer_pKdLOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3(+1)
Soundness
3
Read review
Paper summary

The paper introduces ToA-Eval, which identifies model-specific cases where forced tool use either fixes an incorrect no-tool answer or corrupts a correct one. ToA-Steer extracts a THINK/ACT activation direction from contrastive completions and uses PCA/K-means routing to select signed, cluster-specific intervention strengths. Evaluations cover four models and four reasoning benchmarks. Additional analyses include label reconstruction, full-pool evaluation, decision-point-only steering, routing controls, tuned static steering, and transfer from GPQA calibration to BFCL V4 Web Search.

Critical strengths
  1. The work asks a useful question beyond whether a model can operate a tool: whether tool use helps this model on this input. The bidirectional ACT/THINK construction captures both beneficial and harmful calls. The repeated-label, prompt-paraphrase, and hidden/empty-result controls provide substantive checks on the protocol, while the paper explicitly acknowledges that these are model- and protocol-dependent outcome contrasts rather than intrinsic tool-necessity labels.

  2. The control method is lightweight in terms of model-parameter updates, and the analyses go well beyond comparison with an untuned static vector. Table 5 includes a development-tuned Static-best baseline and a first-action-only intervention; Appendices L and O add random-direction/cluster controls, calibration sensitivity, and transfer without BFCL recalibration. These results support a role for input-conditioned steering and provide useful evidence that the gains are not explained only by globally increasing tool use.

Critical weaknesses
  1. The sample-efficiency and low-cost interpretation is incomplete when presented mainly in terms of 50 contrastive pairs or 100 forward passes. That accounting describes vector extraction, whereas the complete controller also fits routing representations and selects per-cluster magnitudes through answer-supervised generation over a nine-value grid, with additional choices of layer and K. Appendix G distinguishes these roles, but does not give the exact per-model/per-benchmark split counts and total calibration budget. Thus, the absence of parameter updates is clear, but the comparative data and computation requirements versus SFT/GRPO are not yet fully characterized.

  2. The relationship between the selected disagreement benchmark and the full-pool deployment claim needs clearer reporting. The main evaluation intentionally excludes both-correct and both-wrong instances. The full-pool analyses and BFCL transfer are valuable responses to selection concerns, but the population sizes, split relationship, and contribution of the both-wrong group are not sufficiently explicit to reconstruct overall gains. In particular, Table 3 reports the first three outcome groups but not both-wrong, and the BFCL advantage over GRPO is only about one percentage point with run-level variability. This calls for a more precise account of population-level effect size, rather than repeating experiments that are already present.

Questions for authors
  1. Could you tabulate unique question counts for Vector-train, Router-train, Dev/probe, and Test for each model-benchmark combination, along with the number of contrastive pairs? Please reconcile the main text's broad use of "probe" with Appendix G's four disjoint roles, and identify the AIME years/split used in each analysis. This is particularly useful where the available decision-relevant pool is small; I am not assuming test leakage.

  2. Could you give an end-to-end calibration ledger covering vector extraction, routing fits, magnitude sweeps, and layer/K selection, preferably with total generated tokens or wall-clock time? Please separate the 100 vector-extraction forward passes from the full controller cost and state precisely what information and calibration budget are shared with SFT, GRPO, and Static-best.

  3. Using the existing full-pool outputs, could you report the size, accuracy change, and tool-call behavior of all four no-tool/forced-tool outcome groups, including both-wrong, and the resulting population-weighted overall accuracy? Please confirm that these metrics are evaluated on questions held out from every calibration stage. This would make the full-pool evidence easier to interpret without replacing the focused ToA-Eval protocol.

4 comments
AuthorsOpenReview ↗

Response to Reviewer pKdL Part A

Exact split counts and terminology clarification. (Q1, W1)

We thank the reviewer for the precise request. No question used in any calibration stage appears in Test; the table and the terminology mapping below make this checkable.

1. Unique question counts

Two sources fill the four roles: Vector-train and Router-train are unlabeled source questions outside Test, and Dev and Test partition the labeled pool. Vector-train and Router-train only need a query and its hidden state, so they carry no ACT/THINK label; the number of contrastive pairs equals the Vector-train count.

Where the labeled pool has at least 50 instances, Dev is 25 ACT + 25 THINK and Test is the rest. Where it has fewer, we calibrate with 5-fold cross-validation so that every labeled instance is held out exactly once; these settings are marked with * below.

ModelBenchmarkLabeled poolVector-trainRouter-trainDevTest
Qwen3-8BMATH309505050259
Qwen3-8BAIME30505024*30*
Qwen3-8BGPQA8450505034
Qwen3-8BMMLU-Pro627505050577
Qwen3.5-27BMATH415505050365
Qwen3.5-27BAIME14505011*14*
Qwen3.5-27BGPQA42505034*42*
Qwen3.5-27BMMLU-Pro555505050505
Gemma3-12BMATH315505050265
Gemma3-12BAIME34505027*34*
Gemma3-12BGPQA9550505045
Gemma3-12BMMLU-Pro650505050600
Gemma4-31BMATH365505050315
Gemma4-31BAIME12505010*12*
Gemma4-31BGPQA45505036*45*
Gemma4-31BMMLU-Pro623505050573

Table A1: Unique question counts. Vector-train and Router-train are drawn from source questions disjoint from the labeled ACT/THINK pool; Dev and Test are the only splits drawn from that labeled pool. *5-fold cross-validated: Dev is the size of the four calibration folds, Test is the whole labeled pool with each instance held out once

2. Terminology and AIME years

  • Section 3.2 “full probe set” = Router-train (used in PCA and K-means). “Held-out subset P_k” = Dev (used in magnitude calibration). We will use the Appendix G names throughout.
  • AIME Setup. The labeled pool is AIME 2024–2026 with 90 problems. The 50 contrastive queries, and the N > 50 points in Fig. 8, come from AIME 2000–2023 and are never evaluated. The year ranges in Section 2 and Appendix E will be adjusted to match.
  • Test is held out first: Vector-train and Router-train exclude it by construction.

Revision. Add Table A1 and the cross-validation protocol to Appendix G, explain the sample set for each model-dataset split with clear explanations how each split is computed and sourced, replace “probe” in Sections 2–3, update the AIME years, and state the benchmark versions used.

AuthorsOpenReview ↗

Response to Reviewer pKdL Part B — Calibration Ledger

End-end break down of the the full controller costs (Q2, W1)

We thank the reviewer for asking for the complete accounting. The full controller costs 450 rollouts per benchmark for the magnitude grid, plus prefill-only steps that are negligible, and no gradient update.

1. Ledger

StageDataSupervisionLLM callsGenerated tokensWall-clock (1 × H100)
Direction extractionVector-train, 50 queriesnone100 prefill0< 1 min
Routing fit (PCA + K-means)Router-train, 50 queriesnone50 prefill0< 1 min (fit on CPU)
Magnitude gridDev, 50 questionsgold answers9 × 50 = 450 rollouts≈ 3.2 M≈ 1.5 h
K selectionDevgold answers0 (re-reads the grid for K = 2…10)0—
Layer selectionDevnone50 prefill (logit lens)0< 1 min
Total per benchmark450 rollouts + 200 prefill≈ 3.2 M≈ 1.5 h

Table P2: Qwen3-8B, one benchmark, one calibration run, at the average 7000 generated tokens per rollout. Per model, all four benchmarks: ≈ 1,800 rollouts and ≈ 6 GPU-hours.

2. Comparison of each method:

MethodRollouts per benchmarkGradient-trained parametersSame Test as ToA-Steer
ToA-Steer4500yes
Static-best4500yes
Linear router4504,097yes
SFT100 (two per labeled question)43.6 M (LoRA, r = 16)yes
GRPO900 in total: 100 for SFT + 800 (8 rollouts × 2 epochs × 50 prompts)43.6 M (LoRA, on the SFT model)yes

Table P3: Per benchmark, comparison using Qwen3-8B. Every method in Table 2 is calibrated or trained on the same labeled questions and evaluated on the same Test.

Key findings.

  • No gradient training: 450 rollouts and 200 prefill passes, about 1.5 H100-hours per benchmark.
  • Same budget as Static-best and the linear router, which share the same Dev set.
  • Cheaper than GRPO: GRPO needs 900 rollouts on top of SFT and trains 43.6M parameters.

Revision. Replace “cheap to calibrate” in Section 5 with Table P2, add Tables P2–P3 to Appendix G, and state the shared Dev set in Appendix F.

AuthorsOpenReview ↗

Response to Reviewer pKdL Part C — Full-Pool Effect Size

Break down results of each groups and the held-out guarantee (Q3, W2)

We thank the reviewer for this request. On the full pool the population-weighted gain over Base is +3.0 ± 0.8 points. It comes almost entirely from the two decision-relevant groups; The changes on both-correct and both-wrong are near zero relative to their run-level variation. Table 3 omitted the last two rows below.

1. Four outcome groups

Outcome groupnShareBaseGRPOStatic-bestToA-SteerToA − BaseTool-call rate, Base → ToA
ACT-needed1,61710.9%54.8 ± 2.466.0 ± 3.062.5 ± 2.670.4 ± 2.7+15.6 ± 2.30.55 → 0.70
THINK-preferred2,09814.1%71.0 ± 2.264.2 ± 2.972.9 ± 2.479.2 ± 2.5+8.2 ± 2.00.60 → 0.42
Both-correct8,91660.0%97.2 ± 0.696.8 ± 0.797.2 ± 0.597.3 ± 0.6+0.1 ± 0.50.54 → 0.51
Both-wrong2,22915.0%2.8 ± 0.81.9 ± 0.62.5 ± 0.82.8 ± 0.80.0 ± 0.70.61 → 0.63
Overall, population-weighted14,860100%74.6 ± 0.974.6 ± 1.275.5 ± 1.077.6 ± 1.0+3.0 ± 0.80.56 → 0.53

Table P4: Full pool, pooled over four models, mean ± SD over five runs. n counts held-out questions only (the Test column of Table P1 for the two labeled groups

2. Relation to ToA-Eval and held-out

All full-pool numbers are held out from every calibration stage: The ToA-Eval subset of Table 2 is the union of the first two rows. The both-wrong group adds nothing to the overall gain (15% of the pool, accuracy unchanged at 2.8), so the +3.0 overall is the 15.6- and 8.2-point gains on the two decision-relevant groups weighted by their own shares. All full-pool numbers are computed on questions outside Vector-train, Router-train and Dev; in the cross-validated settings each instance is scored in the fold where it is held out.

3. BFCL held-out results

On BFCL, ToA-Steer matches GRPO rather than beating it: As suggested in the paper, the 0.8–1.1-point margin is about one run SD, and ToA-Steer achieves it without parameter updates or re-calibration

Revision. Add the both-wrong and overall rows, group sizes and shares to Table 3, add per-benchmark full-pool results to the appendix, and revise the BFCL sentence in Section 4.2 and Appendix O.

Thank you for clarifying the data splits and breaking down the cost of each calibration step. I can now follow which examples are held out and how much computation the full method requires, so I will raise my presentation score. The full-pool results also clarify the scope of the benefit. Most of the gain comes from questions where the choice between thinking and acting matters, and performance on BFCL is comparable to GRPO. On balance, I will keep my overall rating at 3.

Reviewer_oAtQOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the Think-or-Act decision in LLM agents, focusing on whether an agent should rely on internal reasoning or invoke external tools. The authors introduce ToA-Eval, a model-specific protocol that evaluates whether tool use is beneficial or harmful for individual instances based on observed outcomes. Based on those instances, they further propose ToA-Steer, an efficient cluster-routed steering method. By leveraging clustering, ToA-Steer avoids using a globally fixed steering magnitude and instead assigns different steering strengths to different groups of instances while maintaining a shared semantic direction. Experimental results demonstrate that ToA-Steer consistently outperforms prompting and post-training methods, supported by extensive analyses.

Critical strengths
  1. Clear writing and organization. The paper is well-organized and clearly written, making it easy to follow and understand.

  2. Comprehensive experiments and consistent improvements. The experimental setup is described in detail, and the proposed method demonstrates substantial and consistent improvements across different settings. The authors also provide extensive analyses of key factors, including steering magnitudes, the number of clusters, and the number of contrastive samples.

  3. Interesting findings and insights. The paper also presents several interesting findings, such as the effects of steering in opposite directions (positive vs. negative magnitudes), the impact of steering duration (first-action-only vs. full-generation steering), and the consistent relative layer depths at which THINK/ACT-related signals emerge across different models.

Critical weaknesses
  1. Limited evaluation scale and missing variance. AIME contains at most 90 problems, yet the paper uses per-benchmark N=50 contrastive pairs, a 50-instance dev split, and four disjoint splits. The AIME test set could therefore be very small, but Table 2 reports no variance or confidence intervals, and many “best” entries are separated by only 0.01. The claim of winning all 16 settings is not very persuasive under these conditions.

  2. Overfitting risk in magnitude selection. Per-cluster magnitudes and K are both selected on a dev split of about 50 instances. On GPQA (K=5), this means picking the best of 9 magnitudes from roughly 10 instances per cluster. Especially considering the uniformly consistent test gains. Results over multiple resplits would help.

  3. Limited qualitative analysis. The experimental analysis focuses primarily on quantitative results and numerical comparisons. More concrete examples or qualitative analyses would help readers better understand how steering affects the model's behavior and final outputs.

Questions for authors
  1. Unexpected tool-call rate trends. In Figure 3, the tool-call rates on AIME and GPQA appear to exhibit trends that are inconsistent with the expected semantic direction of steering. Have the authors discussed or further investigated these observations?

  2. Report the exact size of each split for every model–benchmark pair. Could the authors report the exact split size for each model–benchmark pair? In particular, how were the N=100/200 points for AIME in Figure 8 obtained, given that AIME has at most 90 problems?

  3. Further mechanistic understanding of cluster-routed steering. Cluster-routed steering achieves stronger improvements than static magnitude steering and the linear router baseline. Could the authors provide further analysis of why cluster-based routing is more effective? For example, do instances within the same cluster share certain semantic characteristics or exhibit similar sensitivity to different steering magnitudes? What specific properties of these instances within different clusters make them more sensitive or insensitive to the strength of THINK-or-ACT interventions? More broadly, what additional information does cluster-routed steering capture compared to static bidirectional steering?

Comments on the paper

5 comments
AuthorsOpenReview ↗

Response to Reviewer oAtQ Part A

Evaluation scale, split sizes and variance (Q2, W1)

We thank the reviewer for the careful reading. The contrastive pairs do not consume the AIME pool, and every gain holds over five complete reruns. We provide the exact split size table and variance analysis as here.

1. Exact split sizes

  • Only Dev and Test come from the labeled pool: the contrastive pairs and the router use unlabeled questions drawn from the source benchmark, outside Test, so they need no ACT/THINK label. For AIME, the labeled pool is AIME 2024–2026 (90 problems), while the 50 pair queries and the N > 50 points in Fig. 8 use AIME 2000–2023, so the two do not conflict.

  • Small pools use 5-fold cross-validation: where the labeled pool has at least 50 instances, Dev is 25 ACT + 25 THINK and Test is the rest. Where it has fewer, a single split would leave a test set of a few questions, so we calibrate with 5-fold cross-validation: each fold is held out in turn, the other four form Dev, and the held-out predictions are pooled. Every labeled instance is then a test instance exactly once.

ModelBenchmarkLabeled poolVector-trainRouter-trainDevTest
Qwen3-8BMATH309505050259
Qwen3-8BAIME30505024*30*
Qwen3-8BGPQA8450505034
Qwen3-8BMMLU-Pro627505050577
Qwen3.5-27BMATH415505050365
Qwen3.5-27BAIME14505011*14*
Qwen3.5-27BGPQA42505034*42*
Qwen3.5-27BMMLU-Pro555505050505
Gemma3-12BMATH315505050265
Gemma3-12BAIME34505027*34*
Gemma3-12BGPQA9550505045
Gemma3-12BMMLU-Pro650505050600
Gemma4-31BMATH365505050315
Gemma4-31BAIME12505010*12*
Gemma4-31BGPQA45505036*45*
Gemma4-31BMMLU-Pro623505050573

Table A1: Unique question counts. Vector-train and Router-train are unlabeled source questions outside Test. *5-fold cross-validated: Dev is the size of the four calibration folds, Test is the whole labeled pool with each instance held out once

2. Variance over five complete runs

We rerun the full pipeline five times. Each run re-draws the splits (or the folds), re-samples the contrastive pairs, fit the direction and clustering; nothing is carried over between runs. As the following table demonstrates, ToA-Steer consistently yields larger performance gain over the existing method.

ModelToA-Steer, ToA-Eval AvgToA − GRPO, ToA-EvalToA-Steer, full poolToA − GRPO, full poolToA − Static-best, full pool
Qwen3-8B0.721 ± 0.016+0.033 ± 0.01967.3 ± 1.4+0.6 ± 1.0+1.4 ± 1.0
Qwen3.5-27B0.726 ± 0.024+0.026 ± 0.02782.2 ± 1.2+4.4 ± 1.5+2.7 ± 1.1
Gemma3-12B0.720 ± 0.018+0.032 ± 0.02273.8 ± 1.4+2.5 ± 1.6+1.7 ± 1.2
Gemma4-31B0.749 ± 0.023+0.017 ± 0.02686.9 ± 1.1+4.2 ± 1.4+2.6 ± 1.1

Table A2: Mean ± SD over five runs; differences are paired

Key findings.

  • ToA-Steer stays ahead of GRPO on every model: +0.017 to +0.033 on ToA-Eval, although within one SD for Qwen3.5-27B and Gemma4-31B.
  • On the full pool, the margin over Static-best exceeds one SD on every model: +1.4 to +2.7 points.

Revision. Add Table A1 and the cross-validation protocol to Appendix G , explain the sample set for each model-dataset split with clear explanations of how each split is computed and sourced, and add the statistical testing over the existing benchmarks.

AuthorsOpenReview ↗

Response to Reviewer oAtQ Part B — Overfitting in Magnitude Selection

Elaboration on the overfitting problems. (W2)

We thank the reviewer for this concern. Re-selecting K and all magnitudes on four re-splits lowers Test relative to Dev, but ToA-Steer still beats a Static-best re-tuned on the same Dev.

BenchmarkBaseStatic-best, TestToA-Steer, DevToA-Steer, TestDev − TestToA − Static (paired)Sign of m_k stable across re-splits
MATH0.700.76 ± 0.020.82 ± 0.020.785 ± 0.020.035+0.025 ± 0.01595%
AIME0.380.58 ± 0.080.64 ± 0.070.60 ± 0.090.04+0.02 ± 0.0870%
GPQA0.580.68 ± 0.040.80 ± 0.040.73 ± 0.050.07+0.05 ± 0.0478%
MMLU-Pro0.680.72 ± 0.010.76 ± 0.010.735 ± 0.0150.025+0.015 ± 0.01288%
Avg0.5850.685 ± 0.020.77 ± 0.020.713 ± 0.020.057+0.028 ± 0.02—

Table B1: Qwen3-8B, mean ± SD over 4 re-splits. Each re-split re-selects K and all per-cluster magnitudes on its Dev set; Static-best is re-tuned on the same Dev.

Key findings.

  • Dev performance is optimistic than the real performance in test: Test is 0.057 below Dev on average, most on GPQA
  • The gain over Static-best survives re-selection: +0.028 on average and positive on all four benchmarks;
  • Magnitude signs are mostly stable: each cluster keeps its sign in 70–95% of re-splits.

Revision. Add Table B1 to Appendix L and report the per-cluster Dev counts in Appendix K.

AuthorsOpenReview ↗

Response to Reviewer oAtQ Part C — What Cluster Routing Captures

Why cluster routing beats a static magnitude and a linear router (Q3, W3).

We thank the reviewer for this question. A cluster groups questions that respond to the magnitude in the same direction, which neither a static magnitude nor a hard label can express.

1. What instances in a cluster share

ClusterShareACT shareDominant contentBase tool-call rateSelected m_kAcc at m = −40 / 0 / +40
10.250.67physics and chemistry with a numerical step (unit conversion, rate constants)0.74+800.43 / 0.50 / 0.64
20.190.56organic chemistry mechanisms with a counting step0.69+400.52 / 0.55 / 0.63
30.230.47mixed: astrophysics, molecular biology0.6000.58 / 0.63 / 0.59
40.200.35genetics and cell-biology recall0.55−400.69 / 0.61 / 0.55
50.130.27quantum mechanics and relativity, conceptual0.50−400.64 / 0.62 / 0.50

Table C1: GPQA, Qwen3-8B on all labeled instances.

Key findings.

  • A cluster is a region of similar response to the intervention: the ACT share ranges from 0.27 to 0.67, yet each cluster’s accuracy moves one way with m (cluster 1: 0.50 → 0.64 at +40; cluster 4: 0.61 → 0.69 at −40).
  • Content intensity roughly explains the trend: computation-heavy questions concentrate in the clusters that select m > 0 and recall required questions concentrate on m < 0. The mixed cluster in between shows no significant preference over any of the cluster.

2. Qualitative example

In both directions, steering changes the first decision sentence, and that alone flips the answer. Appendix C gives two MATH cases with the model’s original decision sentence. We here give another case using the GPQA dataset:

  • Over-use (THINK-preferred). Q: You overhear two chemists talking as they leave a synthetic organic chemistry lab. One asks “So, how did it go?” The other replies, “Not well, my compounds are on top of each other.” What is the second chemist most likely referring to? (A) similar polarities (B) similar boiling points (C) non-covalent bonding between the compounds (D) similar optical rotations. GT: (A). Base: “Let me search for what ‘compounds on top of each other’ means in organic chemistry.” Wikipedia returns the page on van der Waals forces; the model answers (C) ✗. ToA-Steer: “‘On top of each other’ refers to overlapping spots on TLC, i.e., similar Rf values.” Answers (A) ✓.
  • Under-use (ACT-needed). Q: In a given population, 1 out of every 400 people has a cancer caused by a completely recessive allele, b. Assuming the population is in Hardy–Weinberg equilibrium, what is the expected proportion of individuals who carry the b allele but are not expected to develop the cancer? (A) 1/400 (B) 19/400 (C) 20/400 (D) 38/400. GT: (D). Base: “q² = 1/400, so q = 1/20; heterozygotes are 2pq ≈ 2 × 1/20 = 1/10, closest to (C).” Answers (C) ✗. ToA-Steer: “Let me use Python to compute 2pq exactly.” Python returns 0.095 = 38/400; answers (D) ✓.

Revision. Add Table C1 and the GPQA cases to Appendices K and C with the attached analysis..


From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.