acceptodds
Under review as a conference paper at ICLR 2027

TraceSAE: Selective Sparse Control After Tool Feedback

Abstract

Tool-using language models must revise plans when tool feedback invalidates their prerequisites, while retaining plans that remain supported. We introduce TraceSAE, an inference-time controller that targets unwarranted persistence: continuing to rely on a premise explicitly contradicted by visible tool feedback. TraceSAE separates the decision to intervene from the action it changes. An unexecuted shadow proposal informs a gate that is frozen before the actual action is generated. When the gate opens, the controller edits 16 sparse-autoencoder coordinates toward activation profiles from warranted decisions, preserves the original reconstruction error, and caps each perturbation at 2% of the residual norm. Feature groups and intervention strengths are selected through restored-state continuations that jointly evaluate action correctness, goal progress, preservation, and token cost. The language model and pretrained autoencoder remain fixed. Across Gemma-2-9B-it and Llama-3.1-8B-Instruct on Telecom-Compose and ALFWorld, TraceSAE improves full-task success over a rank-16 dense controller by 4.67–8.21 percentage points under a shared gate, donor bank, edit window, and candidate-search allowance. These results establish the task-level benefit of behaviorally selected sparse-profile control in the four evaluated settings.

Primary area:
interpretability and explainable AI
Keywords:
large language model agents, tool use, sparse autoencoders, activation steering, inference-time control, selective plan revision

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_7BkwOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces TraceSAE, an inference-time method for agents that keep following a plan after a tool observation has contradicted one of its prerequisites. At each step, the agent first generates a "shadow" action that is never executed. Auxiliary classifiers then judge whether the observation invalidates the plan and whether the agent is likely to persist. If both hold, the shadow is discarded and the action is regenerated while 16 SAE features are pushed toward their average values from correct past decisions. The features and steering strength are chosen offline by rollout-based search. On Gemma-2-9B-it and Llama-3.1-8B-Instruct, evaluated on ALFWorld and Telecom-Compose, TraceSAE outperforms a matched dense-steering baseline by 4.7 to 8.2 points in task success.

Critical strengths
  1. Using an SAE to identify concepts and steer model behavior toward correct actions is interesting.
  2. The method shows consistent performance gains.
Critical weaknesses
  1. The plan and evidence adapters determine whether an intervention can happen at all, yet the paper never says whether they are rule-based parsers, fine-tuned models or prompted LLM judges, nor how accurate they are. This contrasts with Appendix D, which gives SHA256 hashes of the SAE weight files. If the adapters are strong external LLMs, much of the "knowing when the plan is broken" comes from outside the method. If they are hand-built rules, the method may not transfer to new domains without substantial engineering.
  2. The risk model, the donor profiles and the calibration objective all depend on step-level labels of "warranted" versus "unwarranted persistence." Appendix A.2 states only that "independent blinded raters" produced them. The paper does not say whether these raters are humans or LLMs; how many steps were labelled across the 1,400 discovery tasks per model; how much the raters agreed; etc.
  3. Selectivity is central to the paper's motivation, yet there are no precision, recall or AUROC figures for the risk model or the evidence classifier, and no report of how often the gate opens at test time. Without these, it is impossible to tell how much of the improvement comes from detecting errors well versus from the edit itself.
  4. Most of the improvement over the guard-only baseline is shared by the dense controller, so it comes from the gate, the donor profiles and the search rather than from the SAE. The paper does not test the shadow-and-regenerate procedure without any edit (re-sampling alone may help); a random group of 16 SAE features. There is also no comparison with prompting, self-critique or reflection baselines (e.g., Reflexion [1]), which are the natural alternatives for this problem.

References

[1] Reflexion: Language Agents with Verbal Reinforcement Learning, Shinn et al., 2023

Questions for authors
  1. What exactly are the plan and evidence adapters (rules, a fine-tuned model, or a prompted LLM)? How accurate are they?
  2. Who produced the process labels? How many steps were labelled, with what inter-rater agreement, and what is the precise training target of the risk model?
  3. What are the gate's precision, recall and opening rate at test time?
Reviewer_1q9pOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

TraceSAE is an inference-time controller for tool-using language models that continue relying on a prerequisite contradicted by visible tool feedback. An unexecuted shadow proposal and a pre-action risk model determine a frozen gate; when it opens, the controller edits 16 SAE coordinates toward context- and token-role-specific profiles from warranted decisions. The additive edit retains the original SAE reconstruction error and is capped at 2% of the local residual norm. Feature groups and doses are selected through restored-state continuations that score action quality, progress, preservation, and token cost. Across Gemma-2-9B-it and Llama-3.1-8B-Instruct on Telecom-Compose and ALFWorld, the complete sparse-profile policy improves full-task success by 4.67–8.21 percentage points over a rank-16 dense-profile controller sharing the surrounding control and search procedure.

Critical strengths
  1. The main dense-versus-sparse comparison is thoughtfully controlled. The two policies share the gate, donor decisions and weights, edit positions, norm cap, candidate count, dose grid, calibration states, and continuation allowance. Both use actual branch-specific environment continuations. This makes the comparison substantially more informative than comparing an SAE intervention only with an unguarded agent. The positive full-task differences occur in all four evaluated model–environment settings; the exact counts in Table 7 agree with Table 5.

  2. The intervention and its limitations are explained precisely. Freezing the gate before fresh actual-action generation avoids a circular gating decision, and the additive operator avoids replacing the residual stream with an imperfect SAE reconstruction. The paper correctly distinguishes preservation of the original additive decomposition from invariance after re-encoding, and a local norm bound from behavioral preservation. It also appropriately limits its empirical claim to the complete sparse representation/encoding policy rather than attributing the improvement to sparsity alone.

Critical weaknesses
  1. The empirical results do not yet establish the selective behavior motivating the method. Tables 5 and 7 report only full-task success; there are no held-out measurements of persistence errors, gate coverage/precision, or disruption of warranted decisions. Even task-level rescues and newly introduced failures are not separated. Consequently, an increase in aggregate success could coexist with substantial harm to previously successful tasks, and the results do not show that the specific feedback-update failure was repaired. The calibration preservation term and full-task veto are useful controls, but they are selection criteria, not held-out evidence of preservation. Section 6 explicitly acknowledges this limitation. Because selective revision is the central reason for introducing this controller, a paired held-out analysis of corrected persistence and preserved warranted behavior would materially strengthen the contribution.

  2. The reliability and reproducibility of the reported advantage are difficult to assess from four aggregate outcome rows. The improvements amount to 14/18 net tasks on Telecom and 11 on each ALFWorld comparison, but task-level pairings and uncertainty are unavailable; Telecom also contains only 60 evaluation families, and the same tasks are reused across models. The manuscript additionally omits the actual selected feature groups/doses and the setting-specific learned assets and custom task manifests needed to replay the results, as its reproducibility statement acknowledges. The precise context schema and process-probe fitting procedure are also not sufficiently operationalized to reconstruct the selector. These gaps matter because the empirical contribution depends on a calibrated policy and a custom Telecom extension. Providing paired outcomes with family-aware uncertainty and the concrete deployed-policy/task specifications would make the claimed advantage auditable.

Questions for authors
  1. Can you report a held-out paired breakdown of persistence errors corrected, warranted decisions disrupted, and task-level rescues versus newly introduced failures, together with how often the gate opens? Please specify the process-label sample size and agreement of the blinded raters described in Appendix A.2.

  2. Can you provide task-level outcomes and paired uncertainty for the dense-versus-sparse differences, accounting for Telecom composition families and avoiding treating the reused tasks across models as independent instances?

  3. Are anonymous implementation/policy artifacts available? In particular, what are the selected coordinate IDs and doses in each setting, the exact context categories and process-probe training target/procedure, and the Telecom composition/split manifests? These details would help determine whether the reported policies can be reconstructed from the supplied assets.

Reviewer_oLq7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TraceSAE, an inference-time controller designed to correct unwarranted persistence in tool-using language models. It combines a feedback-aware intervention gate, SAE-based sparse activation editing toward warranted decision profiles, and behavior-based selection of intervention features and strengths. The language model and SAE weights remain frozen. Experiments on two language models and two interactive environments show consistent improvements in full-task success over a dense steering baseline under a shared controller.

Critical strengths
  • The sparse and dense controllers share the intervention gate, donor records, editing positions, perturbation constraints, and candidate search budget. This design helps isolate the benefit of the sparse-profile policy rather than attributing improvements to additional control components or search effort.
  • The method outperforms the dense baseline across both models and environments. Evaluating complete interactive trajectories rather than only intermediate classification or steering metrics provides meaningful evidence of practical effectiveness.
Critical weaknesses
  • The evaluation primarily focuses on full-task success without directly measuring how often unwarranted persistence is corrected or warranted actions are disrupted. Furthermore, the individual contributions of gating, sparse feature selection, and behavioral calibration are not isolated. Consequently, the results establish the effectiveness of the overall controller but provide limited evidence for the specific mechanism claimed as the main contribution.
  • The reported improvements are based on aggregate success counts without paired confidence intervals or task-level rescue/harm statistics. Given the relatively modest net gains and potential dependencies among evaluation tasks, it is difficult to assess how robust the improvements are.
  • The method primarily combines existing ideas in conditional steering, SAE-based interventions, and behavior-guided feature selection. Without comparisons against feedback-aware prompting or tool-critique methods, it remains unclear whether the additional complexity of internal activation control offers a meaningful advantage over simpler alternatives.
Questions for authors
  • Could the authors report held-out persistence-error reduction, intervention frequency, and the rate of disruption to warranted actions? These would help validate the proposed correction mechanism.
  • Could the authors provide paired confidence intervals and task-level rescue/harm statistics, accounting for dependencies among Telecom task families?
  • Are comparisons with feedback-aware prompting or tool-critique methods available? What advantages does the proposed internal steering approach offer over these alternatives?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.