- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces TraceSAE, an inference-time method for agents that keep following a plan after a tool observation has contradicted one of its prerequisites. At each step, the agent first generates a "shadow" action that is never executed. Auxiliary classifiers then judge whether the observation invalidates the plan and whether the agent is likely to persist. If both hold, the shadow is discarded and the action is regenerated while 16 SAE features are pushed toward their average values from correct past decisions. The features and steering strength are chosen offline by rollout-based search. On Gemma-2-9B-it and Llama-3.1-8B-Instruct, evaluated on ALFWorld and Telecom-Compose, TraceSAE outperforms a matched dense-steering baseline by 4.7 to 8.2 points in task success.
Critical strengths
- Using an SAE to identify concepts and steer model behavior toward correct actions is interesting.
- The method shows consistent performance gains.
Critical weaknesses
- The plan and evidence adapters determine whether an intervention can happen at all, yet the paper never says whether they are rule-based parsers, fine-tuned models or prompted LLM judges, nor how accurate they are. This contrasts with Appendix D, which gives SHA256 hashes of the SAE weight files. If the adapters are strong external LLMs, much of the "knowing when the plan is broken" comes from outside the method. If they are hand-built rules, the method may not transfer to new domains without substantial engineering.
- The risk model, the donor profiles and the calibration objective all depend on step-level labels of "warranted" versus "unwarranted persistence." Appendix A.2 states only that "independent blinded raters" produced them. The paper does not say whether these raters are humans or LLMs; how many steps were labelled across the 1,400 discovery tasks per model; how much the raters agreed; etc.
- Selectivity is central to the paper's motivation, yet there are no precision, recall or AUROC figures for the risk model or the evidence classifier, and no report of how often the gate opens at test time. Without these, it is impossible to tell how much of the improvement comes from detecting errors well versus from the edit itself.
- Most of the improvement over the guard-only baseline is shared by the dense controller, so it comes from the gate, the donor profiles and the search rather than from the SAE. The paper does not test the shadow-and-regenerate procedure without any edit (re-sampling alone may help); a random group of 16 SAE features. There is also no comparison with prompting, self-critique or reflection baselines (e.g., Reflexion [1]), which are the natural alternatives for this problem.
References
[1] Reflexion: Language Agents with Verbal Reinforcement Learning, Shinn et al., 2023
Questions for authors
- What exactly are the plan and evidence adapters (rules, a fine-tuned model, or a prompted LLM)? How accurate are they?
- Who produced the process labels? How many steps were labelled, with what inter-rater agreement, and what is the precise training target of the risk model?
- What are the gate's precision, recall and opening rate at test time?