acceptodds
Withdrawn from ICLR 2027

SALT: State-Augmented Learning from Trajectories for Language Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Language agents complete tasks by interacting with users and acting in an environment over multiple turns. At each turn, they choose an action based on the history of user instructions, tool responses, and earlier actions. Standard supervised fine-tuning trains an agent to predict actions from this history, but does not explicitly supervise the interpretation of what the history establishes about the task. We represent that interpretation as a task state that records the agent’s goal, constraints, facts, plan, and open issues. We introduce SALT, State-Augmented Learning from Trajectories, a method that adds task-state supervision to agent training. SALT pairs each action in a trajectory with a task state and trains an agent to generate this state before predicting an action. Across five agentic benchmarks and three Qwen3 model sizes, SALT outperforms standard supervised fine-tuning on the same trajectories in 12 of 15 settings, with an average relative gain of 7%. On -bench, its relative gain at pass^4 is more than double its gain at pass^1. These results suggest that task-state supervision helps agents choose actions that reflect what their interactions have established.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Tool Use, Language Agents, Data Synthesis

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_P5cKOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes SALT, which augments agent training trajectories with structured task states consisting of goals, constraints, facts, plans, and unresolved issues. The states are automatically generated through a writer–reviser–probe pipeline and used as additional supervision before action prediction. The method improves over standard SFT in 12 out of 15 settings across five benchmarks and three Qwen3 model sizes.

Critical strengths
  • The paper is clearly written and addresses a relevant problem in multi-turn tool-using agents.
  • The empirical evaluation covers several benchmarks, and the improvements over SFT are reasonably consistent, especially for the 4B and 8B models. I also appreciate the intervention experiment showing that the generated states can directly influence the agent's subsequent actions.
Critical weaknesses
  1. Limited novelty relative to existing work.

    • The main idea of explicitly supervising intermediate states before action generation is already well explored. In particular, PARL-MT constructs progress-awareness supervision from trajectories, including history summaries and future plans, and uses action reconstruction to validate the generated information. IACM-RL similarly trains agents to generate structured belief states before tool invocation. Earlier work such as UBAR also jointly models belief states and actions.
    • While SALT uses a different five-field representation and annotation pipeline, I find the conceptual difference relatively incremental. The paper does not sufficiently establish why this particular decomposition offers a meaningful advance over existing state- or progress-aware training methods.
  2. Insufficient controls to isolate the actual source of improvement.

    • SALT introduces substantially more supervised tokens than standard SFT. For example, the supervised token count increases from 0.20M to 1.17M on WebShop and from 1.61M to 11.08M on OfficeBench. The rationale and flattened-state controls are useful, but they are only evaluated on WebShop with the 4B model. It remains unclear how much of the improvement comes from the proposed state structure versus additional intermediate supervision.
    • I am also concerned about the construction of the state targets. Appendix A indicates that the demonstrated tool call can be appended to the plan field, including its arguments and provenance. The example in Appendix A.3 appears to contain an explicit specification of the subsequent gold action. This potentially introduces additional action-level supervision within the state targets. An ablation removing this information would be important for understanding the claimed benefit of task-state learning.
  3. Limited evidence of generality.

    • There is no independent evaluation of generated state correctness. The state intervention experiment demonstrates that actions depend on the states, but does not establish that SALT systematically learns more accurate state representations.
Questions for authors
  1. How does SALT compare with token-matched structured reasoning or progress-awareness SFT baselines across multiple benchmarks?
  2. What happens if the demonstrated tool-call specification is removed from the state targets? Does the method generalize when trained on one set of environments and evaluated on unseen domains?
Reviewer_6E23OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SALT, a method that explicitly supervises task-state inference by training language agents to represent goals, constraints, facts, plans, and unresolved issues before acting. Using automatically annotated trajectories, the authors fine-tune Qwen3 models at three scales and evaluate them on five agent benchmarks. The results suggest that structured state supervision can improve agent performance and reliability.

Critical strengths

1、A simple and well-controlled training intervention. SALT adds explicit task-state supervision to existing trajectories without extra environment interaction or human annotations. Comparing it with SFT on the same trajectories provides a useful test of whether state supervision improves agent performance.

2、Broad evaluation with behavioral analysis. Experiments cover five benchmarks and three model sizes, with improvements in 12 of 15 settings. Analyses of pass^k, state mismatches, and state perturbations further support the role of task states in improving action selection and reliability.

Critical weaknesses

1、Limited novelty. Explicit task-state modeling and progress-aware agent training have been explored in prior work, including Progra/PARL-MT [1], IACM-RL [2], PABU [3], and ABBEL [4]. While SALT combines these ideas through a five-field state representation and an automatic annotation pipeline, the paper does not sufficiently demonstrate the distinct benefits of these design choices over existing approaches. The methodological contribution therefore appears somewhat incremental.

2、Insufficient controlled comparisons and efficiency analysis. The comparison with standard SFT demonstrates the potential benefit of additional state supervision, but does not fully isolate the contribution of SALT's specific representation. The Rationale-SFT and SALT-flat ablations are limited to WebShop with the 4B model, leaving their generalizability unclear. Moreover, although the paper reports token overhead and analyzes total token consumption on τ²-bench, the end-to-end cost-benefit trade-off across benchmarks remains insufficiently characterized. References [1] Chai, H., et al. (2026). Progra: Progress-Aware Reinforcement Learning for Multi-Turn Function Calling. Findings of ACL 2026. [2] Zhu, D., et al. (2026). IACM-RL: Intent-Aware Context Management and Reinforcement Learning for Complex Tool Invocation under Dynamic Intent Fluctuations. arXiv preprint, arXiv:2608.02110. [3] Jiang, H., Ge, L., Cai, H., and Song, R. (2026). PABU: Progress-Aware Belief Update for Efficient LLM Agents. arXiv preprint, arXiv:2602.09138. [4] Lidayan, A., Bjorner, J., Golechha, S., Goyal, K., and Suhr, A. (2026). ABBEL: Learning Natural-Language Belief States for Memory-Efficient Interaction. arXiv preprint, arXiv:2512.20111.

Questions for authors

See the Critical Weaknesses.

Reviewer_QipTOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

SALT adds a task state before each agent action, covering goals, constraints, facts, plans and open issues. A writer, reviser and probe annotate existing trajectories. A single model then learns to generate the state followed by the action, while retaining the full interaction history. Across five benchmarks and three Qwen3 model sizes, SALT outperforms standard SFT in 12 of 15 settings, with an average relative improvement of approximately 7%. The paper also reports repeated success metrics on τ²-bench, state interventions and failure analyses.

Critical strengths
  1. The method is simple and clearly motivated. Comparisons use the same source demonstrations across five benchmarks and three model sizes. Improvements in 12 of 15 settings, with more consistent gains at the larger sizes, provide evidence of practical value.
  2. The paper goes beyond aggregate scores to examine environment state errors, state interventions and concrete failures. The structural ablation and the analysis of errors that first appear in the generated state provide useful evidence about how the method behaves.
Critical weaknesses
  1. PABU already supervises progress and information retention alongside action prediction, while PARL-MT generates intermediate representations and verifies whether they support action recovery. SALT’s additions mainly concern its five-field state and annotation pipeline, making it closer to an integration and refinement of existing ideas. The structural ablation and failure analysis provide useful evidence, but I do not find the demonstrated degree of novelty sufficient for acceptance at ICLR.
  2. Demonstrated tool calls that modify the environment are appended to P in the training states, and the model subsequently predicts the same calls again as actions. Some gains may therefore come from repeated action supervision rather than more accurate state inference. Without a control that removes the appended calls, this explanation remains unresolved.
  3. Appendix C omits a control using standard SFT with the same state prompt. Limited gains from prompting the base model do not establish that a model already trained on the task would also fail to benefit. The contribution of state supervision itself is therefore not sufficiently isolated.
  4. The reproducibility statement says that Appendix B and Table 7 include decoding settings, but the evaluation temperature is not reported. The stated temperature = 0 applies only to the annotation pipeline. This omission makes reproduction more difficult and limits assessment of the reliability claims.
Questions for authors
  1. Which design differences discussed in Appendix D contribute to the gains? Beyond the existing structural ablation, could you compare the five-field state with summary, plan or progress supervision using the same data and training budget to establish the added value of SALT’s specific design?
  2. How much of SALT’s gain remains after removing the demonstrated write calls appended to P? Could you include a control that only repeats supervision of these actions to distinguish its contribution from that of state inference?
  3. How does standard SFT perform with the same five-field state prompt? Please add this control under identical evaluation settings to assess the contribution of state supervision beyond task training and inference-time prompting.
  4. What temperatures and other decoding settings were used for the agent and, where applicable, the user simulator on each benchmark? Were these settings consistent across the compared methods? Please provide the values and correct the corresponding reproducibility statement.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.