acceptodds
Under review as a conference paper at ICLR 2027

Typed Retry Trajectories: Structuring Agent Context for Tool-Use Recovery

Abstract

Agents often need to recover from failed or uncertain actions, using information from earlier steps to decide their next action. However, these histories often combine task instructions, external results, errors, constraints, feedback, and earlier actions in a single text representation, leaving the model to infer the role and reliability of each part. We introduce typed retry trajectories, a structured representation that separates an agent’s decision context into four parts: instruction, data, requirement, and execution. The instruction part states the task goal. The data part contains observations, tool results, earlier actions, and other external information. The requirement part contains rules, limits, and verifier feedback. The execution part describes the next proposed action. We implement this schema in LLMON (Hind et al., 2026), a general-purpose markup language that allows the definition of arbitrary tags or concepts. We leverage a fine-tuned LLMON model pretrained on tags such as instr, data, and exec, and extend it by defining retry-specific tags (e.g., goal, stopping_criteria) and mapping them to LLMON’s existing tags. We evaluate typed action trajectories on the repair of failed tool-use cases from τ-bench (Yao et al., 2024). We compare the same model and the same trajectory information in three formats: typed, plain JSON, and prose logs. Typed formats improve F1 for predicting the next tool-call name by 0.088 (JSON) and 0.060 (prose logs). The structured format also improves tool selection and tool-argument accuracy. These results show that separating instructions, observations, rules, and actions can help agents recover from failed tool calls. More broadly, typed retry trajectories provide a practical way to distinguish trusted task instructions and constraints from external information that may be incomplete, incorrect, or unsafe. This distinction is not limited to tool-use recovery; it can also support other agent actions, including memory writes, record updates, message sending, and delegation to other agents.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agentic AI, Tool-Use Recovery, LLM Agents, Structured Representations

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_yEC3OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces typed retry trajectories, a structured representation of the failed trajectory logs, and uses these typed retry trajectories to fine-tune a model to predict the whole list of correct tool-calls. They divide the failed trajectory into four parts (instruction, data, requirement and execution), each holding one or more retry-specific tags. All arms are fine-tuned on the same base model, an LLMON-pretrained model, which takes the failed trajectories as input and outputs the correct tool-call chain; the typed versions are written in LLMON markup.

The motivation for this work comes from the way failed trajectories are fed to the agents currently. The entire failed trajectory history is fed into the agent as a single unstructured text: the original task, results, errors, feedback and earlier actions all together. This means that the model (driving the agent) needs to "infer the role and reliability of each part" of the trajectory and decide how to proceed.

The trajectories they experiment on come from 460 recorded runs over 115 tasks (with 182 failed trajectories), from the retail domain of τ-bench. The repair task is to take the failed run and output a correct list of tool calls that would have accomplished the task. The paper trains the model on different arms: (1) plain JSON, (2) the same content as prose logs, (3) channels only, with no new syntax, (4) tags only, without the channels, and (5) tagged channels. The full version with tagged channels gives +0.043 and the channel assignment arm gives +0.088 improvement in tool-name F1 score when compared to the plain JSON. The paper also tests markup pretraining, prompt injection and a retry loop against a replay environment.

Critical strengths
  1. Typed retry trajectories for agents are an interesting idea and they may help in certain scenarios (e.g., choosing the right tool or resisting prompt injection).

  2. Cross-validation (avoids relying on one small test split) and matched pairs (compares otherwise identical models) are careful choices in the experimental design.

  3. The authors are honest about the results. The paper says that there was no improvement from the tagged-channels arm over the channel-assignment arm. They also report in detail the pipeline artifacts that led to their earlier inaccurate results (Section 6). These lessons are very valuable and can be helpful to others using similar pipelines to keep in mind.

  4. The idea is worth pursuing with a stronger evaluation or a different experimental design.

Critical weaknesses
  1. The model may be ignoring the failed history and just associating the original task (goal) with the tool list. An ablation that trains with only the goal as input, and the tool list as the label, without the trajectory history, would show how much the failed trajectory contributes.

  2. The gain is modest (+0.088 tool-name F1 over a plain-JSON baseline of 0.497, Table 2) and only in tool names, not in argument accuracy. This suggests the typed representation has limited benefit in practical terms.

  3. There are inconsistencies in the paper: the abstract talks about next tool-call, but the results only deal with the tool-call chain. There are also inconsistencies in how the arms are divided. The paper refers to four arms (line 199), but there appear to be 5 arms. There are references to sub-sections that don't exist (e.g., 4.1, 5.1 and so on). The abstract also says argument accuracy improves, while Table 2 and line 307 say it doesn't.

  4. Lack of details:

    a. The paper does not describe how the original failed runs were generated - which model was used?

    b. verifier_feedback is mentioned but no details of what it entails (or references) are provided.

    c. The paper does not name the base model that is fine-tuned to output the tool-call chain, or the models in the matched pairs ("micro (MoE-hybrid)" and "3B (dense)", Table 6). The LLMON work covers several models of different sizes, so this is not obvious from the citation.

    d. It would be helpful to have an example prompt per arm.

  5. The paper does not take into account related prior work on input reformulation and channel separation defenses. A comparison would clarify what the retry-specific schema adds.

Redesigning the experiments and revising the presentation of the paper to address these points would strengthen the contributions of the method.

Questions for authors
  1. Please share any experiments you have already run that show the models trained on the goal alone, without the failed trajectories, don't do as well as the models trained on the failed trajectories.
  2. Beyond the replay-environment loop mentioned in the paper (near Table 6), have you run any experiments on live τ-bench?
  3. Did you retrain any arm with different random seeds? If so, how much did F1 vary?
  4. Could you correct the inconsistencies in the paper (as discussed in the Weaknesses section)?
  5. Could you provide details on which model generated the failed trajectories and what produces the verifier_feedback? Could you also name the models used for fine-tuning and, if possible, include an example prompt for each arm?
Reviewer_6Ln1OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes "typed retry trajectories," which categorize the decision-making context following a failure into four distinct channels—instructions, data, requirements, and execution—implemented via an existing LLMON tagging schema. Using failure trajectories from the retail domain of the τ-bench benchmark, the authors perform LoRA fine-tuning on various input formats and evaluate the prediction quality of repair-call sequences using a five-fold cross-validation approach with mutually exclusive tasks. While this channel-based partitioning improves the F1 score for tool-name identification compared to raw JSON or log-based formats, the gains in parameter matching remain inconsistent. The study also includes an ablation analysis of input formats, comparisons with pre-trained baselines, an investigation into prompt injection, and an assessment of the evaluation pipeline.

Critical strengths
  1. The design is concise and driven by practical objectives. It distinguishes between goals, external information, constraints, and actions to be executed, providing a clear context for failure recovery.
  2. The experimental control is rigorous. The authors keep trajectory content and fine-tuning configurations constant while analyzing channels, labels, and their combinations; they also employ task-mutually-exclusive cross-validation and paired-interval analysis.
  3. The reporting is notably candid. The authors explicitly disclose performance degradation in certain metrics, the uncertainty surrounding pre-training gains, and failures in closed-loop retries; they further analyze various pipeline issues that could otherwise lead to the appearance of spurious gains.
Critical weaknesses
  1. Generalization and transferability have not yet been fully verified. The data is derived solely from 69 failed tasks within the retail domain of τ-bench, and the use of native role labels failed to replicate the primary performance gains.
  2. Information regarding data processing and reproducibility is incomplete. Please clarify why the 182 failed trajectories were ultimately reduced to 180; explain the relationship between the 24 failed conversions (out of 460) and the respective folds; and describe how inconsistencies in read operations from the reference list were handled.
  3. The current experimental scope—in terms of the models covered and the extent of repeated validation—remains limited; consequently, the observed improvements or declines lack sufficient persuasive weight.
Questions for authors

ref weakness

Reviewer_mQtbOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes a schema for the history an LLM agent reads when it retries a failed tool call. The goal, stopping criteria, attempt count, numbered past calls/results/failures, verifier feedback and the proposed call are each tagged and assigned to one of four channels (instruction, data, requirement, execution), with tool output treated as untrusted data. The schema is realised in the LLMON markup [1]. Validation is an offline repair task built from failed τ-bench retail runs. A LoRA-tuned model maps each failed run to the corrected tool-call list, scored by multiset F1 on tool names or names plus arguments over all or write-only calls, with task-disjoint 5-fold cross-validation and paired bootstrap intervals (n = 180). The channel-assigned rendering gains 0.088 and 0.060 all-action tool-name F1 over plain-JSON and prose-log renderings of the same content. Follow-up experiments separate tag syntax from channel assignment, re-express the assignment as role labels, test markup pretraining in two architecture-matched checkpoint pairs, measure prompt-injection success, and run a closed retry loop. A final section documents the pipeline artifacts behind the authors' earlier, partly false result.

Critical strengths

1. A paired, drift-controlled design. One renderer produces all arms, "so truncation, deduplication, and parse tolerance cannot drift between arms" (p4, L199–200). The LoRA recipe is identical across arms (p5, L216–221). Folds are task-disjoint and intervals are paired BCa (p5, L251–258). Pretraining is compared within architecture-matched pairs (p5, L222–232). Format effects of a few F1 points are easy to confound, so this design is what makes the contrasts interpretable, including the nulls in Tables 4 and 6.

2. Candid reporting of negative results and pipeline artifacts. The paper says where the effect fails ("Where it does not help", p7, L369–376). It reports that its closed loop is "worse than submitting the failed run unchanged on three of four measures" (p8, L399–400). §6 documents artifacts as large as the effects studied:

  • a repetition penalty cut write-action F1 by 0.39 (p9, L435–437);
  • a significant 46-item effect reversed at n = 180 (p8, L423–425);
  • checkpoints were trained on data that was later overwritten (p9, L442–447).

These are transferable lessons for agent evaluation.

Critical weaknesses

1. Against the main baseline, the headline gain is confined to the measure the authors say partly reflects label composition, and the statistics overstate its support.

Plain JSON is "our main baseline" (p4, L202). Against it, Table 2 shows:

  • all-action naming: +0.088*;
  • write-action naming: +0.007 [−0.03,+0.05];
  • write-action arguments: −0.037*.

Write-action naming is flat, so the all-action gain points to read-only calls. Yet the paper says "the references omit read-only calls on some cases and include them on others, so all-action F1 partly measures label composition rather than model behaviour" (p5, L245–247). Pairing on identical references does not fix this. A rendering that changes how many lookups the model emits is rewarded or penalised according to how each reference was written. τ-bench's own check compares the final database state with the goal state [4], which lookups cannot affect. The paper does not use it (p7, L374–376).

The statistics have further gaps:

  • the bootstrap resamples cases (p5, L256–258), but the 180 cases come from 69 tasks (p5, L251–252);
  • no seed replication is reported, and the base model is never named (p6, L303);
  • the promised win–loss–tie counts, Wilcoxon tests and equivalence tests (p5, L258–261) are not reported.

Suggestion: Stratify Table 2 by reference type (write-only vs read-including) and report read-only calls emitted per arm. Score predictions by executing them in τ-bench [4]. Use task-clustered resampling with ≥3 seeds.

2. The mechanism, the recommendations and the abstract contradict the paper's own tables.

The +0.088 arm is "channel assignment with no new syntax" (p6, L309–310); its Table 3 row matches Table 2. The retry-tag realisation the abstract describes is "both" ("Tagged channels", p4, L206). That arm scores +0.043 [−0.00,+0.09] on all-action naming and −0.029* on write-action arguments (Table 3).

Yet practitioners are told to "port the assignment even if they keep their own syntax" (p3, L136–137; also p9, L468–470). The paper's own evidence says otherwise:

  • expressing the assignment as role labels "does not reproduce the gain" (p7, L348–349; Table 4);
  • "transfer to other syntaxes remains unverified" (p3, L160–161);
  • without single-token delimiters, the envelope ends up "parsing only 127 of 180 outputs" (p6, L322);
  • single-token tags lose to undefined "punctuation tags" by 0.077 at n = 180 (p8, L423–425).

This fits the finding that format effects are large and model-specific [5,6].

The abstract's "also improves tool selection and tool-argument accuracy" (p1, L029–030) contradicts Table 2 and the paper's own "Neither contrast improves argument-exact matching" (p6, L307). p5, L247–248 also misstates Table 2.

On injection, prose logs (no channels) have lower attack success than the channel envelope at two of three strengths (19% vs 22%, 25% vs 36%; Table 5), with no intervals or goal-retention numbers (p7, L360). So "The data channel provides a partial defense" (p7, L355) is not isolated. Instruction/data separation is already established [2,3,7] and motivates LLMON itself [1]; the increment is the retry-tag mapping (Table 1).

Suggestion: Restate the claims to match the evidence: an effect of the envelope on one markup-pretrained base, measured on all-action naming. Correct the abstract and p5. Either test syntax transfer (XML or native tool-role messages vs the same JSON baseline) or drop the advice. Add intervals and goal-retention to Table 5, on a standard agent-injection suite [8].

Minor issues

  • There are no numbered subsections, yet the paper cites §3.1 (p7, L356), §4.1 (p4), §4.3 (p6, L306), §5.1–§5.6 and §6.1–§6.6. Table 6 and Fig. 2 are never referenced.
  • Several terms and inputs are undefined: "stems" (p5, L252), "punctuation tags" (p8, L423–424), and how goal, stopping criteria and verifier feedback are derived for recorded τ-bench runs.
  • Table 1 (p4) maps tool_call → exec, while the §3 table puts history calls in data, and attempt_count is missing. Table 1 also lists "requirement" among "LLMON's pretrained tags", but I could not find it as a tag in the cited LLMON paper [1].
  • Counts: 127 + 29 + 22 = 178, not 180, and "fails on most cases" conflicts with 127/180 parsed (p6, L322–323); 182 failed runs (p3, L146) vs 180 stems (p5, L251–252).
  • Mizrahi et al. and Sclar et al. are cited for output-format effects (p2, L098–100), but both study input prompts [9,6]; Tam et al. [10] fits. The two Beurer-Kellner et al. 2023 entries (p10) are the same paper. Typos: "contol" (p2, L099), "This avoid" (p5, L253), and a duplicated sentence (p8, L418–419).
Questions for authors
  1. On cases whose references contain only write actions, and with task-clustered intervals, does the +0.088 over plain JSON survive? How many read-only calls does each arm emit? If the gain holds, Weakness 1 is largely resolved and I would raise Soundness and my rating. If it comes from read-only calls on read-including references, I will read the headline as a labelling effect.
  2. Which base models were used for Tables 2–5, and which agent generated the 460 τ-bench runs? What are the "punctuation tags" (p8, L423–425), and why is that arm absent from Table 3? Do the defects listed in App. A (a delimiter "spelled three inconsistent ways" and 24 conversion failures; p11, L576–578) affect arms differently? A competitive punctuation syntax or arm-specific defects would further weaken the LLMON-specific contribution.
  3. Execute the predicted call lists in τ-bench and score the final database state against the goal state [4], with ≥3 seeds and ideally a second base model. A positive result of that kind is what would move me toward acceptance.
  4. When will the artifacts documented in the Reproducibility statemnt be released?

References

[1] Michael Hind, Basel Shbita, Bo Wu, Farhan Ahmed, Chad DeLuca, Nathan Fulton, David Cox, Dan Gutfreund. "LLMON: An LLM-native Markup Language to Leverage Structure and Semantics at the LLM Interface." arXiv:2603.22519, 2026.

[2] Sizhe Chen, Julien Piet, Chawin Sitawarin, David Wagner. "StruQ: Defending Against Prompt Injection with Structured Queries." USENIX Security Symposium, 2025.

[3] Eric Wallace, Kai Xiao, Reimar Leike, Lilian Weng, Johannes Heidecke, Alex Beutel. "The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions." arXiv:2404.13208, 2024.

[4] Shunyu Yao, Noah Shinn, Pedram Razavi, Karthik Narasimhan. "τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains." arXiv:2406.12045, 2024.

[5] Jia He, Mukund Rungta, David Koleczek, Arshdeep Sekhon, Franklin X Wang, Sadid Hasan. "Does Prompt Formatting Have Any Impact on LLM Performance?" arXiv:2411.10541, 2024.

[6] Melanie Sclar, Yejin Choi, Yulia Tsvetkov, Alane Suhr. "Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting." ICLR, 2024.

[7] Egor Zverev, Sahar Abdelnabi, Soroush Tabesh, Mario Fritz, Christoph H. Lampert. "Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?" ICLR, 2025.

[8] Edoardo Debenedetti et al. "AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents." NeurIPS Datasets and Benchmarks Track, 2024.

[9] Moran Mizrahi, Guy Kaplan, Dan Malkin, Rotem Dror, Dafna Shahaf, Gabriel Stanovsky. "State of What Art? A Call for Multi-Prompt LLM Evaluation." Transactions of the ACL 12:933–949, 2024.

[10] Zhi Rui Tam et al. "Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models." EMNLP 2024 Industry Track.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.