acceptodds
Under review as a conference paper at ICLR 2027

Words, not Walks: An Algebraic Theory of Tool-use Trajectory Reduction for Agent Distillation

Abstract

A language agent that solves a task by calling tools produces a trajectory: a sequence of actions that transforms an environment state. We argue that this process is most naturally described not as a walk on a graph but as a word in a transformation monoid: tools are generators, dependencies are non-commutativity, and an efficient solution is a geodesic—a shortest generator product. This view exposes two failure modes that plague behavioral cloning of agents: state-relative identity sub-words (no-ops and backtracking loops that fix the current state) and non-commutative ordering errors. We introduce Algebraic Trajectory Reduction (ATR), a sound, model-free procedure that rewrites a teacher trajectory to its empirical geodesic before it is used for supervised fine-tuning. ATR unifies loop elimination and observed shared-state shortcuts as shortest-path extraction over the observed transition graph, with provable soundness under deterministic tools and exact state identifiers. We further give a sufficient boundary: on strictly monotone response integration, ATR coincides with deduplication by construction, whereas state revisits in sequential tool-use can make dedup unsound. Both regimes appear in real data. On 44,730 GPT-4/DFSDT TOOLBENCH trajectories, 27.9% of API calls fail (no-information updates) and ATR removes 39.4% of calls, exactly matching the strongest baseline as predicted. On 336 sequential AGENTINSTRUCT/ALFWorld trajectories, ATR removes twice as much as dedup (24.4% vs. 12.0%); under the observation proxy, offline replay fails on 1.5% of ATR paths versus 25.3% for dedup, and the former 1.5% is already present in unreduced data. A conservative certified variant removes only 1.3%. A 3B Qwen student trained on the proxy-reduced supervision achieves closed-loop success comparable to RAW/B2 within seed variation (9.5% vs. 9.2%/10.0%) using 24.4% fewer action targets. In a controlled transformation-monoid environment where teacher verbosity is a manipulated variable, cloning identical policies on ATR-reduced data raises closed-loop task success from 71.4% to 97.2% (matching an oracle-geodesic skyline of 97.8%), cuts non-commutativity errors from 40.9% to 0.7%, and uses 46% fewer supervision transitions; ATR is invariant to how verbose the teacher is, whereas naive cloning degrades monotonically. Our results give a precise algebraic account of what makes an agent trajectory worth imitating.

Primary area:
applications to robotics, autonomy, planning
Keywords:
LLM agents, tool use, trajectory distillation, trajectory reduction, transformation monoids

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_CAHnOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper models tool-use trajectories as words in a transformation monoid and proposes Algebraic Trajectory Reduction (ATR), which extracts shortest paths from observed state transitions before distillation. It provides guarantees under deterministic tools and exact state identification, analyzes ToolBench and ALFWorld data, and trains Qwen2.5 students on raw or reduced trajectories. Its largest success-rate gains come from a controlled study with a small MLP policy.

Critical strengths
  1. The formalization states conditions under which pruning preserves the endpoint of a demonstrated trajectory. This provides a useful basis for reasoning about the validity of trajectory reduction (Section 4).
  2. The controlled MLP study includes redundancy sweeps and an equal-epoch control, providing evidence for the proposed mechanism within that setting (Section 6; Appendix C).
Critical weaknesses
  1. The presentation is difficult to follow. The connection between the algebraic formulation, the observation-based reduction used in practice, and the conclusions about student training is insufficiently clear. Figure 1 has small labels and legends despite substantial unused space, making a central comparison unnecessarily hard to read (p. 8).
  2. The empirical support for LLM-agent distillation is unconvincing. In Table 2, ATR reports 0.0% success at 1.5B versus 1.7% for RAW; this near-floor result does not establish systematic degradation, but provides no positive task-success evidence. At 3B, ATR reaches 9.5% versus 9.2% for RAW and 10.0% for B2, within seed variation. The large 71.4% to 97.2% gain comes from an MLP in a controlled environment (Table 3), leaving the main LLM-agent motivation insufficiently validated despite shorter action supervision.
Questions for authors

1.given the 0 score at 1.5B and the absence of a clear 3B gain over RAW or B2, whether there are more evidence to support your claim? 2. The improvement the MLP does not align the LLM-agent motivation and claims in the introduction. so the analysis and experiments do not enough support the claim.

Reviewer_b6QuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper shortens teacher demonstrations before training a student agent. It removes loops by finding the shortest path through recorded states while preserving the final state under its assumptions.

Critical strengths
  1. The method is simple, with clear guarantees when the full environment state is known.
  2. Controlled experiments show that removing unnecessary actions can substantially improve learning.
Critical weaknesses
  1. Limited benefit in real LLM experiments (§5, Table 2). The 3B student achieves 9.5% success, compared with 9.2% on original demonstrations and 10.0% with deduplication. These results support shorter supervision, but not a clear improvement in task success.
  2. Shortest demonstrations are not necessarily the best teaching examples (§3–4). Reaching the same final environment state does not ensure that useful information is preserved. An agent may acquire knowledge through intermediate observations even when its actions return the environment to an earlier state. The shortest-path guarantee does not establish that removing those steps preserves the information needed by an LLM student.
Questions for authors
  1. Given that the 3B student shows no clear success improvement over RAW or deduplication, what evidence supports a learning benefit beyond reducing the number of training actions?
  2. How does ATR ensure that removing intermediate steps preserves information needed for later decisions, rather than only preserving the final environment state?
Reviewer_9JFkOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper models tool-use trajectories as words in a transformation monoid and proposes Algebraic Trajectory Reduction (ATR), which extracts a shortest path through observed state transitions before distillation. It provides guarantees under deterministic tools and exact state identifiers, with evaluations on ToolBench, ALFWorld, and a synthetic environment.

Critical strengths
  1. Under deterministic tools and exact state identifiers, the method is simple and its endpoint-preservation guarantee is clear.
  2. The paper explicitly reports important limitations, including equivalence to deduplication on ToolBench and the limited coverage of certified reduction on ALFWorld.
Critical weaknesses
  1. The contribution beyond standard trajectory simplification is unclear. ATR primarily runs BFS on the observed state graph; the algebraic formulation does not introduce an additional computational mechanism. The synthetic study lacks ordinary loop-removal baselines, so its large gains do not establish an advantage beyond removing the deliberately injected redundancy.

  2. The real-world validity evidence is insufficient. ALFWorld's 24.4% reduction relies on observation-based state merging, and cached-transition replay cannot rule out hidden-state inconsistencies. The certified variant removes only 1.3%, with no corresponding student results. The reported student experiments also show no clear task-success advantage over the baselines.

Questions for authors
  1. When reduced demonstrations are replayed from their original initial states in the actual ALFWorld simulator, how often do they still complete the task? Please compare observation-proxy ATR and certified ATR.
  2. How does ATR compare with ordinary loop removal and the local rewriting procedure in Appendix A under the same synthetic training setup?
  3. What are the closed-loop student results for the certified reduction variant?
  4. A framework diagram or teaser figure would help readers better understand the method.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.