acceptodds
Under review as a conference paper at ICLR 2027

Compiling Mixed Agent Traces into Guarded Executable Workflows

Abstract

LLM agents often solve repeated tasks by reconstructing similar tool-use proce- dures with different inputs. Although storing reusable local functions can reduce low-level execution effort, the model still needs to reason about how to compose these functions for every new request. We propose Trace2Workflow, a framework that converts successful interaction traces into reusable executable workflows. In- stead of repeatedly planning from scratch, the agent can directly reuse a learned workflow when its applicability conditions are satisfied. This reduces the reason- ing burden of the model and avoids unnecessary LLM calls. When no workflow is applicable, the agent falls back to normal model-based reasoning, which pre- serves robustness on unseen tasks. We evaluate Trace2Workflow on ALFWorld and AppWorld under a continual setting. We compile previously successful task- level compositions, rather than only local skills, into guarded executable work- flows whose bindings and observable effects are checked at runtime. These re- sults show that reusable workflows can amortize repeated composition reasoning across tasks, reducing both model inference burden and execution cost without sacrificing task performance.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Agents, Agent Workflows, Tool Use, Trace Compilation, Executable Workflows

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_dEdKOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Trace2Workflow, which converts agent interaction traces into reusable executable workflows in three stages: (1) extract parameterized local programs from successful and failed traces and keep only candidates with no verified failure and sufficient positive evidence under each local goal's completion check; (2) compile the task-level composition of a successful trace into a workflow W = (x, V, β, g, Φ) with argument bindings, an applicability guard, and observable completion checks; (3) on a new task, execute a matching workflow without model calls and DEFER to the LLM from the current state if any check fails. With Qwen3.5-27B as the backbone and ReAct, CodeAct, and Skill-DisCo as baselines, it reports success rates of 97.86/98.51 on ALFWorld Seen/Unseen with 0.879/0.858 LLM turns per task, and 83.33/70.00 on AppWorld test-normal/test-challenge.

Critical strengths
  1. Clear idea with a sensible safety mechanism. Compiling the composition itself, rather than only local skills, directly targets repeated task-level planning, and the guard + effect check + DEFER design returns control to the LLM from the current state without resetting the environment or budget, which is more practical than all-or-nothing replay. The ablations (Table 3) and the reuse-granularity analysis make each component's role visible.
Critical weaknesses
  1. The strongest evidence comes from a setting where reuse is nearly trivial, while the evidence on the harder benchmark is thin. ALFWorld has six fixed task templates whose action sequences differ mainly in objects and receptacles, so a parameterized workflow from one successful trace is expected to cover most test tasks; achieving zero model calls on 96 (Seen) and 99 (Unseen) tasks therefore says more about the benchmark than about generalization. The Unseen split changes room layouts, not task types. On AppWorld, where API interactions are diverse, only 4/60 and 2/60 tasks are completed without model calls, and the gains over CodeAct are 50 vs. 47 and 42 vs. 37 tasks, i.e., 3–5 tasks per split, on 60-task subsets whose sampling procedure is not described.
  2. The baseline comparison on AppWorld is not interpretable. With the same Qwen3.5-27B backbone, ReAct solves 7/60 and 0/60 AppWorld tasks while CodeAct solves 47/60 and 37/60. ReAct also has the highest average turns (32.2 and 34.4), suggesting most episodes exhaust the action budget, but the budget value is not given. Appendix B.1 provides only a combined "Agent and CodeAct Prompt" describing an interactive Python environment, and no ReAct-specific prompt or action format, so it is unclear how ReAct and CodeAct actually differ in this implementation. Without this, the claim of outperforming ReAct on AppWorld cannot be assessed.
Questions for authors
  1. (Clarification + additional experiment) How were the 60 AppWorld tasks per split sampled? Can you report results on the full test-normal / test-challenge splits?
  2. (Clarification) What prompt, action format, and step budget does ReAct use, and why does it solve 0/60 on test-challenge?
  3. (Clarification) On ALFWorld, what are coverage and zero-LM rates per task type? Are some types never covered by a workflow?
  4. (Clarification) The guard and completion checks are written by the LLM at compile time from a single trace (Appendix B.4). How often does a guard match a task to which the workflow does not actually apply?
3 comments
AuthorsOpenReview ↗

Response to W1 and Q3: Motivation and Workf

We thank the reviewer for this comment. Our main goal is to avoid repeated planning for similar tasks. ALFWorld fits this goal well. Its tasks share similar procedures but differ in objects, locations, and environment states. Compared with Local Programs Only, our method reduces average LLM turns from 2.806 to 0.858 on ALFWorld Unseen. This shows the benefit of reusing task-level workflows.

1. Per-Task-Type Analysis on ALFWorld

We further analyze workflow coverage and Zero-LM across the six ALFWorld task types. Coverage measures the percentage of tasks that execute a workflow. Zero-LM measures the percentage of tasks completed without LLM calls.

Task TypeUnseen Zero-LMSeen CoverageSeen Zero-LM
Pick & Place24/2434/3533/35
Clean & Place31/3126/2726/27
Cool & Place21/2124/2524/25
Heat & Place23/2313/1613/16
Examine in Light0/180/130/13
Pick Two & Place0/170/240/24
Total99/13497/14096/140

On Unseen, workflows directly complete all 99 tasks from the first four types. This shows that workflow reuse is effective for repeated task structures, but its coverage depends on the learned workflow library.

2. Generalization on AppWorld

AppWorld is more challenging because its tasks are more diverse and share fewer common workflows. As a result, fewer tasks can be completed without LLM calls. However, local programs and partial workflow reuse can still reduce repeated planning. Compared with Local Programs Only, our method reduces average LLM turns from 18.350 to 11.017 (40.0%) on test-normal and from 23.217 to 14.950 (35.6%) on test-challenge.

Our goal is not to achieve Zero-LM on every task, but to reduce repeated planning whenever possible. We will clarify that our method focuses on reusing experience across similar tasks, rather than solving arbitrary unseen task types.

AuthorsOpenReview ↗

Response to W2 and Q2: ReAct Baseline

We thank the reviewer for the comments on the ReAct baseline. We provide more details about its implementation and the performance differences.

1. ReAct and CodeAct Implementation

Both ReAct and CodeAct use the same Qwen3.5-27B backbone and task environment. Their main difference is the action format.

In our implementation, ReAct generates one JSON action per turn, including "thought", "app", "api", and "arguments". Each action calls only one public API. In contrast, CodeAct generates executable Python code. It can use variables, loops, and multiple API calls within a single model turn.

We will provide the separate prompts, action formats, step budgets, and stopping rules in the revised appendix to make the comparison clearer.

2. ReAct Performance on AppWorld

AppWorld contains complex tasks that require multiple API calls, data processing, and repeated operations. For ReAct, each API call requires a separate model decision. In contrast, CodeAct can combine several operations into one Python program.

This difference has a large impact on complex tasks. ReAct spends more model turns on low-level API operations, while CodeAct can complete the same operations with fewer model interactions. In addition, invalid action outputs also consume model turns in our ReAct implementation.

These differences lead to ReAct's lower performance, especially on test-challenge. The reported 0/60 result reflects the difficulty of completing all task requirements under this execution setting.

3. Baseline Comparison and Fairness

Our main conclusion does not depend on ReAct's performance. Trace2Workflow also outperforms the stronger CodeAct baseline. On AppWorld, success improves from 47/60 to 50/60 on test-normal and from 37/60 to 42/60 on test-challenge. Average LLM turns decrease from 20.517 to 11.017 and from 26.500 to 14.950, respectively.

Furthermore, Local Programs Only and Trace2Workflow use the same local-function library and training evidence. Their main difference is whether task-level workflows are reused. This comparison directly supports the benefit of workflow-level reuse.

We will clarify the baseline implementation details and emphasize these comparisons in the revised paper.

AuthorsOpenReview ↗

Response to Q1 and Q4: AppWorld Evaluation and Guard Reliability

We thank the reviewer for these questions. We clarify the AppWorld sampling method and provide further analysis of workflow applicability.

1. AppWorld Task Sampling and Full Evaluation (Q1)

Our original evaluation uses template-level random sampling. For each split, we randomly select 20 task templates without replacement and include all three instances of each template. This gives 60 tasks per split. We use a fixed random seed of 42. The sampling process uses only task IDs and does not consider model performance or workflow coverage.

The full test-normal and test-challenge splits contain 168 and 417 tasks, respectively. To further evaluate generalization, we conduct a full-split comparison between CodeAct and Trace2Workflow using the same Qwen3.5-27B backbone and evaluation settings.

DatasetMethodSuccessSR (%) ↑Turns ↓Zero-LM ↑
test-normal (168)CodeAct103/16861.3121.3000
Trace2Workflow118/16870.2417.8005
test-challenge (417)CodeAct229/41754.9227.2000
Trace2Workflow283/41767.8722.1004

Trace2Workflow improves the success rate by 8.93 and 12.95 percentage points on test-normal and test-challenge, respectively. It also reduces average LLM turns by 16.43% and 18.75%. These results show that workflow reuse remains effective on the full AppWorld splits.

We will provide the sampling script and task IDs for reproducibility.

2. Guard Applicability and Reliability (Q4)

During offline compilation, the LLM generates request patterns and applicability guards from successful traces. At runtime, the matcher checks input bindings and guards before execution. Only a unique valid match is accepted.

We also verify API responses and observable effects during execution. If a check fails, the workflow returns DEFER, and the LLM continues from the current state.

We evaluate guard reliability by checking whether each accepted workflow matches the task requirements. We distinguish incorrect matches from execution failures. The results are shown below.

Metrictest-normal (168)test-challenge (417)
Accepted Matches2230
False Matches511
False-Match Rate (%)22.7336.67

The false-match rate is higher on test-challenge, showing the difficulty of matching workflows to diverse tasks. These results also show the limitations of generated guards. Runtime checks provide an additional way to stop execution when observable conditions fail. The LLM can then continue from the current state.

We will include this analysis and clarify the role of guards, runtime checks, and fallback in the revised paper.

Reviewer_emrPOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

Trace2Workflow targets the repeated task-level composition bottleneck in tool-using agents. Rather than only storing local skills or replaying successful trajectories, it extracts parameterized local programs from mixed successful and failed traces, validates them against local evidence, and compiles validated programs and primitive tools into workflows with data dependencies, parameter provenance, applicability guards, and observable completion checks. A compatible workflow can execute without LLM calls; if a guard or expected effect fails, control returns to the LLM from the current state rather than restarting the task. In continual ALFWorld and AppWorld experiments, the authors report improved task performance with substantially fewer LLM turns, together with component ablations and executor-transfer results. The central claim is that previously solved task-level compositions can become reusable executable objects, but the generality of this abstraction beyond structured repetitive environments, as well as its amortized lifecycle benefit, remains to be established.

Critical strengths
  1. The paper targets a concrete bottleneck—reconstructing repeated task-level compositions—and proposes a reuse abstraction beyond local skill or macro replay.
  2. The guarded workflow representation preserves bindings, dependencies, and observable effects, while current-state fallback avoids treating a learned workflow as universally valid.
  3. Results on two interactive benchmarks, together with component and executor-transfer analyses, provide initial evidence for both utility and implementation behavior.
Critical weaknesses
  1. The generality of the proposed workflow abstraction remains unestablished. Trace2Workflow assumes repeated compositions, parameterizable bindings, and sufficiently reliable local observations and effect checks. ALFWorld and AppWorld provide structured environments with recurring procedures, but the paper does not show whether the same abstraction transfers to novel composition structures, state distributions, partially observable or irreversible tasks, or domains without clean completion checks. The contribution may therefore be a useful reuse mechanism for a particular class of structured repetitive tasks, rather than yet a broadly general trace-to-workflow method.
  2. More importantly, it is difficult to tell how much of the reported efficiency gain comes from the workflow itself. The Zero-LM comparison counts only runtime calls and omits the work needed to mine traces, extract and validate programs, compile workflows, and maintain the library. Without a baseline that receives the same accumulated experience but uses a simpler replay or macro-reuse strategy, the current results do not cleanly separate the benefit of guarded workflow compilation from the benefit of additional offline processing and task-specific experience.
Questions for authors

Please clarify the intended scope of generalization: what assumptions about repeated composition, parameter binding, local observability, and completion checks are required for a workflow to remain valid, and can the authors evaluate the method on novel composition patterns or state distributions outside those used for workflow induction?

3 comments
AuthorsOpenReview ↗

Response to Weakness 1 and Question: Generalization and Applicability

We thank the reviewer for the helpful comments. We clarify the generalization and limitations of our method below.

1. Workflow Applicability

Our main goal is to avoid repeated planning for similar tasks. A workflow can be reused when three conditions are met: (1) the new task has a similar execution structure, (2) task parameters can be correctly bound, and (3) execution results can be checked through observations.

Our method does not require full observability. It only uses available information. If parameter binding fails, guards are not satisfied, or execution results cannot be verified, the workflow returns DEFER. The LLM then continues from the current state. Thus, workflows are reused only when the required conditions are met.

2. Generalization to Novel Task Compositions

We agree that complete workflows mainly apply to tasks with similar structures. However, Trace2Workflow also learns reusable local programs.

For new task compositions, the LLM can still combine existing local programs even when no complete workflow matches.

AppWorld has more diverse tasks, so fewer tasks achieve Zero-LM. However, local programs can still reduce repeated operations.

Compared with CodeAct, Local Programs Only reduces average LLM turns from 20.517 to 18.350 on test-normal and from 26.500 to 23.217 on test-challenge. Trace2Workflow further reduces them to 11.017 and 14.950, respectively.

These results show that experience reuse remains useful even when complete workflows cannot directly solve new tasks. We focus on reusing learned experience rather than directly handling all unseen task structures.

3. Adaptation to Complex Environment States

Our method uses current task parameters and API results instead of copying object IDs or states from past traces.

In partially observable environments, guards and completion checks use available observations to guide execution. If there is not enough evidence, the workflow returns DEFER, and the LLM takes over.

Our method works best when execution results can be verified. For partially observable or irreversible tasks, reliable checks are especially important. We will clarify these conditions in the revised paper.

AuthorsOpenReview ↗

Response to Weakness 2: Workflow Compilation and Efficiency

We thank the reviewer for this suggestion. We add a new baseline to evaluate the benefit of workflow compilation. We also clarify our efficiency metric.

1. Benefits of Guarded Workflow Compilation

We introduce a Parameterized Trace Replay baseline. It reuses successful traces by replacing task-specific parameters and replaying actions. Unlike Trace2Workflow, it does not use guarded branches or step-level effect checks.

Both methods use the same historical traces, backbone model, test tasks, and action budgets. We evaluate them on ALFWorld Unseen (134 tasks).

MethodSuccessSR (%) ↑Turns ↓Zero-LM ↑
Parameterized Trace Replay116/13486.571.03773
Trace2Workflow132/13498.510.85899

Compared with Trace Replay, Trace2Workflow improves the success rate by 11.94 percentage points and reduces average LLM turns by 17.26%. It also completes 26 more tasks without LLM calls.

These results show that the improvement comes not only from past experience, but also from workflow compilation and execution checks. The guarded workflows can better handle changes in task parameters and environment states.

2. Scope of Efficiency Evaluation

We agree that fewer LLM turns do not always mean fewer tokens, lower latency, or lower total cost.

Our work mainly focuses on reducing repeated planning. Therefore, we use Online LLM Turns as our efficiency metric. It includes failed attempts and LLM fallback.

The costs of trace collection, local-program verification, and workflow compilation are not included. These are offline costs.

We will clarify this difference in the revised paper. We will also limit our efficiency claims to the measured reduction in Online LLM Turns.

Could the authors provide the underlying LLM-turn counts for the 134 ALFWorld Unseen tasks and verify the reported Trace2Workflow average of 0.858, since the comparison with 4.037 turns implies a 78.75% reduction rather than the stated 17.26%?

Reviewer_1tVaOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Trace2Workflow extracts local programs from successful and failed agent traces, checks whether they meet their local goals, and compiles successful task compositions into parameterized workflows. A workflow has applicability guards, input bindings, and checks on observable effects. Once matched, it executes without internal model calls; if a check fails or execution cannot continue, the model resumes from the current state. Experiments on ALFWorld and AppWorld use a frozen workflow library and report higher success rates with fewer online model turns. Ablations examine local-program reuse, direct workflow compilation, and the two verification stages.

Critical strengths
  1. Section 3.4 gives a concrete example of reusing the same procedure with different object bindings and current observations. It explains why task-level workflow reuse can save composition decisions even when the agent already has reusable local functions.

  2. Local program labels depend on whether the local goal is met, which allows the method to use valid fragments from failed episodes. The distinction between unknown evidence and verified failure is sensible. Recording workflow outcomes before fallback, then continuing from the current state with the remaining budget, also helps separate workflow performance from model recovery.

  3. On ALFWorld Unseen, Local Programs Only obtains 96.27% success with 2.806 model turns, compared with 98.51% and 0.858 turns for the full method (Table 3). This is the most relevant comparison for the paper's claim about reuse granularity. The AppWorld results suggest that the savings may extend to less regular tasks. Freezing the workflow library during evaluation also makes it easier to interpret transfer to new tasks.

Critical weaknesses
  1. I could not reconstruct the model-free matcher from the description. Section 3.3 and Appendix B say that request matching and input binding use no model calls, but do not specify the request parser, matching representation, or procedure for resolving ambiguous matches. There is also no concrete AppWorld binding/checking implementation. These details determine what the method can generalize: a parser written for benchmark task templates would support a narrower claim than an induced applicability mechanism. The paper also omits the values of m and ρ in Eq. (5), how validation episodes are constructed, and the resulting library sizes. The prompt templates do not resolve these questions.

  2. The inference-savings numbers do not agree. Figure 4(a) gives 115 Trace2Workflow turns, with 15 on covered tasks and 100 on uncovered tasks. Section 4.5 and Table 5(a) give 110, split into 15 and 95. The main Unseen average, 0.858 over 134 tasks, implies approximately 115 turns and agrees with the figure. A second discrepancy concerns coverage: Figure 4(b) and Table 5(b) give 99 direct successes, 17 deferrals, and two failures, totaling 118 matched tasks. Table 4 defines coverage through matching but reports full-workflow coverage of 73.9%, corresponding to about 99 of 134 tasks. Please reconcile the subsets or definitions behind these numbers. They affect the explanation of where the savings come from, though they do not by themselves establish that the primary success rates are wrong.

  3. Several experimental details needed for reproduction are missing: training-corpus sizes, the mining/validation split, action budgets, decoding settings, and repeated-run uncertainty. The cost claim also needs a clearer scope. Fewer model turns do not necessarily imply fewer tokens or lower latency, and the paper gives no costs for trace collection, program verification, or compilation. Either quantify these costs and estimate when reuse pays for acquisition, or limit the efficiency claim to the online metric actually measured.

Questions for authors
  1. What algorithms perform matching and input binding without model calls? Please give an AppWorld workflow with its guard, bindings, observable checks, and fallback. Which parts are induced and which are supplied manually? How is a candidate workflow trialed before a later successful execution activates it?

  2. Can the authors reconcile the 110/115 turn totals and the 118 matched outcomes versus 73.9% coverage? Task-level records of matches, direct successes, deferrals, final outcomes, and model turns would settle whether different subsets or definitions explain the discrepancies.

  3. Please provide the corpus and split sizes, verification thresholds, library sizes, budgets, decoding and repetition settings, and acquisition/compilation costs. Do Local Programs Only and the full method use the same local-function library and matched training evidence?

2 comments
AuthorsOpenReview ↗

Response to Weakness 1 and Question 1

We thank the reviewer for pointing out the missing implementation details. We clarify the matching, binding, and validation methods below.

1. Workflow Matching and Input Binding

During offline compilation, the LLM generates a matching specification from the source request and successful trace. It includes an operation type, typed input slots, request patterns, and guards. At runtime, a deterministic matcher extracts parameters using the patterns and checks their types and guards. The workflow runs only if a unique valid match is found; otherwise, it falls back to the LLM. Intermediate values come from current API results rather than past traces.

2. Concrete AppWorld Example

Consider the Spotify task: "Keep going to the previous song until reaching a previously liked song."

During offline compilation, the LLM generates a matching specification:

  • Operation: playback_search
  • Typed slots: direction ∈ {previous, next}, target_property ∈ {liked}
  • Request pattern: ^Keep going to the (?P<direction>previous|next) song until reaching a previously (?P<target_property>liked) song.$
  • Guard: direction == previous and target_property == liked, with all required inputs bound.

At runtime, the matcher extracts direction=previous and target_property=liked from the request. If the checks pass and the match is unique, the workflow runs without LLM calls. It retrieves liked songs and moves backward until a liked song is reached. API results and task completion are checked during execution. If any check fails, the LLM continues from the current state.

3. Induced vs. Supplied Components

We provide a common specification format, supported input types and guard operators, tool specifications, and a shared matcher and executor. The LLM automatically generates each workflow's operation type, input slots, request patterns, and guards from the source request and successful trace. The compiler preserves tool calls, execution order, and data dependencies. No task-specific matching rules are manually added. Generalization is limited to new requests covered by the generated patterns.

4. Candidate Validation and Reproducibility

For AppWorld, we use three instances per task template. One is used for workflow learning, and the other two are held out for evaluation.

Successful traces produce candidate workflows. Each candidate is tested through source replay and activated only if it succeeds without LLM fallback.

The AppWorld libraries contain 114 local programs and 5 active workflows. Both libraries are frozen during evaluation.

For local-program verification, we use m=3m=3 and ρ=0.8\rho=0.8 in Eq. (5). Workflow activation is based on successful execution.

AuthorsOpenReview ↗

Response to W2–W3 and Q2–Q3: Experimental Clarifications

We thank the reviewer for the constructive feedback. We clarify the metric definitions, experimental settings, and efficiency claims below.

Response to Weakness 2 and Question 2

1. LLM Turn Accounting

The 115 turns in Figure 4(a) include all online LLM calls, consistent with the average of 0.858 turns over 134 tasks. The 110 turns in Table 5(a) count effective decision turns, excluding five format-repair calls that produced no valid actions. These calls did not lead to tool execution or environment changes. We have verified all five calls in the execution logs.

Thus, the two values reflect different counting rules. We will clearly distinguish total LLM turns from effective decision turns and use consistent metric labels in the revised paper.

2. Workflow Coverage

The 118 matched tasks include 99 direct successes, 17 deferrals, and 2 failures. This gives an initial match rate of 88.1% (118/134). The reported 73.9% (99/134) represents direct-completion coverage, where workflows complete tasks without LLM calls.

The difference is that matching does not guarantee direct completion. Some workflows may stop because of failed checks or insufficient evidence. These tasks may require LLM fallback. We will clarify these definitions and use consistent coverage metrics in the revised paper.

3. Task-Level Records

We will provide task-level records of workflow matches, direct successes, deferrals, final task outcomes, and LLM turns. We will also clarify the counting rules used in the inference-saving analysis.

Response to Weakness 3 and Question 3

1. Experimental Settings and Reproducibility

Our training corpus contains 200 traces for ALFWorld (135 successful, 65 failed) and 97 traces for AppWorld (77 successful, 20 failed). Mining and validation episodes are separated within the training data. Validation instances are selected according to local goals and their preconditions. Candidates for the same goal use the same validation conditions.

For local-program verification, we use m=3m=3 and ρ=0.8\rho=0.8 in Eq. (5). The AppWorld libraries contain 114 local programs and 5 active workflows. The task action budget is 35, and the local-program verification budget is 10.

We will provide the exact mining/validation split sizes, decoding settings, random seeds, and evaluation repetition details in the revised paper. These details will make our experimental settings easier to reproduce.

2. Fairness of the Ablation Study

Local Programs Only and Trace2Workflow use the same local-function library, training evidence, backbone model, and action budgets. Both methods have access to the same reusable local functions.

The key difference is the reuse level. Local Programs Only requires the LLM to compose these functions for each new task, while Trace2Workflow directly reuses compiled task-level workflows. Therefore, this comparison isolates the benefit of workflow-level reuse rather than differences in local-function quality.

3. Scope of Efficiency Claims

We agree that fewer LLM turns do not necessarily mean fewer tokens, lower latency, or lower total computational cost.

Our experiments focus on online LLM turns, including failed attempts and LLM fallback. Offline costs for trace collection, local-program verification, and workflow compilation are not included in this metric.

We will therefore limit our efficiency claims to the measured reduction in online LLM turns. We will also explicitly acknowledge the offline acquisition costs as a limitation. Our main conclusion is that workflow reuse reduces repeated online model interactions while maintaining or improving task success.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.