acceptodds
Under review as a conference paper at ICLR 2027

OpWeave: Self-Expanding Agentic Workflows with Hierarchical Transferable Memory

Abstract

Automated agentic workflow design typically searches a fixed operator library, limiting adaptation when required capabilities are absent and forcing repeated search across tasks. We present OpWeave, a framework that self-expands, optimizes, and reuses a verified language of agentic workflows. From a small development subset, OpWeave profiles the task, retrieves procedural evidence and prior experience, and uses this evidence to propose typed operators and reusable workflow patterns. OpWeave then optimizes an initial workflow using development-set feedback and execution traces, where hierarchical attribution localizes failures at progressively finer granularity to keep each repair targeted. Verified elements and compact experience summaries are retained for reuse across tasks. Across six benchmarks spanning three task categories, OpWeave outperforms the strongest automated workflow design baseline on five of six tasks, with an average gain of 3.1 points. In a search-free setting, OpWeave uses memory priors to guide workflow generation and matches the strongest baseline with only 0.02% of the tokens of a full run. These results show that OpWeave enables adaptive and reusable workflow design by expanding and retaining its workflow language across tasks.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agentic Workflows, Automated Workflow Design, LLM Agents, Transferable Memory, Hierarchical Optimization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_tudLOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes OpWeave for automated agentic workflow design. The framework expands its operator space through verified Operators and reusable Pattern cards. It uses execution traces and hierarchical attribution to guide edits at the Block, Path, or multi-block level. Short-term memory supports the current search, and long-term memory stores capabilities and priors for later tasks. Experiments cover six benchmarks in reading comprehension, code generation, and mathematical reasoning, with a matched executor and search budget. The results show competitive performance and substantial token savings from search-free transfer. The continual-accumulation experiment shows clearer benefits for reuse and search efficiency than for accuracy.

Critical strengths

The main contribution is allowing the workflow language to expand and persist across tasks. Verified Operators add capabilities, and Pattern cards store useful local compositions. This gives the system a way to reuse both individual operations and workflow structures. The optimization design is also well motivated. Execution traces guide the scope of an edit, from a Block to an executed Path or a larger region. Results on DROP, HumanEval, and MATH suggest that these localized edits can improve search under the same evaluation budget.

The evaluation considers search cost as well as final performance. Using a matched executor and search budget makes the comparisons easier to interpret. Search-free transfer is particularly useful in practice: it produces competitive workflows at a small fraction of the token cost of full search. I also appreciate the mixed results. Continual accumulation reduces search cost but does not consistently improve accuracy, and the value of same-type priors varies across tasks. These findings help identify the actual benefits of memory rather than presenting reuse as uniformly helpful.

Critical weaknesses

The continual-learning evidence is still limited. The accumulation experiment uses one task order and apparently one run per step. Its average test score is slightly below the corresponding cold runs. The result therefore supports reuse and efficiency more clearly than improved accuracy. The ablations also leave some component effects unclear. Table 2 adds Briefs, Operators, Patterns, and memory in sequence, despite non-monotonic effects across tasks. Table 3 covers only three of the six benchmarks. Limited repetitions further complicate the interpretation of small differences. The transfer discussion should also separate cross-task memory from the “all tasks + feedback” setting, which includes previous target-task optimization feedback.

The experiments use closed-form benchmarks and one executor model. This leaves the broader motivation for general agentic workflow design only partly tested, especially for long-horizon tool use and open-ended environments. The design phase also combines a strong model, retrieval, and verification. The comparisons do not clearly isolate the benefit of typed Operator/Pattern registration from this additional assistance. A control with the same model, retrieved evidence, and budget would help. Finally, fewer search rounds may not mean lower end-to-end cost. Attribution and verification add overhead, so token use, model calls, and wall-clock latency should be reported separately.

Questions for authors

Q1. Are the continual-accumulation results stable across task orders and random seeds? In particular, does the reported 0.84× average search cost hold across runs, and how much does average accuracy vary? Please also separate transfer from other tasks from reuse of previous target-task feedback. This distinction matters for interpreting the cross-task generalization claim.

Q2. Could you add a leave-one-component-out design ablation and a control with the same design model, retrieved evidence, and token budget, but without explicit Operator/Pattern registration? Do Path restriction, stall escalation, and short-term memory show similar benefits on HotpotQA, MBPP, and GSM8K? End-to-end token use, model calls, and latency would also help assess the cost of these mechanisms.

Reviewer_dBrBOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

OpWeave combines knowledge-driven operator synthesis, hierarchical workflow repair, and reusable memory for automated workflow design. It expands a typed operator library, revises workflows using development feedback, and transfers accumulated experience to later tasks. Experiments on six benchmarks show improved average performance and reduced marginal design costs with memory reuse.

Critical strengths
  1. The framework connects several practical design decisions. Operator synthesis, scoped repair, and persistent memory form a coherent workflow-design pipeline.

  2. The experiments examine multiple components and transfer settings. Shared-initialization optimizer comparisons and distinctions between cross-task and cross-type memory are useful.

Critical weaknesses
  1. The independent value of operator expansion is unclear. The examples are primarily specialized prompt modules that may be expressible through existing generic operators. The ablations do not control for equivalent prompts and retrieved knowledge within a fixed library, while limited acceptance tests do not establish broad functional reliability.

  2. Efficiency and transfer claims need stronger qualification. Equal proposal rounds do not equalize successful evaluations, and full search costs 1.72× MCTS in Table 4. The 0.02% figure excludes memory construction; cross-type transfer underperforms A2Flow, and the single-order continual experiment shows no average accuracy improvement.

Questions for authors
  1. Does operator expansion outperform a fixed library given identical retrieved evidence, model access, and specialized prompts?
  2. How do the optimizers compare under matched token budgets and completed evaluation counts, using held-out test performance?
  3. What do acceptance tests establish beyond executability, and how well do admitted operators work on unseen inputs?
  4. Can transfer be evaluated across multiple task orders against a structural-prior baseline, including memory-construction costs and uncertainty estimates?
4 comments
AuthorsOpenReview ↗

Response to Reviewer dBrB

Dear Reviewer dBrB, thank you for the precise questions, and for noting that the pipeline is coherent and that the shared-initialization comparison and the distinction between cross-task and cross-type memory are useful. We agree with each qualification you ask for in the second weakness and have narrowed the corresponding claims. Below we answer Q1–Q4 together with the weaknesses they relate to, and name the table that carries each number.

1. Q1 and Weakness 1 (first part): does Operator expansion outperform a fixed library?

"Does operator expansion outperform a fixed library given identical retrieved evidence, model access, and specialized prompts?"

Yes, when Operators come with the Pattern cards that compose them: against a fixed-library control, the full design scores 76.5 against 71.5 on the six-benchmark test average (Table R1, General Response G1). We agree with your premise that this is not a gain in expressive power. The admitted Operators are specialized prompt modules, and a generic operator can express them: a registered Operator is a stored prompt executed through the generic implemented_skill operator. The control shows that the Designer does not recover that behavior without the registry.

The control. The "w/o O and P" row of Table R1 (General Response G1) is our fixed-library control. Its run differs from the Full run in two flags only, which remove the admitted Operators and the Pattern cards from the registry section of the Designer prompt. The Designer (GPT-4o-mini, temperature 0.2), the Task Brief, the memory file, and the registry file are identical, and the default library remains available. Recalled memory does not bring the withheld elements back: the memory section that enters the prompt contains no Operator or Pattern name. The Designer is free to write any specialized prompt into the default operators. It scores 71.5 against 76.5 for the full design (MATH 32.2 against 50.2; MBPP 68.2 against 77.7).

Identical specialized prompts. Pasting the registered prompt text verbatim into generic operators is the Full system itself, since a registered Operator is a stored prompt run through implemented_skill. The control therefore withholds the registered prompts and leaves the Designer free to write its own. What registration adds beyond the prompt text is the declared interface, the acceptance fixtures, and persistence across tasks.

Independent value of Operators. Your concern is correct for Operators taken alone. Relative to the control, Pattern cards alone add 2.0 points, Operators alone add 0.1, and the two together add 5.0. We therefore describe the contribution as expansion of the workflow language, meaning Operators plus the Pattern cards that compose them, and no longer as Operator expansion by itself.

Rationale for prompt-backed Operators. This is a deliberate restriction. An admitted Operator can only issue a model call under its declared schema. It cannot call the web, a shell, or code execution, and it adds no tool beyond those of the default library. We chose this for two reasons. First, admission stays cheap and checkable: a prompt-backed Operator has no side effects, so a schema check and 1 to 4 fixtures are enough to enforce its contract (Table R9). Second, the comparison stays clean: the full design and the fixed-library control have identical tool access, so the 5.0-point difference in Table R1 (General Response G1) cannot come from added external capability. The same holds against the baselines: the operators of AFlow and A²Flow have the same tool access as ours, so the margin over them (Table 1; Table R5c, General Response G5) does not come from extra tools either. We acknowledge that this design also limits what Operators can do. What prompt-backed Operators offer in practice is portability. The workflows that contain them run unchanged on Qwen3.5-9B and Gemma 4 12B and stay above single-call prompting (Tables R6b–R6c, General Response G6). Admitting tool- or code-backed Operators would need stronger verification, and we leave it to future work.

AuthorsOpenReview ↗

Response to Reviewer dBrB

2. Q3 and Weakness 1 (second part): what acceptance tests establish

"What do acceptance tests establish beyond executability, and how well do admitted operators work on unseen inputs?"

Scope of the acceptance tests. Admission has two checks beyond running the code.

  • Interface and contract. Each fixture runs once through the production executor on the target model (GPT-4o-mini). The response must pass the Operator's frozen response schema and its side-effect contract: an Operator declared as a non-producer may not emit a candidate answer, a producer must emit one, and records must match their declared types.
  • Semantic judgment. A separate judge model compares every actual output with the fixture's reference result. It must mark the output semantically correct, usable by the next Operator, and consistent with the declared candidate behavior. One failed fixture blocks admission.

Fixtures are short LLM-written cases, each with a reference result, 1 to 4 per Operator and 15 in total for the 8 admitted Operators. They contain no benchmark items, and the implementer never sees them. Acceptance therefore shows that an Operator honors its interface and does what it claims on a few constructed cases. It is not a measure of accuracy on a task distribution. In the paper, "verified" means that an Operator has passed these two checks, and the revision states this definition.

Behavior on unseen inputs. We re-checked the test-time responses of the six final workflows of Table 1, three test passes each, against each admitted Operator's frozen schema.

Table R9. Admitted Operators of the six cold runs: acceptance fixtures, and schema conformance on held-out test inputs.

TaskOperatorTypeFixturesIn final workflowTest-time callsSchema-valid
DROPnumeric_evidence_worksheetnon-producer extractor1yes2,37699.7%
DROPconcise_answer_contract_normalizercandidate producer2yes2,370100%
HotpotQAshort_answer_contract_normalizercandidate producer4yes†8100%
HotpotQAtemporal_numeric_evidence_tablenon-producer extractor1no——
HumanEvalcode_contract_edge_case_extractornon-producer extractor1no——
MBPPpython_answer_contract_normalizercandidate producer2yes1,220100%
GSM8Karithmetic_quantity_extractornon-producer extractor1yes3,165100%
MATHfinal_math_answer_normalizercandidate producer3yes2,915100%

†This Operator sits in a branch that runs only when the critic reports a contradiction, so it was called 8 times.

The six Operators that appear in final workflows return schema-valid outputs in 99.7% to 100% of test-time calls. The only failures are 8 of the 2,376 calls of one DROP extractor, each missing one field. This shows that the contracts hold on unseen inputs. It does not show that the Operators' outputs are correct: test questions have no per-Operator reference, so per-Operator accuracy cannot be computed from existing data, and we state this as a limitation. The accuracy effect of admitted Operators is measured end to end by the "w/o O" row of Table R1 in General Response G1 (3.0 points on the six-benchmark average), which mixes the quality of the Operators with how the Designer uses them.

AuthorsOpenReview ↗

Response to Reviewer dBrB

3. Q2 and Weakness 2: matched tokens, completed evaluations, and held-out test

"How do the optimizers compare under matched token budgets and completed evaluation counts, using held-out test performance?"

We therefore ran both comparisons you ask for. Under either budget, equal search tokens or equal completed evaluations, OpWeave is at or above MCTS, and at MCTS's full 20-round token budget Full is 1.1 points higher on held-out test.

Matched tokens, held-out test (Table R2a, General Response G2). At the token budgets MCTS has spent after 5, 10, and 20 rounds, Base is 0.3, 0.9, and 0.5 points above MCTS on the test average of DROP, HumanEval, and MATH, and Full is 0.0, 0.1, and 1.1 points above. Table R2b (General Response G2) gives the same comparison on development data for all six benchmarks and two search runs. OpWeave is higher in 17 of the 18 comparisons and tied in one. Held-out matched-token results cover three benchmarks; the other three are development data only.

Table R10. Completed evaluations, and best development score at equal completed evaluations. Six benchmarks, first search seed. For the last three rows, each method is given the number of completed evaluations that MCTS reaches by its round 5, 10, or 20 on each benchmark.

QuantityMCTSBaseFull
Completed evaluations in 20 rounds, summed over six benchmarks817797
Best development score at MCTS's round-5 evaluation count76.176.876.5
Best development score at MCTS's round-10 evaluation count76.477.176.9
Best development score at MCTS's round-20 evaluation count76.677.277.1
  • Which budget is stricter. Within the token budget that MCTS spends in 20 rounds, OpWeave completes fewer evaluations than MCTS: 48 to 63 against 81, where the range reflects cached evaluations on HotpotQA, MBPP, and GSM8K. Matched tokens (Tables R2a and R2b, General Response G2) is therefore the stricter budget for our method.
  • Equal completed evaluations. OpWeave is ahead of MCTS in every cell of R10, by 0.4 to 0.9 points across the two search seeds, and the second seed gives the same ordering.

4. Weakness 2: the 0.02% figure excludes memory construction

Agreed. The 0.02% is the marginal cost of one more design once memory exists, and the Abstract now says so. Table R3 (General Response G3) gives the source-inclusive accounting.

5. Weakness 2: cross-type transfer underperforms A²Flow

Agreed. Cross-type transfer scores 72.6 against 74.0 for A²Flow (Table R4a, General Response G4a). It is also below other-tasks transfer on all three models we tested, and on Gemma its MATH score is 60.7 against 80.9. We report this as a boundary condition: in our experiments transfer was useful only when the source pool contained a task from the target's category. The revision no longer presents cross-type transfer as a benefit. Its advantage is cost: it is search-free, and a single Designer call of 14k to 32k tokens (Table R3, General Response G3) comes within 1.4 points of what A²Flow reaches by searching on the target.

6. Q4 and Weakness 2: task orders, structural-prior baseline, and uncertainty

"Can transfer be evaluated across multiple task orders against a structural-prior baseline, including memory-construction costs and uncertainty estimates?"

(a) Multiple task orders. We ran two more orders under one protocol (Table R4c, General Response G4c). In all three orders we observe a lower search cost on later tasks: they use 16%, 56%, and 34% fewer search tokens than cold search under the same 20-round budget, and 14 of 15 later-task searches use fewer tokens. The reduction is largest in Order 2, where tasks of the same type are consecutive. You are right that accuracy does not improve on average: the test differences from cold search are −0.4, +0.1, and +0.2 points, so we claim comparable accuracy. Further orders are running and will be posted.

(b) Memory-construction costs. See point 4 and Table R3 (General Response G3).

(c) Uncertainty. Paired bootstrap intervals over test questions (Table R5b, General Response G5): cold search is above other-tasks transfer by 2.8 points [1.8, 3.9], above other-types transfer by 4.5 [3.4, 5.7], and above all-tasks+FB by 0.7 [−0.4, 1.7]. Three independent design-and-search runs give a per-benchmark SD of 0.67 points (Table R5a, General Response G5). With one run per task order we cannot give an interval across orders, and we state this in the paper.

(d) Structural-prior baseline. We built one, and OpWeave's search-free transfer is above it in all four settings (point 7).

AuthorsOpenReview ↗

Response to Reviewer dBrB

7. Q4: the structural-prior baseline

The Designer works over the default Operator library and receives, in place of memory, workflow samples: the structure of the final workflows produced by separate memory-free searches on the source tasks, run with Gemma 4 12B. A rule contains only built-in Operator types and control flow, such as seq, cond, and branches. It has no prompts, no Operator or Pattern names, no scores, and no dataset names; each source task appears as "Task k (type: …)". The "other tasks" rows use the rules from the five other tasks, and the "other types" rows those from tasks of other types. These rows use no Task Brief and no admitted Operators or Patterns, so they show what structure alone contributes. We evaluated it in the search-free transfer setting, not inside the continual chains of point 6(a).

Table R4b. Workflow output rules over the default library, test scores. "No rules" is the same setup without them. The named model is Designer and executor. ± is the standard deviation over three test executions.

GPT-4o-mini

SettingHotpotQADROPHumanEvalMBPPGSM8KMATHAvg.
No rules72.2 ± 0.178.9 ± 0.889.8 ± 0.973.7 ± 0.688.3 ± 0.333.1 ± 0.972.7 ± 0.3
Output rules, other types30.0 ± 0.88.9 ± 0.591.9 ± 1.669.1 ± 0.789.1 ± 0.435.9 ± 2.254.1 ± 0.3
Output rules, other tasks69.2 ± 0.46.2 ± 0.789.6 ± 0.968.4 ± 0.991.4 ± 0.338.7 ± 0.060.6 ± 0.3

Gemma 4 12B

SettingHotpotQADROPHumanEvalMBPPGSM8KMATHAvg.
No rules77.6 ± 0.186.0 ± 0.198.7 ± 0.486.2 ± 0.595.1 ± 0.173.1 ± 0.886.1 ± 0.2
Output rules, other types77.1 ± 0.186.8 ± 0.496.7 ± 1.270.2 ± 0.293.4 ± 0.563.2 ± 1.081.2 ± 0.3
Output rules, other tasks77.8 ± 0.284.0 ± 0.196.2 ± 0.879.7 ± 1.994.9 ± 0.380.4 ± 0.885.5 ± 0.1

OpWeave's transfer against the baseline. For the same source tasks, OpWeave's search-free transfer (Table R4a, General Response G4a) is above the output-rule baseline in all four settings: 74.3 against 60.6 and 72.6 against 54.1 with GPT-4o-mini, and 86.8 against 85.5 and 84.0 against 81.2 with Gemma.

Output rules alone do not help. Against the same setup without rules, the average falls from 72.7 to 60.6 and 54.1 with GPT-4o-mini, and from 86.1 to 85.5 and 81.2 with Gemma.

Failure analysis. With GPT-4o-mini, three cells collapse (DROP at 6.2 and 8.9, HotpotQA at 30.0), and we traced them to a single cause. The Designer copied the programmer Operator from the rule of a mathematics task into reading workflows. programmer executes code and returns the printed result only on mathematics tasks; on other tasks it returns the code itself, so the grader received Python instead of an answer. On DROP, 763 of 800 test questions (95%) return code with other-types rules and 792 of 800 (99%) with other-tasks rules. Gemma does not make this mistake (DROP at 86.8 and 84.0), but with other-types rules it still loses 16 points on MBPP and 10 on MATH relative to no rules. OpWeave's transfer does not show the misapplication: its DROP scores with GPT-4o-mini are 79.7 and 78.7 (Table R6a, General Response G6).

Conclusion. Workflow structure transferred without further context does not help and can be misapplied. OpWeave's search-free pipeline, which gives the Designer the recalled priors knowledge, is above this baseline in every setting we tested.

Revision. The structural-prior baseline and its failure analysis are added to the transfer analysis.

We hope these responses address your concerns, and we are happy to discuss any that remain.

Reviewer_pUmWOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes OpWeave, a framework for automated agentic workflow design that expands its operator library, optimizes workflows through hierarchical failure attribution, and reuses experience through a dual-memory mechanism. Experiments on six benchmarks covering reading comprehension, code generation, and mathematical reasoning demonstrate improvements over existing workflow design methods.

Critical strengths

The paper addresses an important problem in automated agentic workflow design. The integration of operator expansion, hierarchical workflow optimization, and persistent memory provides a reasonable framework for improving workflow adaptability. The experiments cover six benchmarks across three domains and include comparisons with existing methods, component-level ablations, and computational cost analysis.

Critical weaknesses

The methodological novelty is not sufficiently demonstrated. The proposed framework mainly integrates existing techniques for workflow construction, hierarchical optimization, and experience-based improvement. Its technical distinction from prior automated workflow design methods remains unclear, and stronger comparisons are needed to establish its independent contribution. Table 4 shows that Full OpWeave consumes 1.72 times the total search tokens of MCTS. Furthermore, the reported search-free transfer cost of approximately 0.02% excludes the substantial cost of constructing the source memory. The evaluation is also limited to one executor model, with limited repetitions and no reported confidence intervals. These limitations make it difficult to assess the robustness and overall computational efficiency of the proposed framework. The overall presentation quality also needs significant improvement. The figures are visually cluttered and difficult to read, and several parts of the manuscript contain unusual spacing and inconsistent formatting. These avoidable issues suggest insufficient attention to manuscript preparation and proofreading, raising concerns about the overall quality and rigor of the submission.

Questions for authors

Q1. Cross-task transfer and methodological contribution How can the authors demonstrate that the proposed memory mechanism provides effective transfer to genuinely unseen tasks? Given the limited Operator reuse and the decrease in average test performance in Table 6, what evidence supports the claimed benefits of continual memory accumulation? Q2. Experimental fairness and computational efficiency Could the authors provide comparisons under matched total token budgets rather than only matched workflow evaluation counts? How does the claimed search-free efficiency change when the cost of constructing and maintaining the source memory is included? Q3. Statistical reliability and reproducibility Could the authors report performance variance or confidence intervals across independent runs and different task orders? Additionally, could the authors clarify why the final average scores for Base and Full OpWeave differ between Tables 3 and 4 despite both describing optimization performance under the same search budget?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.