acceptodds
Under review as a conference paper at ICLR 2027

Inherit-MAS: Test-Time Evolution of Multi-Agent Systems through Workflow and Execution Inheritance

Abstract

Multi-agent systems (MAS) built from large language models coordinate specialized agents to tackle complex tasks, but effective workflows are difficult to design in advance. Test-time evolution refines workflows using execution feedback, yet broad revisions can disturb useful components, while re-executing unchanged requests can incur redundant computation. Inspired by the interplay of inheritance and selection in biological evolution, we introduce Inherit-MAS, which makes inheritance explicit at the workflow and execution levels. A meta-model first synthesizes a workflow of worker agents with declared roles, communication inputs, and tool permissions, and a separately prompted judge scores each executed candidate and diagnoses its deficiencies. In ordinary refinement rounds, workflow inheritance starts from the latest completed candidate, may discard removable nodes judged unhelpful, and applies a validated edit to address the diagnosed deficiency. When the new candidate executes, execution inheritance inherits eligible stored results only if the complete resolved request and execution context match, avoiding redundant model and tool calls. With GPT-4o-mini workers, Inherit-MAS achieves 55.4% completion on WorkBench and 49.7% joint F1 on HotpotQA FullWiki, outperforming EvoAgent, EvoMAS, and TacoMAS. With Qwen3-32B workers, it also exceeds these evolving-MAS baselines on both benchmarks. Compared with rerunning the same controller with execution inheritance disabled, execution inheritance reduces worker-token usage by 29.1% on WorkBench and 34.6% on HotpotQA, and total token usage by 5.3% and 18.1%.

Primary area:
foundation or frontier models, including LLMs
Keywords:
multi-agent systems, test-time evolution, workflow optimization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_zQX4OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Inherit-MAS, an approach that enhances the test-time evolution of multi-agent systems (MAS) through the inheritance of both workflows and execution results. Based on review feedback, the method retains useful nodes, implements constrained modifications to workflows, and reuses historical results whenever the request and execution context match perfectly. Experiments on WorkBench and HotpotQA demonstrate that this approach outperforms existing evolutionary multi-agent baselines while simultaneously reducing the inference overhead of operational nodes.

Critical strengths
  1. The problem addressed is of practical significance. It specifically considers the loss of valid components—often resulting from iterative modifications to workflows—as well as the overhead of redundant execution, directly aligning with the real-world need for optimization in multi-agent testing.
  2. The rules for result reuse are rigorous. Reuse decisions are based on comprehensive validation—encompassing the full request, execution context, and the integrity of recorded data—while explicitly excluding state-modifying operations, thereby establishing clear safety boundaries for reuse.
  3. The experimental analysis is highly transparent. It covers two distinct task categories and two operational models, distinguishes between the overheads of individual work nodes and the controller, and reports on phenomena such as candidate degradation, ranking errors, and limitations in overall cost-effectiveness.
Critical weaknesses
  1. The current experiments cover only office-tool tasks and multi-hop question answering; validation for the optimization of general multi-agent workflows remains insufficient. Could additional tests be included?
  2. The approach requires intermediate nodes to be read-only, with all write operations performed exclusively by the final execution node. Please explain how this is adapted for WorkBench.
  3. The distinctions between this method and existing workflow optimization and result-caching mechanisms require further clarification; it is recommended to include targeted comparisons to more clearly highlight its unique contributions.
  4. Several evolutionary multi-agent baselines underperform compared to "Single ReAct" on the WorkBench benchmark, for reasons that remain unclear. It is recommended to provide further clarification on whether this performance drop stems primarily from the collaboration and evolutionary mechanisms themselves, or from differences in baseline adaptation and configuration, to help better interpret the comparative results.
Questions for authors

refer weakness

2 comments
AuthorsOpenReview ↗

Rebuttal - Part 1

W1: Only office-tool tasks and multi-hop QA; could additional tests be included?

To add more tests, we ran experiments on BIG-Bench Extra Hard (BBEH), which contains a suite of 23 general reasoning tasks with no tools, retrieval, or environment state, a different regime from both WorkBench and HotpotQA. Following the same experiment settings, the comparison against EvoMAS is shown below.

BBEH, 115 questionsAccuracyWorker tokensTotal tokens
EvoMAS13.0%5.71M8.04M
Inherit-MAS17.4%3.15M6.21M

Inherit-MAS is 4.3 points more accurate than EvoMAS while using 45% fewer worker tokens and 23% fewer total tokens.

W2: Intermediate nodes are read-only and the final node performs all writes: how is this adapted for WorkBench?

The condition applies only to WorkBench, the one evaluated benchmark with mutable state. On HotpotQA, no node changes any state, so the rule is never exercised. It is a condition for a node's result to be eligible for inheritance, not a limit on what a workflow may do.

WorkBench fits it with almost no adaptation, because the benchmark already has this shape. WorkBench itself defines 13 of its 27 tools as read-only and 14 as state-changing, and its reference solution for a task is an ordered list of state-changing calls. Worker nodes are the only nodes with tools, and they may call only the read-only tools declared for them. Integrators and verifiers have no tools: they emit the proposed changes as one JSON batch in the benchmark's own write signatures, which a verifier may approve or revise. A single deterministic, prompt-free executor then validates the batch against the official tool names and argument schemas and applies the surviving calls in order. The benchmark evaluator then scores the final environment state. "Gather evidence, then commit one validated batch" is thus the native form of a WorkBench task.

W3: Distinctions from existing workflow optimization and result-caching mechanisms; targeted comparisons.

  1. Distinctions against existing workflow optimization and result-caching mechanisms.

    • Versus workflow optimization. Offline optimizers such as AFlow search workflows on a validation set and then apply one fixed workflow to all test tasks. Evolving-MAS methods such as EvoAgent, EvoMAS, and TacoMAS adapt at test time, but a single update can change several components, and they carry forward configurations, experience, or earlier outputs without checking which stored node results remain valid. Inherit-MAS explicitly segments the useful and failed parts of the parent workflow. It keeps the nodes that the judge found useful and applies one validated edit, so the rest of the workflow is inherited unchanged.
    • Versus result caching. Methods such as Helium, StepCache, and agentic plan caching reuse operator results, similar responses, or plan templates to speed up given requests or workflows. But they do not revise a workflow. In Inherit-MAS, reuse is part of the search: a node restores a stored result only when its complete request matches, including upstream results and the execution context, and state-changing actions are never reused.
    • What is unique about Inherit-MAS. The coupling of the two inherit mechanisms. Because each update is one scoped edit to an inherited workflow, most nodes repeat their requests, and verified reuse avoids re-running them. A workflow optimizer alone re-executes every candidate, and a cache alone finds little to reuse when candidates are not inherited.
  2. Targeted comparisons. We ran both comparisons on a HotpotQA subset (GPT-4o-mini workers, same retriever and settings as the paper).

SystemJoint F1Worker tokensTotal tokensWorker tokens reused
Inherit-MAS45.30.67M1.57M34.5%
Our search + an existing cache (provider prompt caching, our store off)40.31.04M1.94M0%
An existing search (EvoMAS) + our store43.41.14M1.86M6.1%

The result shows that Inherit-MAS achieves the best F1 score with the fewest tokens used. Both comparisons point to the same conclusion: the savings come from coupling scoped edits with the store, not from either part alone. With our search-and-only provider prompt caching, the provider recognizes repeated prompt prefixes (21.7% of worker prompt tokens) but only discounts them, leading to marginal token savings. With our store attached to EvoMAS, only 6.1% of worker tokens are reused, compared with 34.5% for Inherit-MAS, because one EvoMAS update can change several components and few requests repeat. Therefore, our workflow and execution inheritance are distinct from existing approaches, and their combination uniquely brings performance and efficiency benefits.

AuthorsOpenReview ↗

Rebuttal - Part 2

W4: Several evolving-MAS baselines underperform Single ReAct on WorkBench. Why? Mechanism or adaptation?

To understand why several multi-agent baselines underperform compared to "Single ReAct" on WorkBench, we traced every failed task for every system in the saved WorkBench logs. We find that the gap mainly comes from multi-agent collaboration.

Where the gap is. 25 of the 130 tasks require no state change, and every system solves 20 to 23 of them. The whole gap is in the 105 tasks that need a write action.

System (GPT-4o-mini workers)Solved of 105FailedFailed: record never retrievedFailed: record retrieved by an agent, but lost before the final actionFailed: other
Single ReAct347154017
EvoAgent228359177
EvoMAS2085541417
TacoMAS2481412614

"Other" covers tasks whose writes need no record, and writes that used the right record with a wrong argument.

Collaboration (the evidence is found but lost between agents). Retrieval failures are similar for Single ReAct and the baselines (41 to 59 tasks): this is the shared limit of the worker model. The baselines add the fourth column. In 14 to 26 tasks, one agent retrieved the right record, but the final action did not carry it because the integrating agent issued no write or wrote a placeholder instead. Single ReAct has no handoff and loses none. These losses are as large as the gap itself: the baselines fail 10 to 14 more tasks than Single ReAct.

Evolutionary mechanism. Evolution does not lower quality, but its gain is not returned. EvoMAS's two initial candidates solve 30 tasks each, and its three mutations solve 32 to 35. Some candidates solve the task in 60 of 130 cases (46.2%, above Single ReAct), yet the label-free selection returns only 40.

Reviewer_7jqiOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper introduces Inherit-MAS, a framework for test-time optimization of multi-agent workflows that combines workflow inheritance with execution inheritance. A meta-model first constructs a workflow specifying agent roles, tool permissions, and communication links. Guided by execution traces and judge feedback, ordinary refinement starts from the most recently completed candidate, selectively removes dispensable nodes judged unhelpful, and applies a validated edit within a declared scope while preserving the remaining configuration. During execution, eligible read-only results are reused only when the complete resolved request, including upstream records and execution context, matches a stored record that passes integrity checks. Other nodes execute live, and state-changing operations are always performed anew. After search, the system returns a stored candidate output selected using execution status and judge scores. Experiments on WorkBench and HotpotQA evaluate task performance and inference cost, with additional analyses of candidate ranking, edit scope, and result reuse.

Critical strengths
  1. The framework addresses the practical connection between workflow revision and execution reuse. Execution feedback guides component selection and scoped edits, while structural validation enforces constraints on roles, connections, and tool permissions. This gives workflow changes explicit, inspectable boundaries and creates opportunities to reuse completed computation during subsequent refinement.

  2. The conditions for execution reuse and the treatment of state are carefully specified. Matching covers the complete request, upstream records, and execution context, accounting for cases where a node's configuration remains unchanged but its inputs differ. For stateful tasks, each candidate starts from the same initial environment, and state-changing actions are confined to a sink that always executes live. This establishes a clear boundary between reusable read-only computation and operations with external effects.

  3. The evaluation covers both retrieval-based question answering and stateful tool use with different worker backbones. The accounting separates live worker, meta-model, and judge usage from inherited token equivalents, and includes independent reruns without execution inheritance as well as consistency checks under a deterministic runtime. The additional analyses of candidate quality, ranking, and reuse across edit types make the relationship between performance and execution cost easier to examine.

Critical weaknesses
  1. As written, the final ranking rule can prefer an invalid output over a valid one. Equation (7) first minimizes the sum of recorded node errors and an invalid-output penalty, and only then considers output validity. Since an invalid output adds just one error, a candidate with no node errors but an invalid final output would outrank a candidate with two recoverable node errors and a valid final output. This ordering does not adequately distinguish an unusable final result from intermediate failures that the workflow can recover from. The authors should clarify the rationale for this priority and check whether it can cause the system to return an invalid output despite having a valid candidate available.

  2. The incremental contribution over existing workflow optimization and execution-reuse mechanisms is not sufficiently isolated. AFlow already uses single-step workflow modifications, Evoflux combines execution feedback with typed edits, and Helium supports result reuse through operator-input signatures. The paper acknowledges these works, but does not compare against existing search methods equipped with the same result cache, or independent workflow generation followed by the same judge under a comparable budget. Appendix B.1 shows that retaining and ranking candidates contributes substantially to the final results, while Appendix B.2 does not establish a consistent relationship between edit locality and quality improvement. These observations leave the specific benefit of inheritance-guided editing difficult to separate from ordinary memoization and selection among additional candidates.

  3. Ordinary refinement always continues from the most recently completed candidate, without selecting parents based on their historical quality. Earlier high-scoring candidates are considered for final return and a possible last-round restart, but are not revisited as parents during normal refinement. If an edit damages a useful configuration, subsequent attempts may spend their limited budget repairing a degraded branch. A controlled comparison with selecting higher-ranked parents or retaining multiple search branches is missing, leaving the rationale and effectiveness of this search policy unclear.

  4. The system-level quality–cost advantage would benefit from more competitive reference points. On WorkBench, most token usage comes from the meta-model and judge, whereas execution inheritance primarily reduces worker calls, so worker savings translate into a relatively small reduction in total usage. The controller also uses a fixed five-candidate budget without early stopping or adaptive allocation, and cost is excluded from final ranking. Comparisons with a stronger single agent or repeated execution followed by selection under a matched budget would help establish whether this controller uses inference resources effectively.

  5. Support for stateful tasks relies on restrictive execution assumptions. Each candidate starts from the same initial environment, intermediate nodes are read-only, and state-changing operations are deferred to the final executor. This provides a clear basis for reuse, but the evaluation does not cover tasks requiring interleaved reads and writes, replanning based on the results of write operations, or continued interaction with a changing external environment. The implications of these requirements for the method's applicability deserve clearer discussion.

Questions for authors
  1. How does the judge distinguish an unhelpful node from one whose useful output is not being used correctly downstream? Could controlled node-removal experiments, with the remaining configuration held fixed, assess the accuracy of keep/discard recommendations, particularly when nodes make complementary contributions or their value emerges only through several downstream steps?

  2. Execution inheritance fixes a previously sampled node output across candidates. In remote-model settings where identical requests do not guarantee identical fresh outputs, how does this affect search diversity and error propagation? In particular, when the first output is incomplete or incorrect, how does the system recover, and what would justify selectively re-executing the node?

  3. In Table 2, Inherit-MAS leads in answer F1 but trails TacoMAS in supporting-fact F1. What explains this difference? A breakdown of failures into missing retrieved evidence, incorrect supporting-sentence selection, and omissions during final synthesis would help clarify which capabilities workflow optimization improves and whether the judge reliably identifies gaps in the evidence chain.

4 comments
AuthorsOpenReview ↗

Rebuttal - Part 1

W1: The final ranking rule can prefer an invalid output over a valid one.

We consider both error count and final-output validity essential to evaluating workflow quality. Final-output validity is important because it ensures that the returned result satisfies the required schema and can be accepted by the downstream evaluator. However, validity alone does not capture the reliability of intermediate execution. We prioritize execution errors because they provide a finer-grained signal of workflow reliability, identifying failures that may remain hidden behind a valid final output.

Empirical check: no invalid output was returned while a valid candidate existed, and placing validity first changes nothing. We re-ranked each trajectory with validity ahead of the error count:

BenchmarkTask trajectoriesCompleted candidatesInvalid output returned while a valid candidate existedReturned candidates changed by the output-validity-first order
WorkBench1,5607,14900
HotpotQA1,8008,88500

The table includes Inherit-MAS runs in Tables 1–2. The configuration in the reviewer's example is that an invalid candidate with a lower bkb_k than a valid one never happens. The experimental results demonstrate that the performance is insensitive to this ordering.

W2: The incremental contribution over existing workflow optimization and execution-reuse mechanisms is not isolated.

1. Inheritance-guided editing versus memoization plus selection among additional candidates. We ran a study on WorkBench varying whether later candidates inherit the workflow (Select + Edit on the latest candidate) and whether exact node results may be inherited. Everything else remains the same.

SettingsCompletionTotal tokensUSDReuse
Independent synthesis, no store57.711.38M9.200.0%
Independent synthesis + same result store63.111.22M9.120.4%
Workflow inheritance, no store55.410.93M8.150.0%
Both (Inherit-MAS)60.010.26M7.9828.0%

The table shows that memoization alone improves performance while also slightly reducing token cost. Workflow inheritance alone trades small portions of performance for efficiency (token and monetary cost). Together, they provide a balanced performance-efficiency trade-off. Thus, selection, editing, and memoization work well together within the two inheritance mechanisms.

2. An existing search method equipped with the same result store. We apply the identical exact-request store to our HotpotQA adaptation of TacoMAS on a subset, without changing other settings.

TacoMASStore offStore on
Joint F144.548.6
Worker requests served from the store0167 / 1,124 (14.9%)
Inherited share of worker-token equivalents0%9.6%
Live worker tokens1.32M1.27M (−3.7%)

The store transfers well on a search-based method, and it helps performance. But it finds little to reuse without workflow inheritance. The reuse a search method can obtain is set by how much of the workflow its operator keeps between candidates: nothing under independent synthesis, a few agents under TacoMAS's fast and slow updates, and much more on Inherit-MAS. That is the specific benefit of inheritance-guided editing we claim, and the one a cache alone cannot supply.

W3: Ordinary refinement always continues from the most recently completed candidate.

To show the effectiveness of the search policy, we ran the ablation of refinement from highest-ranked-parent against latest-parent, with other settings unchanged.

BenchmarkLatest-parent: reported Qwen referenceHighest-ranked-parent
WorkBench (completion)46.2%48.5%
HotpotQA (joint F1)42.743.0

Inheriting from the highest-ranked parent provides modest gains against the latest parent, and spends 4% and 11% more worker tokens. We also observed that refinement from a parent whose official score falls below an earlier candidate accounts for 3.7% of overall attempts. Such continuations therefore occupy a minority of the refinement budget, and their occurrence alone does not establish that those attempts were wasted.

These results support latest-parent as a competitive default setting, balancing sequential refinement and execution reuse. Earlier candidates remain available for final return under the archive ranking.

AuthorsOpenReview ↗

Rebuttal - Part 2

W4: The system-level quality–cost advantage would benefit from more competitive reference points.

We address this concern in the four points below.

1. Inherit-MAS already spends less than the evolving baselines and scores higher. Tokens are totals in millions over the full test sets (Table 7):

WorkerMeta/judgeTotalCostScore
WorkBench
EvoMAS168.485.11173.59$36.0030.8
TacoMAS222.4114.18236.59$50.7234.6
Inherit-MAS1.448.9210.35$8.0755.4
HotpotQA
EvoMAS103.818.47112.28$29.7439.2
TacoMAS9.008.3717.37$13.8747.8
Inherit-MAS3.604.598.18$6.2849.7

The overall cost of Inherit-MAS is low, although the meta/judge costs more in some cases. The baselines use 2.5 to 154 times more worker tokens, more meta/judge tokens in three of the four comparisons, and 2.1 to 22.9 times more tokens in total. The meta/judge share of Inherit-MAS is high on WorkBench because its worker usage is already small (1.44M), not because the controller is expensive.

2. Early stopping is optional, and the five-candidate budget is the default. The controller is flexible to adding early stopping. It can stop after any candidate without other changes. We fixed five candidates to match the baselines' budget, and it still costs fewer total tokens and performs better. We ran a small test on early stopping and found that it aligns with Inherit-MAS's core design and improves efficiency. Applying a simple rule to the stored trajectories (stop at the first valid candidate whose judge quality reaches a threshold) keeps 54.6% completion at 6.16M tokens on WorkBench (55.4% at 10.35M with five candidates) and 49.3 joint F1 at 7.14M on HotpotQA (49.7 at 8.18M).

3. Cost and the final ranking. The ranking makes no model call: it sorts judge scores that are already counted in the meta/judge column above. It chooses among candidates that have already been paid for, so cost is controlled by the budget and the stopping rule and not by the ranking. The total cost considers everything and is reported in Table 7 in the paper.

4. Repeated generation followed by selection under a matched budget. We ran this control on WorkBench: five independently generated workflows per task with the same judge, the same return rule, and the same budget. It completes 58.1% of tasks at $9.20, against 59.2% at $7.98 for Inherit-MAS with both inheritances. Inheritance therefore shows slightly better performance than repeated generation with selection at 13% lower cost.

W5: Support for stateful tasks relies on restrictive execution assumptions.

This constraint applies specifically to WorkBench, the only evaluated benchmark involving mutable state. WorkBench tasks are evaluated through an ordered sequence of state-changing operations, which our deterministic sink executes against the initial environment. Thus, the intermediate agents can perform their reasoning and read operations without modifying the environment. In HotpotQA, all operations are read-only, so this constraint does not apply. Together, these evaluations demonstrate that Inherit-MAS supports both stateful tasks with deferred state changes and stateless workflows, although workflows requiring interleaved reads and writes remain outside the scope of our current implementation.

AuthorsOpenReview ↗

Rebuttal - Part 3

Q1: How does the judge distinguish an unhelpful node from one whose useful output is not used correctly downstream? Controlled node keep/removal experiments?

  1. How the judge distinguish the two cases apart. The judge sees the workflow graph, every node's recorded output, tool events and errors, and the final output. A node whose output is correct but ignored downstream shows useful content in the trace, so the critique is written against the consumer, and the fix is an edit to the consumer. The selector discards a node only when the audit or its recorded output shows it contributes nothing or harms the result.

  2. Controlled node keep/removal experiments. We ran this kepp/removal control on a HotpotQA subset. For each workflow that Select judged, we removed each removable node in turn and re-executed only the nodes that depend on it. We restored every other node result from the record and applied no edits. The control re-executes the same nodes with nothing removed.

Select's decisionNodesJoint F1, node present (control)Joint F1, node removedRemovals that lower / raise joint F1
Discarded4348.648.34 / 4
Kept4647.838.815 / 3

As shown in the table, removing a node that Select discarded leaves the final result unchanged, whereas removing a node that Select retained reduces joint F1 by 8.9 points. To address the reviewer’s specific case, we examined removable nodes that uniquely retrieved a gold page, meaning a gold page retrieved by no other node. Their outputs therefore contained relevant evidence regardless of whether downstream nodes ultimately used it. Select retained most of these nodes. Every discarded instance came from a single question, which remained incorrect both with and without those nodes. Thus, Select can occasionally discard uniquely relevant evidence, but this failure mode was rare in our analysis and did not affect the final answer in the observed case.

Q2: Execution inheritance - Explain effect on remote request diversity and error propagation, recovery (incomplete, incorrect), and when to re-execute?

1. Effects on search diversity and error propagation. Execution inheritance fixes a node's output only while its complete request is unchanged. Search diversity comes from the judge-guided edits, not from resampling unchanged requests. With inheritance disabled, so that every node is resampled, quality does not improve: WorkBench completion is 55.4% in both settings, and HotpotQA joint F1 with Qwen3-32B workers is 42.8 against 42.7. We answer the other subquestions below.

2. How an incomplete or incorrect output is handled. After each candidate runs, the judge audits the final output and the recorded output of every node, and its critique names the nodes that did useful work and those that did not. When an output is incomplete or incorrect, this feedback guides the next round: Select discards the nodes judged unhelpful, and Edit applies one validated change that targets the reported defect. The result of a discarded or edited node is not carried forward, since a discarded node's result is dropped and an edited node executes live under its new request. Only kept nodes with unchanged requests inherit their results, and any recorded errors in them still count against the candidate in the ranking (Eq. 7). There is no separate recovery step for the first workflow. If the candidates remain weak after the refinement rounds, the final round re-synthesizes a different workflow and runs it live.

3. What justifies node re-execution. Re-execution is decided by the difference between the inherited workflow and the edited one. Nodes in the affected region containing every new or changed node and all nodes downstream of them (Eq. 3) execute live whenever their request has changed. All other nodes reuse their stored results after their complete request is verified. Selective re-execution is thus justified by the diagnosis: a node runs again when the judge-guided edit changes it or anything upstream of it.

AuthorsOpenReview ↗

Rebuttal - Part 4

Q3: In Table 2, Inherit-MAS leads in answer F1 but trails TacoMAS in supporting-fact F1. What explains this difference?

First of all, we clarify that the source code of TacoMAS only provides implementations for the design without valid execution harness for downstream tasks such as WorkBench and HotpotQA. Directly running the source code of it leads to broken results. In detail, TacoMAS's release produces a text answer and has no stateful executor, so we compile its final plan into official write calls (Appendix D.1). We believe our adaptation makes the reproduction and comparison fair and faithful.

Why Inherit-MAS leads in answer F1. This is because Inherit-MAS improves several capabilities better than TacoMAS during the evolving process, including turning the retrieved evidence into the exact answer (correct rate with complete or partial evidence), and grounding what it cites (citation precision). These post-retrieval capabilities are decoupled from the coverage of evidence searching that affects SP F1.

The table below shows each system's refinement from the first to the final workflow. Inherit-MAS improves exact-answer correctness on both evidence conditions and citation precision. TacoMAS trails on transferring evidence into correct exact answers. Refinement therefore improves what the gold-free judge can verify (whether an answer is supported by observed sentences, query quality, etc.).

What's improved (first workflow → final workflow)Inherit-MASTacoMAS
Exact-answer correctness (complete evidence)73.4 → 75.472.9 → 68.4
Exact-answer correctness (partial evidence)45.6 → 46.234.3 → 32.7
Citation precision75.1 → 79.676.2 → 78.4
Joint F144.8 → 49.946.2 → 47.8

Why Inherit-MAS trails in SP F1. We explain the trail by identifying two causes. Then we propose one useful revision to the existing implementation.

  • Cause 1: Inherit-MAS under-utilizes the supporting evidence budget. We observed that across all 4 allowed retrievals per task, Inherit-MAS uses only 1.8 on average, while TacoMAS uses 3.8. The main cause is that TacoMAS starts every task with a heavier topology that includes more researcher subagents than Inherit-MAS. The system prompt for each researcher never exposes the allowed retrieval times or explicitly asks for a second-hop search, so it stops voluntarily after 1-2 queries. While each researcher behaves similarly across the two methods, TacoMAS wins by integrating more subagent researchers that scale retrieval times, bringing more supporting evidence without affecting answer F1.
  • Cause 2: Inherit-MAS's current gold-free judge hardly identifies which supporting evidence is missing without seeing the gold answer. The judge can verify that cited sentences are observed and support the answer, but it cannot identify which sentences are missing without the gold answer. It does not provide feedback on "allowed vs used retrievals" due to the system prompt issue, either. Thus, refinement rounds did improve the overall coverage through better queries, but not more queries.
  • The above 2 causes are supported by the evidence in the table below. It shows that Inherit-MAS has more missed retrievals compared to TacoMAS, leading to the slightly lower SP F1.
Fate of each gold supporting sentenceInherit-MASTacoMAS
Cited51.4%56.4%
Related documents never retrieved23.8%20.1%
Other failures24.8%23.5%

Proposed fix: making the prompts retrieval-aware.

Because the deficit is unused allowance, we made two prompt changes that expose it, leaving everything else the same.

  • We make a research subagent's prompt state its retrieval allowance and ask for a follow-up search when a bridge entity or one part of the question is still unsupported by an observed sentence.
  • We report the 'used retrieval v.s. allowed per subagent' to the judge and the meta-LLM.

We validate the fix on the same test set. As shown in the table below, after making the prompts retrieval-aware, Inherit-MAS improves on SP F1, Answer F1, and Joint F1. After adapting it to TacoMAS, Inherit-MAS matches the SP F1 and further enlarges the Answer/Joint F1 gaps. This means the retrieval-aware prompts improve not only evidence coverage but also overall performance.

MetricInherit-MAS (Original)Inherit-MAS (Retrieval-aware prompts)TacoMAS (Retrieval-aware prompts)
Supporting-fact F161.767.767.7
Answer F173.375.264.6
Joint F149.754.548.9
Reviewer_eCkgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Inherit-MAS proposes a test-time workflow refinement framework with two forms of inheritance. Workflow inheritance starts from the latest completed candidate, optionally removes nodes judged unhelpful, and applies one validated edit to the retained workflow. Execution inheritance reuses eligible node results when their complete resolved requests and execution-context fingerprints match stored records. A separately prompted judge provides candidate scores and node-level feedback, and the final output is selected from the archive of completed candidates.

Critical strengths
  • Useful problem formulation.
  • Detailed implementation disclosure.
  • Good supporting analysis.
Critical weaknesses
  • Workflow inheritance is not adequately isolated from candidate generation, restart, and historical selection.
  • The execution mechanism’s contribution beyond exact-request memoization remains limited.
  • Baseline adaptations, unequal resource allocation, and missing uncertainty analyses limit the strength of the comparative conclusions.
Questions for authors
  1. What is new beyond applying exact-request memoization to workflow search?
  2. Why raw error count should take precedence over final-output validity?
  3. Intermediate agents only read a fresh copy of the initial environment, and a single deterministic sink performs all state-changing actions. This makes inheritance eligibility easier to establish, but does not cover workflows in which an agent writes, observes the resulting state, and conditions later reads or actions on that outcome.
  4. A stronger single-agent or fixed-workflow control should receive comparable access to the controller model, reflection, and candidate-selection budget.
  5. The term “Initial” should be standardized across Figure 3 and Appendix B.1. Figure 3 starts from the first candidate attempt, counting invalid proposals as failures, whereas the appendix’s “initial workflow” comparison uses the first successfully executed synthesis; this explains the different WorkBench gains of 9.2 versus 6.2 percentage points. The distinction is documented, but it should be visible in the figure caption so that recovery from invalid initial proposals is not confused with improvement over an executable initial workflow.
  6. Are there existing results for refinement without Select, without final-round restart, or with independent synthesis under the same judge and budget?
  7. What HotpotQA joint F1 was obtained when execution inheritance was disabled?
  8. Did Eq. (7) ever select an invalid final output over a valid candidate with more recorded internal errors?
  9. How do outcomes change when refining the highest-ranked candidate rather than the latest completed candidate?
5 comments
AuthorsOpenReview ↗

Rebuttal - Part 1

We appreciate the valuable feedback. We address each concern below.

W1: Workflow inheritance is not adequately isolated from candidate generation, restart, and historical selection.

Clarifying workflow inheritance versus candidate generation and historical selection. Our workflow inheritance mechanism operates at the node level, distinct from candidate generation and historical best-of-N selection (e.g., EvoMAS). In each ordinary refinement round, Select uses the judge’s node-level critique and execution trace to determine which components of the latest completed workflow to retain or discard (Section 4.3). Edit then applies one validated, diagnosis-guided modification to the retained workflow, preserving all other components. This enables targeted refinement while retaining useful prior structure. Separately, execution inheritance reuses stored node results only when the complete resolved request and execution context match a verified record (Section 4.4), avoiding redundant computation. Initial synthesis generates the first workflow, while a conditional final-round restart permits fresh synthesis without workflow inheritance. Historical candidate ranking (Eq. 7) serves only to select the final output and inform the exceptional restart; ordinary refinement always inherits from the latest completed candidate, regardless of its historical rank.

To distinguish workflow inheritance from candidate generation, restart, and historical selection, we analyze these mechanisms separately using the existing trajectories, as summarized in the table below.

Candidate pool (GPT-4o-mini workers, same ranking)WorkBench completionHotpotQA joint F1
Initial workflow only49.244.7
+ inheritance rounds R1–R3 (no final round)54.649.0
+ final round, restart candidates excluded53.849.1
+ restart candidates (reported)55.449.7
  • Workflow inheritance: rounds R1–R3 add 5.4 and 4.3 points under the same ranking, with no restart.
  • Restart: adds 0.8 to 1.5 points on WorkBench and 0.6 to 0.7 on HotpotQA.
  • Candidate generation: workflows are synthesized from scratch only at initialization and in the restart. It is the entry point of the system rather than a part of workflow inheritance, and after it no workflow is generated from scratch except the final-round resynthesis.
  • Historical selection: the ranking is identical in every row. This is not encouraged to be removed from the workflow inheritance loop by design, we provide a study on selecting latest workflow candidate against best workflow candidate in the discussion below (refer to our answer to Q9).

In summary, these results show that workflow inheritance drives most of the improvement, while restart provides smaller additional gains. Candidate generation and historical selection serve separate roles, with the ranking policy held constant throughout.

AuthorsOpenReview ↗

Rebuttal Part 2

W2: The execution mechanism’s contribution beyond exact-request memoization remains limited.

We clarify that exact-request matching is a verification step in execution inheritance, rather than the full mechanism. Inherit-MAS goes beyond conventional memoization in three key aspects: how workflow refinement creates opportunities for reuse, how eligible results are identified and verified, and how inherited results are integrated into subsequent execution. We elaborate on these distinctions from three perspectives below.

  1. Execution reuse is coupled with workflow inheritance, enabling substantial token savings. A stored result can be reused only when a subsequent candidate issues an identical request. Workflow inheritance creates these opportunities through Select and Edit, which preserve useful workflow components while modifying others. To quantify this effect, we compare workflow inheritance against independent workflow synthesis on WorkBench, using the same request-verified result store in both settings.
SettingWorker token reuse rate
Independent workflow synthesis + the same store0.35%
Workflow inheritance (Select + Edit) + the same store28%

According to the results, without workflow inheritance, independently generated candidates rarely produce matching requests, yielding only 0.35% reuse. In contrast, workflow inheritance preserves unchanged nodes, including their prompts, inputs, and tool configurations, while selectively modifying other components. This increases reuse to 28%, demonstrating that the token savings arise from coupling workflow inheritance with request-verified execution reuse, rather than memoization alone.

  1. Execution inheritance enables diagnosis-guided, selective reuse rather than blind caching. Conventional memoization reuses any matching request. In Inherit-MAS, reuse opportunities are deliberately shaped by execution feedback. The judge identifies useful nodes and deficiencies to address; Select preserves useful components, while Edit applies a targeted modification. Change localization then identifies the affected subgraph, including modified nodes and their downstream dependencies. This process guides which computations are preserved and which may require re-execution, while exact-request verification determines whether reuse is valid. Table 3 illustrates this distinction: prompt edits inherit 47.3% of worker-token equivalents on WorkBench and 75.0% on HotpotQA, compared with only 2.0% and 0.0% for restarts. Thus, workflow inheritance deliberately preserves useful computation, while execution inheritance avoids recomputing verified, unchanged requests. The two mechanisms work together to enable targeted refinement with lower inference cost.

  2. Execution inheritance preserves consistent evidence across refinement rounds, avoiding re-sampling variability. Identical requests do not necessarily produce identical outputs from remote LLMs, even at temperature zero. In our full-execution reruns, repeated requests within the same task produced byte-identical outputs only 54–57% of the time on WorkBench and 23% on HotpotQA. Without execution inheritance, unchanged nodes would be re-executed, introducing sampling variability that could confound the evaluation of workflow edits. Inherit-MAS instead verifies matching requests and restores their stored outputs, tool observations, and errors. This preserves consistent evidence for unchanged computations, allowing the judge to assess the effects of targeted edits with less interference from re-sampling noise.

AuthorsOpenReview ↗

Rebuttal - Part 3

W3: Baseline adaptations, unequal resource allocation, and missing uncertainty analyses limit the strength of the comparative conclusions.

1. Baseline adaptations. We aim to preserve the original evolution mechanisms of all baselines while ensuring comparable evaluation conditions. For EvoAgent and EvoMAS, we use their released implementations and preserve their core evolution procedures, including population management, mutation, crossover, and relevant hyperparameters, with benchmark-specific adaptations where necessary. We standardize model assignments where supported, using GPT-5.4-mini for meta-model/judge roles and GPT-4o-mini or Qwen3-32B for workers.

TacoMAS requires additional adaptation because its released implementation does not directly support the benchmarks' required execution and output interfaces. On WorkBench, we preserve its native evolution controller and compile its final plan into official write operations executed by the benchmark's deterministic executor. On HotpotQA, we adapt its evolution mechanisms to the shared runtime and structured answer format. These adaptations enable meaningful evaluation under the official metrics while preserving the core design of TacoMAS. We document the implementation details and deviations from the original releases in Appendix D.1.

2. Unequal resource allocation. The resource differences in our experiments favor the baselines: EvoMAS and TacoMAS consume 2.2×–6.3× more provider cost than Inherit-MAS, yet achieve lower performance on both benchmarks.

GPT-4o-mini workersWB tokensWB completionHP tokensHP joint F1
EvoMAS173.59M30.8112.28M39.2
TacoMAS236.59M34.617.37M47.8
Inherit-MAS10.35M55.48.18M49.7

This efficiency follows from our design: ordinary refinement preserves useful workflow components and applies a scoped edit, rather than evolving a population of candidates. Execution inheritance further eliminates redundant computation. Although these comparisons do not constitute a strictly matched-budget evaluation, the results demonstrate that Inherit-MAS achieves higher accuracy with substantially lower inference cost under the evaluated configurations.

3. Uncertainty analysis. To quantify uncertainty in the pairwise performance differences, we analyze the per-instance paired outcomes using exact two-sided tests. This is because all methods are evaluated on the same held-out instances, allowing the tests to control for instance difficulty and measure whether the observed advantage is consistent across instances rather than driven by a small subset. Specifically, we use McNemar’s exact test for WorkBench’s binary completion outcomes and an exact paired sign test for HotpotQA’s per-question joint-F1 differences. The results are listed below.

Inherit-MAS vs.WorkBench Δ\Delta completionpHotpotQA Δ\Delta joint F1p
Single ReAct+13.80.006+3.70.038
EvoAgent+21.5<0.001+14.9<0.001
EvoMAS+24.6<0.001+10.5<0.001
TacoMAS+20.8<0.001+1.90.69

The improvements over EvoAgent and EvoMAS are statistically significant on both benchmarks, as is the WorkBench improvement over TacoMAS. On HotpotQA, the observed +1.9 difference over TacoMAS is not statistically distinguishable from zero (p = 0.69).

Q1: What is new beyond applying exact-request memoization to workflow search?

Please refer to our reply to W2 above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.