acceptodds
Under review as a conference paper at ICLR 2027

MASOpt: Online Prompt Optimization for Multi-Agent Systems with Closed-Loop Control Theory

Abstract

Large language model (LLM)-based multi-agent systems (MAS) have emerged as a powerful paradigm for solving complex tasks through role specialization and inter-agent collaboration, while their performance remains highly sensitive to the prompts that govern individual agents and their interactions. Existing MAS prompt optimization methods mainly optimize prompts before deployment and then keep them fixed during execution, making them unable to adapt to errors and state changes revealed by the ongoing trajectory. In this work, we propose MASOpt, a framework for online prompt optimization in multi-agent systems, inspired by classical closed-loop feedback control. MASOpt treats each prompt modification as a control action whose effectiveness must be validated by the subsequent system response: a lightweight meta-MAS continuously diagnoses the current execution state, proposes local prompt repairs, and verifies their actual effects, allowing ineffective updates to be reverted and unresolved problems to trigger further optimization. To reduce online exploration, we further introduce Cross-Actuation Relative Advantage (CARA), which compares alternative repairs from matched execution states and learns state-conditioned repair preferences from historical trajectories without updating the underlying LLM parameters. Experiments across question answering, code generation, mathematical reasoning, and interactive decision-making benchmarks show that MASOpt achieves state-of-the-art performance in multi-agent prompt optimization.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multi-Agent Systems, Prompt Optimization, Closed-Loop Control Theory

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_bV8tOpenReview ↗
Rating
1(−1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes MASOpt, a framework that changes prompts during a task within a fixed multi-agent workflow. An observer diagnoses visible failures, a repair module edits an agent's prompt or interface instruction, and a feedback gate decides whether to keep the edit after replaying the affected part of the workflow.

Cross-Actuation Relative Advantage (CARA) compares candidate interventions from the same execution prefix during offline calibration. The resulting repair memory stays fixed during deployment. Prompt edits apply only to the current task.

Across six non-interactive benchmarks, MASOpt reports an average score of 83.40, versus 80.58 for MASPOB. Its average success across ALFWorld seen, ALFWorld unseen, and TextCraft is 64.85, versus 39.23 for AFlow. Additional experiments remove experience components and test different models and workflows.

Critical strengths
  • The repair procedure is documented in enough detail to inspect its decisions. Appendices B.2 and B.5 to B.7 specify which invocation receives an edit, record the original and edited prompts, and rerun affected downstream calls over the same comparison horizon. Appendix C.3 groups experience from each independent task. These records allow the effects of an edit to be traced and the repair process to be reproduced.

  • The interactive-task results show substantial performance gains. Table 2 reports 89.55% success on ALFWorld unseen tasks, versus 54.48% for the strongest baseline. MASOpt also solves 24/80 TextCraft tasks, versus MASPOB's 11/80. The controller keeps online observations separate from offline reward labels, and additional experiments test other models and workflows.

Critical weaknesses
  • The claimed guarantee is broader than the acceptance test supports. Property 1 says measured error cannot increase across checkpoints. Yet a new observation can reveal a new violation even when the prompts stay unchanged. Appendix B.7 only compares edits at the same saved state and horizon, and B.6 notes that replay results can vary. Property 1 should state these conditions. Appendix A.4 also says reducing measured error cannot sacrifice correctness. However, agents can agree on a wrong answer, and a program can pass public tests while failing hidden ones. Appendix F.2 acknowledges this limitation. The paper should measure how often accepted edits worsen final task performance.

  • The experiments do not show how much CARA adds beyond extra computation. Appendix A.6 reports 888 TextCraft calls for MASOpt versus 684 for MASPOB, about 29.8% more. Using the same model and environment-step limit still allows MASOpt to make more calls. Token and latency comparisons are also incomplete across benchmarks. Figure 3 removes experience components, but does not compare CARA with raw utility ranking or alternative repair locations using the same candidates and replay budget. The acceptance gate is not separately tested under these controls. Without these comparisons, the gains cannot be clearly attributed to CARA and the controller rather than extra computation.

Questions for authors
  • When does Property 1 hold, and how often do accepted edits worsen final task performance?

  • How does MASOpt compare with retries or feedback-based repair using the same inference budget and visible observations?

  • How is environment state restored for interactive replay, and do the 35/20-step limits include all replayed actions?

  • How much do CARA normalization, repair location, and the acceptance gate each add when candidates and replay budgets are matched?

  • Which task sets, initial prompts, and configurations explain the MBPP scores of 73.314 in Figure 3 and 84.16 in Table 1?

11 comments
AuthorsOpenReview ↗

Response to Reviewer bV8t: Guarantees, Costs, and Component Analyses

Part 1/5: Q1

Thank you for your detailed review and for recognizing the traceability of the repair procedure. The added controls support a higher average score under shared inference ceilings and a TextCraft advantage when all environment actions are charged. We narrow the guarantee to matched-checkpoint retention, quantify when accepted edits reduce final scores, and explain the MBPP configuration discrepancy. Your seven points are answered individually below.

Q1 (Weakness 1). The stated guarantee exceeds the acceptance test, and accepted edits may worsen final correctness.

A1: Thank you for identifying where the theoretical statement needs more precise conditions. We will restrict Property 1 to verified retention at a matched checkpoint: the controller keeps an edit only after a completed comparison demonstrates a lower public residual under the same state, horizon and checks. It also requires non-decreasing progress and valid constraint/budget checks. Otherwise, it retains the reference, as specified in Appendix B.7.

This design uses calibration on downstream development-task outcomes to guide local repair choices, but the acceptance rule itself establishes a local observable property. A later observation can reveal a new violation, and public-test agreement need not imply hidden-test correctness. We will state the matched-state conditions directly in Property 1 and align Appendix A.4 with the replay and scoring distinctions in Appendices B.6–B.7 and F.2.

To answer the empirical part, we replay accepted edits from the same checkpoint both with and without the edit and compare their final scores:

Table B1. Final-score decline after an accepted edit.

BenchmarkDecline rate (%)
HumanEval0.00
MBPP0.00
GSM8K0.00
MATH0.00
HotpotQA32.15
DROP25.33
ALFWorld (verified subset)0.00
TextCraft0.00

Accepted edits can satisfy the public acceptance test while reducing final correctness. The declines on HotpotQA and DROP demonstrate why Property 1 must be stated as a matched-checkpoint guarantee. The zero entries do not establish universal safety: the HumanEval, TextCraft and ALFWorld audits contain only four, two and six evaluable pairs, respectively. We will report the local property and final-task outcomes separately.

Part 2/5: Q2, Q3

Q2 (Weakness 2). Extra computation, incomplete cost reporting, and missing component controls prevent attribution to CARA and the controller.

A2: Thank you for asking us to separate resource use from component contributions. Following your suggestions, we report the shared-budget comparison and evaluate the component ablations separately. The original TextCraft result is 30.00% versus 13.75% success with 888 versus 684 calls (Appendix A.6, Table 4).

Following your suggestion, we compare MASPOB, fixed-prompt retry, feedback-based output repair and MASOpt using the same initial workflow, model, task inputs and visible feedback. Each benchmark uses common per-task call and total-token ceilings; diagnosis, generation, verification and replay are included in the budget. All scores are on a 0–100 scale, using the task metrics in Appendix A.1, Table 3.

The registered per-task ceilings (calls / total tokens) are HumanEval 27 / 35,097; MBPP 27 / 24,677; GSM8K 50 / 87,540; MATH 21 / 85,029; HotpotQA 30 / 55,776; DROP 29 / 18,795; both ALFWorld splits 644 / 2,086,652; and TextCraft 83 / 205,110.

Average is the unweighted mean of the nine displayed benchmark/split scores, with ALFWorld seen and unseen counted separately.

Table B2. Controlled task scores under shared inference ceilings.

BenchmarkMASPOBRetryOutput repairMASOpt
HumanEval91.2291.3790.8491.60
MBPP71.9971.5571.8571.85
GSM8K93.6591.0393.2293.84
MATH54.4252.4554.6354.84
HotpotQA73.8274.0460.8771.85
DROP81.2980.9358.5770.50
ALFWorld seen45.8644.1445.1471.25
ALFWorld unseen48.7546.5849.3076.83
TextCraft15.0012.5011.2518.50
Average64.0062.7359.5269.01

The results support a performance advantage under shared inference ceilings, with the clearest gains on interactive tasks. MASOpt achieves the highest average score even when the baselines receive feedback and correction opportunities. The advantage is not uniform: MASPOB remains stronger on MBPP and DROP, and retry leads on HotpotQA. This addresses the computation concern by showing where instruction adaptation adds value under the controlled budget, without claiming equal actual consumption or superiority on every benchmark.

To clarify actual resource consumption, Table B3 reports measured inference costs. All controller and replay calls are included in the token accounting.

Table B3. Measured inference cost for completed non-ALFWorld conditions.

BenchmarkRetry tokens/taskOutput repair tokens/taskMASOpt tokens/taskMASOpt median latency (s)
HumanEval13,04012,90213,535169.40
MBPP9,67210,0979,752120.39
GSM8K31,04728,80032,172132.33
MATH46,21241,75844,611111.87
HotpotQA22,94424,61822,74064.72
DROP8,3969,3509,640141.87
TextCraft90,71641,38390,035272.57

The cost results support a performance–cost trade-off rather than uniform savings. On TextCraft, MASOpt improves success with slightly fewer tokens than retry, but consumes more tokens than output repair. Shared ceilings therefore control the available budget without forcing equal actual expenditure. Together with Table B2, this addresses both performance and resource use; A6 (Part 4) separately examines the component contributions.


Q3 (Question 1). When does Property 1 hold, and how often do accepted edits worsen final performance?

A3: Thank you for requesting the precise condition and frequency. Property 1 applies to the recorded reference/candidate comparison at the same saved state, horizon and public checks, with both continuations completed and the gate conditions satisfied. Rejected candidates leave the incumbent unchanged. It does not extend that ordering to a new state or a newly sampled stochastic replay.

Table B1 (Part 1) in A1 reports the requested frequencies. The task-averaged decline rates are 32.15% on HotpotQA and 25.33% on DROP. No decline was observed in the evaluated HumanEval, MBPP, GSM8K, MATH and TextCraft pairs, or in the verified ALFWorld subset. These are conditional measurements of accepted edits, not whole-system error rates. They motivate reporting final correctness independently of the local residual guarantee.

Part 3/5: Q4, Q5

Q4 (Question 2). How does MASOpt compare with retries or feedback-based repair under the same inference budget and visible observations?

A4: Thank you for asking whether the gains persist with equal access to feedback and inference opportunities. Yes, MASOpt retains an aggregate advantage over both runtime baselines. In Table B2 (Part 2), the average gains over retry and output repair are 6.27 and 9.49 points, respectively, calculated before rounding.

The clearest benefits occur on the interactive tasks, while the comparison also identifies benchmarks where a simpler runtime baseline remains competitive. Because the baselines share the observations and resource ceilings and can already retry or revise outputs, their lack of correction opportunities cannot explain the overall advantage. The conclusion is an empirical benefit from instruction adaptation under these controls, with task-dependent gains and potentially different actual token consumption.


Q5 (Question 3). How is interactive state restored, and do the 35/20-step limits include every replayed action?

A5: Thank you for asking us to separate trajectory limits from total interaction costs. The environment is reconstructed by resetting the same native task and seed and executing the saved action prefix. No, the 35/20-step limits apply to the retained trajectory, not to all replayed actions. Before branching, the runner verifies the reconstructed public state, step count and state fingerprint. All six checked prefixes matched the saved checkpoints. Appendices B.2 and B.6 specify the checkpoint and replay scope.

The measured distinction between the retained trajectory and all executed actions is:

Table B4. MASOpt environment actions per task in the main controlled comparison.

BenchmarkRetained trajectoryAll executed actions
ALFWorld seen34.861,203.33
ALFWorld unseen34.761,209.43
TextCraft16.2751.92

Retained-trajectory length substantially understates total environment interaction. Prefix reconstruction and alternative continuations account for the gap, so they must be included when evaluating interaction cost. Additional diagnostic audits are accounted for separately. Following your suggestion, we impose a common ceiling on all TextCraft actions, including reconstruction and replay:

Table B5. TextCraft under a shared ceiling on all environment actions.

MethodSuccess (%)
MASPOB13.12
Retry12.50
Output repair11.25
MASOpt15.88

MASOpt retains its TextCraft advantage when reconstruction and replay actions are charged. It outperforms all three baselines under the total-action ceiling, so the gain in this setting is not explained solely by allowing extra environment interaction. Table B4 reports actual action use under the main inference-budget protocol, while Table B5 controls total actions directly. The corresponding ALFWorld comparison remains incomplete. We will make these accounting definitions explicit in Appendix A.3 and the cost analysis.

Reviewer_o8tVOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies online prompt optimization for multi-agent systems (MAS), aiming to dynamically improve individual agents' prompts during task execution rather than optimizing them only before deployment. The authors propose MASOpt, a closed-loop framework that uses a meta-MAS to diagnose execution errors, modify agent prompts, and verify the effectiveness of these modifications. Within this framework, a main part Cross-Actuation Relative Advantage (CARA) is proposed to learn state-conditioned preferences over prompt repair actions from historical trajectories to guide online prompt modifications more efficiently. Experiments across many different tasks, question answering, mathematical reasoning, code generation, and interactive long-term tasks demonstrate improvements over existing MAS prompt optimization methods. Compared with prior offline prompt optimization approaches, the main contribution is enabling runtime prompt adaptation based on execution feedback.

Critical strengths
  1. Most existing MAS prompt optimization methods focus on offline optimization before deployment. This paper investigates whether individual agents' prompts can be dynamically optimized during execution based on runtime feedback. This is an interesting direction, particularly for long-horizon interactive tasks where failures may emerge gradually and require online adaptation.

  2. The authors evaluate the proposed framework across diverse tasks, including question answering, mathematical reasoning, code generation, and long-horizon interactive decision-making.

  3. The paper provides a relatively comprehensive set of ablation studies to investigate the contributions of different components of the proposed framework.

Critical weaknesses
  1. Concerns about the fairness of experimental comparisons. My main concern is whether the reported performance improvements can be attributed to online prompt optimization itself rather than additional runtime computation and correction feedback opportunities. Existing baselines primarily optimize prompts offline and execute with fixed prompts, whereas MASOpt can detect failures, modify prompts, regenerate intermediate outputs, and perform additional verification during execution. For example, in code generation, the baseline produces multiple candidate solutions and tests them, while MASOpt may perform additional prompt repairs, regeneration, and testing after observing intermediate failures. Such additional opportunities could improve performance even without modifying system prompts. Moreover, MASOpt incurs higher inference costs in some settings (e.g., 888 versus 684 API calls for MASPOB on TextCraft). Therefore, the current comparisons do not clearly isolate the improvements attributable to online prompt optimization. A stronger evaluation would include compute-budget-matched baselines equipped with comparable runtime feedback, self-correction, and retry mechanisms, but without modifying system prompts.

  2. Insufficient clarity and consistency in MAS configurations. The paper does not clearly specify the detailed MAS configurations used for the two long-horizon interactive decision-making tasks, ALFWorld and TextCraft, including the number of agents, their roles, and their communication topology. More generally, different tasks use different MAS architectures, without sufficient justification for these choices. I understand that different tasks may naturally require different MAS designs. However, these variations introduce additional uncertainty into the experimental findings, making it difficult to determine how much the reported improvements depend on the particular MAS configurations rather than the proposed optimization method. It will be great to have more comprehensive experiments on some fixing MAS to further eliminate such uncertainty of the results.

Questions for authors
  1. How does MASOpt determine whether an intermediate execution step has failed? What feedback is available to the diagnosis and verification agents for each benchmark, and does any of this feedback require access to ground-truth answers or test outcomes? For many realistic tasks, an incorrect answer may not produce an observable error signal. How does MASOpt handle such cases, and how sensitive is its performance to inaccurate or incomplete failure detection?

  2. MAS configurations and experimental reproducibility. Could the authors clarify the MAS architectures used for ALFWorld and TextCraft, including the number of agents, their roles, and workflow topologies? What motivates the choice of different MAS configurations across tasks? How sensitive are the improvements to these architectural choices?

6 comments
AuthorsOpenReview ↗

Response to Reviewer o8tV: Fairness, Architecture, and Feedback

Part 1/3: Q1

Thank you for your constructive review and for requesting clearer controls and workflow definitions. MASOpt leads the shared-budget comparison on six of the nine benchmark/split entries and improves on Initial in 17 of the 24 architecture comparisons. We provide the configurations below and explain both the use and the limitations of public feedback. Each of your four points is addressed separately.

Q1 (Weakness 1). Are the gains attributable to prompt optimization, rather than additional computation and correction opportunities?

A1: Thank you for raising the concern about additional computation and correction opportunities. Following your suggestion, we add feedback-enabled baselines under common resource ceilings; the results favor MASOpt overall. The original TextCraft comparison reports 30.00% versus 13.75% success with 888 versus 684 calls (Appendix A.6, Table 4). The new experiment directly addresses the stronger fairness question.

Following your suggestion, we compare MASPOB, fixed-prompt retry, feedback-based output repair and MASOpt using the same initial workflow, model, task inputs and visible feedback. Each benchmark uses common per-task call and total-token ceilings; diagnosis, generation, verification and replay are included in the budget. All scores are on a 0–100 scale, using the task metrics in Appendix A.1, Table 3. In code generation, both runtime baselines can observe public test failures, and output repair can revise the program without changing its system prompt.

The registered per-task ceilings (calls / total tokens) are HumanEval 27 / 35,097; MBPP 27 / 24,677; GSM8K 50 / 87,540; MATH 21 / 85,029; HotpotQA 30 / 55,776; DROP 29 / 18,795; both ALFWorld splits 644 / 2,086,652; and TextCraft 83 / 205,110.

Average is the unweighted mean of the nine displayed benchmark/split scores, with ALFWorld seen and unseen counted separately.

Table O1. Feedback-enabled comparisons under shared inference ceilings.

BenchmarkMASPOBRetryOutput repairMASOpt
HumanEval91.2291.3790.8491.60
MBPP71.9971.5571.8571.85
GSM8K93.6591.0393.2293.84
MATH54.4252.4554.6354.84
HotpotQA73.8274.0460.8771.85
DROP81.2980.9358.5770.50
ALFWorld seen45.8644.1445.1471.25
ALFWorld unseen48.7546.5849.3076.83
TextCraft15.0012.5011.2518.50
Average64.0062.7359.5269.01

The results support a performance advantage under shared inference ceilings, with the clearest gains on interactive tasks. MASOpt achieves the highest average score even when the baselines receive feedback and correction opportunities. The advantage is not uniform: MASPOB remains stronger on MBPP and DROP, and retry leads on HotpotQA. This addresses the computation concern by showing where instruction adaptation adds value under the controlled budget, without claiming equal actual consumption or superiority on every benchmark.

Part 2/3: Q2

Q2 (Weakness 2). The interactive MAS configurations are unclear, and architectural variation may confound the findings.

A2: Thank you for asking how architectural differences are controlled. Following your suggestion, we compare MASOpt with Initial within each fixed workflow and extend the architecture analysis across the full benchmark suite. The controller is separate from the target-agent roles:

Table O3. Target roles and communication in the controlled interactive workflows.

TaskTarget rolesCommunication
ALFWorldTask interpreter; state tracker; planner; selectorTask contract once; state → planner → selector → environment each step
TextCraftState tracker; planner; selectorGoal, recipes and inventory → state → planner → selector → environment

The controller adapts instructions within a fixed role graph. This keeps the target-agent organization explicit and prevents the addition of the controller from being confused with a change to the underlying workflow.

The configurations are as follows. Original retains the submitted executable workflows for the six non-interactive benchmarks; for the added interactive controls, it denotes the role graphs in Table O3. Simplified removes the ensemble stage in HumanEval, MBPP, GSM8K and HotpotQA; merges passage solving and formatting in DROP; and uses two reasoning branches in MATH. Two-candidate generates two candidate solutions or answers for selection; in MATH, the two branches use reasoning and program execution, respectively. For ALFWorld and TextCraft, Simplified merges state tracking with planning, while Two-candidate produces two plans for the selector. The main paper's earlier architecture study is described in Section 5.3, Figure 5 and Appendix A.5; Table O4 is the expanded controlled comparison defined here. Within each architecture, Initial and MASOpt share the model, task inputs and resource limits.

Table O4. Paired MASOpt − Initial gains within each fixed workflow (percentage points).

BenchmarkOriginalSimplifiedTwo-candidate
HumanEval+1.53+1.91+1.15
MBPP+2.05+0.88+2.05
GSM8K+0.43−0.47−0.57
MATH+0.31−2.37−1.03
HotpotQA+1.73+2.10−0.85
DROP+9.18+0.79−4.85
ALFWorld seen/unseen+5.28+2.45+6.53
TextCraft+0.63−1.25+6.88

MASOpt improves multiple architectures, although the benefit depends on the task and information flow. HumanEval, MBPP and ALFWorld improve under all three variants, while several other tasks are more sensitive to the topology. TextCraft benefits most from the two-candidate design but declines under the simplified workflow. These within-architecture comparisons show that the method is not tied to one graph, while identifying architectural choices that can weaken its benefit. The ALFWorld row combines its seen and unseen splits.

Part 3/3: Q3, Q4

Q3 (Question 1). What detects an intermediate failure, does the feedback expose hidden answers, and how sensitive is MASOpt to missed or inaccurate detections?

A3: Thank you for asking how failures are detected when feedback is imperfect. Online diagnosis and verification use public task and execution evidence; hidden evaluation answers and private test outcomes are not supplied to the controller. Appendices A.4 and B.1–B.3 specify the feedback:

  • HumanEval and MBPP: syntax, exceptions, interfaces, and executions of examples/tests exposed in the problem statement.
  • GSM8K and MATH: answer parsing, symbolic validity, computation signals, and disagreements among workflow branches.
  • HotpotQA and DROP: supplied passages, answer types, evidence consistency, and message-schema checks.
  • ALFWorld and TextCraft: observations, admissible/valid actions, visible preconditions, inventory dependencies, and lack of progress.

For the role of calibration, development trajectories connect these observations to the downstream outcomes of candidate repairs. The resulting memory records useful and harmful interventions and is frozen before online execution (Appendices C.1–C.3). For example, an ALFWorld action may require opening a container first; a TextCraft recipe may require an intermediate material absent from inventory. The repair addresses the visible prerequisite, and the next environment response verifies whether it was satisfied. Appendix F.3 illustrates this mechanism. Calibration helps select a response to observable evidence; it does not supply otherwise hidden truth.

To evaluate detection itself, we use an independent model assessment of the evidence available at each checkpoint:

Table O5. Independent assessment of visible-evidence diagnoses.

BenchmarkMethodGrounded diagnosis (%)Visible-issue miss rate (%)
ALFWorld seenOutput repair24.8790.85
ALFWorld seenMASOpt25.2793.41
ALFWorld unseenOutput repair27.2787.86
ALFWorld unseenMASOpt28.0890.66
TextCraftOutput repair60.1448.17
TextCraftMASOpt62.0947.43

MASOpt's diagnoses are more often grounded in visible evidence, but better grounding does not ensure complete failure detection. The TextCraft assessment favors MASOpt on both measures, whereas ALFWorld still shows higher miss rates. This identifies detection coverage as a practical limitation. The task-averaged judgments concern public evidence, not hidden-answer correctness; output repair is the detector-level baseline because MASPOB does not expose a corresponding diagnostic output.

To test incomplete signals, we additionally mask semantic feedback:

Table O6. TextCraft success under masked semantic feedback.

Masked feedbackMASOpt (%)Output repair (%)
0%12.5011.25
10%11.8811.25
30%13.1211.25
50%11.8811.25

The TextCraft advantage persists under the tested levels of missing semantic feedback. Performance changes only modestly and remains above output repair, supporting tolerance to these masking conditions. This does not establish robustness to every feedback error: silent failures can remain undetected, and the attempted false-positive perturbation did not change any assessments. We will report this boundary explicitly. This sensitivity experiment is evaluated separately from Table O1 (Part 1).


Q4 (Question 2). What motivates the ALFWorld/TextCraft architectures, how are they initialized, and how sensitive are improvements to those choices?

A4: Thank you for requesting the role definitions, initialization and sensitivity analysis. The controlled interactive workflows initialize from the corresponding MASPOB seed prompts and compare methods on the same frozen role graph. Table O3 (Part 2) gives the roles and communication; Table O4 (Part 2) reports the effects of architecture.

ALFWorld adds a task interpreter to the state-tracking, planning and action-selection pipeline; TextCraft exposes recipes and inventory dependencies directly. All target roles use gpt-4o-mini-2024-07-18. Table O4 (Part 2) evaluates the simplified and two-candidate variants defined in Q2 (Part 2).

The architecture results in Table O4 (Part 2) support transfer beyond a single topology. ALFWorld improves under all three graphs, while simplifying TextCraft's information flow reduces its benefit. The role decomposition is therefore a meaningful design choice, rather than an incidental implementation detail. We will include the role prompts and graph definitions and distinguish the submitted workflows from the controlled variants.

Thanks again for your careful and valuable comments, hope our response addresses your concerns. If our responses answer your questions well and reassure your concerns, we would appreciate if you could reassess our work and increase the rating.

Reviewer_eSWCOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MASOpt, which adapts agent prompts during execution while keeping the underlying models and workflow fixed. A controller diagnoses failures, proposes local prompt edits, and retains them when observable errors decrease. Offline calibration uses CARA to compare repairs from matched execution states and build a reusable memory. They report experiments on 8 benchmarks and show improvements across all eight, with particularly large gains on interactive tasks.

Critical strengths
  • The proposed method addresses a practical limitation of fixed prompts. Failures become apparent during execution and may require changes to upstream instructions or agent interfaces. Comparing candidate repairs from the same saved prefix is a sensible way to reduce confounding from different preceding trajectories.
  • The evaluation covers diverse tasks and reports substantial interactive improvements. Separating offline reward-based calibration from online checks that exclude hidden labels is also a useful design choice for deployment.
Critical weaknesses
  • The experiments do not sufficiently separate the proposed method from additional inference and feedback-based retries. The main comparisons emphasise offline optimisers, while MASOpt adds diagnosis, repair, verification, and replay. The appendix reports 888 TextCraft calls versus 684 for MASPOB, but this cost accounting does not replace a comparison at matched computation. Also, removing diagnostic or verification experience does not isolate CARA's contribution over ranking repairs by unnormalised utility.
  • Equation 6 guarantees that an accepted repair improves the measured error relative to its comparator at a matched checkpoint. It does not by itself establish a non-increasing error sequence across an evolving episode, where later execution can expose new violations. Furthermore, reducing observable violations does not guarantee preserved task correctness, it cannot explain non-negative benchmark gains by itself.
Questions for authors
  1. Do the gains persist against a feedback-based retry or output-repair baseline given the same runtime observations and total call or token budget?
  2. What does CARA contribute beyond the paired utility in Equation 7 when candidate generation, memory size, and calibration budget are held fixed?
3 comments
AuthorsOpenReview ↗

Response to Reviewer eSWC: Budget Controls, CARA, and Guarantees

Part 1/2: Q1

Thank you for your careful review and for recognizing the value of runtime adaptation and matched-prefix comparisons. The additional experiments support a higher average score under common inference ceilings and an incremental benefit from CARA. We also clarify that the guarantee applies to the recorded matched-checkpoint comparison. We address your four points individually below.

Q1 (Weakness 1). The comparisons do not separate prompt optimization from additional inference, and the experience ablations do not isolate CARA.

A1: Thank you for raising the questions of computational fairness and CARA attribution. The additional comparison supports an overall advantage under shared inference ceilings, and the experience ablation separately examines how repairs are selected. In the original TextCraft setting, MASOpt achieves 30.00% success versus 13.75% for MASPOB, a 16.25-point gain, with 888 versus 684 calls. The additional comparison below addresses the stronger requirement of controlling inference opportunities.

Following your suggestion, we compare MASPOB, fixed-prompt retry, feedback-based output repair and MASOpt using the same initial workflow, model, task inputs and visible feedback. Each benchmark uses common per-task call and total-token ceilings; diagnosis, generation, verification and replay are included in the budget. All scores are on a 0–100 scale, using the task metrics in Appendix A.1, Table 3.

The registered per-task ceilings (calls / total tokens) are HumanEval 27 / 35,097; MBPP 27 / 24,677; GSM8K 50 / 87,540; MATH 21 / 85,029; HotpotQA 30 / 55,776; DROP 29 / 18,795; both ALFWorld splits 644 / 2,086,652; and TextCraft 83 / 205,110.

Average is the unweighted mean of the nine displayed benchmark/split scores, with ALFWorld seen and unseen counted separately.

Table E1. Controlled task scores under shared inference ceilings.

BenchmarkMASPOBRetryOutput repairMASOpt
HumanEval91.2291.3790.8491.60
MBPP71.9971.5571.8571.85
GSM8K93.6591.0393.2293.84
MATH54.4252.4554.6354.84
HotpotQA73.8274.0460.8771.85
DROP81.2980.9358.5770.50
ALFWorld seen45.8644.1445.1471.25
ALFWorld unseen48.7546.5849.3076.83
TextCraft15.0012.5011.2518.50
Average64.0062.7359.5269.01

The results support a performance advantage under shared inference ceilings, with the clearest gains on interactive tasks. MASOpt achieves the highest average score even when the baselines receive feedback and correction opportunities. The advantage is not uniform: MASPOB remains stronger on MBPP and DROP, and retry leads on HotpotQA. This addresses the computation concern by showing where instruction adaptation adds value under the controlled budget, without claiming equal actual consumption or superiority on every benchmark.

For experience construction, we compare single-model direct extraction, Untrained, raw/centered utility and CARA/Full. Untrained removes calibrated diagnostic and verification experience, including CARA-derived memory. The scoring-rule comparison holds candidate generation, memory capacity and calibration budget fixed, and evaluates the resulting memories with the same task-scoring protocol.

Table E2. Experience construction and scoring ablations.

BenchmarkDirect extractionUntrainedRaw / centered utilityCARA / Full
HumanEval91.6091.6091.6096.95
MBPP71.7070.8571.9273.14
GSM8K88.8392.6391.5894.41
MATH55.3563.2961.8463.58
HotpotQA70.4971.8571.8574.13
DROP69.6070.5076.2485.66
ALFWorld seen47.8650.7551.4375.00
ALFWorld unseen51.4352.6754.4889.55
TextCraft12.5012.5012.5030.00
Average62.1564.0764.8375.82

Calibrated experience contributes beyond direct extraction, and CARA provides a further benefit under the controlled scoring comparison. CARA/Full leads on every displayed benchmark/split, with particularly large improvements on interactive tasks. This supports using matched interventions to construct repair memory and using CARA to rank their evidence. A4 (Part 2) explains the normalization mechanism; the results establish a downstream benefit without isolating every step of that mechanism. Table E2 is a separate ablation setting from Table E1.

Part 2/2: Q2, Q3, Q4

Q2 (Weakness 2). Why should a locally accepted repair help the overall task, and what does Equation (6) actually guarantee?

A2: Thank you for asking us to distinguish the motivation from the guarantee. We agree that Equation (6) alone does not establish monotonic improvement over an evolving episode or guarantee final correctness. Its guarantee concerns the matched checkpoint comparison. Offline calibration provides the broader task-level evidence used to choose a local repair:

  • Diagnosis uses workflow context. The observer reads the task contract, upstream messages, role obligations, and visible execution evidence, rather than only the latest output (Appendices B.1–B.3).
  • Repair construction considers downstream dependencies. A repair intent specifies the behavior to restore and the constraints to preserve, then compares legal producer, interface, and receiver locations (Appendix B.4).
  • Calibration measures consequences. Candidate interventions start from the same saved prefix, and downstream execution is completed when needed to evaluate development-task reward. Useful and harmful outcomes are aggregated into the frozen memory (Appendices C.1–C.3). This is experience calibration, not backbone-weight training.

For the mathematical claim, Equation (6) guarantees the following recorded matched-state comparison, subject to completed replays, the same horizon and checks, non-decreasing progress, and valid budget/constraint checks:

Es,hrec(retained)≤Es,hrec(reference).E^{\mathrm{rec}}_{s,h}(\mathrm{retained})\leq E^{\mathrm{rec}}_{s,h}(\mathrm{reference}).

New observations may expose new violations, and satisfying a public constraint does not ensure final correctness. The apply/omit audit indeed finds final-score decline rates of 32.15% on HotpotQA and 25.33% on DROP among evaluable accepted-edit pairs, averaged by task. We will therefore make Property 1 and Appendix A.4 consistent with Appendices B.6–B.7: calibration supplies task-informed repair preferences; the gate certifies a local observed improvement; final task gains are established empirically.


Q3 (Question 1). Do gains persist against feedback-based retry or output repair with the same observations and call/token budget?

A3: Thank you for asking whether the gains persist with equal access to feedback and inference opportunities. Yes, MASOpt retains an aggregate advantage over both runtime baselines. In Table E1 (Part 1), the average gains over retry and output repair are 6.27 and 9.49 points, respectively, calculated before rounding.

The clearest benefits occur on the interactive tasks, while the comparison also identifies benchmarks where a simpler runtime baseline remains competitive. Because the baselines share the observations and resource ceilings and can already retry or revise outputs, their lack of correction opportunities cannot explain the overall advantage. The conclusion is an empirical benefit from instruction adaptation under these controls, with task-dependent gains and potentially different actual token consumption.


Q4 (Question 2). What does CARA add beyond the paired utility in Equation (7), with candidates, memory size and calibration budget fixed?

A4: Thank you for asking what CARA contributes beyond paired utility. Paired utility measures improvement over a matched no-operation response; CARA evaluates how distinctive that improvement is relative to the other interventions available at the same state. Equation (8) uses their mean and dispersion, while absolute reward, harm and support checks remain part of memory admission (Appendix C.2).

Following your suggestion, Table E2 (Part 1) holds candidate generation, memory capacity and calibration budget fixed. CARA/Full improves the average score over raw/centered utility by 11.00 points. Its consistent advantage across the displayed tasks supports an incremental benefit from the scoring rule, beyond simply providing calibrated experience.

The purpose of normalization is to distinguish the strength of the evidence for a repair from the general responsiveness of the checkpoint. The same absolute gain may be routine when most alternative edits help, but distinctive when the alternatives fail. Centering and scaling express this distinction before observations are aggregated into memory (Appendices C.2–C.3). The larger interactive gains are consistent with this motivation, although the ablation does not directly establish that mechanism. The combined raw/centered column also does not distinguish those two rules from each other. We will keep this targeted comparison separate from Untrained, which removes the full calibrated-experience pipeline.

Thanks again for your careful and valuable comments, hope our response addresses your concerns. If our responses answer your questions well and reassure your concerns, we would appreciate if you could reassess our work and increase the rating.

AuthorsOpenReview ↗

Looking Forward to Your Feedback (Reviewer eSWC)

Dear Reviewer eSWC, thank you again for pushing us to separate the effects of additional inference from the contributions of runtime adaptation and CARA. When you have a chance during the discussion period, we would appreciate hearing whether the comparisons under shared inference budgets, the CARA ablation, and our clarification of Equation (6) address your concerns. We look forward to your perspective and would be glad to clarify any point that remains open.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.