- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This work focuses on prompt optimization for multi-agent systems (MAS), particularly addressing the challenge of credit assignment in complex, interconnected agent systems. To improve optimization, the authors propose CEPO, a framework based on two principles: (1) coordinator-executor prompt decomposition, which separates each agent’s prompt into a shared coordination prompt and an agent-specific execution prompt; and (2) role-aligned optimization, which provides distinct optimization guidance for coordination and execution prompts. The proposed framework aims to improve prompt optimization by explicitly separating system-level coordination from agent-level execution. Experimental results across multiple domains demonstrate performance improvements when applying CEPO to existing prompt optimizers, compared with unoptimized systems.
Critical strengths
- Relevant and important research problem. Prompt optimization is important but challenging in multi-agent systems, particularly due to the difficulty of credit assignment in complex and tightly coupled agent interactions. Investigating how to improve credit assignment to guide more effective prompt optimization is a meaningful research direction, and this paper addresses this problem through structured prompt decomposition and role-aligned optimization.
Critical weaknesses
-
Insufficient justification of the proposed design principles. The two proposed principles are not sufficiently justified. Regarding the first principle, the assumption that all agents should share a common coordination prompt is not well motivated. In many real-world multi-agent systems, a shared coordination prompt is not necessary. For example, orchestration-based multi-agent systems are widely adopted in industry, where individual agents maintain their own prompts and coordination objectives are communicated through context rather than predefined shared instructions. Although the proposed decomposition may be applicable to certain multi-agent architectures, this strong architectural assumption limits the generalizability of the framework. Regarding the second principle, while distinguishing coordination-level and execution-level optimization is reasonable, the proposed role-aligned optimization primarily relies on LLM-based optimization guided by role-specific meta-instructions. The paper does not sufficiently justify how this mechanism achieves better credit assignment or provides generalizable insights for designing future optimization methods.
-
Insufficient experimental validation of the proposed framework’s superiority. Although the experimental results demonstrate performance improvements, the comparisons in Tables 2 and 3 do not fully establish the superiority of the proposed framework. In particular, Table 2 primarily compares CEPO-enhanced systems against their unoptimized counterparts, making the observed improvements less convincing given the additional optimization process. While Table 3 includes comparisons with MASPO, a more comprehensive evaluation against other existing prompt optimization approaches would strengthen the claims. Considering the paper’s focus on improving prompt optimization, comparisons with methods such as TextGrad (Yuksekgonul et al., 2024), MiCRO (Opsahl-Ong et al., 2024), and more recent approaches would provide stronger evidence of its effectiveness.
Questions for authors
- How does the proposed framework compare with existing prompt optimization methods for multi-agent systems, particularly under comparable optimization budgets?
- How sensitive is the proposed framework to the capabilities of the optimizer model? Specifically, what happens when a weaker model is used as the optimizer, particularly when its capabilities are comparable to or weaker than those of the executor models?
- How stable are the optimization results across different runs? Since verbalized prompt optimization can be less stable than numerical parameter optimization, evaluating its robustness across multiple runs is important. Could the authors report the mean performance and standard error over 3 or 5 independent optimization runs to demonstrate the stability and reliability of the proposed method?
2 comments
Dear Reviewer bX3n,
With one day remaining in the discussion period, we would appreciate your feedback on our response above, including the optimization-budget controls, weaker-optimizer experiments, and three-run stability results (Tables R10–R12), as well as our clarification of the shared coordinator and role-aligned guidance (W1). Could you let us know which concerns, if any, remain unresolved? We would be happy to provide further clarification.
Thank you for your time and consideration.
Response - (1)
Response Summary: Thank you for the detailed questions. We address orchestration and role-aligned guidance (W1, Tables R8 and R9) and provide optimization-budget controls (W2/Q1, Table R10). Added optimizer-model comparisons show gains over PE2 (Q2, Table R11, Appendix B.6). For MDocAgent, we report three-run means and standard errors (Q3, Table R12, Appendix B.7). The manuscript clarifies guidance in Section 3.2 and summarizes robustness in Section 4.3.
W1, architecture. Applicability to orchestration.
CEPO improves a system that coordinates through context, matching the orchestration mechanism described in the review (Table R8, paper Table 7). In LongVideoAgent, the master dynamically chooses which agent to invoke, constructs vision queries, and accumulates agent responses in its conversation context. Grounding and vision agents execute requests using their role prompts. Thus, coordination is communicated through requests and observations in context. CEPO improves this system by optimizing rules for requests, evidence reporting, and evidence sufficiency while retaining the same interaction and routing mechanism.
Table R8. Orchestration in LongVideoAgent.
W1, guidance. The design lesson.
Separate the optimized functions and provide feedback appropriate to each function. In our MDocAgent implementation, coordinator updates use the full interaction trace. Executor updates focus on the target agent's input and output with task and outcome feedback. The swap ablation shows that matching guidance to its role matters (Table R9). Our contribution here is a functional decomposition with corresponding optimization guidance, rather than a separate credit assignment algorithm. We clarified this feedback scope and design lesson in Section 3.2.
Table R9. Guidance alignment on MMLB.
W2/Q1. Existing optimizers and matched budgets.
CEPO's gain is not explained by additional optimization calls. We evaluate it with PE2, ProTeGi, OPRO, and MASPO. Table R10 shows that both comparable and lower budgets favor CEPO. This addresses the concern that extra optimization alone explains the improvement. These budget controls are already in paper Table 5 and Appendix B.1. We made PE2 explicit in the method labels and added the common optimizer comparison in Appendix B.5.
Table R10. Controls on optimization calls on MMLB. Iteration pairs denote (TC,TE). MAS calls count internal agent LLM API requests.
Q2. Weaker optimizers and the same model as executors.
CEPO remains effective with a weaker optimizer and when the optimizer uses the same model as the executors. We added both conditions on MDocAgent+MMLB (Table R11). All settings are identical to the default GPT-5.4 experiment, except for the optimizer model. The additional gains over PE2 are larger in these conditions than in the original GPT-5.4 setting. The observed benefit therefore does not require a stronger optimizer. We present this as the observed result on MMLB, without claiming that absolute performance is invariant to the optimizer model. We added these comparisons to Appendix B.6 and paper Table S12, with the main finding summarized in Section 4.3.
Table R11. Sensitivity to the optimizer model on MDocAgent+MMLB.
Q3. Stability across runs.
CEPO has higher mean accuracy on every benchmark across three independent optimization runs on MDocAgent (Table R12). Table R12 reports mean accuracy and standard error. We added the reporting protocol to Section 4.1 and the statistics to Appendix B.7 and paper Table S13, with a summary in Section 4.3.
Table R12. Stability across three independent optimization runs on MDocAgent.