- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies a planning agent and an acting agent working together on long tasks. The authors ask both agents to describe the current state, detect when their descriptions conflict, and let them revise their plans or actions. ConPAct uses this procedure during a task and also trains agents on examples of successful correction. The experiments cover four interactive environments and report both task results and measures of disagreement.
Critical strengths
- The paper gives a concrete way to observe when two agents disagree about the same task state. This makes a plausible source of failure easier to study.
- ConPAct substantially improves the delegated Plan-Act system on some tasks. For GPT-5.6-sol/terra on MiniGrid, success rises from 38.6% to 54.4%, although the stronger single-agent result limits what this gain establishes.
Critical weaknesses
-
Belief-state representations and communication for world-model alignment are established ideas, so the paper’s novelty rests on its empirical claim, and a confound weakens that claim. Reduced action conflicts do not by themselves imply better coordination, since conflicts can also fall when agents act less effectively, so reduced conflict alone cannot explain the gains. ConPAct also gives the actor richer state interpretations and corrective feedback from the stronger planner. Both can improve the actor’s decisions and reduce disagreement as a consequence. The paper does not separate coordination from information transfer.
-
The ReAct (Planner) baseline, which uses only the stronger model, reinforces this concern.
- E.g., on MiniGrid, GPT-5.6-sol alone reaches 54.5% SR. Delegating execution to terra lowers this to 38.6%, and ConPAct raises it back to 54.4%. The headline gain therefore only recovers what delegation lost.
- Across the primary metrics, ReAct (Planner) outperforms ConPAct-I in 14 of 16 comparisons. The results therefore cannot distinguish a real coordination benefit from the weaker actor simply being brought up to the stronger model's existing level.
The authors should add ablations that vary the communication token budget, with and without conflict resolution, to test whether consensus adds value beyond the stronger planner’s guidance.
Questions for authors
-
At equal communication-token budgets, how much does explicit conflict resolution improve success compared with giving the actor the same amount of planner guidance without a conflict check?
-
Can the authors separate better coordination from improved actor state understanding, e.g., by providing the same state information to the actor in both conditions?
-
When, if ever, does ConPAct-I outperform ReAct (Planner) under a matched model-call or monetary budget?
4 comments
Response to Reviewer JKtJ (2/3)
W2 & Q1: Matched Communication Budgets With vs. Without Explicit Conflict Resolution
Excellent point! Under equal communication rounds and call budgets, explicit conflict resolution adds substantial value beyond planner guidance or extra inference tokens alone. Our controlled ablations in Tables 18 and 19 directly test this by holding the communication budget fixed while removing programmatic conflict resolution:
- Same communication rounds without conflict pinpointing (No Conflict Feedback, Table 18): Both roles still exchange up to three rounds of planner guidance and actor feedback under the same trigger, but without programmatic identification of the conflicting state fields. Without explicit conflict pinpointing, residual state mismatch nearly triples and task success drops markedly across both environments (e.g., by nearly 9 points on MiniGrid for GPT-5.6-sol terra), showing that generic planner guidance cannot replace targeted conflict resolution.
- Explicit state reporting or extra call budgets without conflict discussion (Assertion & Resampling, Table 19): Likewise, having both roles output explicit state descriptions without a conflict check (Assertion) barely improves over standard Plan-Act, and spending the exact same extra call budget to redraw planner–actor responses whenever they disagree (Resampling) yields no gain at all—falling more than 18 points behind ConPAct-I on MiniGrid. Explicit conflict detection and field-level resolution are therefore the active ingredients driving the improvement.
Response to Reviewer JKtJ (3/3)
W2 & Q3: ConPAct-I vs. ReAct (Planner): Same-Model Gains and Matched Monetary-Budget Comparison
Another excellent point! We are glad you brought this up. ReAct (Planner) is a baseline we improve upon rather than an upper bound: when the same model serves both roles, ConPAct-I consistently beats single-agent ReAct, and when execution is delegated to a weaker actor, ConPAct-I outperforms ReAct (Planner) under a matched dollar budget in 5 of 6 comparisons.
First, in the same-model setting (GPT-5.6-sol sol and GPT-5.6-terra terra, Table 19), there is no capability gap between roles and thus no stronger-to-weaker information transfer. Yet ConPAct-I substantially outperforms each model’s own single-agent ReAct across both Sokoban and MiniGrid (by +8 to +20 points). This demonstrates that explicit state reconciliation yields a genuine coordination gain over the single-model agent itself, rather than merely recovering what delegation lost. We will move both Table 19 and the ReAct (Planner) baseline into the main tables (Tables 1 and 2).
Second, in the delegated setting (Exp 2), pairing a strong planner with a weaker actor is motivated by monetary cost: because ConPAct-I runs most calls on the cheaper actor while ReAct (Planner) queries the expensive planner at every step, unconstrained ConPAct-I is 25%–62% cheaper in USD per episode (OpenRouter pricing). To compare both methods at equal expenditure, we keep the task setup identical and cap ReAct (Planner)’s per-call response budget so its per-episode dollar spend matches ConPAct-I. Under this matched monetary budget, the ordering reverses: ConPAct-I outperforms ReAct (Planner) across all three model pairs on Sokoban and for both GPT-5.6 and Kimi on MiniGrid. The sole exception—Opus-5 Sonnet-5 on MiniGrid—is bottlenecked by the actor itself (Sonnet-5’s standalone ReAct and Plan-Act sit near the same score in Table 15), reflecting the widest planner–actor capability gap among all evaluated pairs.
Exp 2: Performance and Inference Cost (Full vs. Matched Dollar Budget)
Sokoban
| Planner Actor | Method | Success rate avg@8 (%) | Inference cost (USD/ep.) | Model calls/ep. | Output tokens/ep. |
|---|---|---|---|---|---|
| Opus-5 Sonnet-5 | ReAct (Planner) | 95.4 | 1.145 | 13.7 | 2,843 |
| Opus-5 Sonnet-5 | ConPAct-I | 95.4 | 0.476 | 31.2 | 7,087 |
| Opus-5 Sonnet-5 | ReAct (Planner), matched budget | 88.6 | 0.474 | 15.8 | 2,921 |
| GPT-5.6-sol GPT-5.6-terra | ReAct (Planner) | 83.8 | 1.296 | 16.4 | 4,183 |
| GPT-5.6-sol GPT-5.6-terra | ConPAct-I | 86.6 | 0.489 | 33.5 | 8,334 |
| GPT-5.6-sol GPT-5.6-terra | ReAct (Planner), matched budget | 76.1 | 0.488 | 19.2 | 3,841 |
| Kimi K3 Kimi K2.6 | ReAct (Planner) | 68.9 | 0.242 | 19.8 | 15,047 |
| Kimi K3 Kimi K2.6 | ConPAct-I | 65.8 | 0.152 | 37.7 | 16,513 |
| Kimi K3 Kimi K2.6 | ReAct (Planner), matched budget | 54.1 | 0.152 | 22.9 | 9,618 |
MiniGrid
| Planner Actor | Method | Success rate SR (%) | Inference cost (USD/ep.) | Model calls/ep. | Output tokens/ep. |
|---|---|---|---|---|---|
| Opus-5 Sonnet-5 | ReAct (Planner) | 60.0 | 0.719 | 16.3 | 3,142 |
| Opus-5 Sonnet-5 | ConPAct-I | 36.7 | 0.494 | 29.3 | 7,531 |
| Opus-5 Sonnet-5 | ReAct (Planner), matched budget | 42.8 | 0.494 | 18.5 | 2,904 |
| GPT-5.6-sol GPT-5.6-terra | ReAct (Planner) | 54.5 | 0.694 | 18.0 | 4,320 |
| GPT-5.6-sol GPT-5.6-terra | ConPAct-I | 54.4 | 0.473 | 29.7 | 8,226 |
| GPT-5.6-sol GPT-5.6-terra | ReAct (Planner), matched budget | 46.2 | 0.473 | 19.8 | 3,564 |
| Kimi K3 Kimi K2.6 | ReAct (Planner) | 42.3 | 0.196 | 19.9 | 15,124 |
| Kimi K3 Kimi K2.6 | ConPAct-I | 34.2 | 0.147 | 34.5 | 19,812 |
| Kimi K3 Kimi K2.6 | ReAct (Planner), matched budget | 32.1 | 0.147 | 21.8 | 9,679 |
Kind reminder to Reviewer JKtJ
Dear Reviewer JKtJ,
We sincerely appreciate your valuable comments! We understand that you may be too busy to check our rebuttal. We have clarified:
- How ConPAct separates bidirectional coordination from one-way information transfer, using the same-model setting and the Actor-only ablation (see Section 4 and Appendix F).
- We also clarified the ablations under matched communication budgets (see Section 4 and Appendix F).
- In addition, we added an analysis that separates agreement from state correctness (see Section 3 and Appendix D), as well as a cost–performance comparison with ReAct (Planner) at both full and matched monetary budgets (see Section 4 and Appendix H).
We sincerely hope that these refinements address your concerns. Please let us know if you have any further concerns, and we will be open to all possible discussions.
Best regards,
Authors of Submission 2275
Response to Reviewer JKtJ (1/3)
Thank you for your time devoted to reviewing this paper and your insightful comments! We are delighted that you find our framework provides a concrete way to make a plausible source of failure easier to study and that ConPAct substantially improves the Plan-Act system. Please find our response to your concerns below:
W1 & Q2: Separating Bidirectional Coordination from One-Way Information Transfer and Actor State Understanding
We are glad you bring up such an inspiring question! We would like to clarify that ConPAct’s gains come from bidirectional state reconciliation toward the true state, rather than one-way planner-to-actor information transfer, improved actor state reading alone, or passive conflict reduction:
Decomposition of Training Gains on Qwen3-VL-8B (Tables 2 & 20)
Exp 1: Separating Agreement from State Correctness (Expanding Table 14)