acceptodds
Under review as a conference paper at ICLR 2027

Consistent Plan-Act for Long-Horizon Agentic Tasks

Abstract

Long-horizon agentic tasks demand strong reasoning and efficient execution across successive interactions with dynamic environments. A common approach decouples high-level planning from low-level execution through separate planner and actor roles. To investigate coordination failures in these tasks, we prompt both agents for structured state assertions and compare their reports programmatically to detect explicit contradictions. Our analyses reveal systematic disagreement about the same task-relevant state facts, a phenomenon we term planner-actor state mismatch. We further find that providing agents with task-relevant state information reduces mismatch and improves coordination and task performance. Based on the systematic analysis of the state mismatch, we propose Consistent Plan-Act (ConPAct), which feeds detected contradictions back to both agents to form consistent state interpretations and fine-tunes them on curated consistent interactions for better coordination. ConPAct improves performance across various environments and model configurations, e.g., increasing MiniGrid success rate from 38.6% to 54.4% with GPT-5.6-sol/terra as planner and actor respectively, demonstrating that state consistency can guide both inference-time correction and coordination training.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Planner–Actor State Mismatch, Plan-Act Agent, Long-Horizon Agentic Task

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_JKtJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies a planning agent and an acting agent working together on long tasks. The authors ask both agents to describe the current state, detect when their descriptions conflict, and let them revise their plans or actions. ConPAct uses this procedure during a task and also trains agents on examples of successful correction. The experiments cover four interactive environments and report both task results and measures of disagreement.

Critical strengths
  1. The paper gives a concrete way to observe when two agents disagree about the same task state. This makes a plausible source of failure easier to study.
  2. ConPAct substantially improves the delegated Plan-Act system on some tasks. For GPT-5.6-sol/terra on MiniGrid, success rises from 38.6% to 54.4%, although the stronger single-agent result limits what this gain establishes.
Critical weaknesses
  1. Belief-state representations and communication for world-model alignment are established ideas, so the paper’s novelty rests on its empirical claim, and a confound weakens that claim. Reduced action conflicts do not by themselves imply better coordination, since conflicts can also fall when agents act less effectively, so reduced conflict alone cannot explain the gains. ConPAct also gives the actor richer state interpretations and corrective feedback from the stronger planner. Both can improve the actor’s decisions and reduce disagreement as a consequence. The paper does not separate coordination from information transfer.

  2. The ReAct (Planner) baseline, which uses only the stronger model, reinforces this concern.

    • E.g., on MiniGrid, GPT-5.6-sol alone reaches 54.5% SR. Delegating execution to terra lowers this to 38.6%, and ConPAct raises it back to 54.4%. The headline gain therefore only recovers what delegation lost.
    • Across the primary metrics, ReAct (Planner) outperforms ConPAct-I in 14 of 16 comparisons. The results therefore cannot distinguish a real coordination benefit from the weaker actor simply being brought up to the stronger model's existing level.

The authors should add ablations that vary the communication token budget, with and without conflict resolution, to test whether consensus adds value beyond the stronger planner’s guidance.

Questions for authors
  1. At equal communication-token budgets, how much does explicit conflict resolution improve success compared with giving the actor the same amount of planner guidance without a conflict check?

  2. Can the authors separate better coordination from improved actor state understanding, e.g., by providing the same state information to the actor in both conditions?

  3. When, if ever, does ConPAct-I outperform ReAct (Planner) under a matched model-call or monetary budget?

4 comments
AuthorsOpenReview ↗

Response to Reviewer JKtJ (1/3)

Thank you for your time devoted to reviewing this paper and your insightful comments! We are delighted that you find our framework provides a concrete way to make a plausible source of failure easier to study and that ConPAct substantially improves the Plan-Act system. Please find our response to your concerns below:

W1 & Q2: Separating Bidirectional Coordination from One-Way Information Transfer and Actor State Understanding

We are glad you bring up such an inspiring question! We would like to clarify that ConPAct’s gains come from bidirectional state reconciliation toward the true state, rather than one-way planner-to-actor information transfer, improved actor state reading alone, or passive conflict reduction:

  1. Inference-time coordination vs. one-way information transfer (Tables 18 & 19): In Table 19, when the same model plays both planner and actor, there is no capability gap and thus no stronger-to-weaker knowledge transfer, yet ConPAct-I still substantially outperforms single-agent ReAct. Moreover, in Table 18, Actor-only correction gives the actor the exact same planner state assertions and conflict feedback as ConPAct-I, while disabling only the planner’s revision turn. If the gain came from one-way state transfer to the actor, Actor-only correction would match ConPAct-I; instead, the mismatch repair rate collapses by nearly two-thirds and task success drops markedly, proving that both roles must jointly revise subgoals and actions.
  2. Training-time coordination vs. learning to read the state (Table 2 & 20): The same separation holds during training (table below). In Table 20, Baseline and ConPAct-R both train Qwen3-VL-8B on the exact same structured JSON state descriptions. While adding state descriptions alone (Plan-Act SFT →\to Baseline) improves task success by teaching the model to read the screen, adding bidirectional reconciliation supervision (Baseline →\to ConPAct-R) yields a further large gain across both environments. We will clarify this in our revision.

Decomposition of Training Gains on Qwen3-VL-8B (Tables 2 & 20)

Training StageState DescriptionsReconciliation & DiscussionSokoban avg@8 (%)Δ\Delta vs. Plan-Act SFTMiniGrid SR (%)Δ\Delta vs. Plan-Act SFTAttribution
Plan-Act SFT (Table 2)NoNo35.6—7.8—No state supervision
Baseline (Table 20)YesNo40.7+5.118.0+10.2Learning to read state
ConPAct-R (Table 20)YesYes49.1+13.5 (+8.4 over Baseline)24.8+17.0 (+6.8 over Baseline)Coordination & discussion
  1. Why reduced conflict reflects correct consensus rather than passive behavior (Table 23 & Exp 2): Against environment ground truth, Table 23 shows that ConPAct-I lowers disagreement specifically by raising both-correct agreement (CC) while reducing both-wrong agreement (CW). To directly test whether conflict reduction alone could explain the gain (Exp 1, expanding Table 14), we injected synthetic corrupted state descriptions into both roles at each handoff—either the identical corrupted state (Same-wrong) or two different corrupted states (Independent-wrong). While Same-wrong forces agreement (lowering DD) and raises subgoal compliance (HH), task success (SS) plunges well below the unguided baseline because the actor faithfully executes a plan built on a false state. Conflict reduction improves performance only when it aligns both roles on the correct state (Opus-5 and Oracle).

Exp 1: Separating Agreement from State Correctness (Expanding Table 14)

State SourceAgreementCorrectnessDD (%)AplanA^{\text{plan}} (%)AactA^{\text{act}} (%)HH (%)SS (%)
NoneLowLow61.952.024.177.347.2
Same-wrongHighLow14.68.37.181.221.4
Independent-wrongLowLow78.39.16.869.428.7
Opus-5HighHigh21.378.279.190.880.1
OracleHighHigh7.392.090.590.884.1
AuthorsOpenReview ↗

Response to Reviewer JKtJ (2/3)

W2 & Q1: Matched Communication Budgets With vs. Without Explicit Conflict Resolution

Excellent point! Under equal communication rounds and call budgets, explicit conflict resolution adds substantial value beyond planner guidance or extra inference tokens alone. Our controlled ablations in Tables 18 and 19 directly test this by holding the communication budget fixed while removing programmatic conflict resolution:

  • Same communication rounds without conflict pinpointing (No Conflict Feedback, Table 18): Both roles still exchange up to three rounds of planner guidance and actor feedback under the same trigger, but without programmatic identification of the conflicting state fields. Without explicit conflict pinpointing, residual state mismatch nearly triples and task success drops markedly across both environments (e.g., by nearly 9 points on MiniGrid for GPT-5.6-sol →\rightarrow terra), showing that generic planner guidance cannot replace targeted conflict resolution.
  • Explicit state reporting or extra call budgets without conflict discussion (Assertion & Resampling, Table 19): Likewise, having both roles output explicit state descriptions without a conflict check (Assertion) barely improves over standard Plan-Act, and spending the exact same extra call budget to redraw planner–actor responses whenever they disagree (Resampling) yields no gain at all—falling more than 18 points behind ConPAct-I on MiniGrid. Explicit conflict detection and field-level resolution are therefore the active ingredients driving the improvement.
AuthorsOpenReview ↗

Response to Reviewer JKtJ (3/3)

W2 & Q3: ConPAct-I vs. ReAct (Planner): Same-Model Gains and Matched Monetary-Budget Comparison

Another excellent point! We are glad you brought this up. ReAct (Planner) is a baseline we improve upon rather than an upper bound: when the same model serves both roles, ConPAct-I consistently beats single-agent ReAct, and when execution is delegated to a weaker actor, ConPAct-I outperforms ReAct (Planner) under a matched dollar budget in 5 of 6 comparisons.

First, in the same-model setting (GPT-5.6-sol →\rightarrow sol and GPT-5.6-terra →\rightarrow terra, Table 19), there is no capability gap between roles and thus no stronger-to-weaker information transfer. Yet ConPAct-I substantially outperforms each model’s own single-agent ReAct across both Sokoban and MiniGrid (by +8 to +20 points). This demonstrates that explicit state reconciliation yields a genuine coordination gain over the single-model agent itself, rather than merely recovering what delegation lost. We will move both Table 19 and the ReAct (Planner) baseline into the main tables (Tables 1 and 2).

Second, in the delegated setting (Exp 2), pairing a strong planner with a weaker actor is motivated by monetary cost: because ConPAct-I runs most calls on the cheaper actor while ReAct (Planner) queries the expensive planner at every step, unconstrained ConPAct-I is 25%–62% cheaper in USD per episode (OpenRouter pricing). To compare both methods at equal expenditure, we keep the task setup identical and cap ReAct (Planner)’s per-call response budget so its per-episode dollar spend matches ConPAct-I. Under this matched monetary budget, the ordering reverses: ConPAct-I outperforms ReAct (Planner) across all three model pairs on Sokoban and for both GPT-5.6 and Kimi on MiniGrid. The sole exception—Opus-5 →\rightarrow Sonnet-5 on MiniGrid—is bottlenecked by the actor itself (Sonnet-5’s standalone ReAct and Plan-Act sit near the same score in Table 15), reflecting the widest planner–actor capability gap among all evaluated pairs.

Exp 2: Performance and Inference Cost (Full vs. Matched Dollar Budget)

Sokoban

Planner →\rightarrow ActorMethodSuccess rate avg@8 (%)Inference cost (USD/ep.)Model calls/ep.Output tokens/ep.
Opus-5 →\rightarrow Sonnet-5ReAct (Planner)95.41.14513.72,843
Opus-5 →\rightarrow Sonnet-5ConPAct-I95.40.47631.27,087
Opus-5 →\rightarrow Sonnet-5ReAct (Planner), matched budget88.60.47415.82,921
GPT-5.6-sol →\rightarrow GPT-5.6-terraReAct (Planner)83.81.29616.44,183
GPT-5.6-sol →\rightarrow GPT-5.6-terraConPAct-I86.60.48933.58,334
GPT-5.6-sol →\rightarrow GPT-5.6-terraReAct (Planner), matched budget76.10.48819.23,841
Kimi K3 →\rightarrow Kimi K2.6ReAct (Planner)68.90.24219.815,047
Kimi K3 →\rightarrow Kimi K2.6ConPAct-I65.80.15237.716,513
Kimi K3 →\rightarrow Kimi K2.6ReAct (Planner), matched budget54.10.15222.99,618

MiniGrid

Planner →\rightarrow ActorMethodSuccess rate SR (%)Inference cost (USD/ep.)Model calls/ep.Output tokens/ep.
Opus-5 →\rightarrow Sonnet-5ReAct (Planner)60.00.71916.33,142
Opus-5 →\rightarrow Sonnet-5ConPAct-I36.70.49429.37,531
Opus-5 →\rightarrow Sonnet-5ReAct (Planner), matched budget42.80.49418.52,904
GPT-5.6-sol →\rightarrow GPT-5.6-terraReAct (Planner)54.50.69418.04,320
GPT-5.6-sol →\rightarrow GPT-5.6-terraConPAct-I54.40.47329.78,226
GPT-5.6-sol →\rightarrow GPT-5.6-terraReAct (Planner), matched budget46.20.47319.83,564
Kimi K3 →\rightarrow Kimi K2.6ReAct (Planner)42.30.19619.915,124
Kimi K3 →\rightarrow Kimi K2.6ConPAct-I34.20.14734.519,812
Kimi K3 →\rightarrow Kimi K2.6ReAct (Planner), matched budget32.10.14721.89,679
AuthorsOpenReview ↗

Kind reminder to Reviewer JKtJ

Dear Reviewer JKtJ,

We sincerely appreciate your valuable comments! We understand that you may be too busy to check our rebuttal. We have clarified:

  • How ConPAct separates bidirectional coordination from one-way information transfer, using the same-model setting and the Actor-only ablation (see Section 4 and Appendix F).
  • We also clarified the ablations under matched communication budgets (see Section 4 and Appendix F).
  • In addition, we added an analysis that separates agreement from state correctness (see Section 3 and Appendix D), as well as a cost–performance comparison with ReAct (Planner) at both full and matched monetary budgets (see Section 4 and Appendix H).

We sincerely hope that these refinements address your concerns. Please let us know if you have any further concerns, and we will be open to all possible discussions.

Best regards,

Authors of Submission 2275

Reviewer_27SyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies planner–actor state mismatch and proposes ConPAct, which detects disagreements between structured state assertions and addresses them through inference-time correction or training-time coordination. While the experiments are extensive, the novelty, motivation, and contribution beyond existing coordination methods remain unclear.

Critical strengths
  1. The paper evaluates multiple environments, model families, and both inference-time and training-based methods.
  2. The controlled experiments show that task-relevant state information can improve execution, beyond simply adding tokens or changing the prompt format.
Critical weaknesses
  1. Planner–actor mismatch is related to existing work on inter-agent failures and belief alignment (Cemri et al., 2025; Li et al., 2023; Qiu et al., 2024; Dongre & Hakkani-Tür, 2026). Although the paper identifies its focus on state facts at planner–actor handoffs, it does not sufficiently explain the limitations of prior methods or why the proposed mechanism is necessary.
  2. The introduction mixes empirical findings, metric definitions, and method descriptions without a clear logical progression. For example, the observation that mismatch increases with the capability gap is not sufficiently analyzed, and the connection between this finding, the mismatch metric, and the proposed design remains unclear.
  3. The state-assertion fields appear manually designed, but their selection and sensitivity to alternative schemas are not investigated. Moreover, the contribution of programmatic conflict detection over ordinary discussion or feedback-based correction remains unclear.
  4. The selected baselines mainly compare different agent workflows and training strategies, rather than methods specifically designed to detect and resolve inter-agent inconsistencies. This makes it difficult to determine whether the gains come from the proposed mismatch-specific mechanism or simply from additional discussion and inference-time computation.
Questions for authors
  1. What specific limitations of existing multi-agent misalignment methods does ConPAct address?
  2. How does the observed relationship between capability gap and mismatch motivate the proposed design?
  3. Why were these state-assertion fields selected, and how sensitive are the results to alternative schemas?
8 comments
AuthorsOpenReview ↗

Response to Reviewer 27Sy (1/5)

Thank you for your time devoted to reviewing this paper and your constructive suggestions! We are grateful that you find our evaluation extensive and appreciate that our controlled experiments demonstrate how task-relevant state information can improve execution. Please find our response to your concerns below:

W1 & Q1: Prior Multi-Agent Alignment Methods' limitations and Why ConPAct Is Necessary

We appreciate your insightful question! We are glad to clarify why prior multi-agent alignment methods fall short at planner–actor handoffs: they assume agents either have an external post-hoc judge (Cemri et al. [1]) or already know their information is asymmetric and can align beliefs through free-form dialogue (Li et al. [2]; Qiu et al. [3]; Dongre & Hakkani-Tür [4]). In hierarchical Plan-Act systems, however, both roles observe the exact same screen and communicate only through compact subgoals (e.g., "push the upper box right"). Because both roles implicitly assume they share the same perception, any visual misreading creates false common ground: the contradiction is hidden behind high-level subgoals, so neither role knows a perceptual conflict exists or which side is wrong.

This structural blind spot makes each component of ConPAct strictly necessary, as confirmed by our controlled ablations:

  1. Why structured state assertions are needed: In standard Plan-Act, state beliefs remain implicit, so perceptual discrepancies silently corrupt execution. However, merely prompting agents to output explicit belief states [2] without cross-checking (e.g., Assertion in Table 19; ECoT in Table 2) barely improves over Plan-Act, because each role still acts on its own unverified belief.
  2. Why programmatic conflict detection is needed over model-inferred belief alignment: Asking agents to infer belief discrepancies in dialogue or discuss without pinpointing the disputed fields (No Conflict Feedback in Table 18) drops MiniGrid success severely, because models frequently miss subtle spatial contradictions in free-form text.
  3. Why bidirectional reconciliation is needed: Because either role can misread the state, one-way correction (Actor-only in Table 18) collapses the repair rate drastically, e.g, from 98.0% to 37.7%. Programmatic field-level comparison is therefore essential to deterministically expose silent contradictions and focus both roles on re-inspecting the exact disputed facts.
AuthorsOpenReview ↗

Response to Reviewer 27Sy (2/5)

W2 & Q2: Logical Progression of the Introduction and How the Capability-Gap Finding Motivates ConPAct (1/2)

Thank you for pointing out the potential for misunderstanding in our Introduction! We are glad you brought this up: the capability-gap finding is the direct logical bridge between why Plan-Act delegation is used, why our mismatch metric is needed, and how our propsoed ConPAct works:

  1. Why pair models with a capability gap?: In practice, delegating execution from a strong planner to a weaker actor is done to reduce inference cost. We compare the monetary cost between methods in Exp 1 using OpenRouter API prices to estimate the cost. Specifically, we compare ConPAct-I against running the strong planner alone (ReAct (Planner)) under both unconstrained budgets and a matched dollar budget (where we keep the task setup identical and cap ReAct (Planner)’s per-call response budget so its per-episode USD spend matches ConPAct-I). Because ConPAct-I runs most calls on the cheaper actor, it cuts per-episode cost by 25%–62% at full budget, and outperforms ReAct (Planner) across nearly all pairs under matched dollar spend.

Exp 1: Performance and Inference Cost

Sokoban

Planner →\rightarrow ActorMethodSuccess rate avg@8 (%)Inference cost (USD/ep.)Model calls/ep.Output tokens/ep.
Opus-5 →\rightarrow Sonnet-5ReAct (Planner)95.41.14513.72,843
Opus-5 →\rightarrow Sonnet-5ConPAct-I95.40.47631.27,087
Opus-5 →\rightarrow Sonnet-5ReAct (Planner), matched budget88.60.47415.82,921
GPT-5.6-sol →\rightarrow GPT-5.6-terraReAct (Planner)83.81.29616.44,183
GPT-5.6-sol →\rightarrow GPT-5.6-terraConPAct-I86.60.48933.58,334
GPT-5.6-sol →\rightarrow GPT-5.6-terraReAct (Planner), matched budget76.10.48819.23,841
Kimi K3 →\rightarrow Kimi K2.6ReAct (Planner)68.90.24219.815,047
Kimi K3 →\rightarrow Kimi K2.6ConPAct-I65.80.15237.716,513
Kimi K3 →\rightarrow Kimi K2.6ReAct (Planner), matched budget54.10.15222.99,618

MiniGrid

Planner →\rightarrow ActorMethodSuccess rate SR (%)Inference cost (USD/ep.)Model calls/ep.Output tokens/ep.
Opus-5 →\rightarrow Sonnet-5ReAct (Planner)60.00.71916.33,142
Opus-5 →\rightarrow Sonnet-5ConPAct-I36.70.49429.37,531
Opus-5 →\rightarrow Sonnet-5ReAct (Planner), matched budget42.80.49418.52,904
GPT-5.6-sol →\rightarrow GPT-5.6-terraReAct (Planner)54.50.69418.04,320
GPT-5.6-sol →\rightarrow GPT-5.6-terraConPAct-I54.40.47329.78,226
GPT-5.6-sol →\rightarrow GPT-5.6-terraReAct (Planner), matched budget46.20.47319.83,564
Kimi K3 →\rightarrow Kimi K2.6ReAct (Planner)42.30.19619.915,124
Kimi K3 →\rightarrow Kimi K2.6ConPAct-I34.20.14734.519,812
Kimi K3 →\rightarrow Kimi K2.6ReAct (Planner), matched budget32.10.14721.89,679
AuthorsOpenReview ↗

Response to Reviewer 27Sy (3/5)

W2 & Q2: Logical Progression of the Introduction and How the Capability-Gap Finding Motivates ConPAct (2/2)

  1. Why the capability gap motivates our metric and ConPAct-I/S/R: While delegation saves cost, the capability gap between planner and actor sharply increases state mismatch at handoffs. To analyze what happens inside these mismatches, Exp 2 evaluates every base-agent handoff against environment ground truth and groups them into four correctness outcomes (Both right, Planner right / Actor wrong, Planner wrong / Actor right, and Both wrong). Across frontier pairs, most of all mismatches have exactly one role right and the other wrong, most often the strong planner right and the weaker actor wrong, which cuts task success by more than half.
    • Motivation for ConPAct-I: Because one side already holds the true state in most frontier mismatches, a ground-truth-free JSON conflict check can detect the discrepancy online and trigger bidirectional discussion to recover the correct state.
    • Motivation for ConPAct-S/R: Conversely, when small open-weight models (Qwen3-VL-8B) serve both roles, Exp 2 shows that both sides misread the screen ("one right, one wrong" is only 5.1%), motivating our training-time consistency distillation (ConPAct-S/R) to improve the state interpretation and role coordination.

To make this progression clear, we will revise the Introduction as follows: (i) Phenomenon (cost-driven delegation →\to capability gap →\to asymmetric state mismatch), (ii) Metric & Causal Diagnosis (structured state assertions and controlled interventions), and (iii) Method (ConPAct-I for online recovery and ConPAct-S/R for training).

Exp 2: Breakdown of State-Assertion Correctness in Base Agents

Planner →\rightarrow ActorState-assertion correctnessTask success rate SS (%)Proportion of handoffs (%)
Opus-5 →\rightarrow Sonnet-5Both wrong5.39.9
Opus-5 →\rightarrow Sonnet-5Planner wrong, actor right32.82.5
Opus-5 →\rightarrow Sonnet-5Planner right, actor wrong20.452.6
Opus-5 →\rightarrow Sonnet-5Both right60.435.0
GPT-5.6-sol →\rightarrow GPT-5.6-terraBoth wrong31.045.9
GPT-5.6-sol →\rightarrow GPT-5.6-terraPlanner wrong, actor right61.22.1
GPT-5.6-sol →\rightarrow GPT-5.6-terraPlanner right, actor wrong47.630.0
GPT-5.6-sol →\rightarrow GPT-5.6-terraBoth right79.322.0
Kimi K3 →\rightarrow Kimi K2.6Both wrong3.118.9
Kimi K3 →\rightarrow Kimi K2.6Planner wrong, actor right14.812.5
Kimi K3 →\rightarrow Kimi K2.6Planner right, actor wrong10.238.8
Kimi K3 →\rightarrow Kimi K2.6Both right28.929.8
Planner →\rightarrow ActorExactly one role right (% of all handoffs)Share of all mismatches (%)P(agree∣both wrong)P(\text{agree} \mid \text{both wrong}) (%)
Opus-5 →\rightarrow Sonnet-555.186.110.4
GPT-5.6-sol →\rightarrow GPT-5.6-terra32.151.935.3
Kimi K3 →\rightarrow Kimi K2.651.376.115.9
Qwen3-VL-8B →\rightarrow 8B5.16.183.1
Reviewer_imJ1OpenReview ↗
Rating
2
Confidence
3
Informativeness
3(+1)
Presentation
3
Soundness
2
Read review
Paper summary

The paper looks at agents with two roles. A planner sets short goals, and an actor takes the actions. Both roles are asked to describe the current screen in a fixed JSON format, and the authors compare the two descriptions with code. They find that the two roles often disagree about the same screen. They call this "state mismatch". They then propose ConPAct. When the two descriptions disagree, ConPAct sends the disagreeing fields back to both roles, and the roles talk it over before the next action. The authors also use this process to build training data and fine-tune Qwen3-VL models. They test on Sokoban, Crafter, Procgen, MiniGrid/BabyAI and OSWorld.

Critical strengths
  1. The problem is easy to understand. The examples in Appendix G show clearly how one misread cell can break a plan.
  2. The experiments are broad. They cover five environments, four model families, and both inference and training.
  3. The method needs no ground truth to find disagreements. The paper also checks whether the roles agree on the right answer, not just whether they agree (Table 23).
Critical weaknesses
  1. An important baseline is missing from the main tables: the planner model working alone with ReAct. Its results are only in Table 15, in grey. It often beats ConPAct-I. On MiniGrid, it scores 60.0 vs. 36.7 with Opus-5, 42.3 vs. 34.2 with Kimi, and 22.8 vs. 13.5 with Qwen. On Sokoban, it scores 68.9 vs. 65.8 with Kimi and 22.6 vs. 15.7 with Qwen. It is also cheaper: 14 calls per episode, while ConPAct-I uses 33 (Table 24). So why pair a strong planner with a weaker actor at all? Please compare cost and performance together. When the same model plays both roles, ConPAct-I does beat ReAct (Table 19). This case may be the stronger story.

  2. It is hard to tell mismatch apart from simple misreading. In Table 10, Qwen3-VL-8B describes the state fully correctly only 2.6% (planner) and 3.3% (actor) of the time. So its 83.6% mismatch mostly means that both roles read the screen wrong. Even GPT-5.6-sol is right only about half the time. The claim that 85% of failed runs contain a mismatch is also weak. Mismatch already happens at 40-90% of handoffs, so almost every long run will contain one. Fig. 2b shows the same thing: 82.9% of runs where the actor fully follows the plan also contain a mismatch. For the base agents, please show how often one role is right and the other is wrong. Please also compare mismatch rates in successful and failed runs.

  3. The intervention experiments do not separate "agreeing" from "being right". In Tables 13 and 14, both roles get correct state information, so they agree and are right at the same time. We cannot tell which one helps. A simple test is missing: give both roles the same wrong description. If agreement alone helps, success should still go up. Also, in Table 13, the rate of handing back blocked goals jumps from 0.4% to 94.5%. The control is just unrelated text of the same length. If the Opus-5 facts say directly that the goal is blocked, they give away the answer. Please show some examples of these facts.

  4. The training gains may come from learning to read the state, not from better teamwork. ConPAct-S/R train the model to write state descriptions, and the filtering likely keeps the correct ones. State accuracy goes from about 45% to 83-94% (Table 21). On MiniGrid, the front_cell field is almost the decision itself. A baseline is needed: Plan-Act fine-tuning with state descriptions but no discussion. Please also say whether the "Baseline" in Table 20 includes state descriptions. Finally, the ConPAct-S data comes from a stronger teacher setup than the ReAct data, so the data quality is not the same.

Questions for authors
  1. Can you show examples of the Opus-5 facts used for blocked goals in Table 13?
  2. Does the "Baseline" in Table 20 include state descriptions?
  3. How much do the results depend on the JSON format? For example, what happens with fewer fields?
  4. Please explain the four number mismatches in Minor 1.
5 comments
AuthorsOpenReview ↗

Response to Reviewer imJ1 (1/5)

Thank you for your time devoted to reviewing this paper and your constructive comments! We are delighted that you find our presentation clear and easy to follow, our studied problem easy to understand, our method needs no ground truth to detect disagreements, and our experiments broad. Please find our response to your concerns below:

W1 & Q4: Cost–Performance Comparison Between ReAct (Planner) and ConPAct-I

Thank you for bringing up this important point and for the great suggestion! Pairing a strong planner with a weaker actor is driven by monetary cost: ConPAct-I matches or beats ReAct (Planner) under a matched dollar budget while costing significantly less at full budget. Meanwhile, when cost is not a constraint and the same strong model plays both roles, ConPAct-I outperforms that model’s own ReAct across both environments (Table 19). We agree this same-model result is the stronger headline story and will move the ReAct (Planner) rows into the main table.

Specifically, ReAct (Planner) is a baseline we improve upon, not an upper bound. In the same-model setting (Table 19), there is no capability gap between roles, ruling out one-way knowledge transfer. Table 18 further confirms that the gains come from bidirectional coordination: Actor-only correction (pure planner-to-actor guidance) and discussion without conflict-specific feedback both sharply degrade repair rate and task success.

In the delegated setting, raw call counts are misleading because most ConPAct-I calls run on the cheaper actor, whereas ReAct (Planner) calls the expensive planner at every step. Measured in actual API spend (OpenRouter USD/episode; Exp 1), unconstrained ConPAct-I is much cheaper per episode across all pairs. When we match per-episode dollar spend by capping ReAct (Planner)’s per-call response budget to ConPAct-I’s level, the ordering reverses: ConPAct-I outperforms ReAct (Planner) across all three pairs on Sokoban and for GPT-5.6 and Kimi on MiniGrid. The only exception is Opus-5 →\rightarrow Sonnet-5 on MiniGrid, which is bottlenecked by the actor itself, i.e., Sonnet-5’s standalone ReAct and Plan-Act sit near the same score (Table 15), reflecting the widest planner–actor capability gap among all pairs.

Exp 1: Performance and Inference Cost

Sokoban

Planner →\rightarrow ActorMethodSuccess rate avg@8 (%)Inference cost (USD/ep.)Model calls/ep.Output tokens/ep.
Opus-5 →\rightarrow Sonnet-5ReAct (Planner)95.41.14513.72,843
Opus-5 →\rightarrow Sonnet-5ConPAct-I95.40.47631.27,087
Opus-5 →\rightarrow Sonnet-5ReAct (Planner), matched budget88.60.47415.82,921
GPT-5.6-sol →\rightarrow GPT-5.6-terraReAct (Planner)83.81.29616.44,183
GPT-5.6-sol →\rightarrow GPT-5.6-terraConPAct-I86.60.48933.58,334
GPT-5.6-sol →\rightarrow GPT-5.6-terraReAct (Planner), matched budget76.10.48819.23,841
Kimi K3 →\rightarrow Kimi K2.6ReAct (Planner)68.90.24219.815,047
Kimi K3 →\rightarrow Kimi K2.6ConPAct-I65.80.15237.716,513
Kimi K3 →\rightarrow Kimi K2.6ReAct (Planner), matched budget54.10.15222.99,618

MiniGrid

Planner →\rightarrow ActorMethodSuccess rate SR (%)Inference cost (USD/ep.)Model calls/ep.Output tokens/ep.
Opus-5 →\rightarrow Sonnet-5ReAct (Planner)60.00.71916.33,142
Opus-5 →\rightarrow Sonnet-5ConPAct-I36.70.49429.37,531
Opus-5 →\rightarrow Sonnet-5ReAct (Planner), matched budget42.80.49418.52,904
GPT-5.6-sol →\rightarrow GPT-5.6-terraReAct (Planner)54.50.69418.04,320
GPT-5.6-sol →\rightarrow GPT-5.6-terraConPAct-I54.40.47329.78,226
GPT-5.6-sol →\rightarrow GPT-5.6-terraReAct (Planner), matched budget46.20.47319.83,564
Kimi K3 →\rightarrow Kimi K2.6ReAct (Planner)42.30.19619.915,124
Kimi K3 →\rightarrow Kimi K2.6ConPAct-I34.20.14734.519,812
Kimi K3 →\rightarrow Kimi K2.6ReAct (Planner), matched budget32.10.14721.89,679
AuthorsOpenReview ↗

Response to Reviewer imJ1 (2/5)

W2: State Misreading vs. Planner–Actor State Mismatch

Great observation! We are glad you brought this up, as it helps clarify the relationship between the two: misreading and mismatch are not competing explanations. Misreading is the underlying perceptual error, while mismatch is its ground-truth-free online signal. Because an agent has no oracle at test time, a single role cannot detect its own misreading. However, whenever the planner and actor disagree on a state field, at least one role must be wrong—turning a silent perception error into an explicit trigger for targeted re-inspection. Across frontier model pairs, most mismatches occur precisely when one role is right and the other is wrong, and controlling for trajectory length reveals that the per-handoff mismatch rate in failed runs is nearly double that of successful runs.

Specifically, Exp 2 breaks down base-agent handoffs by whether each role's state assertion is correct. For Opus-5 →\rightarrow Sonnet-5, GPT-5.6, and Kimi, "one right, one wrong" accounts for 51.9%–86.1% of all mismatches (32.1%–55.1% of all handoffs). When the planner is right but the actor is wrong, task success drops by more than half compared to when both are right (e.g., 60.4% →\to 20.4% for Opus/Sonnet; 79.3% →\to 47.6% for GPT). Because one side already holds the true state in these cases, inference-time reconciliation (ConPAct-I) can recover performance. Conversely, we agree with the reviewer that Qwen3-VL-8B mostly fails because both roles misread the screen ("one right, one wrong" is only 5.1%). This explains why zero-shot discussion (ConPAct-I) yields limited gains on 8B models, and directly motivates our training-time ConPAct-S/R, which teaches the model to read states accurately (raising state accuracy to 83%–94%, Table 21).

Comparing successful and failed runs in Exp 3, we also agree that the trajectory-level incidence (≥1\ge 1 mismatch) is high in both groups (85.2%–88.7% vs. >99%) simply because episodes span 13–32 handoffs. However, when we remove this length confound and measure the per-handoff mismatch rate (DD), failed runs exhibit nearly twice the mismatch rate of successful runs across all pairs. Finally, we clarify that the 85% figure cited in our paper (67.9%–85.3% across pairs, Table 12) does not merely count failed trajectories containing a mismatch; rather, it measures the fraction of failures where mismatch was identified as the direct cause of failure (Mismatch only + Both in Table 12). We will clarify this wording in the revision.

Exp 2: Breakdown of State-Assertion Correctness in Base Agents

Planner →\rightarrow ActorState-assertion correctnessTask success rate SS (%)Proportion of handoffs (%)
Opus-5 →\rightarrow Sonnet-5Both wrong5.39.9
Opus-5 →\rightarrow Sonnet-5Planner wrong, actor right32.82.5
Opus-5 →\rightarrow Sonnet-5Planner right, actor wrong20.452.6
Opus-5 →\rightarrow Sonnet-5Both right60.435.0
GPT-5.6-sol →\rightarrow GPT-5.6-terraBoth wrong31.045.9
GPT-5.6-sol →\rightarrow GPT-5.6-terraPlanner wrong, actor right61.22.1
GPT-5.6-sol →\rightarrow GPT-5.6-terraPlanner right, actor wrong47.630.0
GPT-5.6-sol →\rightarrow GPT-5.6-terraBoth right79.322.0
Kimi K3 →\rightarrow Kimi K2.6Both wrong3.118.9
Kimi K3 →\rightarrow Kimi K2.6Planner wrong, actor right14.812.5
Kimi K3 →\rightarrow Kimi K2.6Planner right, actor wrong10.238.8
Kimi K3 →\rightarrow Kimi K2.6Both right28.929.8
Planner →\rightarrow ActorExactly one role right (% of all handoffs)Share of all mismatches (%)P(agree∣both wrong)P(\text{agree} \mid \text{both wrong}) (%)
Opus-5 →\rightarrow Sonnet-555.186.110.4
GPT-5.6-sol →\rightarrow GPT-5.6-terra32.151.935.3
Kimi K3 →\rightarrow Kimi K2.651.376.115.9
Qwen3-VL-8B →\rightarrow 8B5.16.183.1

Exp 3: Mismatch Comparison Between Successful and Failed Runs

Planner →\rightarrow ActorOutcomeTrajectory share (%)Per-handoff mismatch DD (%)Trajectories with ≥1\ge 1 mismatch (%)Judge-attributed failure cause (%)Mean handoffs
Opus-5 →\rightarrow Sonnet-5Success33.241.288.7—13.1
Opus-5 →\rightarrow Sonnet-5Failed66.868.999.485.330.2
GPT-5.6-sol →\rightarrow GPT-5.6-terraSuccess47.238.485.2—14.5
GPT-5.6-sol →\rightarrow GPT-5.6-terraFailed52.871.799.179.730.9
Kimi K3 →\rightarrow Kimi K2.6Success15.045.686.9—14.2
Kimi K3 →\rightarrow Kimi K2.6Failed85.069.199.667.932.1
AuthorsOpenReview ↗

Response to Reviewer imJ1 (3/5)

W3 & Q1: Separating "Agreeing" from "Being Right" and Clarifying Table 13

Thank you for such an inspiring suggestion! Following your idea to separate "agreeing" from "being right", we verify that blind agreement does not improve task success: agreement drives plan compliance (HH), whereas task success (SS) requires agreeing on the correct state. In ConPAct, agreement is not an end in itself; rather, disagreement serves as a runtime trigger to re-examine disputed fields so both roles converge toward the true state (verified in Table 23). Furthermore, the Opus-5 facts in Table 13 only describe raw object locations without access to the planner's subgoal, and restricting those facts to bare JSON fields still recovers most of the gain.

Following your suggestion, we expanded Table 14 (Exp 4a) with Same-wrong (both roles receive the identical wrong state) and Independent-wrong (different wrong states). Same-wrong lowers mismatch DD and raises plan compliance HH, confirming that shared state beliefs align the actor with the planner. However, because the shared state is false, the actor faithfully executes a flawed plan and task success drops severely. Task success rises only when both roles agree on the correct state.

For Table 13 (Q1), which locally probes whether accurate perception helps the actor recognize an impossible subgoal in a single turn, Opus-5 is never shown the subgoal and never states whether a path is blocked. It only reports raw layout—for example, e.g., in Sokoban (Appendix G): "The box is at the top of the player, a wall is directly above the box, and the target is at the top-left of the player." The actor must infer on its own that pushing the box upward is blocked. To strictly rule out phrasing cues, we added Schema-only facts (Exp 4b), which feeds the actor only our fixed JSON state fields with no free-form text. Valid handback still improves a lot, confirming that the gain comes from state perception rather than answer leakage. We will add the additional results in Table 13/14 for clarification.

Exp 4a: Separating Agreement from Correctness (Expanding Table 14)

State SourceAgreementCorrectnessDD (%)AplanA^{\text{plan}} (%)AactA^{\text{act}} (%)HH (%)SS (%)
NoneLowLow61.952.024.177.347.2
Same-wrongHighLow14.68.37.181.221.4
Independent-wrongLowLow78.39.16.869.428.7
Opus-5HighHigh21.378.279.190.880.1
OracleHighHigh7.392.090.590.884.1

Exp 4b: Actor Single-Step Probe with Schema-Only Facts (Expanding Table 13)

Actor ModelConditionBlocked: Valid handback (%)Feasible: First-step progress (%)
GPT-5.6-solControl text0.469.7
GPT-5.6-solSchema-only facts (JSON only)71.388.6
GPT-5.6-solOpus-5 state facts94.594.5
GPT-5.6-terraControl text0.141.7
GPT-5.6-terraSchema-only facts (JSON only)64.883.4
GPT-5.6-terraOpus-5 state facts91.090.2

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to Reviewers and Revision Submitted

We would like to thank the reviewers for their helpful and constructive comments. We are very glad to see that the reviewers find our presentation clear and easy to follow (imJ1), our studied problem easy to understand with examples that show clearly how state misreading breaks a plan (imJ1). We are delighted that they recognize our framework provides a concrete way to observe inter-agent state disagreement and makes a plausible source of failure easier to study (JKtJ), while our detection mechanism needs no ground truth (imJ1) and our controlled experiments verify that task-relevant state information can improve execution (27Sy). Besides, we are encouraged that the reviewers find our experiments broad and extensive (imJ1, 27Sy), evaluating across multiple environments, model families, and both inference-time and training-based methods (imJ1, 27Sy, JKtJ), where ConPAct substantially improves the delegated Plan-Act system (JKtJ).

We have addressed the reviewers' comments and concerns in individual responses to each reviewer. The reviews allowed us to improve our manuscript and the changes have been updated in the manuscript as summarized below:

For Reviewer imJ1:

  • Added the cost–performance comparison with ReAct (Planner) (see Section 4 and Appendix H).
  • Clarified the relation between state misreading and mismatch (see Section 1 and Appendix D).
  • Added the analysis separating agreement from state correctness (see Section 3 and Appendix D).
  • Clarified the Opus-5 state facts and added the schema-only probe (see Section 2 and Appendix D).
  • Clarified the training baseline to isolate coordination gains (see Section 4 and Appendix F).
  • Added the sensitivity analysis on the JSON schema size (see Section 2 and Appendix F).

For Reviewer 27Sy:

  • Clarified the limitations of prior multi-agent alignment methods (see Section 1 and Appendix F).
  • Revised the logical progression of the Introduction (see Section 1).
  • Clarified the motivation from the capability-gap finding (see Section 1, Appendix D, and Appendix H).
  • Added the sensitivity analysis on the state-assertion schema (see Section 2 and Appendix F).
  • Added the comparison with the belief-alignment baseline MindDial (see Section 4 and Appendix F).

For Reviewer JKtJ:

  • Clarified the separation of coordination from information transfer (see Section 4 and Appendix F).
  • Added the analysis separating agreement from state correctness (see Section 3 and Appendix D).
  • Clarified the ablations under matched communication budgets (see Section 4 and Appendix F).
  • Added the matched-budget comparison with ReAct (Planner) (see Section 4 and Appendix H).

All changes have been highlighted in blue in the manuscript.

We appreciate your comments and time! We have tried our best to address your concerns and revised the paper following the suggestions. Would you mind checking it and confirming if you have further questions?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.