acceptodds
Under review as a conference paper at ICLR 2027

Pnyx: Robust Communication and Dynamic Authority for Collaborative VLA Agents

Abstract

Vision-language-action (VLA) models provide a language-conditioned interface between visual reasoning and closed-loop control, but are typically deployed as egocentric policies without mechanisms for sharing observations, distributing subtasks, or scheduling around task conflicts. To demonstrate the benefit of collaborative VLA agents, we study VLA-driven multi-agent systems (VLA-MAS), augmenting the conventional dual-system VLM and action expert pairing with a collaborative layer operating between user prompting and the model's existing natural-language interface. To support decentralized coordination, we introduce a novel system, PNYX, combining a structured, fault-tolerant communication protocol with dynamic task leadership. The protocol tracks message purpose, freshness, receipt, and agreement, while the leader-based architecture temporarily centralizes proposal formation and preserves local review, execution, and fallback. We evaluate PNYX across six closed-loop cooperative-driving tasks and show improvement in nominal joint success from 51.2% for the strongest canonical baseline to 58.3%, while increasing route completion from 75.9% to 80.7%. Under severely degraded communication, PNYX reaches 55.4% joint success, compared with 42.5% for centralized and 53.8% for free-form decentralized planning. Our work highlights that effective VLA collaboration depends not only on communication volume, but also on explicit commitment and network-aware adaptation in information and decision authority.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action (VLA) Models, Multi-Agent Systems & Collaboration, Embodied AI, Decentralized Planning & Control, Robust Multi-Agent Communication, Language-Grounded Coordination, Hierarchical & Dual-System Architectures, Fault-Tolerant Decision-Making, Dynamic Topology & Leader Election

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_sm85OpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces PNYX, a coordination framework for multi-agent vision-language-action systems. It combines a structured communication protocol with task-scoped leader election. The protocol distinguishes information sharing from action negotiation, supports explicit proposal acceptance and rejection, tracks information freshness, bounds retransmissions, and terminates coordination exchanges at a deadline. The leadership mechanism uses deterministic candidate ranking, voting, heartbeats, and replacement procedures. The authors evaluate PNYX on six cooperative-driving scenarios in CARLA 0.9.14, using two VLM planners and a controlled, frozen execution interface. Each architecture–condition cell contains 240 episodes pooled across the two planners. The paper reports a nominal joint success rate of 58.3% for PNYX, compared with 51.2% for the centralized and free-form decentralized baselines, and a route completion rate of 80.7%, compared with 75.9% for the strongest baseline. Under severe degradation, PNYX achieves 55.4% joint success.

Critical strengths

Explicit coordination semantics. The paper distinguishes message receipt, action proposals, participant acceptance, and commitment. This is a meaningful design choice because exchanging intentions does not necessarily establish a mutually accepted joint action. The protocol provides a clear framework for representing these states. The paper evaluates six driving scenarios under nominal, mild, and severe degradation profiles and includes ablations of state representation, proposal rejection, and leader selection.

Critical weaknesses
  1. Limited methodological novelty and unclear VLA-specific contribution. The proposed method largely combines established mechanisms from multi-agent communication and distributed systems, including FIPA-ACL-style message typing, truncated ARQ, age-of-information tracking, and Raft-inspired leader election and failure detection. The paper does not sufficiently identify what fundamentally new coordination principle or capability emerges from this combination, or clearly distinguish PNYX from existing LLM-based multi-agent planning and coordination frameworks. More importantly, the proposed mechanisms operate primarily at the level of language-mediated planning and system orchestration, rather than exploiting characteristics specific to VLA models. The evaluation uses VLM planners with a controlled, frozen execution interface, without demonstrating that the proposed design addresses challenges unique to VLA representations or action generation. Consequently, it remains unclear whether this work constitutes a substantive advance in collaborative VLA research, rather than an application of general multi-agent coordination techniques to a driving simulator.

  2. The experimental evaluation lacks a coherent end-to-end demonstration of how the proposed method works. The paper reports aggregate success rates, several component ablations, and a textual failure trace, but does not provide a sufficiently clear, end-to-end case study connecting the initial scene and observations to information exchange, leader selection, proposal generation, participant acceptance, action execution, and the final outcome. Figure 7 compares free-form decentralized planning with centralized planning, rather than showing how PNYX resolves the same scenario through its proposed protocol. Moreover, the trace contains repeated discrepancies between agents' natural-language statements and the listed executor commands, and the authors explicitly acknowledge that the collision cannot be attributed solely to coordination failure. As a result, the qualitative evidence does not establish the causal mechanism behind the observed failures or demonstrate how PNYX's individual components change the decision process. The quantitative results are also presented mainly as aggregate comparisons across scenarios and degradation profiles, without a sufficiently clear connection to representative execution traces. This leaves the experimental narrative fragmented: the reader can observe reported performance differences, but cannot adequately understand when, why, and through what sequence of interactions the proposed method succeeds or fails.

  3. Insufficient experimental specification and runtime characterization. The paper does not provide enough detail to independently reconstruct the evaluation, including the concrete CARLA maps and scene configurations, task prompts, planner inputs, obstacle and occlusion geometry, and complete task-level success and termination conditions. The reported distinction between “risky” and randomized trials is not sufficiently defined. In addition, although the planners are stated to operate at 2 Hz, the paper does not report the hardware configuration, per-call inference latency, end-to-end negotiation latency, communication rounds, and message counts per decision, retransmission statistics, or the frequency and cost of leader replacement. The reported route-completion time is not a substitute for coordination latency. Finally, communication and sensing factors are varied jointly across degradation profiles, preventing the reported robustness gains from being attributed specifically to communication effects. These omissions substantially limit reproducibility and make it difficult to assess whether the proposed protocol can meet the real-time constraints of the target application or whether its additional coordination overhead is justified.

Questions for authors
  1. Task specification and inputs: For the overtaking scenario in Figure 7, is the overtaking objective specified by the benchmark or task prompt, or is it inferred autonomously by the planner? Please provide the actual task prompts, planner inputs, scenario initialization, and the division of responsibility between the scenario generator, planner, coordination layer, and executor.

  2. Scene reproducibility: Please specify the CARLA maps, initial vehicle states, road geometry, obstacle placement, occlusion construction, and termination rules for the six scenarios(BEV View or actually car input). How are the reported “risky trials” generated, and how do they differ from randomized trials? Are scene-generation scripts and configuration files available?

  3. Execution and message handling: How are planner outputs parsed and validated against the protocol? If a generated message is malformed or inconsistent with the current protocol state, is the model invoked again, is the message rejected, or is it repaired programmatically? How are these cases counted in planner calls, token usage, and coordination deadlines?

  4. Runtime and communication overhead: The paper reports planners operating at 2 Hz but does not provide per-call inference latency or end-to-end negotiation latency. Please report the hardware configuration, inference latency distributions, communication rounds and messages per negotiation, retransmission counts, timeout rates, and the time from task initiation to committed action.

  5. Leader election and replacement: How often does leader replacement occur in the main evaluation? What are the measured election and handoff latencies, and how much of the performance gain can be attributed to dynamic replacement rather than the structured proposal protocol? The severe-degradation election ablation performs worse with a ranked election than with a random leader, so additional evidence is needed to clarify the practical benefit of this mechanism.

  6. Controlled degradation: Please provide the numerical settings for bandwidth, communication delay, packet loss, communication range, and sensing noise in the nominal, mild, and severe profiles, especially the sensing noise, as I do not see any ego view in this article. Since these factors are varied jointly, can their individual effects be isolated through controlled ablations?

  7. Baselines and task difficulty: How does PNYX compare with a simple rule-based coordination strategy and existing general-purpose LLM multi-agent coordination approaches under the same initial conditions? Such comparisons would help determine whether the observed gains require the proposed framework and whether the tasks adequately test its claimed VLA-specific contribution. Also, the definition of "success" should be mentioned, as well as other detailed metrics.

5 comments
AuthorsOpenReview ↗

Part 1 of 4

Thank you for your thoughtful comments and suggestions, which have helped us significantly improve our manuscript. Below, we explain Pnyx's contribution as the composition of established mechanisms from multi-agent systems, networking, and distributed consensus, adapted for physically coupled agents acting under deadlines. We have reframed the paper away from VLA-specific claims. To address the evaluation concerns, we have added an end-to-end case study, full task prompts and scenario details, the degradation parameters, output-validation and runtime statistics, and an analysis of leader replacement showing its benefit on the episodes where it occurs. Please see the revised PDF, with changes in blue. We also directly respond to your specific points below.

Limited methodological novelty and unclear VLA-specific contribution. We build Pnyx from established mechanisms in various fields deliberately. FIPA-ACL act typing comes from agent communication languages in multi-agent systems research. Truncated ARQ comes from wireless link-layer error control. Age of information comes from status-update networking, where it was introduced for vehicular networks. Raft-style terms and leader election come from distributed consensus. None were explicitly designed for LLM agents or collaborative route planning, but each gives a well-defined solution to the problem of making the purpose of a message explicit, recovering from message loss, tracking how old shared information is, and keeping a single leader when agents fail. Reusing them gives Pnyx semantics that are already well understood instead of new mechanisms that would need their own validation. The contribution is how they are composed around the weaknesses of collaborative route planning with general LLM multi-agent frameworks and large central models, and how each one changes once agents are physically coupled and act under deadlines:

  • Majority vs. unanimity. Raft commits on a majority because a lagging replica can catch up later. A vehicle left out of a decision cannot, since it may already occupy the space the maneuver needs, so Pnyx requires unanimity.
  • Retrying vs. falling back. Raft blocks and retries when agreement fails. In driving, waiting is itself an action, so Pnyx falls back to a safe local default before the control deadline.
  • Trusting the log vs. rebuilding state. A new Raft leader trusts its log. World state goes stale regardless, so a new Pnyx leader rebuilds state from every participant, and leadership is ranked by freshness of information.
  • Retransmission. FIPA-ACL assumes messages arrive. Pnyx retransmits only the messages that close a commitment, and stops at the deadline.

On the VLA question, please see the general response. We agree that Pnyx does not exploit VLA-specific representations, and the revised paper no longer presents it as a VLA-specific contribution. The problem it addresses sits above the executor. A language-conditioned executor acts on one instruction per decision period, so when two planners disagree, their executors carry out incompatible actions. Pnyx depends only on that interface and treats the executor as a black box, so the same layer applies to any executor that follows language instructions.

AuthorsOpenReview ↗

Part 2 of 4

The experimental evaluation lacks a coherent end-to-end demonstration of how the proposed method works. We have added an end-to-end case study in the revised appendix that follows one Pnyx episode on a seed where the independent, centralized, and free-form decentralized baselines all fail. It connects the initial scene and each vehicle's camera views to the state exchange, leader election, the leader's proposal, the participant's acceptance and commit, the instructions and executed commands, and the outcome. This shows the sequence of interactions through which Pnyx resolves the conflict the baselines do not, and gives a concrete trace behind the aggregate gains in that scenario. The quoted statements are messages between free-form planners, which write their peer messages and their own vehicle's instruction as separate outputs that need not agree. The remaining mismatches come from the executor's one-period lag. In steady state, every architecture's instructions are executed correctly more than 97% of the time. The disagreement between what a free-form planner tells its peer and what it instructs its own vehicle is itself the failure Pnyx's commit step prevents, where an agent's instruction follows the plan its peers approved.

Task specification and inputs. The overtaking objective is specified by the benchmark's task prompt: for example, "Overtake the stopped vehicle by passing on the left, when it is safe to do so." The planner decides only when and how to carry it out. The scenario generator (CARLA) defines the map, spawn poses, routes, hazards, task prompts, and success and termination checks. Each planner (VLM) receives its vehicle's task prompt, a text description of the sensed scene, and three camera images, and outputs a natural-language instruction. In Pnyx it also produces the content of queries, proposals, and approve/reject decisions. The coordination layer is deterministic code that handles message typing, acknowledgment and retransmission, state freshness, leader election, and commitment. The executor converts the latest instruction into atomic driving actions along the route. The revised appendix now gives every agent's task prompt in all six scenarios, the full planner prompt template, an example scene description, and the camera views each planner receives.

Scene reproducibility. Our scenarios build on the open-source TalkingVehicles benchmark, whose scene-generation scripts and configuration files are publicly available. We use its six scenarios, maps, routes, and task prompts, with three changes: the stopped truck in the overtake scenarios is placed 25 m ahead of the ego vehicle, the safe overtake trial includes both focal vehicles, and the highway-exit time limit is extended to 35 s. Highway exit and highway merge use Town06. Both overtake scenarios use Town01, and the unprotected left turn and red light use Town03. Each scenario has two configurations from TalkingVehicles, a risky one and a random counterpart. Every evaluation cell contains 10 trials of each per scenario on fixed seeds, for 120 episodes per planner. Risky trials place a conflict that the ego vehicle cannot resolve from its own observations, so an uncoordinated vehicle is likely to collide:

  • in the highway exit and merge, the two vehicles start exactly side by side, so neither has priority and both must agree who goes first;
  • in the overtakes, an oncoming car approaches in the passing lane, hidden from the ego vehicle by the stopped truck;
  • at the left turn, an oncoming car is hidden by a stopped truck;
  • at the red light, a car runs the red light at 29–31 km/h from a direction hidden by a queue of trucks.

The random trials keep the same map, routes, tasks, and occluders but remove the forced conflict. In the highway scenarios, the second vehicle and its follower are shifted 2–8 m forward or backward at random, so one vehicle has a natural lead. In the overtake, left-turn, and red-light scenarios, the hidden hazard vehicle is not spawned (TalkingVehicles' "safe" configurations). In both configurations, vehicle models and hazard speeds are also sampled per seed. An episode ends when every focal vehicle has completed its route or collided, or at the time limit (25 s; 35 s for highway exit). The simulation runs at 20 Hz. We did not record a bird's-eye view. Exact spawn poses, routes, and occluder placement follow the TalkingVehicles configuration files. We include the relevant details in the revised manuscript to improve reproducibility of our results

AuthorsOpenReview ↗

Part 3 of 4

Execution and message handling. We answer each individual question below and provide detailed explanations in the revised appendix.

  • Parsing. every planner call uses strict JSON-schema constrained decoding. The model fills only the fields its current protocol role requires; protocol messages, checksums, and digests are built by deterministic code, so the model never writes a raw protocol message.
  • Failed calls. output that does not parse is discarded, with no retry or repair. The previous instruction stays in force, and the call still counts toward planner calls, tokens, and its planner period.
  • Stale outputs. output for an exchange that has since been superseded is dropped and counted as stale.
  • Channel damage. damaged messages are handled by the protocol. A lost header means no acknowledgment, so the sender retransmits; a digest mismatch triggers a fallback.
  • Frequency. 6 of about 28k planner calls in the main evaluation failed to parse.

Runtime and communication overhead. Experiments ran on 2× AMD EPYC 9254 CPUs with 8× NVIDIA L40S (48 GB) GPUs. Planners were served with vLLM on three GPUs, with several episodes running in parallel. This information has been added to section 6.

Mean planner latency per call (s):

IndependentCentralizedDecentralizedPnyx-ProtoPnyx
1.72.74.64.03.6

Pnyx negotiation statistics by profile:

MetricNominalMildSevere
Negotiations completed86%82%50%
Negotiation latency (median)1.25 s1.65 s5.15 s
Messages per negotiation, excluding ACKs (median)668
Retransmitted share of messages0.5%3.3%18.1%
Time to first committed plan (median)2.0 s2.9 s9.8 s

Model calls are locked to simulated time, meaning each result takes effect exactly one planner period after the call is made regardless of how long the call took in wall-clock time. Wall-clock latency therefore does not affect outcomes, as it is largely dependent on hardware capability and usage. Instead, we report relevant hardware-agnostic metrics, such as the number of planner calls.

Leader election and replacement. Leader replacement occurs in 5.2% of Pnyx episodes in the main evaluation. Elections take 0.3 s on average under the nominal and mild profiles. Under severe they take 3.2 s, because candidacy messages also carry each vehicle's state. The benefit of the structured proposal protocol alone shows in Pnyx-Proto, which has no leader. Adding leadership raises joint success over Pnyx-Proto at every profile (+3.7, +6.3, and +2.1 points). Within Pnyx, this benefit comes mostly from replacement. On the seeds where a handoff occurred, Pnyx succeeds in 51.1% of episodes, against 34.0% on the same seeds with a fixed leader (n=47n = 47, paired p=0.039p = 0.039). On seeds without a handoff, the two perform the same (58.9% vs. 58.0%). Replacement therefore matters in the minority of episodes where the initial leader stops being effective. These situations are hard: there, the fixed leader's success falls to 34%, against 56% overall. Under severe degradation, the difference between ranked election and the random leader (53.9% vs. 56.1%) is not statistically significant (p=0.60p = 0.60).

Reviewer_aNwTOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents PNYX, a coordination system that combines a structured communication protocol with dynamic task leadership for collaborative VLA agents. The method requires explicit acceptance from all participants before commitment and uses deterministic task-scoped leader election to assign responsibility for proposal formation. The authors evaluate PNYX on six closed-loop cooperative-driving tasks in CARLA using a fixed VLA-emulation executor.

Critical strengths

1.This paper addresses a meaningful problem in collaborative VLA systems by combining structured communication with dynamic task leadership to support coordination under limited and unreliable communication.

2.The coordination system operates through the existing language interface without requiring modification or collaborative fine-tuning of the execution policies.

3.PNYX demonstrates improved overall performance in closed-loop CARLA experiments under both nominal and degraded conditions.

Critical weaknesses

1.If an unavailable leader remains in the participant set and cannot vote, the election terminates in local fallback. However, the authors do not explain how leader replacement proceeds in this case or report experiments evaluating recovery from such failures.

2.Fig. 6 compares ranked election with a random leader assigned before execution without election messages. This comparison does not isolate the effect of leader selection from election overhead, and the effectiveness of leadership handoff is not directly evaluated. In addition, task success rates for PNYXProto are not reported, so the contribution of dynamic task leadership to task success beyond the protocol alone remains unclear.

3.The paper presents PNYX as a coordination system for collaborative VLA agents. However, PNYX is not evaluated with existing pretrained driving VLA models, leaving its applicability to such models unverified.

Questions for authors

1.How does PNYX complete leader replacement if the unavailable leader cannot vote?

2.Does leadership handoff improve performance when the initial leader and startup communication are held fixed?

3.Can you provide a comparison of task success rates between PNYX and PNYXProto?

4.Do the reported gains hold when PNYX is integrated with existing pretrained driving VLA models?

2 comments

Thank you for your thoughtful comments and suggestions, which have helped us significantly improve our manuscript. We provide new analyses of our existing ablation runs to separate election overhead from leadership handoff. On seeds where no handoff occurs, ranked election and a fixed random leader perform similarly (58.9% vs. 58.0%, p=0.69p=0.69), so the election itself does not cost success. On the 47 episodes where a handoff was triggered, ranked election reaches 51.1% joint success against 34.0% for the fixed leader (p=0.039p=0.039). Pnyx-Proto results, the VLA evaluation, and the revised leader-replacement procedure are covered in the general response. Please see the revised PDF, with changes in blue. We also directly respond to your specific points below.

Comparison does not isolate the effect of leader selection from election overhead. On the ablation seeds, election messages are only a small fraction of protocol traffic and alone have limited overhead. At severe, the byte share is higher because candidacy records carry the sender's STATE, which then does not need to be sent separately. On seeds where no handoff occurs, the election and its overhead are the only difference between the arms, and the two perform the same: 58.9% vs. 58.0% joint success, paired episodes, p=0.69p = 0.69.

ConfigurationEpisodesElection share of messagesElection share of bytesElection messages / episodeMean time spanned by election messages
Ranked (nominal, mild)6917.2%3.3%3.61.3 s
Ranked (severe)23117.5%29.4%4.23.3 s

Does leadership handoff improve performance when the initial leader and startup communication are held fixed? Further analysis into our collected results show the effectiveness of handoff. A ranked Pnyx episode contains a leadership handoff if a new election opens while a leader is already in place, when the leader departs, stops responding, has its proposals repeatedly rejected, or when a follower's planner requests leadership. Handoffs occur in 6 to 8% of episodes. Every ablation seed was also run with the fixed random leader, which cannot hand off, allowing us to compare the two on exactly the situations where a handoff was triggered. The fixed leader's success falls to 34% on these seeds, against 56% overall, and ranked election recovers about half of that loss through handoff.

Paired subset (ablation seeds)Ranked joint success (%)Fixed random leader joint success (%)Ranked-only / random-only successesExact McNemar pp
Episodes with a handoff4751.134.010 / 20.039
Episodes without a handoff63358.958.082 / 760.69
All68058.456.392 / 780.32

Task success rates for Pnyx-Proto are not reported and Pnyx is not evaluated with existing pretrained driving VLA. Please see our general response where we address these weaknesses of our work.

Thank you for your response and explanation. After going through all the responses and comments from other reviewers, my concerns are not fully addressed; I will keep my score for now.

Reviewer_GsRrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents PNYX, a coordination layer for collaborative agents with language-conditioned action interfaces. PNYX combines: (1) a structured communication protocol with typed messages, freshness tracking, bounded retransmission, and explicit acceptance of joint-action proposals; and (2) task-scoped leader election based on task initiation, information freshness, and a deterministic tie-breaker. The system is evaluated in six CARLA cooperative-driving scenarios against independent, centralized, and free-form decentralized planning.

Critical strengths

(1) this paper identifies a meaningful failure mode in free-form coordination. The message analysis shows that agents frequently communicate intentions but rarely establish explicit precedence or commitment. Separating message receipt from acceptance of a specific joint assignment is a sensible design.

(2) the evaluation includes paired routes, multiple planners, degraded network settings, ablations, and resource accounting. The paper is also reasonably transparent about costs: PNYX uses more prompt context and produces slower successful routes than some baselines.

Critical weaknesses

(1) Section 6.1 uses a “controlled VLA emulation”: a frozen Gemma VLM maps observations and language instructions to a small set of atomic commands, which are then executed by an existing command follower. This is substantially different from evaluating an end-to-end VLA policy such as SimLingo. It also conflicts with Section 3’s statement that the baselines use “end-to-end VLA policies for execution.” The title and conclusions therefore overstate the demonstrated scope.

(2) What are the instruction-to-atomic-action error rates for each architecture? Could PNYX benefit simply because its structured outputs are easier for the Gemma executor to parse?

(3) How can a failed leader be replaced when Algorithm 1 requires a candidate record and vote from every member of the fixed participant set?

(4) Are the centralized and decentralized baselines given the same grounded assertions, images, ARQ, deadlines, and profile-specific parameter adaptation?

Questions for authors

There is a paper: https://arxiv.org/pdf/2605.22504 seems to be relevant as well, maybe should discuss the relevant part.

I'm willing to discuss and raise my score.

2 comments

Thank you for your thoughtful comments and suggestions, which have helped us significantly improve our manuscript. We have reframed Pnyx as a general coordination layer for language-conditioned executors, added results with a pretrained end-to-end driving VLA, and corrected Section 3's description of the baselines. We also report per-architecture executor error rates, which show that Pnyx's gains do not come from its instructions being easier to execute, and we now discuss LACO in the related work. The revised leader-replacement procedure is described in the general response. Please see the revised PDF, with changes in blue. We also directly respond to your specific points below.

Controlled VLA emulation and scope. Please see our general response where we address this weakness of our work. Section 3 also wrongly said that the baselines use end-to-end VLA policies; all architectures use the same frozen executor, and we have corrected the text.

Relation to LACO. We now discuss LACO in the related work. LACO shares shallow-layer key-value states between pretrained driving VLAs in place of language, which reduces payload size and decoding latency. Each vehicle still decides on its own how to act. LACO does not assign anyone authority to propose a joint plan, and it does not establish agreement on one. It is also evaluated over an ideal channel. Pnyx targets the problem we measure in Section 4, where agents share information but never settle on one accepted joint assignment. We see the two as complementary, where LACO-style latent payloads could be carried as State content in Pnyx.

Instruction-to-atomic-action error rates, and whether Pnyx benefits from being easier for the executor to parse. We measured executor fidelity for every architecture directly from the episode traces. Each saved frame (2 Hz) logs both the planner instruction in force and the atomic command the executor issued. We map each instruction to the action types it allows: hold (stop / slow down), go (lane follow / speed up), or a lane change left or right. A frame counts as an error when the issued command is not one of them. The executor acts on a new instruction one decision period after it arrives. We therefore also report a steady-state rate that excludes frames where the instruction has just changed.

ArchitectureScored framesError, all frames (%)Error, steady state (%)Executor JSON parse failures
Independent14,91310.00.80 / 50,488
Centralized17,6183.41.10 / 72,668
Decentralized (AutoGen)16,3739.00.40 / 59,661
Pnyx-Proto20,6966.62.80 / 71,263
Pnyx19,5497.32.80 / 65,474

Parsing does not explain Pnyx's advantage. First, the executor never sees Pnyx's structured messages. The State/Proposal/Commit protocol runs only between route planners. In every architecture, including Independent, the executor receives the same kind of input: one free-form natural-language instruction for its own vehicle. The executor's prompt and model are also identical across architectures. Across about 320k executor calls, the executor's own JSON output never failed to parse in any architecture. Second, Pnyx's instructions are not easier to follow. In steady state, every architecture's instructions are executed correctly more than 97% of the time. Pnyx and Pnyx-Proto have a slightly higher steady-state error (2.8%) than Independent (0.8%) and Decentralized (0.4%). This is consistent with their instructions more often being conditional (e.g., "stop behind the truck until car2 has passed, then overtake"). The differences in the all-frames rates come almost entirely from the one-period lag, so they reflect how often an architecture changes its instruction, not how parseable that instruction is. If ease of execution drove the results, Centralized would lead (3.4% overall), yet Pnyx outperforms it under degraded communication.

Baseline inputs and mechanisms. All architectures receive the same grounded assertions, images, planner models, executor, and channel profiles. ARQ, coordination deadlines, and per-profile parameter selection are part of Pnyx’s design, so the baselines do not have them; the baselines are meant to represent centralized and free-form decentralized coordination as they are typically built. Figure 2 partly separates the components. Moving from the baselines to Pnyx-Proto adds the full protocol layer, including ARQ and deadlines (51.2% to 54.6% joint success). Moving from Pnyx-Proto to Pnyx adds leadership with the transport unchanged (54.6% to 58.3%).

Thank you for your explanation. After going through all reviews and comments by other reviewers, I can only maintain my score for now. If there were an option of boardline, I would choose. I will further consider update my score in the AC-reviewer discussion phase.

Comments on the paper

2 comments

Dear AC and Reviewers,

We would like to briefly note that we have incorporated the reviewer feedback into the revised manuscript, including clarifications in the introduction and related work, additional experiments, and supplementary material. The updates are marked in blue and address each concern raised during the review.

There are three weaknesses addressed by multiple reviewers that we would like to answer together within this comment.

Lack of end-to-end VLA policies in evaluation and VLA-specific novelty. All three reviewers mentioned the lack of end-to-end VLA policies in our evaluation. We agree that the emulated executor is not a sufficient substitute for an end-to-end VLA policy. What the evaluation does show is that Pnyx works with language-conditioned executors other than VLA policies. Pnyx is a general coordination layer that sits between user prompting and action generation. We have revised the title, abstract, and conclusions to describe the design more accurately. Section 6 also now includes results with a state-of-the-art pretrained driving VLA, SafeDriveVLA, which extends SimLingo.

Gap in leader replacement procedure. Reviewers aNwT and GsRr correctly identify a gap in our leader-election algorithm. As submitted, Algorithm 1 collects a candidate record and a vote from every member of the fixed participant set CqC_q. An unavailable leader remains in CqC_q, so a replacement election could only terminate in local fallback, and the paper did not support its claim of leader replacement. We revise the election to run over an active participant set P=Cq∖{ℓprev}P = C_q \setminus \{\ell_{\mathrm{prev}}\}, where ℓprev\ell_{\mathrm{prev}} is the leader whose suspicion triggered the election (∅\varnothing for the first election of a task). The suspected leader therefore never votes on its own replacement. Reconfiguration is valid only if PP retains a strict majority of CqC_q. Each vote is bound to a digest that includes PP, so any two certificates for the same term share a voter, and a single vote can belong to only one of them. At most one leader is therefore certified per term, even when followers disagree about which agent has failed. Unanimity still governs commitment (Eq. 2) within PP. Excluding the former leader does not require it to cooperate. Participants in the new term reject messages from earlier terms. A former leader that is partitioned rather than crashed therefore cannot obtain the unanimous acceptance Eq. 2 requires, and it can at most fall back locally. Because it remains physically present, the new leader treats it as an uncontrolled agent: its last committed assignment constrains new proposals until TbindT_{bind} expires, after which its last reported state, discounted by its age of information, is treated as an obstacle. If it reconnects, it observes the later term, steps down, and is readmitted at the next term boundary.

Missing Pnyx-Proto results. Reviewers aNwT and sm85 correctly point out that results for Pnyx-Proto are not reported and make it difficult to identify the contribution of the protocol versus the leader election process towards the improved task success rates. We now report Pnyx-Proto task success alongside the other architectures in Figure 2. Moving from the baselines to Pnyx-Proto adds the full protocol layer, including ARQ and deadlines (51.2% to 54.6% joint success). Moving from Pnyx-Proto to Pnyx adds leadership with the transport unchanged (54.6% to 58.3%).

We believe these revisions have improved the framing and motivation of the work, strengthened the empirical evidence, and clarified how our contributions relate to existing literature. We would be very grateful for any additional comments or discussion during the remaining rebuttal period. In particular, since these changes, including several experiments, were made directly in response to the reviews, it would be extremely helpful to know whether the main concerns now feel adequately addressed; we are happy to clarify any remaining points. Thank you again for the time and care invested in reviewing and for the feedback that guided these revisions.

Dear reviewers,

Thanks for serving as a reviewer. As the discussion period comes to a close and the authors have submitted their rebuttals, I kindly ask you to take a moment to review them and provide any final comments.

If you have already updated your comments, please disregard this message.

Thank you once again for your dedication to the OpenReview process.

Best,

Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.