acceptodds
Under review as a conference paper at ICLR 2027

Convention Grounding: Language as a Coordination Prior for Open Coordination Points

Abstract

Connected autonomous vehicles from different manufacturers must coordinate at open coordination points, a four-way intersection at simultaneous arrival, or a merge where each stack assumes a different rule, where local information fails to resolve a single order; rules frozen in each proprietary stack leave humans no way to alter which convention governs a mixed fleet. We introduce convention grounding: an operator broadcasts a convention as a natural-language sentence, and each receiving vehicle combines that text with its own observation to produce its role-differentiated action, under a reward that never observes the convention. In SUMO, a continuous traffic simulator, we provide, to our knowledge, the first controlled causal demonstration of this mechanism: freezing the scene and flipping only the convention text flips the agent's committed action (flip rate 1.00, zero collisions), while swapping the projected vectors of the two conventions maintains the flip rate but inverts correctness. At a four-way intersection, four independently trained policies, sharing no gradients, policy, training episodes, or projection parameters, coordinate through a common convention signal and their own observations. Generalization to the tested held-out phrasings fails: the role distinction remains unrecovered from the projected representation, and runtime selection is shown only within the grounded convention set. No large generative model is invoked during driving: encoding each broadcast costs one forward pass through a frozen encoder.

Primary area:
reinforcement learning
Keywords:
Multi-agent reinforcement learning, Connected and autonomous driving, language grounding, natural language, zero-shot coordination

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_fXb7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper asks whether independently trained vehicle policies can coordinate using a shared natural-language convention. Each policy combines its observation with a projected sentence embedding to choose a yielding action or crossing rank. Training uses scripted partners that follow the convention and a reward based on traffic outcomes. Swap experiments show that the convention representation controls the learned action, and independently trained policies coordinate successfully on the tested conventions even when their encoders differ. The method does not generalize to the tested held-out paraphrases, and the main coordination experiments use two conventions per scenario under controlled traffic conditions.

Critical strengths

-The exchange control is the strongest part of the paper. Swapping the trained representations preserves action switching but reverses correctness, which makes the role of the convention input clear.

-The mixed-encoder experiments show that the policies do not need a common embedding space to coordinate on the conventions they have learned.

-The authors report the failed paraphrase test and the identity-position confound openly. These results help define the limits of the positive findings.

Critical weaknesses

-I am not yet convinced that the experiments establish much beyond learning a shared convention identifier. The core setting has two trained conventions, their embeddings are explicitly separated, and the tested unseen phrasings fail. A two-entry mapping from signal to role could produce the same behavior. Human-readable messages are useful, but the paper needs a clearer account of what the language representation contributes to learning.

-The convention still enters the training problem through the scripted partners, even though it does not appear directly in the reward. The authors acknowledge this, and the setup is valid, but successful alignment is less surprising once that supervision path is made explicit. The co-learning result is also expected under a reward that permits a fixed role allocation to achieve full return. It is a useful control, but adds limited evidence for a new coordination mechanism.

-The independence result has a narrow scope. Policies are trained separately, but share the training framework, architecture, and execution rules, and identities remain tied to road positions. It is therefore unclear whether they learn to interpret identities or simply associate positions with actions under each convention.

-The paraphrase failure deserves further study before drawing a strong conclusion about its cause. There are only four held-out mirror pairs. The failed probes show that the tested readouts do not generalize, but do not establish whether role information is absent, distorted by the projection, or simply difficult to learn from this split.

Questions for authors

-How do the language embeddings compare with separately trained one-hot and distinct random-code representations, including the one-hot arm mentioned in Appendix A?

-What happens when identities are reassigned to physical positions and the convention set includes more than two crossing orders?

-Does the paraphrase result persist on a larger, lexically balanced test set or with an order-sensitive encoder?

-What information does the rank-execution layer need about other vehicles' chosen ranks, and how would each vehicle obtain it independently?

3 comments
AuthorsOpenReview ↗

Response to Reviewer fXb7

We thank the reviewer. Short answers: one readable broadcast served differently built receivers on new wordings of the trained rules (W1, Q1); distributed execution of previously committed ranks passes a locked simulation test (Q4); we do not claim identity resolution or four orders (W3, Q2).

W1 and Q1, beyond a shared identifier. In a post-review extension with a frozen 7B encoder and a name-swap readout, 16 of 16 new mirror pairs on a separate locked fresh panel were correct at each of three seeds, on all 100 scenes per pair (Section 4.7); the readout was designed after the original readout's locked classifier gate failed with the same encoder (1/4 original merge pairs at every seed). At the intersection, scripted-partner policies trained the same way were correct on 15 or 16 of 16 fresh pairs at every approach and seed, and the four learned policies together were jointly correct on all 8 pairs of another fresh panel for each same-seed team (threshold 6 of 8) and, on committed ranks, for all 78 cross-seed teams. A frozen text-to-code gate feeding one-hot, projected one-hot or private-code policies matched the language policy (16/16 pairs per seed, post-hoc and locked), and a language and a private-code policy coordinated live on all 16 pairs: a shared readable interface across different internal representations, not superiority over an agreed code standard. On trained wordings, direct one-hot, matched one-hot and private random codes reproduced MiniLM's audited ramp results; private codes also matched live-pair outcomes intact, shuffled and exchanged; zeroing failed for both, with different outcome splits (code table).

W2, supervision through partners. The learner receives the projected convention input directly. Scripted partners follow the convention, so action consequences carry a convention-dependent training signal even though the reward never reads the text; the projection also has separate supervision. Section 4.3 now states the co-learning result as observed: the code swap changed the logits at one traced scene but no greedy action of either MAPPO actor on 100 scenes at any of three seeds. Each seed's fixed ordering completed every rollout under either code, so the reward could be met without reading the code.

W3 and Q2, independence, identity and more orders. Independence is bounded to separate runs, seeds and parameters, and different encoders in the mixed arms; framework, architecture and execution rule are shared. With mixed encoders at the four-learner intersection, every assignment reached F_M = 1.00 intact and 0.00 zeroed. In the original design tags are a function of position: when we made them conflict at evaluation, merge policies always chose the position-implied role, and at the intersection 82 to 96 percent of choices by approach followed position. A new north learner trained with letters permuted over approaches failed its locked letter criterion (0.8333 against 0.95; 0 of 8 fresh pairs; training reward plateaued at 3.3 to 7.0 of 10), although it never chose the position rank in conflict cases (0 of 23,040 records per seed). We therefore claim convention-dependent actions at trained positions, not identity resolution. We also tested four trained crossing orders in one intersection with a newly supervised four-class projection and new policies. The language arm failed its locked all-seed criterion (O1/O2 pairs 2/8, 0/8, 3/8; O3/O4 2/8, 3/8, 4/8; one of 12 runs ended training at reward 3.65); an oracle-code control, given one code per order, was correct on all 80 scenes per order and seed (live execution not run). In 7,680 live episodes, 3,291 completed in the instructed order, 2,561 in a different trained order, and 1,828 ended in a conflict-box collision after non-canonical rank vectors. This is an unsuccessful extension of the tested single-swap readout, not evidence of four-order grounding.

W4 and Q3, paraphrase failure. On 16 pairs balanced by first-mentioned vehicle, MiniLM correctness was 0.469 to 0.500 by seed; SGPT depended on mention order. A recreated MiniLM classifier scored 1.000 on training phrasings and 0.500 on the original held-out pairs.

Q4, rank execution. Frozen policies committed ranks as before; to execute them, each vehicle used only its own state, simulated local sensing within 70 m and once-per-second rank messages, held on missing or duplicate ranks, and entered only after every car ranked ahead of it had cleared. All three teams passed the locked test (1,280/1,280 completions each, no conflict-box collision, deadlock or order mismatch), and independent 10% and 30% loss of rank messages still gave full completion without collision or deadlock. Every tested rank-2 to rank-4 non-compliance intervention triggered the conflict-box collision criterion. This is a simulation check of a cooperative executor that assumes a shared rank-message format and compliant vehicles, not fault-tolerant autonomy.

Thanks for running the extra experiments! The cooperative execution tests answer my question about ranks, and it’s good to see the improved paraphrase results with the 7B encoder and new readout.

I still have the same concern about the core result. The code baselines match the original ramp results, so the policies seem able to coordinate by learning convention IDs. The new language results cover different wordings of trained rules, while the identity and four-order extensions didn’t succeed. That leaves the contribution seems to be relatively narrow.

AuthorsOpenReview ↗

Response to Reviewer fXb7 follow-up

Thank you for the follow-up and for confirming that the execution tests answer the rank question and that the transfer results improved. We address the remaining concern directly.

  1. Codes and convention IDs. We agree with the observation, and the paper states it: on trained wordings, the policy maps the convention input to a role, and the tested one-hot and private random-code representations reproduce the ramp results (Appendix E). Our claim does not rest on language outperforming codes on trained wordings. It rests on two distinct findings: causal evidence that the broadcast controls learned actions, which is not unique to language, and a language interface that handles new wordings of trained rules across receivers with different internal representations.

First, the causal evidence. Under a reward that never observes the convention, the committed action on a frozen scene follows the broadcast. Zeroing the convention input removes switching, and exchanging the two projected vectors, the control the review called the strongest part of the paper, keeps switching at 1.00 but drives correctness to 0.00 (Section 4.2). Policies trained in separate runs, with different encoders in two configurations, coordinate through the broadcast without a scripted partner, with F_M of 1.00 intact and 0.00 without the broadcast (Section 4.6). This shows what the policy uses, whatever representation carries the convention.

Second, the interface. In our architecture, the policy does learn a convention identity. The language layer maps new wordings of trained rules to learned convention identities, so receivers with different internal codes can act on the same readable broadcast without an agreed code table. The code-based receivers share one frozen gate, while the direct-language policy uses its own projected language input. This is the setting the paper targets: vehicles from different manufacturers that do not share internal codes. On 32 new wordings of the two trained merge rules, exact lookup of training sentences abstained on all 32, while the frozen 7B encoder with the name-swap readout supported correct actions: 16 of 16 pairs at every seed for the direct-language policy and for receivers using three different internal code representations, and a language policy and a private-code policy coordinated live on all 16 pairs (Section 4.7, Appendix O). At the intersection, policies with scripted partners were correct on 15 or 16 of 16 fresh pairs at every approach and seed, and the four learned policies acting together live under the central executor were jointly correct on 8 of 8 further fresh pairs with no conflict-box collision (Section 4.7). The receiver comparison was designed after the direct-language result and is labeled post hoc, and we claim no advantage over semantic parsing or an agreed code standard.

  1. Scope and the failed extensions. We agree that the transfer result covers new wordings of trained rules, and the paper claims no transfer to rules absent from training (Sections 5 and 6). The four-order failure is specific: grounding four trained crossing orders within each tested geometry, at the merge and the intersection (Appendices I and L). Rank selection over three and four vehicles under each of the two trained conventions succeeded on all held-out scenes (Sections 4.4 and 4.5). Identity reassignment failed (Appendix M). Because identity tags coincide with road position in our scenarios, we claim convention-dependent actions at trained positions, not identity resolution. We report these failures against their criteria rather than leave them out.

The contribution is a controlled causal demonstration of convention grounding with its limits measured. Larger convention sets, identity separated from position, and rules absent from training are the next steps the paper names, and this work gives a tested setup in which to study them.

Reviewer_qW3ZOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies how independently trained vehicles follow a shared traffic convention written in natural language. Each vehicle combines the convention with vehicle states and identity labels to choose whether to go or yield, or to select its place in the crossing order. The policies are trained with PPO against scripted vehicles that follow the convention. Experiments in SUMO show that independently trained policies can coordinate under a shared convention. They can also switch between learned conventions without retraining.

Critical strengths
  1. The experimental design is careful and directly tests the proposed mechanism. The authors change the convention while keeping the physical scene fixed. They also exchange the convention vectors to check whether actions change as expected.

  2. The paper provides valuable evidence for coordination between independently trained policies. The evaluation includes four-vehicle settings and combinations of different text encoders. These results support the feasibility of coordination without joint training or a shared text encoder.

Critical weaknesses
  1. The paper is hard to follow, and the overall framework and training process lack clear visual explanations. It contains no figures. The roles of different components and the relationship between training and evaluation are explained across several sections, requiring readers to piece the information together.

  2. The added value of natural language as a coordination prior is not sufficiently demonstrated, which limits the paper’s contribution. Each main scenario contains only two conventions, and the main experiments use wordings seen during training. On unseen wordings, accuracy drops to 50% and the action flip rate is 0. These results do not demonstrate additional capabilities of natural language over simple convention codes.

Questions for authors
  1. The paper presents operator interpretability as the main advantage of natural language. Could the author explain its practical value in this task? Could the author also report a comparison with non-linguistic convention representations under the same training and evaluation setup?

  2. In Section 4.7, accuracy on unseen phrasings drops to 50%, and the action flip rate is 0. Does this generalization failure depend on the choice of text encoder? Could the author further clarify the main reasons for this failure?

3 comments
AuthorsOpenReview ↗

Response to Reviewer qW3Z

We thank the reviewer. Short answers: W1 is addressed by a new Figure 1. On W2 and Q1, codes match language on trained wordings, and the tested natural-language interface supports new wordings of trained rules across receivers with different internal representations. On Q2, a frozen 7B encoder with a name-swap readout passed a locked fresh-panel test.

W1, presentation. A new Figure 1 traces the broadcast to the actor and, through the parser, to the scripted partners during training; shows where the projection is supervised and where the gradient stops; separates the frozen-scene swap from the exchanged-vector control; and marks where independently trained checkpoints are paired. The figure brings the training and evaluation paths together. Appendix D now includes Figure 2 showing the merge and four-way intersection layouts.

W2 and Q1, value over non-linguistic codes. For the single ramp learner with a scripted partner, matched one-hot and private random codes reproduced MiniLM's outcomes in every audited control (new code table). The natural-language interface demonstrates two capabilities. First, the policies act on new wordings of the two trained merge rules (Q2 below). Second, one readable broadcast serves differently built receivers (post-hoc, protocol-locked): receivers converting the sentence to their own code through the frozen gate also scored 16/16 pairs at every seed, and a language policy and a private-code policy coordinated live on all 16 pairs, without identical code tables. The receivers share one semantic gate, and we claim no advantage over an agreed code standard. Regarding R2, the broadcast is human-readable text rather than an opaque numerical identifier, so its intended meaning can be inspected without a separate code table. This is a design property; we did not measure operator usability or compare it with a code interface that has human-readable labels. On more conventions: with four conventions in one merge geometry, every tested training variant failed the all-seed criterion (Appendix). A newly trained four-order intersection system, with supervised four-class projection training, failed its locked criterion at every seed, while an oracle-code control given one code per order committed correctly on every held-out scene (Appendix). The language arm also produced conflict-box collisions in 1,828 of 7,680 live episodes. Larger convention sets and new rules remain open.

Q2, encoder dependence and cause. With a frozen 7B encoder and a name-swap readout, a locked test on a fresh panel passed, with all 16 mirror pairs correct at each of three seeds. The tested pipelines differ in encoder, readout, training inventory and projection training, so we do not attribute the change to one component. MiniLM gives mirror pairs cosines of 0.975 to 0.984, and a recreated family classifier scored 1.000 on training phrasings and 0.500 held out, giving both members of each pair the same label, so the failure is visible before the policy. Other tested encoders below 7B did not fix it; with SGPT, accuracy depended on which vehicle was named first. With a frozen 7B encoder, the original classifier missed the locked gate (1/4 original pairs at every seed); an exploratory policy extension then scored 12/16, 12/16 and 13/16 on a previously unopened panel. In a later locked diagnostic, the 7B classifier's scores correctly ordered a sentence against its name-swapped twin on four original and 16 previously inspected pairs. The name-swap readout was built on that comparison after the gate failure. In a descriptive ablation, MiniLM with the same readout flipped 5 of 16 Set 1 pairs in the wrong direction at every seed, whereas the 7B policies flipped none; the cells also differ in input formatting and initialization, so this does not isolate the encoder alone (Appendix). We report the threshold explanation as a hypothesis.

Thank you for the explanations and additional experiments. The new figures address my presentation concern. However, my main concerns about the paper’s contribution remain unresolved:

  1. The added value of natural language remains unclear. Simple codes reproduce the main coordination results, and language-to-code conversion matches direct language input. A human-readable interface is useful, but its practical benefit over simpler alternatives is not sufficiently demonstrated.

  2. The revision goes beyond further validation of the original results. It corrects an earlier interpretation of the mechanism and introduces a modified method to improve generalization. Key comparisons assessing the added value of natural language were also added at this stage. These changes improve the paper, but I still find that the method’s justification and the contribution need further development.

  3. The main contributions are not clearly identified in the Introduction. Some of the six listed points describe supporting experiments or scenario extensions. It remains unclear which represent the paper’s main advances over prior work.

These updates do not sufficiently change my assessment of the contribution. I therefore maintain my rating.

AuthorsOpenReview ↗

Response to Reviewer qW3Z follow-up

Thank you for reading the revision and for confirming that the figures resolve the presentation concern. We answer the three points in order.

  1. Value of language over codes. The setting the paper targets is a mixed fleet: vehicles from different manufacturers run proprietary stacks that do not share an internal code table (Introduction, R1 and R2). Appendix O tests a controlled version of this setting using receivers with different internal representations and one shared semantic gate. On 32 new wordings of the two trained merge rules, receivers using three different internal code representations and a direct-language policy were all correct on 16 of 16 pairs at every seed, while an exact lookup of training sentences abstained on all 32. A language policy and a private-code policy also coordinated live on all 16 pairs. The language-to-code receivers show that direct language input to the policy is unnecessary: a shared frozen semantic gate can translate the same broadcast for different internal representations. Exact sentence lookup could not handle the new wordings, but we do not claim that a code standard combined with semantic parsing would fail. On trained wordings, codes match language, and the abstract says so. This comparison was designed after the direct-language result and is labeled post hoc. We did not measure operator usability, and we claim no superiority over semantic parsing or over an agreed code standard.

  2. Method. The method is the same in every experiment: a frozen pretrained encoder reads the broadcast, a per-vehicle projection maps it to the policy input, and the policy is trained with a reward that never observes the convention. All causal results in Sections 4.2 to 4.6 come from the submitted configuration and are unchanged. Section 4.7 replaces the frozen front end (a 7B encoder with a vehicle-name swap readout) and changes the training inventory and projection training, to test whether the failure on held-out wordings was a limit of the mechanism or of the 22M encoder pipeline. The extension passed on locked fresh panels, showing that the general mechanism can support transfer to new wordings of trained rules. Because the encoder, readout, training inventory and projection training changed together, we cannot isolate which change produced the improvement. This is an extension of the same method, reported separately with its failed attempts (Appendix H, Table 15), not a replacement. The Section 4.3 change corrects our interpretation of an unchanged observation: swapping the code changed logits but not either actor's greedy action, so that test does not show convention-conditioned action selection.

  3. Contributions. We agree the list mixed main results with supporting ones. The revised Introduction now states three main contributions: (i) the problem and mechanism, namely the taxonomy of open coordination points and convention grounding under a convention-blind reward; (ii) controlled causal evidence that the committed action follows the broadcast, including the exchanged-vector control and policies trained in separate runs; (iii) the scope of the language interface: codes match language on trained wordings, the same method transfers to new wordings of trained rules with a stronger frozen front end, one broadcast serves receivers with different internal codes, and the four-order and identity extensions failed. The multi-vehicle and intersection results and distributed execution are listed as supporting evidence.

Reviewer_izFvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work addresses coordination failure among heterogeneous connected autonomous vehicles (CAVs) at open coordination points—traffic scenarios where locally available rules fail to determine a unique crossing order. The authors propose a structural taxonomy that partitions open coordination points into symmetric (zero valid orders implied by default rules) and divergent (multiple conflicting orders implied by heterogeneous rule sets).

To resolve symmetric coordination failures without joint training or standardized codebooks, the paper introduces convention grounding: a natural-language convention is broadcast as a public coordination signal; each receiving vehicle encodes the text via a frozen pretrained sentence encoder, projects it through an independently trained projection head, and conditions its control policy on the projected embedding alongside ego-centric observations. The reward function contains no convention, role, or identity labels; convention following is induced through the scripted partner’s behavior, while the projection itself is trained with convention-level pair supervision.

The core empirical contribution is a controlled causal demonstration (the swap test) in SUMO simulation: on frozen scenes where all physical inputs are held constant, flipping only the convention text induces a complete reversal of committed actions (switch rate 1.00, zero collisions in the intact condition). The paper further shows that independently trained policies—with disjoint parameters, training episodes, and, in some arms, different encoder architectures—can coordinate via the same broadcast signal. Finally, the authors identify a representational limitation: the mechanism fails to generalize zero-shot to held-out paraphrases of the trained conventions, and probes do not recover the role-reversal distinction from the projected representation.

Critical strengths
  1. Rigorous causal methodology in multi-agent language grounding. Most prior work on language-conditioned MARL reports correlational performance improvements; this paper’s interventionist design—freezing the physical scene and manipulating only the text input—establishes a direct causal link between the linguistic signal and agent behavior. The exchanged-vector control (swapping embeddings preserves switching but inverts correctness) is particularly elegant, and the gradient-isolation check is carefully executed. This methodological standard will benefit the subfield.

  2. A conceptually promising paradigm for heterogeneous multi-agent coordination. Unlike emergent communication protocols and role-decomposition methods, which require joint training and thus do not directly apply to heterogeneous fleets, convention grounding satisfies training independence, operator interpretability, and runtime switchability in principle. This is a well-motivated direction for further study, even though the paper explicitly does not claim deployment readiness.

Critical weaknesses
  1. The representational ceiling fundamentally limits the utility of natural language. The mechanism as demonstrated operates as an interpretable codebook, not a compositional language interface. Because generalization to unseen paraphrases fails completely, the set of usable conventions must be fixed at training time—just like a numerical codebook. The primary advantage of language (compositionality, zero-shot generalization to new phrasings) is not realized. This diminishes the conceptual novelty relative to standardized codebooks: the gain is chiefly in human readability, not in expressive power or scalability. The paper acknowledges this, but it remains a first-order limitation.

  2. Strong idealizations and an ambiguous source of effect limit external and internal validity. All scenarios are single-step contextual bandits with oracle state observations, constant-speed approaches, scripted partners, and perfect communication. These assumptions strip away perceptual uncertainty, dynamic arrival timing, non-compliant agents, message loss, and multi-vehicle queuing; divergent coordination points are not evaluated. In addition, the projection is trained with convention-level pair supervision, and Appendix B.2 indicates that the auxiliary objective dominates the policy gradient. The paper therefore does not yet establish how much of the reported grounding is attributable to the RL signal rather than to supervised separation. These are first-order limitations for the paper’s framing.

Questions for authors

Clarification requests (answers could change my assessment)

  1. Does the MAPPO baseline receive the convention embedding? Section 4.3 says that "the complete two-vehicle commitment observation is given to both actors," and Table 3 lists the MAPPO baseline's projection class as "none." Please state the exact actor input (dimension and composition) and whether zcz_c is included and varied across episodes. If it is, please report a swap test on the trained MAPPO actors in which zcz_c is exchanged or randomized with the scene held fixed. If zcz_c was not an input, the claim that co-learning agents "ignore the broadcast" (Section 4.3) is not supported by this comparison. This point bears most directly on my soundness rating.

  2. One-hot control results. Table 3 lists an "E1 (one-hot control)," but I could not find its results in the main text or in Appendix E. Please report switch rate, FMF_M, completion, collision, and deadlock for this arm, with projection supervision and capacity matched to the MiniLM arm. A one-hot codebook satisfies R1 and R3, so this comparison determines whether the language channel adds anything behaviorally or only interpretability (R2). If the two arms are equivalent, I would expect the abstract to state that the empirical advantage over a codebook is limited to R2.

  3. What does an "assignment" in Table 2 permute? Table 2 reports 6 assignments for M=2M=2 and 24 ("all 4!=244! = 24 all-distinct seed assignments") for M=4M=4. Are seeds permuted among fixed vehicle positions, or are policies moved to vehicle identities and positions they were not trained on? These support different claims: robustness across seeds versus transfer across roles. Please report the training and evaluation position of each arm and revise the caption if necessary.

  4. Is the Section 4.7 failure a limit of the encoder or of the projection? The projection in this experiment is pretrained with a family classifier on 22 phrasings per family. Please report the classifier's held-out accuracy, the projection's training versus held-out gap, and the confusion matrix over the four mirror pairs. If the classifier is already at chance on held-out phrasings, the bottleneck may lie in the projection objective rather than in the frozen encoder, and the "representational ceiling" interpretation would need to be narrowed.

  5. Identity versus position. The Limitations state that the paper cannot determine whether the policy resolves the identity token or the road position it always co-occurs with, yet Appendix G lists checks.tag_position_independence. Please report the result of this check, or evaluate the existing checkpoints with identity tags permuted independently of position. If the policy keys on position, the "role-differentiated" claim should be softened.

Additional analyses that would strengthen the paper (I do not expect all of them during rebuttal)

  1. Source of the effect. Appendix B.2 reports that the auxiliary objective "determines the outcome and absorbs the policy gradient as a perturbation." A frozen random projection without LprojL_{\text{proj}}, trained with the same policy and reward, would show whether the reward alone can learn the convention mapping or whether the result is mainly supervised separation. This is inexpensive relative to what it would clarify.

  2. Outcome decomposition. The motivating example is a deadlock and the reward includes a deadlock penalty, but Tables 1 and 10 report only collision and completion. Please report collision, deadlock, and completion by arm and by convention, especially for the zeroed, shuffled, and exchanged controls.

Suggestions for future work (not needed for my assessment)

  1. Evaluation at divergent coordination points, which the taxonomy introduces but the experiments do not test.
  2. Robustness to partial compliance (a subset of vehicles that ignores or misses the broadcast) and to broadcast dropout, together with a specified fallback behavior.
  3. Convention sets larger than two, with disjoint train and test conventions, to test the runtime-selection claim (R3) beyond two-way discrimination.
2 comments
AuthorsOpenReview ↗

Response to Reviewer izFv

We thank the reviewer and answer first the questions marked as able to change the assessment.

Q1, MAPPO input. Both actors received a 12-value commitment observation and a 2-value one-hot convention code, resampled each episode, with no sentence embedding and no learned projection. Swapping the code changed logits in one traced scene, but neither actor changed its greedy action on 100 held-out scenes at any of three seeds. Section 4.3 now reports this instead of "ignore the broadcast", and "byte-identical trajectories" becomes matching per-step digests of IDs, positions and speeds.

Q2, code control. Direct one-hot inputs and separately trained, capacity-matched one-hot and private random codes matched MiniLM in every audited single-ramp control cell (intact, zeroed, shuffled, exchanged; new code table). Intact ramp switching was 300/300. In a separate four-order test, oracle codes (one per order) produced correct commitments on 80/80 scenes per order and seed; language failed. This shows policy capacity with perfect codes, not an isolated encoder defect.

Q3, assignments. All 72 audited evaluations preserved training positions. Table 2 reports 66 across five rows; the other six were additional four-learner assignment checks. Policies and scenes were reused.

Q4, encoder versus projection. The original classifier was not saved, so the submitted result cannot be decomposed exactly. A recreated classifier and projected-vector probes scored 1.000 on training phrasings and 0.500 on four held-out mirror pairs; raw-encoder probes reached at most 0.625 (Appendix F). A 7B classifier also missed a locked gate (1/4 original pairs per seed); an exploratory policy extension scored 12/16, 12/16 and 13/16 on a previously unopened panel. A later locked diagnostic of the 7B classifier correctly ordered mirror scores on four original and 16 previously inspected pairs. With a name-swap difference readout, a separate locked test passed all 16 fresh mirror pairs at three seeds. The threshold explanation remains a hypothesis; the encoder and projection effects are not isolated.

Q5, identity versus position. The earlier checks.tag_position_independence is an audit, not an intervention. Conflicting tags yielded position-driven merge actions and 82 to 96% position-driven intersection choices. A north policy trained with letters permuted over all 24 assignments failed both locked criteria (0.8333 letter agreement versus 0.95; 0/8 fresh pairs; training reward plateaued at 3.3 to 7.0 of 10), although it never chose the position rank in conflicts (0/23,040 per seed). We make no identity-invariance claim.

Q6, effect source. With the projection frozen at initialization and its auxiliary loss removed, the vectors stayed at cosine about 0.96 and only one of three seeds switched correctly under both conventions (100/300). Both changes occurred together, so we did not establish grounding from reward alone.

Q7, outcomes. The new table separates completion, collision and deadlock. In the single ramp learner, exchange produced 300/300 collisions under C_A and 300/300 deadlocks under C_B. Live exchange completed all 1,200 episodes but had zero joint correctness across 600 paired scenes.

W1, codebook. The demonstrated capability is one readable broadcast serving differently built receivers on new phrasings of the trained rules, including live, without identical internal code tables. A frozen text-to-code gate matched the language policy (16/16 pairs per seed, post-hoc and locked), so direct language input was not necessary. On trained wordings, codes match language, and the rule set is not expanded.

W2, idealizations. Commitment remains single-step with oracle state, constant-speed approaches and scripted training partners. Joint evaluation uses learned policies without scripted partners. With 70 m simulated local sensing, once-per-second rank messages and fail-closed holds, each team completed 1,280/1,280 episodes in each condition (lossless, 10% and 30% independent rank-message loss), without conflict-box collisions or deadlocks. Every tested rank-2 to rank-4 non-compliance intervention triggered the conflict-box criterion. Real perception, convention-broadcast loss, dynamic arrivals, queuing and divergent coordination points remain untested. Two uncached 7B encodings took 2,456.5 ms median on a laptop GPU with 12 of 32 layers resident; cached lookup, projection and actor took 0.431 ms on one CPU thread. Neither establishes real-time latency.

AuthorsOpenReview ↗

Follow-up for Reviewer izFv

Before the discussion closes, here is where the revised PDF answers each point of the review.

  • Representational ceiling and codebook (W1): Section 4.7 and Appendix H. With a 7B encoder and a name-swap readout, a locked test on 16 fresh mirror pairs was correct at each of three seeds. Appendix J reports the intersection (15 or 16 of 16 fresh pairs per approach and seed). The scope is new wordings of the trained rules.
  • Q1 (MAPPO input): Section 4.3. The actors receive a two-value code; swapping it changed logits in a traced scene but no greedy action on 100 held-out scenes at any seed.
  • Q2 (code control) and Q7 (outcomes): Appendix E, Tables 10 and 11. One-hot and private codes matched the language results in every audited ramp control cell, and outcomes are split into completion, collision and deadlock.
  • Q3 (assignments): Table 2 caption.
  • Q4 (encoder versus projection): Appendices F and H; not isolated, stated as a hypothesis.
  • Q5 (identity versus position): Appendices K and M; no identity-invariance claim.
  • Q6 (effect source): Section 3.2 and Table 12.
  • W2 (idealizations): Section 5 and Appendices C and N.

If any concern remains open, we will answer it before the discussion closes.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Response to all reviewers

We thank the reviewers. Summary: one readable broadcast let separately trained and differently built vehicles act correctly on new wordings of the trained rules, including live. On trained wordings, codes match language. The failures below mark the limits.

Our central claim is a controlled demonstration that a shared natural-language convention signal can produce convention-dependent, role-differentiated decisions at trained positions across independently trained policies in the tested coordination settings. We do not claim unrestricted language understanding, general four-order coordination, or invariance to arbitrary identity reassignment. Post-review experiments are reported separately from the submitted method. For protocol-locked ones, datasets or generation rules, metrics, thresholds and configurations were fixed before the relevant runs; new evaluation panels were labeled by one author with the key hidden, except one screening set whose labels follow from its construction.

New wordings of the trained rules (post-review extension). Unlike a direct code input, the tested language pathway accepts new phrasings of trained rules. A protocol-locked test passed: 16 new mirror pairs, written after the training phrasings were frozen and overlap-checked, were correct at each of three seeds, each pair on all 100 scenes. The pathway keeps a frozen 7B encoder (e5-mistral-7b-instruct) and adds a name-swap readout, which subtracts the embedding of the sentence with the vehicle names exchanged; the projection was fitted and then frozen. The readout was designed after the original readout's locked classifier gate failed with the same encoder and training inventory (1/4 original pairs at every seed); the original MiniLM policies also failed on held-out merge pairs (Section 4.7). At the intersection, policies trained the same way chose the correct rank under both wordings for 15 or 16 of 16 fresh pairs at every approach and seed (threshold 12 of 16). The four learned policies acting together were jointly correct on all 8 pairs of another fresh panel for each same-seed team (threshold 6 of 8), in 3,840 SUMO episodes (160 distinct scene-order rollouts) without a conflict-box collision (two vehicles within 7 m of the box centre in one step) or deadlock. The scope is new wordings of the trained rules, each naming every vehicle.

One sentence, differently built receivers (post-hoc, protocol-locked, same 16 pairs). Receivers converting the sentence to their own code via the frozen gate (one-hot, projected one-hot, private code) were also correct on 16/16 pairs at every seed; an exact lookup abstained on all 32 new wordings. Live, a 7B language ramp policy and a gate-driven private-code mainline coordinated on all 16 pairs for each same-seed team (9,600 episodes, 200 distinct scene-order rollouts, no conflict-box collision or deadlock). A new wording thus served differently built receivers without identical code tables. Direct language input was not necessary, the parser arms share one gate, and we claim no advantage over an agreed code standard.

Distributed execution. With commitment unchanged, execution with simulated local sensing and rank messages passed its locked test, and independent 10% and 30% loss of rank messages caused no conflict-box collisions or deadlocks; every tested rank-2 to rank-4 non-compliance intervention triggered the conflict-box criterion.

Codes on trained wordings. Direct one-hot inputs, and separately trained one-hot and private random codes with matched projection and supervision, reproduced MiniLM's outcomes for the single ramp learner in every audited control cell; independently trained private-code policies also coordinated live (600/600 paired scenes intact). On trained wordings our results do not establish an advantage over the tested codes.

Limits. A newly trained four-order language system and an identity-permutation extension failed their locked criteria; an oracle-code four-order control, given one code per order, committed correctly on every held-out scene. In the four-order live evaluation, 1,828 of 7,680 episodes ended in a conflict-box collision after non-canonical joint commitments. An exploratory probe found the original two-order policies mostly reuse their learned orders. No completed result was retrained, rerun or replaced; four launches stopped before any output were restarted (see chronology).

Corrections. Both MAPPO actors received an episode-resampled two-value code. Exchanging it changed traced logits but not either actor's greedy action on 100 scenes per seed; Section 4.3 now reports this. The abstract now distinguishes intact convention switching from exchanged-vector behavior: switching remains observable under exchange, but convention-correct coordination is lost (F_M = 0.00).

The revision adds Figure 1, code and outcome tables, tag-conflict tests and all post-review results, including failures and verification coverage.

AuthorsOpenReview ↗

Summary of changes in the revised PDF

The revised PDF preserves the submitted findings and adds the following. Section and appendix numbers refer to the revised PDF.

  • Abstract and Introduction: the abstract now separates intact convention switching from convention-correct coordination, reports that the base MiniLM policies failed on held-out wordings, and summarizes the extension and its failures. The introduction states the co-learning result as observed and the scope of R2 and of matched codes.
  • Figures: Figure 1 (new, Section 3, page 6) shows (A) training against a scripted partner, (B) two evaluations on the same frozen merge scene, and (C) the name-swap extension with a frozen 7B encoder. Figure 2 (new, Appendix D, page 19) shows the merge and four-way intersection layouts.
  • Section 4.3: the co-learning result is stated as observed: swapping the convention code changed logits at a traced scene, but neither actor's greedy actions changed on any of the 100 held-out scenes at each of three seeds.
  • Section 4.7: a frozen 7B encoder with a name-swap readout was tested on locked fresh panels. It was correct on all 16 new mirror pairs of the two trained merge rules at three seeds and, with scripted partners, on 15 or 16 of 16 new pairs of the two trained intersection orders at every approach and seed. Four independently trained intersection policies acting together live, under the central executor, counted 8 of 8 further fresh pairs with no conflict-box collision. The receiver comparison used the 16 merge pairs.
  • Section 5 (Limitations): the identity-tag versus position conflict, the failed four-order and identity-reassignment extensions, the non-compliance result of distributed execution, and the scope of the tested phrasings.
  • New appendices:
    • H: transfer to new phrasings, protocol and results (Table 16), with a provenance paragraph and Table 15, which lists all 23 experiments run after the reviews, including failed and exploratory ones, with locked status, outcome and verification reported separately.
    • I: four conventions in one geometry (failed).
    • J: new phrasings at the intersection, with scripted partners and with four learned policies live (both passed their locked criteria).
    • K: identity tags and trained positions (descriptive).
    • L: four trained orders at the intersection (language arm failed; oracle-code control succeeded; an untrained-order probe is exploratory).
    • M: identity versus position (failed).
    • N: distributed execution (passed without message loss; with independent rank-message loss and with one non-compliant vehicle the results are descriptive: loss was tolerated, non-compliance was not).
    • O: one broadcast, differently built receivers. Language-to-code receivers matched direct language, so the direct-language superiority criterion was not met. Live mixed-receiver coordination passed its locked criterion. Both comparisons were designed after earlier results were known.
  • Other changes: outcome and code tables in Appendix E (Tables 11 and 12) and live pairing with private codes (E.3), runtime of the 7B pipeline in Appendix C, development selection and the 16 further merge mirror pairs in Appendix F (F.1, F.2), commands and verifiers for the extension experiments in Appendix G, and updated Ethics and Reproducibility Statements. The supplement contains the reported code, logs and verification artifacts.
  • Page limit: To keep the main text within 10 pages, Algorithm 1 moved to Appendix A, and the conflict-gate and validation details of Sections 4.1, 4.4 and 4.5 moved to Appendix D, word for word.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.