- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper asks whether independently trained vehicle policies can coordinate using a shared natural-language convention. Each policy combines its observation with a projected sentence embedding to choose a yielding action or crossing rank. Training uses scripted partners that follow the convention and a reward based on traffic outcomes. Swap experiments show that the convention representation controls the learned action, and independently trained policies coordinate successfully on the tested conventions even when their encoders differ. The method does not generalize to the tested held-out paraphrases, and the main coordination experiments use two conventions per scenario under controlled traffic conditions.
Critical strengths
-The exchange control is the strongest part of the paper. Swapping the trained representations preserves action switching but reverses correctness, which makes the role of the convention input clear.
-The mixed-encoder experiments show that the policies do not need a common embedding space to coordinate on the conventions they have learned.
-The authors report the failed paraphrase test and the identity-position confound openly. These results help define the limits of the positive findings.
Critical weaknesses
-I am not yet convinced that the experiments establish much beyond learning a shared convention identifier. The core setting has two trained conventions, their embeddings are explicitly separated, and the tested unseen phrasings fail. A two-entry mapping from signal to role could produce the same behavior. Human-readable messages are useful, but the paper needs a clearer account of what the language representation contributes to learning.
-The convention still enters the training problem through the scripted partners, even though it does not appear directly in the reward. The authors acknowledge this, and the setup is valid, but successful alignment is less surprising once that supervision path is made explicit. The co-learning result is also expected under a reward that permits a fixed role allocation to achieve full return. It is a useful control, but adds limited evidence for a new coordination mechanism.
-The independence result has a narrow scope. Policies are trained separately, but share the training framework, architecture, and execution rules, and identities remain tied to road positions. It is therefore unclear whether they learn to interpret identities or simply associate positions with actions under each convention.
-The paraphrase failure deserves further study before drawing a strong conclusion about its cause. There are only four held-out mirror pairs. The failed probes show that the tested readouts do not generalize, but do not establish whether role information is absent, distorted by the projection, or simply difficult to learn from this split.
Questions for authors
-How do the language embeddings compare with separately trained one-hot and distinct random-code representations, including the one-hot arm mentioned in Appendix A?
-What happens when identities are reassigned to physical positions and the convention set includes more than two crossing orders?
-Does the paraphrase result persist on a larger, lexically balanced test set or with an order-sensitive encoder?
-What information does the rank-execution layer need about other vehicles' chosen ranks, and how would each vehicle obtain it independently?
3 comments
Thanks for running the extra experiments! The cooperative execution tests answer my question about ranks, and it’s good to see the improved paraphrase results with the 7B encoder and new readout.
I still have the same concern about the core result. The code baselines match the original ramp results, so the policies seem able to coordinate by learning convention IDs. The new language results cover different wordings of trained rules, while the identity and four-order extensions didn’t succeed. That leaves the contribution seems to be relatively narrow.
Response to Reviewer fXb7 follow-up
Thank you for the follow-up and for confirming that the execution tests answer the rank question and that the transfer results improved. We address the remaining concern directly.
- Codes and convention IDs. We agree with the observation, and the paper states it: on trained wordings, the policy maps the convention input to a role, and the tested one-hot and private random-code representations reproduce the ramp results (Appendix E). Our claim does not rest on language outperforming codes on trained wordings. It rests on two distinct findings: causal evidence that the broadcast controls learned actions, which is not unique to language, and a language interface that handles new wordings of trained rules across receivers with different internal representations.
First, the causal evidence. Under a reward that never observes the convention, the committed action on a frozen scene follows the broadcast. Zeroing the convention input removes switching, and exchanging the two projected vectors, the control the review called the strongest part of the paper, keeps switching at 1.00 but drives correctness to 0.00 (Section 4.2). Policies trained in separate runs, with different encoders in two configurations, coordinate through the broadcast without a scripted partner, with F_M of 1.00 intact and 0.00 without the broadcast (Section 4.6). This shows what the policy uses, whatever representation carries the convention.
Second, the interface. In our architecture, the policy does learn a convention identity. The language layer maps new wordings of trained rules to learned convention identities, so receivers with different internal codes can act on the same readable broadcast without an agreed code table. The code-based receivers share one frozen gate, while the direct-language policy uses its own projected language input. This is the setting the paper targets: vehicles from different manufacturers that do not share internal codes. On 32 new wordings of the two trained merge rules, exact lookup of training sentences abstained on all 32, while the frozen 7B encoder with the name-swap readout supported correct actions: 16 of 16 pairs at every seed for the direct-language policy and for receivers using three different internal code representations, and a language policy and a private-code policy coordinated live on all 16 pairs (Section 4.7, Appendix O). At the intersection, policies with scripted partners were correct on 15 or 16 of 16 fresh pairs at every approach and seed, and the four learned policies acting together live under the central executor were jointly correct on 8 of 8 further fresh pairs with no conflict-box collision (Section 4.7). The receiver comparison was designed after the direct-language result and is labeled post hoc, and we claim no advantage over semantic parsing or an agreed code standard.
- Scope and the failed extensions. We agree that the transfer result covers new wordings of trained rules, and the paper claims no transfer to rules absent from training (Sections 5 and 6). The four-order failure is specific: grounding four trained crossing orders within each tested geometry, at the merge and the intersection (Appendices I and L). Rank selection over three and four vehicles under each of the two trained conventions succeeded on all held-out scenes (Sections 4.4 and 4.5). Identity reassignment failed (Appendix M). Because identity tags coincide with road position in our scenarios, we claim convention-dependent actions at trained positions, not identity resolution. We report these failures against their criteria rather than leave them out.
The contribution is a controlled causal demonstration of convention grounding with its limits measured. Larger convention sets, identity separated from position, and rules absent from training are the next steps the paper names, and this work gives a tested setup in which to study them.
Response to Reviewer fXb7
We thank the reviewer. Short answers: one readable broadcast served differently built receivers on new wordings of the trained rules (W1, Q1); distributed execution of previously committed ranks passes a locked simulation test (Q4); we do not claim identity resolution or four orders (W3, Q2).
W1 and Q1, beyond a shared identifier. In a post-review extension with a frozen 7B encoder and a name-swap readout, 16 of 16 new mirror pairs on a separate locked fresh panel were correct at each of three seeds, on all 100 scenes per pair (Section 4.7); the readout was designed after the original readout's locked classifier gate failed with the same encoder (1/4 original merge pairs at every seed). At the intersection, scripted-partner policies trained the same way were correct on 15 or 16 of 16 fresh pairs at every approach and seed, and the four learned policies together were jointly correct on all 8 pairs of another fresh panel for each same-seed team (threshold 6 of 8) and, on committed ranks, for all 78 cross-seed teams. A frozen text-to-code gate feeding one-hot, projected one-hot or private-code policies matched the language policy (16/16 pairs per seed, post-hoc and locked), and a language and a private-code policy coordinated live on all 16 pairs: a shared readable interface across different internal representations, not superiority over an agreed code standard. On trained wordings, direct one-hot, matched one-hot and private random codes reproduced MiniLM's audited ramp results; private codes also matched live-pair outcomes intact, shuffled and exchanged; zeroing failed for both, with different outcome splits (code table).
W2, supervision through partners. The learner receives the projected convention input directly. Scripted partners follow the convention, so action consequences carry a convention-dependent training signal even though the reward never reads the text; the projection also has separate supervision. Section 4.3 now states the co-learning result as observed: the code swap changed the logits at one traced scene but no greedy action of either MAPPO actor on 100 scenes at any of three seeds. Each seed's fixed ordering completed every rollout under either code, so the reward could be met without reading the code.
W3 and Q2, independence, identity and more orders. Independence is bounded to separate runs, seeds and parameters, and different encoders in the mixed arms; framework, architecture and execution rule are shared. With mixed encoders at the four-learner intersection, every assignment reached F_M = 1.00 intact and 0.00 zeroed. In the original design tags are a function of position: when we made them conflict at evaluation, merge policies always chose the position-implied role, and at the intersection 82 to 96 percent of choices by approach followed position. A new north learner trained with letters permuted over approaches failed its locked letter criterion (0.8333 against 0.95; 0 of 8 fresh pairs; training reward plateaued at 3.3 to 7.0 of 10), although it never chose the position rank in conflict cases (0 of 23,040 records per seed). We therefore claim convention-dependent actions at trained positions, not identity resolution. We also tested four trained crossing orders in one intersection with a newly supervised four-class projection and new policies. The language arm failed its locked all-seed criterion (O1/O2 pairs 2/8, 0/8, 3/8; O3/O4 2/8, 3/8, 4/8; one of 12 runs ended training at reward 3.65); an oracle-code control, given one code per order, was correct on all 80 scenes per order and seed (live execution not run). In 7,680 live episodes, 3,291 completed in the instructed order, 2,561 in a different trained order, and 1,828 ended in a conflict-box collision after non-canonical rank vectors. This is an unsuccessful extension of the tested single-swap readout, not evidence of four-order grounding.
W4 and Q3, paraphrase failure. On 16 pairs balanced by first-mentioned vehicle, MiniLM correctness was 0.469 to 0.500 by seed; SGPT depended on mention order. A recreated MiniLM classifier scored 1.000 on training phrasings and 0.500 on the original held-out pairs.
Q4, rank execution. Frozen policies committed ranks as before; to execute them, each vehicle used only its own state, simulated local sensing within 70 m and once-per-second rank messages, held on missing or duplicate ranks, and entered only after every car ranked ahead of it had cleared. All three teams passed the locked test (1,280/1,280 completions each, no conflict-box collision, deadlock or order mismatch), and independent 10% and 30% loss of rank messages still gave full completion without collision or deadlock. Every tested rank-2 to rank-4 non-compliance intervention triggered the conflict-box collision criterion. This is a simulation check of a cooperative executor that assumes a shared rank-message format and compliant vehicles, not fault-tolerant autonomy.