- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies human–machine coordination among several connected autonomous vehicles crossing an intersection together, and proposes CG-HMT. It places every vehicle and one remote human supervisor in a directed trust graph. Each vehicle has three things: its belief about the scene, its own driving intent, and the uncertainty of these two. Each vehicle aggregates the others’ beliefs and intents according to trust weights. Trust values are updated automatically from three indicators: each vehicle’s performance, how consistent their judgments are with one another, and calibration. The supervisor can fine-tune trust values. When a vehicle’s judgment deviates from the group, its uncertainty is too high, or the intent it expects from other vehicles does not match what they actually report, the system prompts the supervisor. Training uses PPO. Each vehicle’s actor loss is multiplied by the average trust it receives times its confidence. Experiments are run at one unsignalized intersection in the CARLA simulator, with three vehicles and three seeds in total. Under clean conditions, CG-HMT reaches a success rate of 0.75, versus 0.20 for the strongest baseline (MAPPO). On a single seed, the paper also tests three perturbations: stale camera frames, adverse weather, and one vehicle driving randomly. The results show that trust toward the faulty vehicle drops to about 0.19. Five operators rated the supervision dashboard 4.83/5.
Critical strengths
The problem the paper addresses is genuinely interesting, and under clean conditions CG-HMT’s success rate is clearly higher than the baselines’ (0.75 vs. 0.20 for MAPPO). The authors are candid about the limitations of their evidence.
Critical weaknesses
-
The trust mechanism does not explain the gain. The trust mechanism never enters the policy; it only serves as a weight on the actor loss during training. In the paper’s own ablation, success remains 0.55 after removing trust weighting and human input, so most of the gain cannot be attributed to the trust mechanism. The main gap actually comes from baselines timing out at low speed at a symmetric intersection. If vehicles have different weights, that alone could break this “nobody goes first” deadlock, but the paper includes no shuffled-weight control.
-
The described system differs from the implemented one. The belief encoder and the vehicle-to-vehicle trust edges described in the main text do not exist in the implementation. The so-called evidential uncertainty is in fact a function that decreases deterministically with the time step. Readers cannot tell which system the experiments actually evaluate.
Questions for authors
-
Since trust is not an input to the actor, can supervisor labels change vehicle behavior at execution time? If not, where does human–machine teaming take effect at deployment?
-
With trust weighting and human input removed, the ablation still reaches 0.55 SR versus 0.20 for MAPPO. What accounts for this gap?
-
Can a control with shuffled or random per-vehicle constant weights reproduce the 0.75 SR? This would test whether the gain comes from breaking the symmetric-intersection deadlock rather than from trust.
-
Which system do the experiments evaluate? Specifically, how is the belief μ computed (is there a learned encoder?), and how are agent-to-agent trust edges computed?
2 comments
Thank you for the detailed response, the revised PDF, and for running the shuffled-weight, time-schedule, matched-pipeline and retuned-baseline controls.
W1 (does the trust mechanism explain the gain). The new controls answer my Q2 and Q3, and they change the headline. The revised Section 6 now states that the 0.55 SR of the "no human feedback" variant reflects the training pipeline, and that much of the gap in Table 1 does as well. Against matched or retuned MAPPO, CG-HMT's margin is 0.13 to 0.20 SR, where Table 1 showed 0.55. With 3 seeds of 20 episodes, the pooled tests give p = 0.12 against the time schedule (Appendix B.4) and p = 0.17 against the best retuned baseline (general response). The revision itself calls this evidence "suggestive rather than conclusive". I agree with that reading, which is more cautious than the summary in the general response.
W2 (described vs. implemented system). The revision now states in Section 5 that the belief is a moving average of simulator position and speed with no learned encoder, that trust is one weight per agent, and that trust never enters the actor's input. Stating this is the right fix. It also means that what was evaluated is a per-agent, time-varying weight on the actor loss, which is much narrower than the trust graph and mental models the paper presents.
I hope these comments are useful for the revision.
Response to Reviewer pDqS
We thank the reviewer for a detailed summary and for proposing the shuffled-weight control, which we have run. We answer the four questions in order.
Q1. Can supervisor labels change vehicle behavior at execution time? Where does HMT take effect at deployment? CG-HMT incorporates human input through training: operator labels (420 of 12,600 prompts across three seeds) adjust the trust weights that scale the actor loss, and this shapes the learned policy. At deployment, the dashboard provides monitoring and divergence alerts to the supervisor, while the policy executes without trust as an input. The revision states this scope explicitly in §5 and the conclusion, and identifies trust-conditioned policies as a direct extension for execution-time intervention.
Q2. What accounts for 0.55 versus 0.20? The training pipeline: CG-HMT's pipeline differs from the original baseline configurations in rollout workers, parameter sharing across agents, learning rate and critic setup. Under the CG-HMT pipeline, MAPPO reaches 0.550 and IPPO 0.483 over three seeds; with every baseline retuned at an equal budget, the best (MAPPO) reaches 0.617. CG-HMT (0.750) is the strongest method in every case, and the revision reports these matched comparisons as the headline result (new Table 3).
Q3. Can shuffled or random per-vehicle constant weights reproduce 0.75? No. Shuffled constant per-agent weights reach 0.567 SR (0.55 / 0.55 / 0.60), against 0.750 for CG-HMT (+0.18, seed-level 95% interval [0.08, 0.29]). CG-HMT's lowest seed (0.70) is above every seed of the control, so arbitrary asymmetric weighting does not account for the result. We also ran a time-schedule-only control (see Q4), which reaches 0.600. Both results show that trust-based weighting provides a gain beyond symmetry breaking. We thank the reviewer for suggesting this test; it is reported in Table 3.
Q4. Which system do the experiments evaluate? The CARLA instantiation is a compact special case of the general formulation:
The revised main text (§3, §5 items 6–7, Fig. 1 caption) states this instantiation explicitly. We hope these answers and the new controls address the reviewer's concerns.