acceptodds
Withdrawn from ICLR 2027

HADP: Game-Grounded Hierarchical Adaptive Dialogue Policy for Aligning Actions and Utterances in MOBA Team Coordination

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Competitive multiplayer games require AI teammates not only to choose useful actions but also to honor what they tell players. We study this say–do gap in MOBA team coordination and propose Hierarchical Adaptive Dialogue Policy (HADP). HADP grounds player requests in a shared Mission representation, generates coordinated action and dialogue responses, and reviews candidate outputs against evolving context and memory. We evaluate alignment at three levels: semantic consistency between utterances and issued plans, execution consistency between activated plans and agent behavior, and interaction consistency of commitments across turns. In the evaluated closed-loop scenarios in Honor of Kings, HADP achieves 91.5%, 86.3%, and 76.5% consistency at these levels, respectively, compared with 81.3%, 67.5%, and 51.5% for a flat policy with budget-matched self-refinement. Module ablations show that removing Understanding lowers semantic consistency by 18.0 percentage points, while removing Adaptive Review lowers interaction consistency by 15.3 points. These results support complementary roles for mission grounding and contextual review within the tested setting. Supporting trace-based experiments assess macro-action prediction and command quality after supervised fine-tuning, GRPO, and command-channel self-distillation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Anthropomorphic AI, dialogue policy learning, game AI, hierarchical reinforcement learning, multi-agent systems

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_A8wCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies the say–do gap of LLM teammates in MOBA games — what the agent says diverges from what it does. HADP is a three-stage hierarchy: Understanding grounds player requests and game state into a structured Mission; the Response Generator jointly produces action and dialogue from that Mission and an RL policy's tactical proposal; Adaptive Review screens candidates against short- and long-term memory before execution. In closed-loop Honor of Kings evaluation, HADP scores SC 91.5%, EC 86.3%, and IC 76.5%, vs. 81.3% / 67.5% / 51.5% for a budget-matched flat baseline.

Critical strengths

S1. Problem formulation — a useful diagnostic taxonomy. Decomposing the say–do gap into SC (utterance–plan), EC (plan–execution), and IC (commitments across turns) is clean and reusable, separating types of inconsistency from persona consistency and human-likeness. The taxonomy itself is a strength — the paper's most reusable contribution; W2/W4 concern its validation, not its definition.

S2. Architecture matched to the problem, corroborated by ablations. Each stage targets one failure source, and the ablations dissociate as predicted: removing Understanding hurts SC most (−18.0 pts), removing Review hurts IC most (−15.3 pts). The budget-matched baseline, under the same backbone, data, and scenarios, makes the comparison interpretable.

S3. Multi-level, practically relevant evidence. Closed-loop evaluation, offline probes, command-quality ratings, and production deployment complement each other; the deployment evidence in particular supports the practical motivation.

Critical weaknesses

W1. Mechanism not isolated. Integration is the claimed contribution, yet its parts are not separated. Review is pre-execution only, so nothing monitors EC once a plan runs; IC rests on memory-based LLM review, with no explicit pending-commitment state or trigger tracking. The RL proposal and joint action–dialogue generation are never ablated independently. Worse, conditioning both channels on one Mission allows common-mode failure: a wrong Mission can be executed consistently and still score high on SC.

W2. Consistency is not shown to imply good decisions. The gap is validation, not definition: consistency, defined independently of task success, is never reconnected to it. The Mission (player intent) and the RL proposal (tactical judgement) can conflict, yet arbitration among cooperation, refusal, and delayed cooperation is left to the SFT generator, with no criteria, conflict frequency, or outcomes reported. A compliant but tactically poor agent could therefore score highly. No closed-loop task success is given to rule this out, nor the diagnostics that would locate failures: Review pass/fail, suppression and regeneration rates, or SC/EC/IC failures split by successful vs. failed trajectories.

W3. Latency uncharacterised, despite being a hard constraint. A decision is only useful within the validity window of its input state; delay itself creates say–do inconsistency. HADP chains at least three sequential LLM calls, plus RL inference and retrieval, with regeneration adding tail latency — yet only an approximate P99 for the deployed command-channel model is reported: no end-to-end or per-module latency, no wall-clock comparison with the baseline (budget matching equalises compute, not time), and no latency vs. SC/EC/IC analysis, despite the paper noting replies going stale in team fights.

W4. Evidence is not auditable or generalisable. The closed-loop study uses a proprietary game, data, and stack over a narrow templated design (5 roles × 8 commands × 10 trials), with scenario construction unspecified and a single game/patch throughout. The supporting studies do not close this gap: the offline probe measures agreement with one logged action, not decision quality; production results omit cohort sizes, absolute rates, and uncertainty; Appendix G's key quantities — repair rate r, damage rate f — are never estimated.

Questions for authors

Q1. When Mission and RL proposal conflict, what criteria govern cooperation / refusal / delayed cooperation? How often do conflicts occur, which side prevails, and with what task outcomes?

Q2. How are SC/EC/IC adjudicated, and what is the inter-annotator agreement? Please report closed-loop task success alongside consistency.

Q3. What are the end-to-end and per-module latencies (including regeneration), how do they compare with the baseline's wall-clock time and the state-validity window, and how do SC/EC/IC degrade with latency?

Q4. How were scenarios constructed and sampled, and what protocol, data, or code can be released? Please also give cohort sizes, absolute rates, and uncertainty for the production results.

Q5. Is there an explicit pending-commitment representation, and what monitors temporal triggers and interrupted instructions? What are Review's pass/fail, suppression, and regeneration rates, and can r and f in Appendix G be estimated?

Reviewer_3wmpOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work proposes the Hierarchical Adaptive Dialogue Policy (HADP), a system consisting of fine-tuned or pretrained neural networks alongside a harness, aimed to resolve the "say-do gap" in agentic interactions in MOBA games.

The challenge is that agents typically dont follow cooperative isntructions well, even if they communicate that they intend to.

HADP's system consists of an Understanding module and an Adaptive Review module around the response generator. The former is aimed to ground the incoming messages in observations and short term memory, to produce a clear mission to the response generator. The latter's job is to use the response, the predicted next action and the long term memory to produce a synchronised action-utterance pair.

Another contribution of the work is an evaluation system, which shows that the HADP framework functions better than naive baselines. This is a meaningful contributioin as this aspect of agentic operation has no established measurement or baseline - it is a new area.

Other than the proposed evaluation system, the authors also demosntrate that HADP functions well in offline trace-based benchmarks (macro-action prediction, QA, and command quality) and a production A/B rollout.

Critical strengths
  • Clear isolation of an important problem: this work pinpoints a specific and significant failure mode of these systems and explicitly aims to measure it and treat it. For this, a sensible-looking taxonomy of the say-do gap's evaluation is introduced. Further, the Mission interface and Adaptive review create a clear and separated fix for this problem.

  • Robust empirical validation with thorough ablations: Demonstrating the system's utility in live deployments and other metrics makes the results more trustworthy. The authors took time to explain which components of the system contribute what, including finetuning methods and GRPO or self-distillation, and different base models and model sizes. Meeting real-life latency budgets with LLM-s in the loop is an impressive engineering feat as well.

Critical weaknesses

Because scoring SC, EC, and IC requires manually hand-crafting dozens of arbitrary task-specific rules and deadlines for every new environment, it is both prohibitively labor-intensive to set up and impossible for the research community to standardize—leaving any future authors who use these metrics unable to prove to reviewers that their pass/fail thresholds were not simply tuned to favor their own method.

  • Informal Metrics: The paper claims the formalization of Semantic, Execution, and Interaction Consistency as its first core contribution (Section 1). However, mathematically, the only "formal" equations provided for these metrics are a standard binomial average (Eq. 1) and a textbook Wilson score interval (Eq. 2). The actual operationalization that maps a trajectory to a binary 0/10/1 decision (ziv,Lz_i^{v,L}) relies entirely on undisclosed, hand-crafted "behavioral predicates and deadlines" across 40 position-command combinations, combined with manual human (or LLM? This remains undisclosed) judgement for "ambiguous cases" and subjective judgments of what constitutes a "justified revision" under IC (Section 4.1, Table 5). While Appendix H (Table 13) attempts to frame this as a feature—separating the "reusable core" (the conceptual distinction between SC, EC, and IC) from "environment-specific adaptation" (manually redefining all behavioral predicates, trigger conditions, observation windows, and validity rules for every new task)—this admission highlights the core limitation of the metric framework:
    • Why this is consequential: A metric whose binary pass/fail boundary must be manually hand-engineered by domain experts (and arbitrated by humans) for every individual command, role, level, game phase and game release is heavily dependent on subjective designer taste (e.g., choosing a 5-second vs. 15-second deadline, or deciding which state changes "justify" breaking a commitment). Even if the authors release the 40 bespoke predicates for Honor of Kings in the appendix, this approach requires prohibitive manual labor to port to any new domain and prevents meaningful comparisons across environments, roles, or difficulty levels. Consequently, SC, EC, and IC function as a domain-specific QA rubric rather than a generalizable, transferable evaluation metric for the multi-agent community.
  • Irreproducibility: The primary closed-loop experiments (Section 4.1) and live deployment (Section 4.6) are inextricably tied to an internal Honor of Kings engine runtime that serializes privileged game-state telemetry into JSON (Figure 4) and dispatches macro-actions directly into a proprietary pre-trained RL controller (Section 3). While the Reproducibility Statement offers to release offline scenario statistics and rubrics, external researchers cannot access the closed-loop simulator, the underlying RL execution policy, or the exact evaluation predicates needed to verify or build upon the primary results in Table 1. This opacity is compounded by presentation inconsistencies regarding the system's operational boundaries—such as Figure 1 depicting pixel-level visual frame extraction and audio processing (alongside scrambled callout text) when the pipeline actually consumes structured engine JSON and text, Figure 2 omitting state inputs to the pre-trained RL policy, and the text overclaiming that an LLM-based parser "guarantees deterministic goal specification".
    • Why this is consequential: When both the execution environment and the metric evaluation rules are proprietary and closed-source, the paper's core empirical claims (the 2,000 closed-loop trajectory trials) are completely unverifiable and irreproducible by external researchers. Consequently, the work reads more like an internal, game-specific engineering case study than a reusable, scientifically verifiable framework for the broader research community.
Questions for authors
  1. Metric Standardization, Setup Labor, and Threshold Tuning (SC, EC, IC): How are the binary 0/10/1 decisions (ziv,Lz_i^{v,L}) in Section 4.1 computed across the 40 position-command combinations (please provide the full list of 40 commands, behavioral predicates, and deadlines)? More broadly, because applying SC, EC, and IC to any new environment requires manually engineering dozens of bespoke predicates and deadlines from scratch (Table 13), how can the community standardize these metrics across studies, and how can future researchers prove to reviewers that their chosen thresholds were not simply tuned to favor their own method?
  2. Closed-Loop Reproducibility: Given that the internal Honor of Kings simulator, JSON state extractor, and pre-trained RL controller are proprietary, how can external researchers independently verify the 2,000 closed-loop trials in Table 1 or evaluate new policies under your closed-loop protocol?
  3. Figure 1 & 2 Corrections: Does the system ever take raw pixel frames as input, or is Figure 1 purely illustrative of the human player's UI? In the same figure, the surronding text seems incomplete or wrongly arranged. Did you mean to say

Visual Perception: The AI

extracts game frame data to

perceive the player’s field

of view, positional

information, and the state

of the game environment.

requests.

Auditory Processing: The

AI must process the

player’s auditory signals

and speech, and interpret

the underlying intent or requests.

Please also clarify the inputs to the Pre-trained MOBA RL module in Figure 2.

  1. Ambiguity Adjudication & Human Rater Details: Who were the "two model-blind annotators" in Section 4.1 (and the expert raters in Section 4.3), what criteria flagged a trial as an "ambiguous case", what percentage of the 2,000 trials required human adjudication, and what was the inter-annotator agreement?
Reviewer_7vn9OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This work introduces HADP, a hierarchical framework that improves LLMs' consistency between what they say and do in Honour of Kings. The paper evaluates the model's consistency across turns and gauges agreement between dialogues/plans and action executions. Experiments with Qwen3-8B and 14B using SFT, GRPO and command-channel self-distillation show improvements in offline action prediction and command quality. In the reported closed-loop comparison, HADP attains SC/EC/IC of 91.5%/86.3%/76.5%, compared with 81.3%/67.5%/51.5% for the compute-matched flat self-reflection baseline.

Critical strengths
  1. The work addresses a practical problem of cooperation between AI agents or Human-AI interactions.
  2. Evaluation covers both decision-making and communication. The work talks about action prediction, open-ended game questions, command actionability, and wording quality. It also includes blind expert ratings and report agreement between the experts and the LLM judge, rather than relying entirely on automated evaluation.
Critical weaknesses
  1. The work does not establish why the proposed architecture is needed over a simpler action-grounded dialogue system. The cited Hierarchical Language Agent [1] already interprets human instructions, passes the inferred intention to an action generator, tracks completion, and generates corresponding dialogue. HADP’s contribution therefore needs to be more specific than separating understanding, actions, and communication. A relevant baseline would select an executable plan first, generate dialogue conditioned on that plan, and maintain a structured record of pending and cancelled commitments. The reported flat and self-refinement baselines do not establish whether HADP improves over this simpler way of preventing contradictions. Without this comparison, it is unclear how much the gain comes from the proposed Mission-and-review design rather than simply tying language generation more tightly to the action plan.

  2. Table 9 has serious inconsistencies with Table 2 of Think in Games [2]. All four numerical rows are identical, but the first two are reported for Qwen3-8B instead of Qwen2.5-14B. Several benchmark labels are also different. For example, the original paper reports 95.7 on SchoolChinese and 3.05 on CharacterEval, whereas HADP reports these as MMLU and BBH scores. This raises concerns about whether Table 9 contains results from the models and evaluations claimed in this paper.

  3. Frontier models' performance on the task is not reported. HADP evaluates GPT-4o and DeepSeek-R1 on supporting offline tasks, and the main closed-loop consistency comparison contains the flat policies and HADP variants. It does not show how a recent frontier model performs at maintaining and executing commitments to validate whether we need HADP in frontier-scale training.

[1] Jijia et al. LLM-Powered Hierarchical Language Agent for Real-time Human-AI Coordination https://ifaamas.org/Proceedings/aamas2024/pdfs/p1219.pdf

[2] Yi Liao et al. Think in Games: Learning to Reason in Games via Reinforcement Learning with Large Language Models https://arxiv.org/abs/2508.21365v1

Questions for authors
  1. Do the model's gains transfer to other scenarios?
  2. Why are all four rows of Table 9 numerically identical to Table 2 of Think in Games, despite different model names and benchmark labels? Were these benchmarks independently evaluated on the HADP checkpoints? Could the authors provide the evaluation configurations and correct the table?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.