Understanding AI Teammates through Grounded Dialogue

Effective human–AI coordination requires people to understand and anticipate AI teammates' behavior. Yet plausible post-hoc explanations may not faithfully reflect the policy that selected an action. We present PolicyLens, an interactive system that provides evidence-grounded explanations of AI teammates' actions and the possible consequences of alternative human actions. Before deployment, its regularity-constrained policy distillation module extracts compact executable summaries of neural behavior; during collaboration, PolicyLens receives open-ended questions and constructs answers from interaction records, available program paths, or isolated simulations using the same fixed controller. We implement PolicyLens in Warehouse, Cooperative Pong, and an Overcooked-inspired Cooperative Kitchen, matching explanations to the controller used in each environment. In a 60-participant study across the three environments, 53.3% of the explanation group gave positive ratings (5–7 on a seven-point scale) on the item about understanding when their teammate waited or changed plans, compared with 30.0% of the control group, a 77.8% higher proportion. This overall result provides preliminary support for improving participants' perceived understanding of AI teammates. PolicyLens offers a concrete approach to supporting human–AI collaboration through dialogue grounded in verifiable evidence.