The Allocentric Shift: Multi-Ego Contrastive Perceptual Alignment from Coordinated Behavior

Efficient Human-Agent Teaming (HAT) benefits from implicit coordination. Most teaming research relies on explicit communication, but in fast, reactive environments communication is often constrained, so each member must infer where teammates are and what they are doing from its own first-person view. Taking a video-understanding perspective, we ask whether this allocentric awareness can be learned from the observations coordinated teams naturally produce. We introduce X-Ego-CS, 273 hours of synchronized first-person gameplay from 101 high-skill Counter-Strike 2 matches, with a 33-task benchmark measuring what one player's view reveals about the player, teammates, opponents, and game. Aligning teammates' synchronized representations with Cross-Ego Contrastive Learning (CECL) produces a consistent Allocentric Shift, a trade-off in which other players' locations become more predictable across encoders and horizons at a modest cost to self-centric prediction, especially of the observer's own motion. Controlled experiments on held-out matches show that the gain grows with how much team state paired views share, not with alignment itself, and that aligned teammates become similar even when far apart. Without text supervision, a vision-language encoder's concept rankings also move away from self-centric phrases, and in a two-agent ViZDoom task the objective improves the sample efficiency of a multi-agent reinforcement learning baseline. These results suggest that the natural correspondence in recorded teamwork could offer a scalable path toward agents that sense their team's situation and coordinate without explicit communication.