PAIMON: An Aerial World Tracking Model Bridging Simulation to Reality
Abstract
Scaling high-quality supervision for aerial target tracking is challenging, as real-world flights rarely provide synchronized geometric and action supervision. Although simulation offers such supervision at scale, the appearance gap hinders sim-to-real transfer. We present PAIMON, a dual-expert MoT world action model that decouples action-relevant geometric learning from appearance adaptation. PAIMON translates simulated RGB observations toward real-world appearance with a structure-preserving image editor, while action-conditioned future-depth prediction provides geometric supervision during training. At deployment, PAIMON directly predicts actions from current observations, visual and ego-motion histories, and target instructions, without explicit video rollout. We further introduce the Aerial Tracking Platform (ATP) for rule-based expert data collection and closed-loop evaluation with high-fidelity 3D Gaussian-splatting scenes. PAIMON achieves the highest average closed-loop success and tracking rates among the evaluated baselines. Real-world flight tests further demonstrate transfer of simulation-learned tracking behavior to RGB-based closed-loop tracking under varying flight conditions. Code and data will be released upon publication.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.