Foresight on Demand: Learning What and When to Predict for Autonomous Driving
Abstract
World action models predict the future to guide planning, but not all predicted information is useful for planning and not every scene benefits from additional prediction. Foresight can waste computation when history already supports a reliable plan, and can even degrade planning when its predictions are inaccurate. We present FD-WAM, which couples what to predict with when to predict for better planning, with the added benefit of reducing unnecessary foresight computation. Specifically, a Decision-Relevant Foresight Representation uses planning supervision to shape future slots learned through latent prediction, then compresses predictions from multiple future horizons into a compact set of tokens for planning. Utility-Aware Selective Foresight trains a router with the realized planning score difference between the same planner's foresight and no-foresight paths, enabling selective prediction from observed history alone. Evaluations on NAVSIM v1, NAVSIM v2, and zero-shot HUGSIM show that FD-WAM achieves competitive planning scores at lower inference cost than state-of-the-art baselines under aligned training scale. Controlled ablations further show that the compact foresight representation and selectively invoked foresight improve planning over the no-foresight baseline with little additional inference cost.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.