AutoFlow: Autoregressive Normalizing Flows for Next Vector Prediction

Next-token prediction gives language models a simple interface for both understanding and generation. Extending this idea to continuous signals such as images and robot actions is challenging because these signals are continuous vectors rather than discrete tokens. We introduce Autoflow, a framework for next-vector prediction in autoregressive foundation models. The key idea is to equip a language model with a conditional normalizing flow that predicts the next continuous vector while leaving the autoregressive backbone unchanged. We find that this simple formulation better connects generation and understanding than diffusion-based alternatives: models trained for image generation transfer more effectively to visual understanding, while also supporting interleaved image–text reasoning and robot action prediction. Autoflow further generates continuous outputs in a single forward evaluation and provides their likelihood directly. Our results suggest that next-vector prediction via Autoflow is a promising interface for extending autoregressive foundation models beyond discrete tokens.