Colouring motion: forward-process design for multi-agent trajectory diffusion

Diffusion models are a popular choice for multi-agent trajectory generation. Typically, the noising process uses stochastic white noise, even though real trajectories evolve smoothly over time, with the position at one time step affecting the next. We argue that this mismatch presents an extra burden to the denoising process. In this paper, we replace white noise with a range of temporally correlated noise fields that can be tuned to suit the underlying process being modelled. Sport provides us with large standardised datasets to work with. Compared to the same model trained with white noise, correlated noise models reduced averaged scene error by 5.1% on association football and 2.3% on American football. Our best model beats the current state-of-the-art on a public sport trajectory generation benchmark, improving averaged and best-of-20 scene error on Bundesliga football by 9.6% and 5.9%. We further explore the properties that make temporal noise fields effective, showing that slower-varying noise helps performance. The training objective is equally important. Whitening the loss has a negative effect on the results compared to plain squared loss. We also demonstrate that at 25Hz, varying the noise by agent type makes the ball more realistic. These results establish the importance of tuning the forward process in multi-agent diffusion models.