Learning Perspective Matters in Latent World Models

Latent world models often organise prediction along different latent directions, yet the perspective governing these directions and their use is typically predefined rather than learned. This leaves an important design choice outside the learning process, even though both the choice of directions and their relative weighting can shape predictive computation. We introduce orthogonal subspace modulation, which jointly learns an orthonormal basis and adaptive gains conditioned on the current state and action. The basis determines where modulation acts, while the gains determine how strongly each direction is used. We formalise their interaction through Gain–Basis Coupling, showing that the latent perspective becomes functionally consequential when different subspaces receive different gains Across four visual-control tasks, learning this perspective improves control performance over strong latent world-model baselines. Detailed analyses further reveal the empirical validity of Gain–Basis Coupling, functional coordination among subspaces, and physical-dynamics accessibility from subspace combinations. These results show that learning perspective can improve how latent representations are organised and used for prediction and control.