Fixed-Point Reasoning: Learning Optimal Physical Representations of Dynamical Systems

Learning dynamical systems from observations is fundamental to scientific modeling, prediction, and control, while recurring structures across heterogeneous systems motivate foundation models that learn general representations. However, existing representation learning approaches lack a principled way to preserve physical fidelity. We introduce a core concept, optimal physical representation, defined by minimizing the inconsistency between the learned representation and the inferred physical dynamics. Specifically, we endow tokens with explicit physical meanings via a novel physics-aware encoding constructed from the estimated dynamics. This creates a critical feedback link for representation refinement, formulated as a cooperative Refiner–Estimator loop. To guarantee convergence, we further develop a novel Fixed-Point Transformer with explicit contractive constraints, driving the loop toward a self-consistent equilibrium. More broadly, this establishes fixed-point reasoning as a new paradigm for physical representation learning, where the model reasons over physical estimates through recurrent refinement until equilibrium. Extensive experiments across diverse dynamical systems demonstrate improved predictive accuracy, zero-shot generalization, stable long-horizon rollout, and progressive refinement toward the fixed-point equilibrium.