HealthSAW: Scaling Health World Model with a Forecast-Driven Bottleneck

Longitudinal patient healthcare records are widely considered a key source of clinical intelligence, but incorporating them into foundation models through next-token prediction has achieved limited success. This paper introduces HealthSAW, a Health State–Action World Model that represents patient healthcare records in a learned higher-order representation space, using local encoders to abstract tokens about a patient's health observations into health state vectors and treatment tokens into action vectors. These state–action representations enable a new pretraining paradigm in which the world model can autoregressively predict future events, including diagnoses, treatments, and responses to treatment. To make these vectors capture the patient's health rather than trivial token patterns, HealthSAW introduces a forecast-driven bottleneck: local decoders generate the tokens of the next state/action rather than reconstructing the current ones, and access prior patient history only through these higher-order state–action representations. Our experiments on about 19 million patients and 28 billion healthcare record tokens establish scaling laws for HealthSAW and demonstrate its superior performance across a wide range of disease-risk and treatment-outcome prediction tasks, including generalization to health systems unseen during pretraining. The world model architecture also enables new zero-shot patient simulation capabilities for predicting post-treatment laboratory outcomes, while being $1.8\times$ and $3.2\times$ faster than token-level models in pretraining and simulation, respectively. Further studies demonstrate the necessity of abstraction into the higher-order representation space, the benefits of this abstraction for context efficiency, especially with richer patient observations, and the generalization advantage of the forecast-driven bottleneck.