Q-PhysJEPA: Query-Conditioned Physical Abstraction for JEPA World Models

Embodied world models should preserve the physical evidence needed for the task at hand, yet accurate latent prediction and physical-property decoding do not by themselves establish decision relevance. We introduce Q-PhysJEPA, a query-conditioned joint-embedding predictive architecture that incorporates task queries before low-dimensional history compression, rather than only during candidate evaluation. The model aggregates visual, proprioceptive, and action histories into shared evidence slots, reweights these slots through query-conditioned gating, and compresses them into a task-dependent representation. It then predicts future observation representations for candidate control settings and estimates their utilities from the predicted futures. Future prediction, utility regression, and candidate ranking are jointly trained using observed executions. We examine query-relative physical content through property readouts, candidate-future predictions, and utility estimates under matched query switches and physical interventions. On ManiSkill manipulation tests with physical parameters outside the training ranges, Q-PhysJEPA achieves 56.29% top-1 candidate-selection accuracy, averaged equally across four task objectives, compared with 46.83% for a query-independent representation baseline, an improvement of 9.46 percentage points. Controlled analyses further suggest that greater query-dependent physical selectivity does not necessarily translate into better decisions. These findings support task-conditioned history compression while highlighting the distinction between accessible physical information and its usefulness for candidate selection. Codes are available at https://anonymous.4open.science/r/Q-PhysJEPA-30A7\faGithub.