Probabilistic Knowledge Graphs for Embodied AI: Joint Inference via Message Passing over Entities, Relations, and Affordances

Embodied agents that operate in open worlds must keep structured beliefs about objects, their relations, and their actionable properties while observations remain partial and noisy. Recent work encodes such beliefs as knowledge graphs whose nodes and edges carry uncertainty, yet inference stays local. Each entity is filtered independently, beliefs are never propagated along graph edges, and relation evidence is not fed back into entity states. As a result coupled errors are underestimated and the model cannot return calibrated marginals that tell the agent when another observation is worth its cost. We treat the embodied knowledge graph as a factor graph and perform joint inference by message passing. Observation, relation, temporal, and affordance potentials define a joint distribution over entities, relations, and affordances. Closed form Gaussian messages handle the linear potentials, and a variational linearization of the relation energy yields messages that close the loop between entities and relations. The inferred marginals are calibrated and drive active perception and robust anomaly detection. Across three question answering benchmarks, embodied scene understanding, affordance prediction, active perception, and physical adversarial attacks, ProbKG improves accuracy and calibration over a broad set of baselines, needs fewer observations to reach a fixed confidence, and recovers scene semantics faster under attack.