PartIMap: Online Open-Vocabulary 3D Panoptic Mapping as Partition Inference

Online open-vocabulary 3D panoptic mapping is a fundamental task in scene understanding. The central difficulty is fusing partial and noisy 2D masks from many views into consistent 3D instances. Existing online systems fuse with hand-designed rules that lack an explicit objective, which discard the uncertainty of partial observations, lead to potentially sub-optimal merging, and require special treatment if estimated poses change. We present PartIMap, which treats mask fusion as probabilistic inference. PartIMap maintains a probabilistic voxel map in which semantics are von Mises-Fisher distributions on the descriptor sphere, kept per instance and per voxel, and instance geometry is a per-voxel Beta occupancy state. Deciding which masks belong to the same object is cast as maximum a posteriori inference of a partition on a mask graph. This partition model retains probabilistic data association as its first-order relation while capturing higher-order relations such as transitivity, and yields soft membership weights from the same posterior. Every fused quantity is an additive sum over its constituent observations, and the map can be updated effectively and efficiently by re-merging when a live SLAM system closes a loop. PartIMap is zero-shot and requires no supervised training. On ScanNetV2 and Replica it outperforms both online and offline methods, improving the strongest online baseline by 3.2 AP and 3.9 PRQ(T) on ScanNetV2 at comparable speed, degrades the least when poses come from live SLAM, and exposes per-voxel uncertainty. Codes will be released upon publication.