SpatialMind: Test-time spatial pointing via observation dependency graph

This paper presents a test-time framework for spatial pointing with frozen vision-language models. The presented method features a set of zoom procedures, which we call the registry, and treats their outputs as dependent observations. This casts test-time pointing as selection and acquisition. For selection, we propose SpatialMind (SM), a relational answer model that estimates the posterior of correctness of each observation. SM draws on the frozen model’s prefill states and messages over dependency relations, starting from a boosted-tree prior. For acquisition, we propose Adaptive SpatialMind (ASM), a practical extension of SM for reduced call budgets. ASM keeps SM unchanged and adds a cost-conditioned policy that decides what to acquire next and when to stop. Our analysis shows that SM learns to update its prior by Bayes’ rule in log-odds over dependent observations, and that ASM includes full acquisition and every fixed schedule as special cases. The proposed framework is applied to three frozen backbones on five public pointing benchmarks. The experiments demonstrate the accuracy of SM and the cost efficiency of ASM against both training-free and learned test-time methods.