Shape-Bayes: Bayesian Inference of Structured Shapes under Visual Ambiguity

Perceiving structured shapes, such as human faces, from pixels is an inherently ambiguous task under real-world conditions. Yet, shape inference is largely posed as a deterministic regression task, predicting fixed spatial coordinates. We find that deterministic regression is brittle when visual evidence is ambiguous or incomplete; under severe occlusions, deterministic models exhibit structural collapse, predicting incoherent shapes or reverting to generic averages. To address this, we introduce Shape-Bayes, a probabilistic framework that couples uncertainty-aware visual perception with Bayesian shape reasoning. Rather than forcing point estimates, Shape-Bayes dynamically weights confident visual evidence against geometric priors to infer a structurally coherent shape posterior. Using human face shape regression as a rigorous testbed featuring complex non-rigid deformations and strict anatomical constraints, Shape-Bayes comprises: (1) a base model that predicts noisy landmarks and aleatoric uncertainties distilled from a teacher; (2) a lightweight Transformer that encodes these observations into an adaptive prior over a PCA shape manifold; and (3) a differentiable Bayesian solver that computes closed-form posteriors by balancing the noisy predictions against this prior. By guaranteeing complete structural integrity (100% In-Distribution Rate), Shape-Bayes achieves an absolute improvement of up to  34% over state-of-the-art deterministic models. Simultaneously, it yields highly calibrated uncertainty bounds and establishes a new state-of-the-art for robust 2D face shape regression under severe occlusion, reducing relative error by up to 12.5%.