acceptodds
Under review as a conference paper at ICLR 2027

Modeling Uncertainty Propagation in 3D Morphable Models

Abstract

We study the problem of mapping parameter-space variance of a 3D Morphable Model (3DMM) to per-vertex variance. Traditionally, per-vertex variance has been used to visually interpret the learnt parameter uncertainty. However, mapping parameter-space variance to per-vertex using sampling or error propagation is computationally expensive. We address this limitation with Closed-form Local Uncertainty Estimation (CLUE), which transforms the parameter-space distribution of 3DMMs to per-vertex through a closed-form approximation. CLUE emerges from applying the law of total variance to the 3DMM formulation and is modular, computationally inexpensive, and accurate. Subsequently, we study integrating CLUE for in-the-wild human mesh recovery training and inference. We note that CLUE enables probabilistic mesh alignment as a training objective, and uses the geometry of the mesh as an inductive bias when predicting the variance. Through extensive qualitative and quantitative experiments, we show that when used in training, CLUE (1) outpeforms probabilistic and deterministic baselines on metrics evaluating joints, mesh and uncertainty, (2) is at par with SAM 3D Body on 3DPW using fewer training samples, and (3) has the highest uncertainty-to-error spearman correlation, with applications including data mining and interpretable uncertainty. Finally, we share SAM 3D Body (CLUE) which augments the regressor with high resolution uncertainty maps.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Morphable Models, Digital Humans, Uncertainty Estimation, Real-Time

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_a6bhOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces CLUE, a computationally efficient approximation for propagating parameter uncertainty to per-vertex positional variances in 3D morphable models. Using the law of total variance and a first-order treatment of skeletal rotations, CLUE combines shape, expression, and pose uncertainty without explicit Jacobian computation or repeated mesh sampling. The method also enables vertex- and joint-space likelihood objectives for probabilistic human mesh recovery. Validation includes comparisons with Monte Carlo sampling and Jacobian-based propagation on FLAME, SMPL-X, and MHR, alongside reconstruction results on 3DPW, pseudo-ground-truth evaluations on seven datasets, and training-loss ablations. The reported results demonstrate substantial computational savings and improved reconstruction accuracy under the evaluated settings.

Critical strengths

Substantial computational savings for spatially localized uncertainty estimation. CLUE exploits the structure of 3D morphable models to estimate per-vertex variances without repeated mesh sampling or explicit Jacobian computation. On FLAME, the reported runtime is 3.4 ms, compared with 226.4 ms for Jacobian-based propagation, while additional memory decreases from 410.4 MB to 4.3 MB. This efficiency makes localized uncertainty estimation more practical for resource-constrained applications and constitutes a meaningful contribution. Demonstrated benefits of geometry-aware probabilistic supervision. Beyond uncertainty visualization, CLUE enables vertex- and joint-space likelihood objectives that improve reconstruction under the same training protocol. On 3DPW, it reduces MPJPE/PVE from 58.9/69.6 mm with parameter-space NLL to 56.2/66.1 mm. Training-loss ablations further support the contribution of the geometric objectives, providing evidence that the method offers practical value beyond accelerating uncertainty propagation.

Critical weaknesses

Limited validation of uncertainty accuracy and calibration. The statistical evaluation relies primarily on visual error maps, without a clear characterization of approximation error across parameter-variance magnitudes and pose configurations. Moreover, uncertainty evaluation on seven datasets uses SAM 3D Body predictions as pseudo-ground truth, measuring agreement with a reference model rather than directly establishing calibration against true geometry. Although the 3DPW results support improved reconstruction accuracy, stronger independent uncertainty evaluation is needed to substantiate the proposed downstream uses.

Questions for authors

Clarification of approximation accuracy. What parameter means and variance ranges were used in the statistical experiments? How does the error relative to the 10,000-sample reference vary with uncertainty magnitude and pose configuration, particularly when the small-rotation and fixed-joint-location approximations become less accurate? Evidence for uncertainty calibration. Do you have calibration results against independent 3D ground truth, such as empirical coverage of predicted confidence intervals? Since the seven-dataset uncertainty evaluation uses SAM 3D Body pseudo-ground truth, please clarify what evidence distinguishes calibrated uncertainty about true reconstruction error from uncertainty that primarily reflects disagreement with the reference model.

1 comment
AuthorsOpenReview ↗

Response to Reviewer a6bh

Dear Reviewer a6bh, we thank you for your review! Please find our responses below.

Q1. "Clarification of approximation accuracy…"

A1. Thank you for this suggestion. We now include a study of a sweep over the parameter variance and the MAE between the ground truth (10,000 MC samples) and estimates. This study is in Fig. 8, and tabulated in Table 6. The study is also referred to in the main paper at Lines 366–371. We set the pose randomly by setting parameters in {0,+0.3,−0.3}\{0, +0.3, -0.3\}. For each non-zero parameter, we consider its variance as a part of a sweep. The sweep goes from 0 input standard deviation to 0.25 radians collectively for all parameter variances.

The key takeaway is that CLUE tracks first-order error propagation closely while having pragmatic assumptions. The plots also highlight a general limitation with first-order propagation, which overestimates the output variance as the input variance increases. As a result, at larger rotation, shape, or expression variances, CLUE and first-order error propagation are inaccurate. However, the overestimation of output variance can be absorbed by the learnt parameter variances during training.

Q2. "Evidence for uncertainty calibration"

A2. We now also report AUSE on 3DPW and EMDB, which have ground truth SMPL meshes. AUSE compares whether the predicted uncertainty and the ground truth errors rank samples in the dataset in the same way. We note that CLUE has the lowest AUSE across both datasets. We use AUSE as a measure of uncertainty because while SAM 3D Body predicts MHR meshes, the ground truth of 3DPW and EMDB is in SMPL format. Mapping the uncertainty from MHR to SMPL is not trivial.

In general, using SAM 3D Body as the pseudo-ground truth for out-of-distribution data meant that both the ground truth mesh and the predicted mesh are in the same space of MHR.

Method3DPW AUSE ↓EMDB AUSE ↓
SAM 3D Body (released)––
Deterministic Training––
Parameter NLL0.1570.139
Matrix-Fisher0.2320.182
Normalizing Flow0.1450.120
CLUE0.1200.104
Reviewer_MbeJOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper is about turning parameter-space uncertainty from a 3D morphable model into per-vertex uncertainty on the mesh. The authors split the vertex variance using the law of total variance, leading to a closed-form expression that runs in a few milliseconds. They report better accuracy and uncertainty than parameter-NLL, Matrix-Fisher and normalizing-flow baselines.

Critical strengths
  1. This paper solves a practical problem in 3DMM models. Probabilistic human-mesh models predict uncertainty in parameter space. Seeing it on the mesh usually means sampling around 100 meshes, which is slow. A closed-form shortcut is useful.

  2. The derivation is clear. Splitting the variance into a pose part and a shape/expression part with the law of total variance is intuitive. The uncertainty is computed from the kinematic chain and grows with distance from the joint, following the geometric properties of 3DMM models.

  3. The method is fast and computationally cheap. It takes about 3–11 ms compared with 140 ms to about 7 s for sampling or Jacobian propagation, and uses far less memory. It also needs only standard tensor operators, so it fits on-device frameworks.

Critical weaknesses
  1. Would the author be interested in adding an ablation study on a method variant, that replaces CLUE with sampling in the training loop. Train the same model with vertex-NLL, but compute the per-vertex variance by naive Monte Carlo sampling instead of CLUE. If accuracy is similar, CLUE’s value for training is speed, not a better inductive bias. This could justify the claim of the method.

  2. Unproven bound in Assumption 4. The paper claims the neglected shape-induced joint displacement is “bounded by twice the shape-induced variation of the bone lengths,” but no derivation is given. Please either prove it or state it as an approximation.

Questions for authors
  1. How does CLUE’s runtime compare with an efficient Jacobian computation, such as forward-mode JVPs or a hand-written analytic Jacobian, instead of a generic autograd one?

  2. How is the joint-NLL variance computed? Joints are linear combinations of vertices, so this needs vertex covariances, which CLUE drops. Do you simply sum the diagonal terms?

  3. Can you report mean and standard deviation over several seeds? Several gaps in Tables 2 and 4 are under 1 mm.

1 comment
AuthorsOpenReview ↗

Response to Reviewer MbeJ

Dear Reviewer MbeJ, we thank you for your review! Please find our responses below.

W1. "...compute the per-vertex variance by naive Monte Carlo sampling instead of CLUE…"

A-W1. Thank you for this suggestion! However, implementing Monte Carlo during training is computationally infeasible.

We show a quick study with batch size 128, taking 100 MC samples (following Sengupta et al.), and the reparameterization trick for backpropagation. A naive implementation quickly runs out of memory. An efficient version with micro-batching is 15× slower than CLUE. If plugged into an actual training loop, we expect the computational requirements of MC to shoot up significantly.

MethodRig forwardsTime (ms)Time / CLUEPeak memory (GiB)Memory / CLUE
CLUE1281221.03.11.0
Monte Carlo N=100, naive12,928OOM on 80 GB–OOM–
Monte Carlo N=100, micro-batched 3212,9281,8551530.710

W2. "Unproven bound in Assumption 4…"

A-W2. Indeed, this was an assumption which is fortunately not critical for our derivation. We have removed this sentence to avoid any diversion. Thank you for your feedback!

Q1. "How does CLUE's runtime compare with an efficient Jacobian computation, such as forward-mode JVPs…"

A-Q1. Our results are with forward-mode Jacobian. This is updated in Lines 372–373.

Q2. "How is the joint-NLL variance computed?"

A-Q2. To compute joint-NLL, we assume that neighboring vertices are correlated. Hence the joint standard deviation is the weighted average of the vertex standard deviations, σjoint=∑vwv σv\sigma_{\text{joint}} = \sum_{v} w_v \, \sigma_v. These weights are from MHR's vertex-to-joint regressor. In contrast, assuming independence meant that the joint variance was underestimated by 5× to 13×. This is updated in Lines 305–308.

Q3. "Can you report mean and standard deviation over several seeds?"

A-Q3. We acknowledge this request, but performing this would be computationally infeasible. We unfortunately do not have the resources to train foundation HMR models and our baselines several times to give a meaningful statistical result. This is also the case for other works in Human Mesh Recovery, such as SAM 3D Body, which provide results from a single run. However, we understand the request, and ran each probabilistic baseline twice, including an improvement in parameter NLL stabilization during the second run. Our results and the ranking of the baselines remain unchanged.

(Updated text) Context on under 1mm gap: many of the under 1mm gaps for MPJPE and PVE are between SAM 3D Body and CLUE. This is also shows that SAM 3D Body and CLUE are indistinguishable based on 3DPW metrics, with the latter being trained on three times fewer samples. PA-MPJPE and PA-PVE are metrics which rotate and scale the predicted body to have best alignment with the ground truth mesh. As a result the gap on these metrics tends to be smaller than the non-aligned version.

Reviewer_TDoKOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents a nice and interesting idea. The key insight is that uncertainty in parameter estimates of 3DMMs do not directly tell us about the resulting uncertainty in the position of the actual mesh vertices. For example, uncertainty in a shoulder joint rotation leads to high uncertainty in the position of the hand vertices. The method involves adding a small head to estimates variances for 3DMM parameters to any parameter regression network and then propagating these uncertainties in closed form to the vertices of the mesh. The specific mechanism for doing this uses the law of total variance and is much more computationally efficient than Monte Carlo sampling or Jacobian-based error propagation.

In a bit more detail, the idea is:

  1. Because the shape and expression offsets are linear in their parameters, their contribution to vertex variance can be computed analytically. For independent parameters, this is essentially a weighted sum of blendshape basis coefficients multiplied by the corresponding parameter variances. This variance is then transformed into the posed coordinate system using the linear-blend-skinned rotations.
  2. This is the more interesting part. A small rotational perturbation at a joint causes a positional displacement of downstream vertices proportional to their distance from the joint's rotation centre. Using a first order approximation permits the variance contributed by each joint to be computed using simple operations on its angular variance and the corresponding lever arm. Importantly, uncertainty follows the kinematic hierarchy: shoulder uncertainty affects the hand, for example, but not an unrelated body part.

Secondly, the paper shows how to use the uncertainty propagation as part of the training objective. The motivation is that conventional parameter-space NLL does not explicitly encourage the predicted uncertainty to reflect the geometric consequences of parameter errors. By supervising uncertainty in the output mesh space, they introduce a geometrically meaningful inductive bias.

Critical strengths

The paper derives a closed-form approximation for propagating uncertainty from 3DMM parameters to individual mesh vertices. By exploiting the known structure of linear blend skinning and the kinematic hierarchy, it avoids expensive sampling or explicit Jacobian computation. The derivation is mathematically motivated, interpretable and relatively straightforward to implement.

CLUE achieves very significant speed and memory improvements over Monte Carlo sampling and Jacobian-based error propagation, while maintaining a good approximation to the resulting vertex uncertainties. The experiments across FLAME, SMPL-X and MHR demonstrate that these improvements are not specific to a single 3DMM. This makes the approach particularly attractive for real-time and resource-constrained applications.

Perhaps the most compelling aspect is that CLUE enables probabilistic supervision directly in vertex and joint space. This provides a geometrically meaningful inductive bias during training and leads to consistent improvements in reconstruction accuracy over deterministic and alternative probabilistic approaches.

Critical weaknesses

CLUE assumes independent parameters with diagonal covariance matrices and relies on a first-order approximation of rotational uncertainty. While these assumptions are understandable given the focus on computational efficiency, their practical implications are not thoroughly evaluated. A discussion or experimental analysis of the circumstances in which these assumptions become problematic would be useful.

While the paper provides extensive qualitative comparisons against high-sample-count Monte Carlo estimates across three 3DMMs, it does not report quantitative approximation errors. Numerical measures of the discrepancy between CLUE and Monte Carlo estimates, ideally as a function of parameter variance, would help establish the accuracy of the proposed approximation and its limitations.

The proposed training approach introduces additional probabilistic losses in vertex and joint space, and the experimental results demonstrate clear improvements. However, it is not entirely clear how much of this improvement can be attributed specifically to the geometry-aware uncertainty propagation rather than the additional output space supervision. A comparison against alternative approaches to modelling vertex-level uncertainty would help isolate the contribution.

Questions for authors

Could you report quantitative errors between CLUE's per-vertex variance estimates and those obtained using 10,000 Monte Carlo samples, for example using MAE or relative error? It would also be useful to understand how these errors change as the magnitude of the parameter uncertainty increases.

What is the practical impact of the assumptions on diagonal covariance matrices, independent parameter groups and small rotational perturbations, particularly the first-order approximation for rotations? Are there examples of poses or uncertainty levels where CLUE produces substantially inaccurate estimates compared with Monte Carlo sampling?

Did you consider a baseline that directly predicts per-vertex uncertainties, rather than deriving them from parameter-space uncertainties? Such a comparison might help establish whether the improvements arise specifically from the geometric inductive bias introduced by CLUE.

1 comment
AuthorsOpenReview ↗

Response to Reviewer TDoK

Dear Reviewer TDoK, we thank you for your review! Please find our responses to your questions below.

Q1. "Could you report quantitative errors…"

A1. Thank you for this suggestion. We now include a study of a sweep over the parameter variance and the MAE between the ground truth (10,000 MC samples) and estimates. This study is in the Appendix: Fig. 8, and tabulated in Table 6. The study is also referred to in the main paper at Lines 366–371. The key takeaways are:

  1. CLUE tracks first-order error propagation closely while having pragmatic assumptions of independence and diagonal covariance.
  2. First-order propagation overestimates the output variance as the input shape, expression, or rotation variance increases. This is because the derivation is based on the Taylor expansion and is accurate for a small variance.

Q2. "What is the practical impact of the assumptions…"

A2. We continue our response from A1. Since CLUE tracks first-order error propagation, we don't see any major impact of assuming independence or a diagonal covariance. However, as the input parameter uncertainty grows, such as the variance in rotation, CLUE and first-order error propagation overestimate the per-vertex variance and are less accurate. This is evident when the predicted mesh deviates significantly away from the rest pose and has high parameter uncertainty. Practically, sources of large parameter uncertainties can be images with severe (full body) occlusion or rare camera angles, and CLUE overestimates the uncertainty across the entire body. However, for most cases, the learnt parameter variance accounts for the overestimation of variance and scales accordingly.

Q3. "Did you consider a baseline that directly predicts per-vertex uncertainties…"

A3. We did! We noticed that Parameter NLL and Vertex NLL are similar, with small trade-offs between MPJPE and PVE.

Method3DPW MPJPE3DPW PA-MPJPE3DPW PVE3DPW PA-PVEEMDB MPJPEEMDB PA-MPJPEEMDB PVEEMDB PA-PVE
Parameter NLL57.935.568.948.366.441.279.550.9
Vertex NLL58.135.868.448.166.841.579.150.7

However, to avoid any confounding factors, and with improvements in Parameter NLL stability, we drop Vertex NLL completely from our study. This matches the training of SAM 3D Body which supervises on the space of parameters and joints only. We have modified the paper where applicable to reflect this.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.