- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Output-Aware Conformal Geometry (OACG) for constructing prediction sets for structured parametric models such as SMPL and MANO. The main idea is to combine prediction residual statistics with decoder sensitivity using a matrix geometric mean, so that the resulting parameter-space ellipsoids account for the spatial displacement of decoded outputs. The authors also consider two regularization strategies, GM-I and GM-R, for rank-deficient outputs.
The paper provides a theoretical analysis of the proposed geometry and evaluates it on 3DPW, EMDB, and HOT3D with several body and hand reconstruction models. The results show reduced decoded RMS extent compared with additive fusion in most settings, although the relative performance of GM-I and GM-R varies across output types.
Critical strengths
-
The motivation is reasonable. Parameter-space uncertainty does not necessarily reflect uncertainty in the decoded mesh. Using decoder sensitivity to construct the prediction region is a sensible way to address this issue.
-
The paper includes a fairly thorough evaluation. I appreciate the comparisons across different predictors and output types, particularly the analysis of how regularization affects mesh and joint predictions. The authors are also careful to distinguish the theoretical surrogate result from the actual nonlinear decoded extent.
Critical weaknesses
-
The theoretical result is somewhat limited. The optimality result applies to a linearized surrogate rather than the actual calibrated nonlinear extent. It is therefore unclear how much this result explains the improvements observed in practice. I would like to see a more direct connection between the theoretical objective and the reported evaluation metric.
-
The improvements are not consistent across settings. While the results on full meshes are encouraging, GM-I performs worse than additive fusion on all three SMPL joint settings. GM-R performs better on joints, but the preferred variant still depends on the task. This makes it difficult to identify a general advantage of the proposed construction beyond selecting the best variant for each setting.
-
The computational cost is relatively high. The method requires Jacobian computation and matrix square roots. Section 5.6 reports considerable overhead in some settings, especially for HOT3D meshes at larger batch sizes. Given that the improvements are sometimes modest, I am not fully convinced that the additional complexity is justified in all cases.
Questions for authors
-
Can the authors provide an empirical comparison between the surrogate objective and the actual calibrated nonlinear extent? Does a lower surrogate value generally lead to a smaller decoded prediction set?
-
Is there a practical way to decide between GM-I and GM-R without evaluating both on each new task? The results suggest that this choice can substantially affect performance.
-
How does the runtime scale with the dimensionality of the parameter space? A direct end-to-end runtime comparison with the additive and learned-metric baselines would help clarify the practical trade-off.
3 comments
Response to Reviewer AzA4 Continued
2. Reply to W2 & Q2: A general advantage and a practical variant-selection rule
GM-R gives positive mean paired gains over additive in all 12 settings, with a median gain of 11.09%. Eleven meet the prespecified criterion of ≥5% mean gain and ≥16/20 positive splits (Table 28, p. 44). Variant selection remains useful: GM-I trails additive on the three SMPL joint settings but outperforms GM-R on the original participant-disjoint HOT3D mesh and outputs (Table 2, p. 8).
Our restricted search, R490 + I10, retains the full-search choice at about half the development cost. It searches 490 GM-R configurations, then tests ten GM-I ridge strengths using the selected residual-moment hyperparameters. It chooses the smaller exact development extent, refits and freezes the configuration, and calibrates independently (Section 3.4, pp. 4–5; Appendix C.5.3, p. 59).
This reduces the search from 980 to 500 candidates and reproduces the full search’s final variant and configuration in 28/28 runs (Table 47, p. 62). Actual 3DPW executions confirm the same choices and reduce total development time by 48.8% for both mesh and joints (Tables 48–49, p. 62).
3. Reply to W3 & Q3: Total development cost and parameter dimension
OACG constructs geometry analytically, without training an additional geometry network (Eqs. (6)–(7), p. 4). We compare total development time from cached inputs to the final fitted configuration, including search, exact-extent evaluation, and learned C’s training and checkpoint selection. All methods use the same A100, eight CPU threads, and 2,000 cached 3DPW inputs (Appendix C.5.4, p. 61).
| Total time (minutes) | Full OACG | Restricted OACG | Learned C (estimated) |
|---|---|---|---|
| Mesh | 40.38 | 20.66 | 110.22 |
| Joints | 40.04 | 20.49 | 110.29 |
OACG totals are measured. C’s total is estimated by scaling measured training and checkpoint-evaluation costs to 80 configurations × five rotations, then adding measured final training (Tables 49–50, pp. 62–63).
Including training, full OACG reduces total development time by 63.4%–63.7%; restricted OACG increases the savings to 81.3%–81.4% while preserving the selected configurations. At nominal 90%, selected GM-R also achieves smaller extent than rebuilt C: 94.46 vs. 99.21 mm for mesh and 78.13 vs. 78.62 mm for joints (Table 42, p. 60; Appendix C.2, p. 45).
For dimension scaling, CPU GM matrix construction at batch size 1 takes 0.432, 2.495, and 20.240 ms/input at , respectively. Tables 53–54 (p. 64) report the full matrix-cost scan for batch sizes 1 and 128; the development totals above use SMPL’s actual dimension, .
Summary of updates addressing W1–W3 and Q1–Q3
Thank you again for your questions on the theoretical explanation, variant selection, and computational cost. We have added targeted experiments and posted detailed responses. The main findings are: W1 / Q1 — Surrogate versus nonlinear extent. We compare 110 candidates across 20 calibration splits. On 3DPW meshes, surrogate and exact-extent rankings agree closely (Spearman ρ = 0.9794). For joints, the four-stage comparison locates the ranking reversals at nonlinear decoding. A matched-perturbation decomposition shows that small-ridge extent growth comes from nonlinear terms; at the development-selected ridge, these terms are controlled and exact extent remains close to the linear scale (Tables 29–33, pp. 46–49). This explains the roles of the analytical GM core and exact-extent-based ridge selection.
W2 / Q2 — General gains and practical selection. GM-R achieves positive mean paired gains over additive in all 12 settings, with a median of 11.09% (Table 28, p. 44). Our R490 + I10 procedure searches 490 GM-R configurations and only ten GM-I ridge strengths. It reduces the candidate budget from 980 to 500, consistently preserves the full search's optimal choice in 100% of replayed runs (28/28), and reduces measured total development time by approximately 48.8% on both 3DPW outputs (Tables 47–49, p. 62).
W3 / Q3 — Training-inclusive cost and dimension scaling. OACG is fully plug-and-play, requiring no additional geometry-network training. Restricted OACG reduces total development time by 81.3%–81.4% relative to learned C’s estimated training-inclusive cost (Tables 49–50, pp. 62–63). We also added the requested unified full-pipeline comparison and a controlled matrix-cost scan over d = 16 ∼ 256 at batch sizes 1 and 128 (Tables 51, 53–54, pp. 63–64).
We would particularly welcome your assessment of the surrogate analysis and the practical selection procedure. Please let us know whether any concerns remain; we would be happy to discuss them further.
Response to Reviewer AzA4
Thank you for the questions on theory, variant selection, and cost. Extent comparisons use nominal 90% coverage. Page references are to the manuscript with the rebuttal appendix (Appendix C, pp. 44–64).
1. Reply to W1 & Q1: Connecting the surrogate to calibrated nonlinear extent
The surrogate and decoded extent give closely aligned rankings on 3DPW meshes. For joints, the rankings diverge after nonlinear decoding. Using the same residual statistics, we compare 110 candidates: 10 GM-I, 10 GM-R, and 90 additive configurations. We average each candidate’s results across splits, then measure how closely the surrogate and extent rankings agree using Spearman correlation (Appendix C.3.1, p. 46).
These are the ρ(U1,U4) entries of Table 30 (p. 47). The four-stage comparison in Eq. (42) and Table 29 (p. 46) identifies where the ordering changes:
Section 3.3 (p. 4) already identifies the joint-specific mechanism: 14 joints provide 42 coordinates for 82 parameters, leaving directions with zero first-order cost. Ridge controls perturbations in those directions, whose finite response can be nonlinear (Section 4.3, pp. 5–6). We measure that contribution on identical perturbations: five splits per joint task and 64 shared draws per input. Writing L=Jδ and r=g(δ)−g(0)−L, Eq. (44) (p. 46) gives
E2=E∥L∥W2+2E⟨L,r⟩W+E∥r∥W2.
For GM-R on 3DPW joints, Table 31 (p. 48) yields:
Selecting ridge by exact decoded extent gives a consistent choice and improves joint extent. All ten 3DPW joint deletion folds select λ=0.5 (Table 41, p. 59). The full development-selected GM-R configuration reduces joint extent from additive’s 83.18 to 78.13 mm (Table 42, p. 60).
The small-ridge results explain why this selection matters. The all-row linear estimate stays near 68–73 mm, while nonlinear terms drive exact extent to 1,371 mm; shape-only perturbations already produce 1.29 m (Table 31, p. 48). Exact extent closely tracks the nonlinear remainder (ρ=0.90–0.99; Table 33, p. 49). This decomposition locates the ranking reversals in the finite-radius nonlinear response and quantifies the correction in Proposition A.4 (p. 20) and Eq. (45) (p. 47).