SketchHMR: Generative Projection Guidance for Human Mesh Recovery from Freehand Sketches

Reconstructing a 3D human mesh from a freehand sketch is complicated by the inherent sparsity and style variance of hand-drawn strokes, which offer insufficient cues regarding body surface, joint ordering, occlusion, and depth. To bypass the ambiguity of single-stage regressors that map raw strokes directly to parametric body dimensions, we propose SketchHMR, a two-stage paradigm that decouples sketch interpretation from mesh estimation via an intermediate, camera-aligned generated SMPL projection. The first stage, the core component, GeoInk, combines a spatially aligned sketch latent with DINOv2 visual features and input-anchored guidance to translate the sketch into a structured body observation. The second stage iteratively estimates SMPL pose, shape, and weak-perspective camera from the generated projection. To reduce the domain gap between clean rendered projections and generated projections, we pretrain the regressor on clean inputs and subsequently adapt it using paired clean and generated projections. On the Sketch3D dataset, SketchHMR achieves 96.60 mm MPJPE, 66.10 mm PA-MPJPE, and 115.90 mm PVE. Reducing flow sampling from 25 to 10 steps also decreases translation latency by 45.2% without degrading reconstruction accuracy. SketchHMR provides a modular and interpretable approach to sketch-based human mesh recovery by making sketch interpretation an explicit intermediate stage.