acceptodds
Under review as a conference paper at ICLR 2027

SHuman: Bridging SMPL-X and Stereo Matching for Pose-Free 3D Human Reconstruction

Abstract

Recent years have seen rapid development in 3D human reconstruction, a technology that produces digital human models based on image or video inputs. However, existing methods heavily rely on pre-calibrated camera parameters, thus limiting the applicability in consumer-level scenarios. While intrinsic camera parameters can be readily obtained from device specifications, the pose-free 3D human reconstruction framework remains crucial. In this paper, we propose a pose-free 3D human reconstruction framework named SHuman by bridging SMPL-X and Stereo matching. Stereo matching delivers pose-free capability via extensive 3D reconstruction priors, while SMPL-X supplies human priors to guide the enhancement of geometric accuracy and fine-grained texture details. Specifically, SHuman presents two key innovations. On the one hand, a SMPL-X Estimation module is proposed to acquire a unified SMPL-X model from SMPL-X monocular estimations of different views, providing a robust and consistent human prior. On the other hand, a Posemap Adapter module is designed to inject structured human body information from the unified SMPL-X model into the stereo matching network, significantly enhancing both geometric accuracy and textural details in the reconstructed human model. Experimental results on the THuman2.0 and Twindom datasets demonstrate that SHuman is competitive with the pose-aware methods.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Human Reconstruction, SMPL-X, Stereo Matching, Pose-Free Cameras, Pose-Free Reconstruction

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Bc9KOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes S²Human, a pose-free 3D human reconstruction framework combining SMPL-X priors with VGGT-based stereo matching to generate high-quality 3D Gaussian humans without pre-calibrated camera extrinsics. It introduces a multi-view SMPL-X fusion module and a Posemap Adapter to enhance reconstruction quality. Experiments on THuman2.0 and Twindom demonstrate competitive performance against pose-aware methods, supported by ablation and robustness studies.

Critical strengths
  1. The paper introduces a pose-free 3D human reconstruction framework that eliminates the need for pre-calibrated camera extrinsics, potentially improving its flexibility and practicality in real-world scenarios.
  2. The proposed Posemap Adapter integrates structured SMPL-X human priors into the stereo matching network through intra-view and inter-view attention, improving rendering quality.
Critical weaknesses
  1. The Posemap Adapter provides only a modest improvement in PSNR (0.25 dB). Moreover, Table 4 does not report the geometric error of the full model, making it difficult to verify the claimed improvement in geometric accuracy.
  2. The proposed visibility-aware weighted fusion strategy for multi-view SMPL-X estimation is conceptually similar to the joint-wise view weighting strategy used in MUC (Zhu et al., 2025). The paper does not sufficiently clarify the novelty of this strategy compared to existing methods.

Zhu, Y., Wang, S., Xu, M., et al. (2025). MUC: Mixture of Uncalibrated Cameras for Robust 3D Human Body Reconstruction. Proceedings of the AAAI Conference on Artificial Intelligence, 39(10), 11040–11048.

Questions for authors
  1. Since VGGT can already predict camera extrinsics and 3D point maps, could combining VGGT-estimated camera parameters with existing pose-aware 3DGS human reconstruction methods achieve comparable performance to S²Human?
  2. Why is Tri-posemap preferred over alternative representations of SMPL-X human priors? What evidence supports the effectiveness of this design choice?
  3. In Table 5, the two settings with a 60° FOV show significantly different results (27.61 vs. 25.93 PSNR). Could the authors clarify the differences in camera configurations between these two settings and explain the performance gap?
  4. Given the emphasis on practical applications without pre-calibrated camera extrinsics, could the authors clarify whether the method has been evaluated on real-world multi-camera captures and what evidence supports its robustness in such scenarios?
5 comments

We sincerely thank the reviewer for the constructive comments and valuable suggestions. We would like to provide the following clarifications and additional experimental evidence.

W1. Reconstruction Improvement and Geometric Accuracy

The geometric error of the full model was not reported in Table 4 because the scale of the estimated SMPL-X may differ from the ground truth, making direct point-wise distance comparisons unreliable.

Inspired by the reviewer's suggestion and scale-aligned evaluation protocols commonly used in depth estimation, we have conducted an additional geometric evaluation by aligning the scale and translation of the predicted first-stage point clouds with the ground truth before computing the L2 distance. The results are shown in the following table.

Geometric Error Results\mathbf{\text{Geometric Error Results}}

\hline \text{Method} &\text{PSNR}\uparrow &\text{SSIM}\uparrow &\text{LPIPS}\downarrow &\text{L2 Distance(cm)}\downarrow \\\\ \hline \text{w/o adapter} & 25.68 & 0.9268 & 0.0893 & 2.27 \\\\ \text{full method} & 25.93 & 0.9287 & 0.0888 & \textcolor{blue}{1.73} \\\\ \text{full method(GT SMPL-X)} &\mathbf{26.54} &\mathbf{0.9331} &\mathbf{0.0834} &\mathbf{1.27}\\\\ \hline \end{array}$$ The resulting geometric error of the full model is 1.73 cm, which lies between the w/o adapter baseline (2.27 cm) and the full model using ground-truth SMPL-X (1.27 cm) reported in Table 4. This result is consistent with our conclusion that incorporating human structural priors improves geometric prediction, while more accurate SMPL-X priors provide further benefits. The additional geometric evaluation also complements the rendering-quality improvements reported in our ablation study. **W2. Differences Between Our Visibility-Aware Fusion and MUC** We thank the reviewer for highlighting MUC. Both methods employ weighted multi-view fusion to improve SMPL-X estimation, but our method adopts a much more lightweight weighting mechanism. MUC employs a learned Joint Reweighting Network to predict view-dependent joint weights from feature tokens and estimated camera parameters, guided by a joint-distance distribution loss. In contrast, our method directly determines joint visibility through 3D ray-mesh occlusion checks and 2D image-boundary checks, using the resulting explicit visibility information to compute fusion weights. Unlike MUC, our weighting strategy does not require an additional learned reweighting network or a distance-distribution-based training objective. Therefore, our contribution in SMPL-X fusion lies in adopting an explicit, geometry-based visibility criterion rather than learning joint reliability weights. The effectiveness of this strategy is supported by the SMPL-X estimation results in Table 3.
AuthorsOpenReview ↗

Responses to the Additional Questions

Responses to the Additional Questions

Q1. Could VGGT-estimated camera parameters be combined with existing pose-aware 3DGS human reconstruction methods?

Directly combining VGGT-estimated camera poses with existing pose-aware 3DGS human reconstruction methods faces a fundamental challenge of scale inconsistency.

VGGT predicts camera parameters in a normalized geometric scale, which is not guaranteed to match the scale and coordinate conventions used to train existing pose-aware human reconstruction networks. Directly substituting these estimated camera parameters can therefore introduce inconsistent geometric projections and degrade reconstruction quality, particularly for methods that rely on calibrated camera configurations.

In contrast, S²Human explicitly encodes the SMPL-X surface coordinates in Tri-posemap, providing a human-specific geometric scale reference that is integrated into the stereo matching network. This enables the network to incorporate human structural and scale information during reconstruction, alleviating the scale inconsistency associated with directly transferring independently estimated camera parameters to a pose-aware reconstruction pipeline.

Therefore, the suggested combination is not a direct plug-and-play alternative under the existing model configurations.

Q2. Why is Tri-posemap preferred over alternative SMPL-X prior representations?

Tri-posemap provides a dense, spatially aligned representation of human structura l priors that can be directly integrated with stereo matching features Li et al. (2024); Guo et al. (2025). Its three coordinate channels encode the 3D positions of the SMPL-X surface, while the additional channel describes the overlap between the projected SMPL-X body and the actual clothed human region. Compared with global SMPL-X parameters or sparse joint coordinates, this representation provides explicit spatial correspondence with image features.

Moreover, its pixel-aligned structure facilitates integration through the Posemap Adapter. The ablation results in Table 4 demonstrate the effectiveness of this human-prior representation and its integration mechanism.

Li, Z., Zheng, Z., Wang, L., & Liu, Y. (2024). Animatable Gaussians: Learning Pose-dependent Gaussian Maps for High-fidelity Human Avatar Modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 19711–19722).

Guo, C., Li, J., Kant, Y., Sheikh, Y., Saito, S., & Cao, C. (2025). Vid2Avatar-Pro: Authentic Avatar from Videos in the Wild via Universal Prior. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 5559–5570).

Q3. Why do the two 60° FOV settings in Table 5 produce different results?

Although both settings use a 60° FOV, they correspond to different camera configurations. The 60° (original) setting follows the original multi-view experiment in Table 1, whereas the 60° setting belongs to the intrinsic robustness experiment.

As described in Section 4.3.3, the camera-to-subject distances are adjusted in the robustness experiment to maintain approximately consistent human image footprints across different FOVs. Consequently, the adjusted 60° setting has a different camera-to-subject distance and human image coverage from the original setting, resulting in different rendering evaluation conditions and contributing to the observed PSNR difference.

Q4. Evaluation on Real-World Multi-View Captures.

To evaluate our method on real-world multi-camera captures and to provide evidence supporting its robustness in such scenarios, we have evaluated S²Human using real-world multi-view images captured with a consumer smartphone. The images are captured from multiple viewpoints without pre-calibrated camera extrinsics. Using these captured images as input, we reconstruct the 3D human representation and render intermediate viewpoints between the input views for qualitative evaluation. As shown in the video in the supplementary materials, these results provide a qualitative assessment of the reconstruction quality and novel-view synthesis capability of S²Human under real-world consumer-camera capture conditions, further demonstrating its practical applicability beyond synthetic multi-view datasets.

We thank the reviewer again for the constructive feedback.

Reviewer_Bc9KOpenReview ↗

Two Remaining Questions

Thank you for the authors' detailed response, which has addressed most of my concerns. However, I still have two remaining questions.

First, I wonder whether MUC's Joint Reweighting Network could be directly integrated into your framework to replace the proposed visibility-aware weighting strategy. I would like to see a direct comparison under the same experimental settings to better understand the advantages of your method.

Second, could the authors provide a direct comparison between Tri-posemap and other SMPL-X prior representations, such as global SMPL-X parameters or sparse joint coordinates? I think such a comparison would help demonstrate why Tri-posemap is a better choice.

If the authors can adequately address these two concerns with convincing experimental evidence, I would be willing to reconsider and potentially increase my rating.

Reviewer_wzaMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a multi-view 3D human reconstruction framework, S^2 Human, which doesn't require camera extrinsic. The framework recovers a unified SMPL-X model by fusing monocular predictions of each view. It then uses PoseMap Adapter to inject the SMPL-X prior presented by Tri-posemap into the stereo matching network and transforms the stereo features into a detailed 3DGS representation through the 3D Gaussian Regression module. Experiments on THuman2.0 and Twindom show promising results.

Critical strengths
  • removing camera calibration from multi-view reconstruction with good PSNR comparing to other baselines.
  • reasonable experiment coverage
Critical weaknesses
  • marginal novelty and improvement - the contribution is heavily based on the pretrained VGGT backbone.
  • many relevant work are not mentioned or compared, including pose-free Gaussian methods such as Splatt3R, or human reconstruction methods without camera calibration such as EasyRet3D, among many others.
Questions for authors
  • Paper should direct compare with a VGGT-based Gaussian reconstruction baseline without human priors
  • How well does it perform when replacing ground-truth SMPL-X supervision with realistic estimated ones during training?
3 comments

We thank the reviewer for the constructive comments and valuable suggestions. We would like to clarify the following points.

W1. Novelty and Dependence on VGGT

We acknowledge that, as a foundation model for 3D geometry reconstruction, VGGT has been widely adopted in pose-free 3D and 4D reconstruction methods Zhuo et al. (2026); Fang et al. (2026); Sucar et al. (2026), providing a strong geometric foundation for our approach. Using VGGT as backbone, however, our main contribution is not simply fine-tuning VGGT for human reconstruction, but introducing structured human priors into its general-purpose stereo matching framework. Specifically, we introduce a visibility-guided SMPL-X Estimation module that fuses monocular predictions from multiple views into a unified and consistent human prior, alleviating errors caused by occlusions and ambiguities. We further design a Posemap Adapter that represents the unified SMPL-X prior as a Tri-posemap and injects it into the stereo matching network through intra-view and inter-view attention. Together, these components bridge SMPL-X-based human priors and general-purpose stereo matching, improving geometric accuracy and fine-grained texture details in pose-free human reconstruction.

Zhuo, D., Zheng, W., Guo, J., Wu, Y., Zhou, J., & Lu, J. (2026). Streaming Visual Geometry Transformer. In International Conference on Learning Representations.

Fang, K., Zhou, C., Fu, Y., Li, H., & Chen, Y. (2026). IncVGGT: Incremental VGGT for Memory-Bounded Long-Range 3D Reconstruction. In International Conference on Learning Representations.

Sucar, E., Insafutdinov, E., Lai, Z., & Vedaldi, A. (2026). V-DPM: 4D Video Reconstruction with Dynamic Point Maps. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 14502–14511.

W2. Related Work and Baseline Comparisons

Splatt3R is mentioned in Section 2.2 (Pose-Free 3D Reconstruction). We do not include Splatt3R in our quantitative comparisons. Instead, we adopt a VGGT-based Gaussian reconstruction baseline, which is a more recent and mainstream pose-free method. Pose-free Gaussian methods are designed for general-scene reconstruction, with training settings and reconstruction objectives that differ from those of high-fidelity human reconstruction. Directly applying these methods to human reconstruction may be affected by differences in training domains and reconstruction objectives. We therefore fine-tune the VGGT-based Gaussian reconstruction baseline and report the results in Table 4 (w/o adapter configuration).

Regarding EasyRet3D, it focuses on pose-free 3D human pose reconstruction and tracking using parametric SMPL models, whereas our S²Human aims at detailed 3D Gaussian human reconstruction for high-fidelity novel-view synthesis. Due to these differences in reconstruction objectives and output representations, these two methods cannot be directly compared using our rendering-based evaluation metrics (PSNR, SSIM, and LPIPS).

AuthorsOpenReview ↗

Responses to the Additional Questions

Responses to the Additional Questions

Q1. Comparison with a VGGT-based baseline without human priors.

The w/o adapter configuration in Table 4 serves as a VGGT-based Gaussian reconstruction baseline fine-tuned on human data without SMPL-X prior injection. The results demonstrate that introducing structured human priors improves reconstruction quality beyond the VGGT-based baseline. We will include the correspondence results in Table I of the paper for comparison analysis.

Quantitative Comparison of Multi-View Reconstruction on THuman2.0\mathbf{\text{Quantitative Comparison of Multi-View Reconstruction on THuman2.0}}

\hline \text{Method} & \text{PSNR}\uparrow & \text{SSIM}\uparrow & \text{LPIPS}\downarrow \\\\ \hline {\text{GPS Gaussian}} & {23.53} & \mathbf{0.9315} & \mathbf{0.0875} \\\\ {\text{EVA Gaussian}} & { 24.61} & {0.9261} & { 0.1013} \\\\ {\text{GHG}} & {23.39} & {0.9184} & {0.0923} \\\\ {\textcolor{blue}{\text{VGGT}}} & \textcolor{blue}{25.68} & \textcolor{blue}{0.9268} & \textcolor{blue}{0.0893} \\\\ \text{S²Human(Ours)} & \mathbf{25.93} &{0.9287} & {0.0888} \\\\ \hline \end{array}$$ **Q2. Training with estimated rather than ground-truth SMPL-X.** Estimated SMPL-X predictions cannot be used in our current training pipeline because they do not provide absolute scale and global coordinate alignment consistent with the ground-truth geometric supervision. Our training employs ground-truth point maps and camera parameters for geometric supervision, while Tri-posemap explicitly encodes the 3D coordinates of the SMPL-X surface. These representations are expected to be geometrically consistent. However, monocular SMPL-X estimation cannot guarantee the same absolute scale and global alignment as the ground-truth geometry. Directly using estimated SMPL-X would therefore introduce mismatched geometric information between the encoded human priors and the supervised point maps, affecting the learning of geometric correspondences. Consequently, estimated SMPL-X cannot serve as a geometrically equivalent replacement for ground-truth SMPL-X in our current training formulation. We use ground-truth SMPL-X during training to maintain consistent geometric priors and supervision, while estimated SMPL-X is used during inference. The results in Table 4 demonstrate the effectiveness of our framework under this inference setting. We thank the reviewer again for the constructive feedback.

We sincerely thank the reviewer again for the valuable comments and suggestions. We have carefully addressed the concerns raised in the review and provided detailed clarifications in our response. Other reviewers have kindly taken the time to review our responses and provide further feedback. We would also greatly appreciate it if the reviewer could take a moment to review our response when convenient. We welcome any further questions or suggestions.

Reviewer_x1aNOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes a pose-free 3D human reconstruction framework that combines SMPL-X human priors with stereo matching. The method introduces two main components: a visibility-aware SMPL-X Estimation module that fuses monocular predictions from multiple views into a unified human model, and a Posemap Adapter that injects structured human priors into a VGGT-based stereo matching network. The resulting features are used to reconstruct a detailed 3D Gaussian representation. Experiments on THuman2.0 and Twindom demonstrate competitive performance against existing pose-aware methods.

Critical strengths
  • Removing the dependency on pre-calibrated camera extrinsics is valuable for making 3D human reconstruction more accessible in consumer-level scenarios.
  • Combining SMPL-X human priors with general-purpose stereo matching is a reasonable design. The visibility-aware fusion and Posemap Adapter provide a clear mechanism for improving reconstruction quality.
  • The method achieves competitive performance against pose-aware baselines, including the best PSNR on both datasets. The cross-domain experiments and ablation studies further support its effectiveness.
Critical weaknesses
  • The experiments primarily consider human subjects with clean backgrounds. It remains unclear how the method performs in real-world scenes containing complex backgrounds, clutter, or occlusions. Since the pipeline relies on human masks and SMPL-X estimation, background interference could potentially affect reconstruction quality.
  • Although the method targets consumer-level scenarios, the paper does not provide inference-time or computational efficiency comparisons. Such evaluations would help establish its practical advantages.
  • The reconstruction quality depends on the initial SMPL-X predictions. The noticeable performance improvement when using ground-truth SMPL-X suggests that errors in human pose estimation remain a potential bottleneck.
Questions for authors
  • How would the proposed method handle input images containing complex background scenes rather than isolated human subjects? Does the current pipeline require accurate foreground segmentation, and how sensitive is reconstruction quality to imperfect human masks?
  • How does the method perform when the initial SMPL-X estimates are significantly inaccurate, particularly under severe occlusions or challenging human poses?
  • Could the authors provide inference-time and memory-consumption comparisons with existing methods? How practical is the proposed framework for consumer-level applications?
2 comments

We sincerely thank the reviewer for the positive evaluation and constructive suggestions. We appreciate the recognition of the practical value of our pose-free reconstruction framework and the effectiveness of the proposed human-prior integration. We would like to provide the following clarifications and additional experimental results.

W1 & Q1. Robustness to Complex Backgrounds and Imperfect Human Masks

We appreciate the reviewer's concern regarding reconstruction in complex real-world environments.

Our current pipeline requires human foreground masks to identify the target subject, construct the Tri-posemap, and select the corresponding human Gaussian primitives. These masks can be obtained using the pretrained DeepLabV3-ResNet50 model provided by torchvision. While inaccurate masks may affect reconstruction quality, they do not introduce a requirement for pre-calibrated camera extrinsics.

Our method takes foreground images obtained using existing human segmentation methods as input. Therefore, background clutter is largely removed before the reconstruction process and is not involved in subsequent reconstruction, making the method less sensitive to complex background conditions. We have conducted an additional real-world evaluation using multi-view images captured with a smartphone. As shown in the video in the supplementary materials, the captured scene contains a relatively complex background, and our method successfully reconstructs the human subject and renders intermediate viewpoints. This provides additional qualitative evidence of the applicability of S²Human beyond the relatively clean backgrounds.

W2 & Q3. Inference Efficiency and Practical Applicability

We thank the reviewer for raising the important question of computational efficiency.

We have supplemented the inference-time comparison using the same test set with two-view inputs. The forward-pass times of S²Human, EVA-Gaussian, GPS-Gaussian, and GHG are shown in the following Table.

Forward-Pass Time Results \mathbf{\text{Forward-Pass Time Results }}

\hline \text{Method} & \text{GPS Gaussian} & \text{EVA Gaussian} & \text{GHG} & \text{S²Human(Ours)} \\\\ \hline \text{Time(ms)} & 25.10 & 51.12 & 318.39 & 496.24 \\\\ \hline \end{array}$$ As expected, S²Human requires more inference time than the compared methods. This is mainly due to the additional computational complexity of our pose-free reconstruction pipeline, which needs to perform VGGT-based stereo matching without relying on pre-calibrated camera extrinsics. We acknowledge this efficiency trade-off and do not claim computational superiority over existing pose-aware methods. Instead, the primary advantage of S²Human lies in eliminating the need for pre-calibrated camera extrinsics, providing greater flexibility in multi-view image acquisition. We believe this flexibility is valuable in consumer-level capture scenarios, where camera calibration can be inconvenient or difficult to obtain. **W3 & Q2. Sensitivity to Initial SMPL-X Estimation Errors** We agree that the reconstruction quality of S²Human is effected by the initial SMPL-X estimates. The proposed visibility-aware multi-view fusion mechanism helps alleviate estimation errors by exploiting complementary observations across different viewpoints. However, we acknowledge that when the initial SMPL-X estimates are significantly inaccurate across multiple views, particularly under severe occlusions or challenging human poses, the reconstruction quality may be affected. The reviewer's suggestion highlights an important direction for our future research. We will investigate more robust human-prior estimation and integration strategies for challenging scenarios, especially when reliable SMPL-X predictions are difficult to obtain from the available viewpoints. We sincerely thank the reviewer again for the constructive suggestions and positive assessment of our work.
Reviewer_x1aNOpenReview ↗

Comment

I thank the authors for the rebuttal. I am still in support of acceptance and will keep my rating.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Review Status and Author Responses

We sincerely thank the Area Chair and all three reviewers for their time and constructive feedback.

Our submission initially received scores of 3, 2, and 2 from reviewers x1aN, wzaM, and Bc9K, respectively. During the discussion period, reviewers x1aN and Bc9K have reviewed our responses and provided further feedback. Reviewer x1aN maintained their score of 3, while reviewer Bc9K increased their score from 2 to 3 following our clarifications and discussion. As a result, the current scores are 3, 2, and 3.

Reviewer wzaM has not yet responded to our rebuttal. Nevertheless, we have carefully addressed the concerns raised in their review and provided detailed explanations and clarifications.

We greatly appreciate the reviewers' engagement and feedback throughout the discussion process. We respectfully hope that the Area Chair will take our rebuttal and the subsequent discussions into consideration when making the final assessment of our submission.

Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.