acceptodds
Withdrawn from ICLR 2027

SemGS-Avatar: Semantic-Aligned Disentangled Generation of 4D Human Avatars from a Single Image

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Disentangled generation of 4D human avatars from a single image has achieved remarkable progress in recent years. However, existing methods suffer from two challenges: (1) Spatially, they struggle to accurately assemble separate body parts due to unclear rules on how parts fit together, leading to overlapping parts and broken boundaries. (2) Temporally, they fail to maintain smooth motion because estimated poses jump from frame to frame, causing the reconstructed avatar to shake and drift. To address these challenges, we propose SemGS-Avatar, a framework for high-fidelity 4D human avatar generation through semantic-aligned disentangled Gaussian splatting. Specifically, SemGS-Avatar consists of two core modules: Semantic-Aligned Gaussian Composer and Temporal Consistency Pose Optimization. The former explicitly aligns the semantic attributes of 3D Gaussians with the structured somatic manifold of the SMPL-X model to ensure region-consistent part binding and precise structural arrangement, enabling accurate body-part assembly and boundary refinement. The latter integrates a motion interpolation network with sliding-window kinematic smoothing to stabilize dynamic poses, ensuring smooth and coherent animation. Benefiting from the above designs, our method not only produces high-fidelity 4D human representations with sharper textures and cleaner part boundaries, but also reduces pose jitter and improves temporal stability. Extensive experiments demonstrate that our SemGS-Avatar outperforms existing methods in both visual fidelity and video-driven animation stability.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Human Avatars, Disentangled Representation, 3D Gaussian Splatting, Single-Image Generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_LsEZOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents SemGS-Avatar, a framework for single-image 4D human avatar generation. It combines semantic-aligned Gaussian composition with SMPL-X and temporal pose optimization to improve part assembly and animation stability. The method achieves strong reported reconstruction and animation results, but the overall technical novelty and temporal evaluation are somewhat limited.

Critical strengths

The problem is relevant and the overall framework is intuitive. Semantic alignment with SMPL-X improves structural consistency and part boundaries. The reported reconstruction results are strong. The qualitative animation results are visually convincing.

Critical weaknesses

The main components appear relatively incremental.

The temporal evaluation does not directly measure pose jitter or long-term drift.

Prior work[1] has also explored topology-aware structural constraints for Gaussian avatars The authors may consider citing this work.

Temporal ablations mainly use static image-quality metrics, which are not ideal for validating temporal consistency.

[1] Tavatar: Topology-Aware Gaussian Attribute Derivation for Animatable Human Avatars, cvpr, 2026

Questions for authors

Are the training and evaluation identities on THuman3 and CustomHuman strictly disjoint?

Can the authors provide direct metrics for pose jitter, motion smoothness, or long-term drift?

How much improvement does the temporal optimization module provide when applied to existing avatar methods?

Could the authors better clarify the novelty of the Semantic-Aligned Gaussian Composer compared with existing SMPL-aligned Gaussian methods?

Reviewer_VETQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SemGS-Avatar reconstructs an animatable full-body human avatar from a single image and animates it using a driving video. Its two main contributions are:

  1. A semantically aligned Gaussian synthesizer that combines part-level Gaussians with a Gaussianized SMPL-X template through semantically constrained soft matching and reconstruction optimization.
  2. A test-time pose refinement module that completes missing keypoints and smooths the SMPL-X pose and shape trajectories to achieve more stable LBS-driven animation.

The reported experiments show that the method achieves better reconstruction metrics, superior visual quality, and higher user preference compared with previous methods.

Critical strengths
  1. Improved reconstruction fidelity. The comparison in Fig. 4 shows that the method produces finer clothing textures, clearer garment boundaries, and a more natural overall appearance than the compared methods. These visual improvements are also supported by the reconstruction metrics in Table 1. This combination of visual and quantitative gains provides meaningful evidence for the method's practical value, which I find appealing.

  2. Cleaner part separation under articulation. In the poses shown in Fig. 5, the method clearly reduces unnatural connections between clothing regions and between the limbs and the torso. These artifacts directly affect the usability of animatable full-body avatars, so reducing them is a compelling result. These examples support the value of part-wise modeling and semantic alignment in maintaining distinct structures during articulation.

Critical weaknesses

Insufficient evidence for the central claim of temporal stability. Although improved animation stability is a major contribution of this paper, no supplementary video is provided to demonstrate reduced jitter, mitigated drift, or improved continuity of garment deformation. Temporal smoothness cannot be established from sampled frames alone, and FVD and CLIP scores do not reflect these properties in isolation. Moreover, the ablation study of the temporal module in Table 2 reports image reconstruction metrics rather than direct measures of temporal consistency and motion accuracy. This leaves the core claimed advantage without sufficient empirical support.

Limited demonstrative novelty beyond existing binding and pose optimization techniques. Semantic garment representation and SMPL-X-based Gaussian binding have already appeared in Disco4D, and generic pose sequence optimization techniques are well established (e.g., SmoothNet). Although the proposed soft matching mechanism differs from prior binding mechanisms, its specific advantages are not sufficiently isolated and validated. Likewise, the temporal module mainly optimizes the driving keypoints and SMPL-X trajectories rather than additionally modeling the dynamic properties of the avatar. To demonstrate substantive progress beyond combining these components, cross-method comparisons using the same optimized driving poses are needed, along with controlled comparisons against existing binding and optimization strategies.

Minor: Some visualizations are too small (e.g., Fig. 8), making it difficult to discern the differences between methods; I suggest enlarging them. Although some figures mark local regions with red boxes, the marked regions are not enlarged and therefore remain unclear. In addition, the paper does not clearly state what the red boxes are intended to highlight.

Questions for authors
  1. What exactly does the ablation study of the temporal module in Table 2 evaluate: static reconstruction results or animation frames? If the former, why would removing the driving pose optimization affect these metrics? If the latter, how are the driving sequence, reference frame, and evaluation viewpoint defined?

  2. Are the Part Attribute Maps defined in image coordinate space or in canonical UV space? How are they converted into 3D Gaussian center positions (including for unobserved regions)? How are the skinning weights of the garment Gaussians assigned: inherited from a single Skin-GS anchor, interpolated using soft matching weights, or obtained in some other way? This distinction is particularly important for clothing that is far from the body (e.g., skirt hems, loose sleeves).

  3. Given that the THuman3 and CustomHuman datasets are used for both fine-tuning and evaluation, could you specify the identity/scan-level split, and whether different poses or viewpoints of the same subject appear in different data splits?

  4. The experimental results from the back view appear better than those of the compared methods. Could you clarify how the unobserved regions are generated and supervised? What evidence can distinguish the contribution of the part generation prior from that of semantic alignment to this improvement?

Reviewer_x7DgOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper propose SEM-GS Avatar through semantic-aligned disentangled Gaussian Splatting. It consists of two core modules: Semantic-Aligned Gaussian Composer and Temporal Consistency Pose Optimization. While the qualitative results look promising, the method description is difficult to follow and leaves key components unexplained.

Critical strengths

Problem definition is well-motivated. Part-level/disentangled single-image avatars do suffer from inter-part misalignment and texture bleeding at boundaries, and per-frame pose estimation does cause jitter; both are real and worth addressing.

Critical weaknesses

1. Missing description of a core component.\textbf{1. Missing description of a core component.} The “pretrained deep encoder-decoder network” / part denoising U-Net is the central component of the method, yet it is neither cited nor preciesly explained: its architecture, backbone, and whether it is a diffusion model are all unspecified.

2. Limited novelty of the temporal module.\textbf{2. Limited novelty of the temporal module.} Temporal Consistency Pose Optimization is a combination of standard techniques — first- and second-order smoothness penalties on SMPL-X parameters and keypoint interpolation for missing frames. I think it should not be counted as a contribution of this work.

3. Insufficient comparison with related works.\textbf{3. Insufficient comparison with related works.} The comparison set is inconsistent across tables, omits LHM from Table 1 and LHM/IDOL from Table 3, and does not include per-subject optimization methods (e.g., ExAvatar, PERSONA), which are the closest in cost and setting given that the proposed method also performs 15k iterations of per-subject 3DGS optimization.

4. Insufficient ablation table.\textbf{4. Insufficient ablation table.} There is no temporal metric (jitter, acceleration error) for the temporal components anywhere in the paper.

5. Missing citations.\textbf{5. Missing citations.} There are too many missing citations (e.g., 3DGS, SMPL-X, etc.)

Questions for authors

Q1. Where does the “pretrained” part denoising U-Net come from? Is it an external pretrained model?

Q2. What is exact mathematical formulation and input-output of part denoising U-Net?

Q3. Is Skin-GS attached to the entire SMPL-X template (including face, hands, and feet), or only to body regions?

Q4. How does the method handle garments that are far from the body, such as loose skirts or dresses? Is this method generalizable and applicable for these cases?

Q5. Why do the temporal ablations in Table 2 significantly change static reconstruction metrics?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.