acceptodds
Under review as a conference paper at ICLR 2027

HybridAvatar: Photoreal Animatable Humans as Mesh-Native Assets from Multi-view Cameras

Abstract

Mesh-based avatars have enjoyed decades of graphics engine support: physics, shadows, and relighting all assume mesh geometry. Recent 3D Gaussian Splatting (3DGS) avatars deliver higher rendering quality and can be reconstructed from multi-view captures with as few as 10–16 cameras, but their splat representation is incompatible with conventional graphics engines. We present a hybrid pipeline that bridges these two regimes. We train a pose-conditioned avatar with mesh-anchored Gaussian Splatting from multi-view video, then, for every training pose, extract three asset maps (texture, normal, displacement) with a UV-space rasterizer applied to the trained Gaussians. At inference time, geometry for a novel pose is produced by the trained network as a posed mesh, while appearance is produced by a pose-space mixture-of-Gaussians blend over the per-pose asset maps. The output is a textured mesh asset that can be loaded directly into any engine, supports physics and dynamic shadows, and responds to scene lighting, while approaching the rendering quality of 3DGS avatars. The same model can also be rendered directly as Gaussians at quality comparable to 3DGS avatars. In a hybrid mode it is rendered with Gaussians while its mesh, which shares their geometry, serves as a physics and shadow proxy. We evaluate the pipeline on multi-view human captures and show that it produces engine-ready assets whose photometric quality exceeds that of meshes extracted from 3DGS avatars and approaches that of direct renderings of animatable 3DGS avatars. Our project page is available at https://anonymous.4open.science/w/HybridAvatarICLR27/.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Gaussian Splatting, Classical Rendering, Neural Rendering, 3D Humans, AR/VR, Virtual Reality, 3D Reconstruction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_nKbuOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

In this work, the authors present HybridAvatar, an animatable human avatar that supports inference in both mesh mode and Gaussian mode. They achieve this through pose-dependent Gaussian avatar learning, per-pose mesh asset baking, and pose-space blending. Experiments show improved performance over direct mesh extraction from an animatable Gaussian avatar and over the static DeMapGS baseline.

Critical strengths
  1. The problem setting is interesting: learning engine-ready assets with learning based Gaussian avatars.

  2. The method supports multiple modes, including mesh, Gaussian and hybrid mode.

  3. In mesh mode, the method outperforms the static Gaussian baseline (DeMapGS + LBS) and direct mesh export from an animatable Gaussian avatar.

Critical weaknesses
  1. Although baking Gaussians into mesh assets for animatable avatars is interesting, the technical novelty is limited, as most components rely heavily on DeMapGS and Animatable Gaussians. In addition, the paper does not discuss or compare with closely related alternatives that learn a motion-dependent deformable template together with dynamic texture maps (e.g., Habermann et al., Real-time Deep Dynamic Characters, ACM TOG 2021 and Xiang et al., Modeling Clothing as a Separate Layer for an Animatable Human Avatar, ACM TOG 2021). It remains unclear what advantage baking from Gaussians offers over baking appearance directly onto a deformable template, for example by optimizing its texture maps against the multi-view images.

  2. The pose-space blend is the main new component, but it is not sufficiently validated. It is unclear how much quality is lost by blending compared with directly extracting the maps at the test poses. Moreover, since the kNN-based blend cannot extrapolate beyond the appearance observed during training, which limits its applicability to out-of-distribution poses. It would be helpful to report the distribution of distances from the testing poses to the training poses, together with results on unseen motions.

  3. The storage cost of the mesh mode is not reported. Novel poses require the full per-pose map database, whose size grows linearly with the number of training frames (three 1024×1024 maps per pose, likely several to tens of GB per subject), and each frame reads K = 16 map sets.

  4. In the supplementary videos, the mesh mode loses fine clothing wrinkles in some demos (e.g., the shadow-casting scene), while coarser clothing deformation is preserved in the "Animatable Mesh Avatars" results. The authors should analyze the source of this detail loss and discuss in the limitations which scales of clothing deformation the mesh asset can reproduce.

Overall, I think this paper lacks the comparisons and evaluations needed to demonstrate its superiority over simpler baselines, and its discussion of limitations is incomplete regarding the pose distribution, fine wrinkles, and storage.

Questions for authors
  1. What is the advantage of baking from Gaussians over learning a deformable template and directly optimizing its texture maps against the multi-view images?

  2. What is the storage size of the per-pose map database per subject, and what are the total extraction and training times?

  3. What causes the loss of fine wrinkles in some mesh-mode videos (e.g., the shadow-casting scene)?

Reviewer_jZ2oOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This work introduces an approach for building animatable full-body avatars which combines two representations - 3D gaussian splatting and mesh-based textured avatars. The architecture follows closely existing work on personalized 3DGS avatars (AnimatableGaussians, DeMapGS): articulated SMPL-X scaffold, UV-aligned gaussian payload and mesh correctives predicted by a pose-conditioned CNN, trained on multi-view videos of a single identity. The key novelty is an approach for converting gaussian representation into meshes: the method computes texture/normals/displacement on all training samples. This pre-computed database is then used on a novel pose by a k-NN based blended lookup. The key motivation for the work is easier integration of mesh-based representations with existing rendering engines (both for efficient rendering and physical simulations - e.g. for collision detection).

Critical strengths
  • The qualitative results look quite convincing, and the method for generating mesh-based representations seems to be meaningful and conceptually simple. This is consequential because the process of integration of 3DGS rendering in existing game engine pipelines is indeed non-trivial, and this method provides a solution to this.
Critical weaknesses
  • The method is person-specific and is data-heavy. This means that a) one needs an expensive multi-camera setup (10+ synchronized cameras at least), b) it requires a lengthy multi-stage training process to build a single model. This is especially critical since the main contribution of the method is basically a k-NN pose lookup for appearance - so one must assume that the variation of the pose per person is also critical for reasonable performance. There is no evaluation on OOD poses.
  • The main novelty of the paper is an ad-hoc k-NN-based lookup scheme, requiring a cached database of samples per identity. All the underlying machinery for the drivable 3DGS model already exists and is used quite widely. This could be interesting for people working on applications, but it is unclear if this will be of wide interest to the community.
Questions for authors
  • It would be great if the rough numbers for the full training cost of building an avatar would be included.
  • How sensitive is the model to the number of views (e.g. <14 views?)?
  • Can authors provide examples on extreme poses which are very different from training samples?
Reviewer_oEvnOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents HybridAvatar, a framework for reconstructing animatable human avatars from multi-view videos. A pose-conditioned UV-space network predicts mesh deformation and mesh-bound Gaussian attributes. After training, the method extracts texture, normal, and displacement maps for each training pose. For a new pose, the network predicts the mesh geometry, while the asset maps are obtained by blending maps from nearby training poses. The model supports Gaussian rendering, standard mesh rendering, and a hybrid mode using the mesh for collisions and shadows. Experiments on ActorsHQ and AvatarReX evaluate rendering quality, geometry, and compatibility with standard graphics engines.

Critical strengths

S1. This paper addresses a practical problem. Converting high-quality Gaussian avatars into mesh assets is useful for existing graphics pipelines, including animation, shadows, and object interaction. Supporting different rendering modes from one trained model provides useful flexibility.

S2. The paper is clearly written and easy to follow. The results show that the proposed mesh assets outperform the implemented AG-Mesh and DeMapGS + LBS baselines. The pose-dependent maps also improve over static maps.

Critical weaknesses

W1. The main concern is the limited technical novelty and combination of several existing works. Mesh-bound Gaussian avatars have been studied in ASH [1], SplattingAvatar [2], and GoMAvatar [3]. UV Gaussians [4] already learns pose-dependent mesh deformation and Gaussian attributes in UV space. EVA [5] and UMA [6] also combine deformable human meshes with Gaussian appearance models. More directly, DeMapGS [7] already extracts texture, normal, and displacement maps from surface-bound Gaussians, and this paper explicitly follows its 2DGS–3DGS–2DGS training schedule. Gaussian-to-mesh output is also not a new direction: GAvatar [8] extracts textured meshes for text-generated avatars, MCGS [9] reconstructs dynamic human meshes, and Topo4D [10] produces topology-consistent head meshes with texture maps. These methods have different tasks and do not cover the full pipeline proposed here. However, they establish most of its main components. In my opinion, the main additional contribution is extending asset extraction to a pose-conditioned full-body avatar and generating new-pose maps through pose-space blending. This is useful, but the blending itself is standard nearest-neighbor interpolation with Gaussian kernel weights. The paper does not sufficiently establish the technical contribution beyond adapting and combining these existing ideas.

W2. The experimental comparisons are not sufficient to validate the value of the proposed extension. The main mesh baselines are AG-Mesh, which reconstructs a separate mesh and UV layout for each pose, and DeMapGS + LBS [7], which is trained on only one frame and uses fixed maps. These comparisons show improvements over the implemented export baselines, but do not isolate the benefit over a stronger combination of existing dynamic Gaussian avatars and asset extraction. For example, a comparable export procedure applied to a dynamic mesh-bound model such as ASH [1] or UV Gaussians [4], with matched training data and supervision where possible, would provide a more direct comparison. MCGS [9] and Topo4D [10] also deserve a clearer discussion regarding dynamic mesh output, although their output settings and tasks differ.

The ablations leave some questions unresolved. In Table 4, texture-only rendering achieves 27.21 PSNR, compared with 27.24 for the full three-map asset, with the same reported LPIPS. Since the main evaluation uses unlit rendering, these metrics provide limited evidence for the additional normal and displacement maps.

[1] ASH: Animatable Gaussian Splats for Efficient and Photoreal Human Rendering. CVPR 2024.

[2] SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splatting. CVPR 2024.

[3] GoMAvatar: Efficient Animatable Human Modeling from Monocular Video Using Gaussians-on-Mesh. CVPR 2024.

[4] UV Gaussians: Joint Learning of Mesh Deformation and Gaussian Textures for Human Avatar Modeling.

[5] EVA: Expressive Virtual Avatars from Multi-view Videos. SIGGRAPH 2025.

[6] UMA: Ultra-detailed Human Avatars via Multi-level Surface Alignment.

[7] DeMapGS: Simultaneous Mesh Deformation and Surface Attribute Mapping via Gaussian Splatting. SIGGRAPH Asia 2025.

[8] GAvatar: Animatable 3D Gaussian Avatars with Implicit Mesh Learning. CVPR 2024.

[9] Mesh-Centric Gaussian Splatting for Human Avatar Modelling with Real-time Dynamic Mesh Reconstruction. ACM MM 2024.

[10] Topo4D: Topology-Preserving Gaussian Splatting for High-Fidelity 4D Head Capture. ECCV 2024.

Questions for authors

Q1. The technical differences from the above mentioned works need clearer explanation, especially the parts that go beyond adapting and combining their existing components.

Q2. A comparison with an existing dynamic mesh-bound Gaussian avatar followed by a comparable asset extraction procedure would help verify the benefit of the proposed pipeline.

Q3. Additional results on test poses at different distances from the training database would help assess the generalization of pose-space blending.

Q4. The authors should report silhouette IoU and normal error for AG-Mesh and DeMapGS + LBS under the same evaluation protocol. These comparisons are needed to assess the claimed improvement in geometry quality beyond image appearance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.