acceptodds
Under review as a conference paper at ICLR 2027

AvatarDynamizer: From Static to Dynamic Human Avatars via Generative Dynamic Textures

Abstract

Modeling pose-dependent surface dynamics is essential for realistic full-body avatars. Person-agnostic methods recover static 3D avatars from monocular images, videos, or text prompts, but their skeleton-driven animations lack realistic surface dynamics such as clothing wrinkles. In contrast, Person-specific methods capture such dynamics but require costly multi-view acquisition for each identity. Generalizable dynamic methods either produce limited surface dynamics or sacrifice multi-view consistency when introducing them. We present AvatarDynamizer, a generative method that transforms an off-the-shelf static avatar into a controllable 4D avatar with realistic surface dynamics and multi-view-consistent rendering. Our key insight is to represent pose-dependent surface dynamics as dynamic textures, bridging pretrained video diffusion models and 3D Gaussian splatting. Specifically, our Dynamic Texture Generator produces fine-grained, motion-dependent textures, which are decoded by a Generalized Gaussian Decoder into 3D Gaussians for inherently multi-view-consistent rendering. To bridge the gap between identity-rich but motion-limited datasets and motion-rich datasets with few subjects, we collect a high-quality multi-view dataset with dense camera coverage and diverse identities and motions. Experiments across four datasets show that our method outperforms competing generalizable dynamic methods on generative metrics and produces more faithful high-frequency surface dynamics.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
4d human avatar, texture-space human representation, video diffusion models, surface dynamics

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_FQ69OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces AvatarDynamizer, a method for transforming off-the-shelf static human avatars into controllable dynamic avatars. Given a static identity texture and a sequence of pose-conditioned normal maps, a video diffusion model generates dynamic textures, which are then decoded into 3D Gaussians for multi-view-consistent rendering. The authors also collect DynaHuman, a multi-view human dataset with diverse identities and motions, to support learning pose-dependent surface dynamics. Experiments across four datasets demonstrate improved generative quality over the evaluated dynamic baselines.

Critical strengths
  1. The motivation of this paper is clear and well justified. The proposed method enhances static avatars with pose-dependent surface details by combining dynamic texture generation in texture space with neural decoding into 3D Gaussians. This is a sensible design that leverages video generative priors while preserving multi-view consistency.

  2. I appreciate the substantial effort involved in constructing DynaHuman, including dense multi-view capture, non-rigid registration, and dynamic texture preparation. Its combination of diverse identities and rich motion sequences provides valuable data for learning avatar surface dynamics.

Critical weaknesses
  1. The paper does not report the rendering FPS or the per-frame computational cost at inference time, making it difficult to assess its practical rendering efficiency.

  2. Although the authors acknowledge limitations with loose garments and complex clothing deformations, the paper does not show corresponding failure cases of the proposed method. Such examples would help clarify its practical limitations and the range of clothing it can handle reliably.

Questions for authors
  1. What motivates the texture layout used in this work rather than the standard SMPL-X UV layout? An ablation comparing the two layouts would help clarify the benefits of this design.

  2. Have the authors considered adapting the fine-tuned video model using methods such as Self Forcing or Causal Forcing? Would these offer advantages over the current approach in generation efficiency or long-sequence temporal consistency?

3 comments
AuthorsOpenReview ↗

Response to Reviewer FQ69

We thank the reviewer for noting that our motivation is clear and well-justified, highlighting our design is sensible that uses video generative priors while preserving multi-view consistency, and acknowledging the value of our DynaHuman dataset. We provide additional efficiency and texture-layout evaluations, concrete failure cases, an exploratory acceleration study, and clarifications on participant consent and permitted identity use. We also provide figure of failure cases in the new main submission on page 10.

W1 — Rendering efficiency and per-frame cost

We report per-frame Gaussian decoding/rendering time and the estimated cost including DTG. We additionally evaluate a subject-specific avatar distilled from LHM+AD, which predicts the animated avatar directly from motion without video diffusion.

Inference cost on H100 NVL and RTX 3090.

ConfigurationH100 NVLRTX 3090
Gaussian decoding + rendering3.61 ms (277 FPS)5.37 ms (186 FPS)
DTG + decoding/rendering (estimated)130.5 ms (7.7 FPS)408.2 ms (2.4 FPS)
Distilled motion-driven avatar (S76)2.57 ms (390 FPS)4.34 ms (230 FPS)

Measurements use a single GPU, bf16 model inference, 29,159 Gaussians, and one 800×600 view per frame. Decoding/rendering assumes pre-generated textures and includes normal-map creation and network inference. DTG-inclusive costs sum separately measured stages, with texture-generation time amortized over 44 new frames per window.

Texture generation dominates the animation cost. The distilled motion-driven avatar was trained on 6,000 poses from S76. The following table compares its quality with LHM+AD.

Quality comparison on S76:

MethodPSNR ↑SSIM ↑LPIPS ↓FVD ↓
LHM+AD26.7870.722380.1962040.932
Distilled avatar26.8930.733400.2049449.599

Distillation slightly improves PSNR and SSIM but increases LPIPS and FVD, indicating a trade-off between inference speed and perceptual quality.

W2 — Failure case visualization

Fig. 7 illustrates three limitations discussed in Appendix J. Loose clothing: the dress splits around the legs and takes on a trouser-like shape during animation, most visibly in the tug-of-war poses. Severe self-occlusion: the squatting motion produces local texture artifacts around the waist and legs, which become less visible in later frames. Garment removal: the avatar follows the driving motion but keeps its original clothing, failing to reproduce the removal shown in the motion reference.

Q1 — Texture layout

Our layout is designed to allocate texels more uniformly over the body. Because each valid texel anchors a Gaussian, this allocation determines the initial surface-sampling density. Following ASH (Pang et al., 2024), the torso and limbs share a connected UV chart. A dedicated square head region supports separate head and body branches, as in EVA (Junkawitsch et al., 2025).

We evaluate texel allocation and UV distortion on the same reference mesh, using surface-area-weighted statistics over the common body region:

Metric ↓Our UVSMPL-X UV
UV-to-surface area ratio variation (CV)0.3520.621
Mean conformal distortion, ideal = 11.2371.376
Mean absolute angle error (°)5.818.42

CV denotes standard deviation divided by the mean.

These results show more uniform texel allocation and lower local distortion. Decoded body Gaussians also exhibit lower CVs in their largest principal scales (0.369 vs. 0.439) and canonical-space sixth-neighbor distances (0.229 vs. 0.341), averaged over 100 frames per subject on S71/S76 with equal subject weighting.

We also compare layout-specific RDT reconstruction pipelines at 256×256, using a GGD trained for each layout. Results are averaged equally across S71 and S76.

LayoutPSNR ↑SSIM ↑LPIPS ↓
Our UV29.740.8420.121
SMPL-X UV29.430.8350.127

Each pipeline uses its own texture preparation and rendering postprocessing.

The layout-specific reconstruction results favor our UV layout on all three mean metrics, complementing the direct evidence of more uniform texel allocation and lower distortion.

AuthorsOpenReview ↗

Response to Reviewer FQ69

Q2 — Post-training acceleration and long-sequence consistency

Self Forcing and Causal Forcing. We see both methods as promising extensions to DTG. Our current model generates dynamic textures in bidirectional windows. An autoregressive adaptation could generate successive texture chunks conditioned on the static identity, driving poses, and previously generated textures. Their few-step formulation could improve efficiency and enable streaming, while training with generated histories would explicitly target consistency across successive texture chunks. The potential gains should be assessed jointly with fine-detail fidelity and motion control.

Four-step distillation. As a related post-training study, we implemented and evaluated a four-step LoRA-distilled DTG, preserving our identity and pose conditioning within the existing bidirectional window-based generation scheme. This experiment evaluates sampling-step reduction; it does not evaluate the autoregressive adaptations of Self Forcing or Causal Forcing. In the inspected sequences, qualitative comparisons with the original DTG reveal reduced fine-scale wrinkle detail and poorer temporal continuity across generation windows. We therefore retain the 50-step configuration for our main results and report the four-step variant as an exploratory acceleration study.

EC — Participant consent and permitted identity use

Participants provided voluntary written informed consent after receiving information about the research goals, intended uses, and data-security procedures. The agreement permits non-commercial algorithm research and publication of captured data and derived results for scientific dissemination and reproducibility. Other uses require additional explicit consent. Participants may revoke consent at any time, and their contact information is not made public. We will clarify these procedures and permitted uses in the revised ethics discussion.

Thanks for your detailed responses! The explanation of layout is intuitive, and results address my concerns. It maybe better to update the LoRA-distilled results on the final version.

Reviewer_Qm9pOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper presents AvatarDynamizer, a framework that converts a static 3D human avatar into a controllable 4D avatar with pose-dependent surface dynamics. Its key technical idea is to represent motion-dependent deformation in a dynamic texture space: a video-diffusion-based Dynamic Texture Generator predicts temporally varying textures conditioned on the avatar identity and skeletal motion, and a Generalized Gaussian Decoder then maps these dynamic textures together with posed normal maps to texel-aligned 3D Gaussians, including motion-dependent Gaussian displacements, enabling consistent free-view rendering. The paper also introduces the DynaHuman multi-view dataset to provide diverse identities and motions for training. The method is validated across multiple datasets against representative baselines, demonstrating improved generative quality and more detailed motion-dependent surface dynamics.

Critical strengths
  1. Effective representation design. The paper introduces dynamic textures as an intermediate representation that connects pretrained video diffusion priors with 3D Gaussian avatars, enabling detailed motion-dependent dynamics while retaining explicit 3D, multi-view-consistent rendering. This is the main technical strength and provides a clear advantage over purely image-space generation or direct 3D regression.

  2. Strong data and empirical support. The paper contributes the DynaHuman dataset with diverse identities, motions, and dense multi-view capture, and evaluates the method across several datasets with consistent improvements in generative quality and high-frequency surface dynamics. This substantially strengthens the evidence that the proposed representation generalizes beyond a single subject or dataset.

Critical weaknesses
  1. Dynamic geometry is not sufficiently validated. The GGD predicts motion dependent Gaussian positions, but the current training and evaluation mainly rely on rendered images. As a result, it is difficult to tell whether the improved wrinkles come from actual 3D geometry changes or mainly from changes in texture and shading. This distinction is important because, if similar visual quality can be achieved by keeping the avatar geometry fixed and only updating the texture, then one could simply retexture the avatar without introducing dynamic geometry prediction in the GGD. A direct geometry evaluation, such as normal or displacement visualization, together with an ablation that keeps the Gaussian geometry fixed and only changes the dynamic texture, would provide much stronger evidence that the GGD is actually learning meaningful 3D surface deformation.

  2. The method may be limited for loose clothing and large deformations. The representation relies on a fixed SMPL-X template and texel-aligned Gaussians, so it assumes that the surface can be mapped to a relatively stable canonical location over time. This assumption may become weak for loose garments, large folds, strong self-occlusion, or topology changes, where the clothing can move far away from the body and the correspondence itself may become unreliable. More experiments on these cases would help clarify how robust the method is and where its practical limits are.

Questions for authors
  1. For a novel avatar, the method requires an identity fitting optimization before applying the trained generator and decoder. Could the authors report the optimization time for each new avatar and discuss how stable this fitting process is across different avatar inputs?

  2. For the implementation details, could the authors clarify which video diffusion backbone is used in the experiments and how the VAP baseline is implemented?

5 comments
AuthorsOpenReview ↗

Response to Reviewer Qm9p

We appreciate the reviewer’s assessment of our dynamic-texture representation as the main technical strength, offering a clear advantage over purely image-space generation or direct 3D regression. We provide direct geometry evaluations, measurements of identity-fitting efficiency and stability, concrete failure cases, and implementation clarifications. We also provides figures of W1, W2 and Q1 in the new main submission on page 10.

W1 — Dynamic geometry evaluation

We agree that the contribution of geometric deformation should be evaluated directly. To examine whether the wrinkles reconstructed by the GGD arise from appearance changes, geometric deformation, or both, we evaluate depth-derived normals and compare the full GGD with three variants:

  1. w/o displacement (retrained): disables all predicted Gaussian position offsets, while keeping rotations, scales, opacity, and SH appearance learnable.
  2. w/o displacement & rotation (retrained): additionally fixes canonical rotations to template normals, while keeping scales, opacity, and SH appearance learnable.
  3. SH-only on fixed template (inference-only): uses template positions and rotations, fixes scales and opacity, and retains predicted SH appearance.

Both retrained variants use the same training data and budget as the full GGD. We evaluate geometry decoding with RDT and the full pipeline with LHM initialization and generated textures. Results are averaged equally across S71 and S76.

Normal and RGB evaluation of geometry ablations on S71/S76.

SettingDecoderNormal-LPIPS ↓Angular error (°) ↓RGB PSNR ↑RGB LPIPS ↓
RDTFull GGD0.171918.1829.770.1213
w/o displacement0.193419.5129.930.1280
w/o displacement & rotation0.196319.5429.850.1318
SH-only on fixed template0.205520.3529.390.1525
GeneratedFull GGD0.202321.9826.410.2010
w/o displacement0.217022.0426.480.2088
w/o displacement & rotation0.213921.7626.510.2121
SH-only on fixed template0.212721.7726.510.2219

Normals are derived from rendered median depth using identical processing across variants and evaluated against ProbeSDF reference normals.

In the reconstruction setting (RDT), the full GGD improves both normal metrics and RGB LPIPS over all three variants, supporting the contribution of displacement and rotation prediction to geometry decoding and perceptual rendering quality.

With generated textures, the Normal-LPIPS and RGB LPIPS improvements persist, while angular errors remain similar across variants. Angular error measures pixelwise normal agreement with the reference, so spatially misaligned generated wrinkles can incur higher errors.

Fig. 5 visualizes rendered normals of the full GGD and its variants. Together, the quantitative results and normal visualizations support the contribution of geometry prediction to both recovered 3D surface detail and perceptual rendering quality.

W2 — Limitations and failure cases

We provide additional qualitative examples in Fig. 7 to clarify the practical limits of our method under loose clothing, severe self-occlusion, and topological changes.

Loose clothing. The input dress has a continuous skirt silhouette, but the animated results develop a trouser-like shape during waving and tug-of-war motions. This distortion is particularly apparent in the wide-legged tug-of-war poses, demonstrating limited preservation of loose-garment shape in this example.

Severe self-occlusion. When driven by motion with severe self-occlusion, the generated textures and rendered avatar show local artifacts around the waist and legs. These artifacts become less visible in later frames, showing partial recovery.

Topological changes. When driven by motion extracted from a garment-removal sequence, the avatar follows the body motion but retains its initial clothing configuration. Our current pipeline uses a fixed canonical correspondence and does not explicitly model garment removal or changes in garment attachment.

AuthorsOpenReview ↗

Response to Reviewer Qm9p

Q1 — Identity fitting time and stability

Identity fitting time. Given the SMPL-X initialization, fitting images, and a surface point cloud, identity fitting takes approximately 12 s on H100 NVL and 18 s on RTX 3090, once per avatar. The fitted avatar is reused across motion sequences. On these GPUs, preparation adds 23.5/39.3 s for single-image input (LHM+NeuS2), or 19.9/32.3 s for multi-view input (NeuS2). Timings exclude model loading, view rendering, and mesh extraction.

Fitting stability. We evaluated six MVHPP subjects with varied body shapes, hairstyles, and clothing, including a long dress, a short skirt, wide-leg trousers, and a loose jacket. Using both input routes, each prepared input was fitted three times on an RTX 3090 with shared optimizer settings; all 36 runs completed successfully. All subjects share one fitting configuration. Fig. 6 shows qualitative results of identity fitting.

Fitting quality and repeatability.

MetricLHM inputMulti-view input
Fitting error (mm) ↓4.57 ± 0.315.35 ± 0.28
PSNR (dB) ↑33.76 ± 2.4525.90 ± 1.51
LPIPS ↓0.102 ± 0.0100.165 ± 0.021
Mean vertex difference across runs (mm) ↓0.560.56
Max. PSNR std. across runs (dB) ↓0.050.17

The first three metrics are mean ± std. across subjects after averaging repeats. RGB metrics use two views withheld from texture fitting, with LHM renderings and captured images as references for the respective input routes.

Consistent fitting errors across subjects, together with low variation in vertex positions and PSNR across repeated runs, support stable and repeatable fitting across the tested inputs with shared optimizer settings.

Q2 — Video backbone and VAP implementation

Video backbone. Our main results use the pretrained Wan-1.3B backbone fine-tuned with LoRA, as stated in Sec. 4. The SVD description and citation in Sec. 3.4 are incorrect and will be corrected.

VAP implementation. As detailed in Appendix F, we implemented a VAP-based regression baseline because the original code and training data were unavailable at implementation time. It shares our GGD’s training data, UNet architecture, and training strategy. To match the GGD parameterization, our baseline uses texture space instead of orthogonal projection space and predicts absolute Gaussian parameters rather than Gaussian deltas. We will clarify this implementation detail in Appendix F. The baseline receives a static identity texture and pose, while the GGD receives per-frame dynamic textures and pose.

Thank you for the detailed response and additional experiments.

The new geometry evaluation provides useful evidence that displacement prediction improves some geometry metrics. However, in the generated texture setting, the improvements in RGB quality are limited, suggesting that much of the visual improvement may come from dynamic textures rather than geometric deformation. While this does not weaken the main contribution of the dynamic texture representation, the importance of dynamic geometry prediction remains unclear. I therefore consider W1 partially addressed.

The additional failure cases clearly explain the limitations of the method, and I consider W2 adequately addressed.

Reviewer_tiS4OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents AvatarDynamizer, a framework that converts static 3D human avatars into dynamic avatars with pose-dependent surface details. The method introduces a dynamic texture representation that connects pretrained video diffusion models with a Gaussian decoder, aiming to generate realistic clothing dynamics while maintaining multi-view consistency. The paper addresses an interesting problem and presents promising results. However, I have concerns regarding the methodological novelty and whether the experiments sufficiently support the main claims. I am open to raising my score if the authors can adequately address these concerns during the rebuttal.

Critical strengths

The paper addresses a meaningful problem of generating realistic surface dynamics for animatable human avatars, with clear motivation and potential practical applications.

The integration of video diffusion priors with 3D Gaussian representations provides a reasonable framework for generating dynamic appearance details while supporting free-viewpoint rendering.

The experiments cover multiple datasets and demonstrate promising generative quality. The newly collected DynaHuman dataset is also a valuable contribution.

Critical weaknesses
  • Limited methodological novelty. The main components, including dynamic textures, pose-dependent Gaussian modeling, and video diffusion, have been explored in prior works [1,2]. The authors should better clarify the technical novelty beyond combining these existing techniques. [1] Animatable Gaussians. CVPR 2024 [2] Real-time Free-view Human Rendering from Sparse-view RGB Videos using Double Unprojected Textures. CVPR 2025.

  • Concerns about baseline comparisons. The VAP baseline is reimplemented with modifications to its original architecture and representation. Moreover, differences in pretrained priors and generative formulations make it difficult to isolate the benefits of the proposed dynamic texture representation.

  • Insufficient evaluation of surface dynamics. Although FID and FVD are useful generative metrics, they do not directly measure whether the generated clothing dynamics are physically plausible and correctly correlated with body motion. More direct evidence would strengthen the central claims.

  • Limited evaluation of multi-view consistency. While multi-view consistency is a key claimed advantage, the paper lacks dedicated quantitative comparisons demonstrating its improvement over image-space generative methods.

  • Missing efficiency analysis. The paper does not report the end-to-end inference time or computational cost of the proposed pipeline, which is important for practical avatar animation.

Questions for authors

See weaknesses.

7 comments
AuthorsOpenReview ↗

Response to Reviewer tiS4

We thank the reviewer for recognizing our approach as a reasonable way to generate dynamic appearance details while supporting free-viewpoint rendering, as well as our promising generative results and the value of the DynaHuman dataset. To address the concerns raised, we provide additional baseline comparisons and evaluations of surface dynamics, multi-view consistency, and efficiency.

W1 — Methodological novelty

Our methodological contribution comprises a dynamic-texture representation that bridges multi-view reconstruction and conditional generation of surface dynamics, and an identity- and motion-conditioned generative model learned in this representation. This design follows from our analysis of where surface dynamics are represented and how they are learned, a key challenge in generating fine-grained surface dynamics for generalizable, identity- and motion-driven 4D human avatars (Introduction; Sec. 3.2).

Identity- and pose-conditioned Gaussian regression learns to infer surface details from a static identity texture and a time-varying pose texture, without an explicit per-frame representation of those details. Both our VAP-based regression baseline and GGD are trained across multiple subjects using the same data and matched network architectures. The GGD receives per-frame dynamic textures together with pose textures. Evaluated on S59, a subject in the training set, explicit dynamic-texture encoding improves reconstruction PSNR from 30.36 to 32.02 and LPIPS from 0.160 to 0.103, without involving video generation or its pretrained prior (Sec. 4.2). These results support the benefit of explicitly encoding observed surface details for reconstruction.

Image-space generation, as in GAS, benefits from pretrained video priors but does not enforce a shared 3D state across views. Our new quantitative comparison on S71 confirms higher multi-view self-consistency than GAS on all three metrics (W4).

The same dynamic textures serve two complementary learning roles: they explicitly encode observed surface dynamics for multi-view-supervised Gaussian reconstruction and provide targets for identity- and motion-conditioned video generation. A shared GGD maps both reconstructed and generated textures to Gaussian geometry and appearance, yielding one 3D state per frame for all rendering views.

Animatable Gaussians [1] learns a person-specific pose-to-Gaussian mapping, while DUT [2] uses person-specific models to reconstruct geometry and appearance from textures unprojected from current-frame images. We instead learn a generalizable generative model in dynamic texture space that synthesizes 4D human avatars with vivid, motion-dependent surface dynamics from a static avatar and driving motion. For unseen identities, our method requires neither person-specific dynamic training sequences, as required by [1,2], nor target-frame image observations at inference, as required by [2].

Our ablations support the value of pretrained video priors and temporal modeling for this formulation (Table 3 of the submission). Furthermore, Ours (SVD1.1) uses the same pretrained SVD1.1 backbone as GAS, with GAS-style conditioning adapted to texture generation, and reduces mean FVD from 124.51 to 70.61 on S71/S76 (W2). This shows that the observed advantage of our generation pipeline is not specific to the Wan backbone.

AuthorsOpenReview ↗

Response to Reviewer tiS4

W2 — Baselines and pretrained priors

As detailed in Appendix F, we implemented a VAP-based regression baseline because the original code and training data are not publicly available. It follows VAP’s core formulation of predicting Gaussians from a static identity representation and pose. We use the same texture-space Gaussian parameterization, training data, UNet architecture, and training strategy as our GGD to control these implementation factors. Both models are trained across multiple subjects. The baseline receives a static identity texture and a time-varying pose texture, while our GGD receives per-frame dynamic textures and pose textures.

The S59 compression-and-reconstruction experiment (Sec. 4.2) evaluates the two representations on S59, a subject in the shared training set, using the same observed multi-view videos. The GGD receives reference dynamic textures optimized from these observations. Compared with the regression baseline, which embeds surface dynamics implicitly in its network parameters, explicit dynamic-texture encoding improves PSNR from 30.36 to 32.02 and LPIPS from 0.160 to 0.103. These results support the benefit of explicitly encoding observed surface details for reconstruction, without involving video generation or its pretrained prior.

To address differences in pretrained priors and generative formulations, we evaluate Ours (SVD1.1): our dynamic-texture formulation using the same pretrained SVD1.1 backbone as GAS, together with GAS’s pose and appearance conditioning modules adapted to UV texture generation. The conditioning inputs are the static UV identity texture and per-frame pose textures; the generated dynamic textures are decoded by our GGD.

To accelerate training, Ours (SVD1.1) uses a global batch size of 16 for 25k optimization steps, matching the total sample exposure of our original Wan schedule (global batch size 4 for 100k steps). For reference, we also retrain our Wan-based generator as Ours (Wan, retrained), using the same training data, identity and motion inputs, texture representation, Gaussian decoder, batch size, and number of optimization steps.

Generation results averaged equally over held-out subjects S71 and S76, using LHM initializations.

MethodPSNR ↑SSIM ↑LPIPS ↓FID ↓FVD ↓
GAS †25.740.7230.25450.03124.51
Ours (paper) †26.430.7290.20222.5856.72
Ours (Wan, retrained)26.450.7290.20021.2955.03
Ours (SVD1.1)26.780.7400.21329.3170.61

† Results reported in Table 1 of the submission. GAS uses its official checkpoint; Ours (SVD1.1) is our UV-generation variant initialized from pretrained SVD1.1 with adapted GAS conditioning.

The retrained Wan variant maintains performance close to our submitted results with this training schedule. Using the same pretrained SVD1.1 backbone as GAS and adapted conditioning modules, Ours (SVD1.1) outperforms GAS on all five metrics averaged over S71 and S76, reducing LPIPS from 0.254 to 0.213, FID from 50.03 to 29.31, and FVD from 124.51 to 70.61. Thus, our improvement persists with an SVD1.1-based texture generator using adapted GAS conditioning and is not specific to the Wan backbone.

These experiments provide complementary evidence: S59 evaluates the encoding and reconstruction of observed surface dynamics without video generation or its pretrained prior, while Ours (SVD1.1) demonstrates that our generation pipeline retains its advantage over GAS with the same pretrained backbone and adapted GAS conditioning.

AuthorsOpenReview ↗

Response to Reviewer tiS4

W3 — Physical plausibility and motion–surface correlation

Perceived physical plausibility. Our method targets perceptually plausible, motion-dependent surface dynamics (Appendix J); we do not claim physically accurate cloth simulation.

Quantitative assessment of physical plausibility is challenging in this setting: our Gaussian representation models geometry and appearance without an explicit cloth-mechanics model, and the available captures lack physical annotations such as material parameters and applied forces. HOOD (Grigorev et al., CVPR 2023, Sec. 4.1) similarly discusses the difficulty of objective evaluation of garment quality and plausibility and uses a perceptual study to assess garment realism.

We evaluate perceived physical plausibility through our user study involving 18 participants and five subjects (Appendix H, Table 5 of the submission). Our method receives a mean rating of 4.003/5, compared with 3.289 for static avatars, 2.622 for the VAP-based baseline, and 1.289 for GAS. These results support improved perceived physical plausibility, complemented by the quantitative motion–surface correlation evaluation below.

Motion–surface correlation. To assess whether the generated surface dynamics are correctly correlated with the driving body motion, we perturb the motion conditioning supplied to the DTG while keeping the LBS motion and decoder pose inputs unchanged. We evaluate the outputs in texture space using LPIPS and Sobel-gradient correlation against reference dynamic textures, and in image space using angular error and LPIPS between depth-derived normal maps and reference normals from ProbeSDF reconstructions.

On two unseen subjects, S71 and S76, we compare the following six variants, using the same identity texture across variants:

  1. Aligned (ours): the original, temporally aligned pose-normal sequence conditions the DTG.
  2. Constant (Local): the first pose-normal condition is repeated within each generation window.
  3. Constant (Global): reuses textures generated under a single fixed pose condition.
  4. Shifted 44 frames: the DTG conditioning sequence is temporally offset by 44 frames.
  5. Shifted 88 frames: the same temporal offset is increased to 88 frames.
  6. Window-shuffled: each conditioning window is replaced with an intact window from elsewhere in the same sequence.

Motion-conditioning results averaged equally across S71 and S76.

VariantTexture LPIPS ↓Texture Sobel corr. ↑Normal error (°) ↓Normal LPIPS ↓
Aligned (ours)0.1440.29721.970.2022
Constant (Local)0.1570.25322.380.2051
Constant (Global)0.1640.22922.320.2059
Shifted 44 frames0.1600.25422.350.2049
Shifted 88 frames0.1620.24722.420.2055
Window-shuffled0.1690.22722.600.2067

Sobel correlation measures spatial Pearson correlation between generated and reference gradient magnitudes over the body region.

Aligned conditioning achieves the best mean values on all four metrics. Both constant variants perform worse, supporting the value of time-varying motion conditioning. Temporal shifts also degrade performance despite preserving the order of the conditioning sequence, with the 88-frame shift yielding worse results than the 44-frame shift on every metric. Window shuffling produces the largest degradation overall, further supporting the importance of temporal alignment.

The advantage is observed in UV textures before Gaussian decoding and in surface normals after decoding, with unchanged skeletal motion and decoder pose inputs. These results provide quantitative evidence that temporally aligned DTG conditioning is associated with better agreement with observed surface dynamics, in both appearance and geometry.

Cross-method normal comparison. To assess the geometric fidelity of generated surface dynamics beyond FID and FVD, we compare surface normals across four methods on S71 and S76 , averaging results equally across the two subjects.

Surface-normal comparison.

MethodAngular error (°) ↓N-LPIPS ↓
Ours21.970.2023
Static avatar23.120.2086
LHM+VAP22.600.2157
GAS*36.750.2579

GAS normals are derived from 3DGS reconstructions fitted to 24 generated views.

Ours achieves lower angular error and N-LPIPS than all three baselines.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.