acceptodds
Under review as a conference paper at ICLR 2027

FACt: Feed-forward Animatable Clothed Human Reconstruction

Abstract

We present FACt, a novel feed-forward framework for animatable clothed human reconstruction from sparse-view images. Unlike optimization-based methods requiring per-subject refinement or feed-forward approaches limited to static capture, FACt leverages 3D Gaussian Splatting to enable high-fidelity avatar reconstruction and adaptive clothing deformation in a single forward pass. Our model's architecture efficiently fuses 3D representation with locally aligned image features via a deformable transformer, effectively preserving geometric and appearance fidelity in the canonical space. To accurately model dynamic clothing, we introduce an adaptive cloth joint learning strategy. This strategy extends the standard Linear Blend Skinning kinematic chain beyond the SMPL-X template, enabling realistic motion control across diverse clothing styles and poses. Experiments on real-world datasets demonstrate that FACt produces realistic and animatable human avatars across novel views and poses, outperforming state-of-the-art methods.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Human Reconstruction, 3D Gaussian Splatting

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_MAdAOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents FACt, a feed-forward framework for animatable clothed human reconstruction from sparse-view images. The method combines deformable feature fusion with an extended LBS formulation based on learned cloth joints. The problem is relevant, and the reported quantitative results are promising.

Critical strengths

The paper addresses a meaningful and challenging problem. Feed-forward reconstruction of animatable clothed humans from sparse-view images is practically valuable, particularly for loose garments with complex non-rigid deformations.

The proposed framework is well-structured. The combination of deformable feature fusion and cloth-joint-based deformation provides a coherent approach to integrating image information with structured garment motion modeling.

The experimental results are promising. The method achieves competitive quantitative performance on THuman2.1 and 4D-DRESS while using relatively few network parameters and Gaussian primitives. The paper also includes ablation studies and visualizations to analyze its main components.

Critical weaknesses
  • Fairness and completeness of baseline comparisons. Table 1 compares the proposed method using two input views and 50K Gaussians against LHM-MINI using only one view and 20K Gaussians. This makes it difficult to determine whether the improvements arise from the proposed architecture or the additional input information. Moreover, although LHM++ is included in the efficiency comparison in Table A2, its reconstruction quality is not reported. I suggest providing comparisons under matched input conditions where possible and including LHM++ in the reconstruction evaluation.

  • Limited methodological novelty of extended LBS. The proposed cloth-joint-based extended LBS shares substantial similarities with existing approaches such as GART (CVPR 2024) and ToMiE (ICCV 2025), which also extend skeletal representations to model non-rigid clothing deformation. Although the feed-forward formulation is a potentially valuable distinction, the paper does not sufficiently clarify the technical novelty of its cloth-joint learning mechanism relative to these methods.

  • Insufficient isolation of the contributions of extended LBS. In Table 3, the effects of the cloth template and extended LBS are not fully disentangled. Furthermore, the deformation MLP contributes substantial improvements, raising questions about how much of the final performance gain is attributable to the proposed cloth joints. More controlled ablations, particularly under novel-pose animation, would strengthen the claims regarding the effectiveness of extended LBS.

  • Generalization evaluation and dataset splitting. According to Appendix B.3, the 4D-DRESS dataset is split at the pose-sequence level, with identities and outfits shared between training and testing. While this evaluates generalization to unseen motion sequences, it does not establish generalization to unseen identities or clothing. A subject-disjoint evaluation or a clearer discussion of this limitation would be valuable, especially given the claimed generalizability of the framework.

  • Limited evidence for reconstruction fidelity and alignment robustness. Despite using additional input information compared with LHM-MINI, the proposed method does not consistently show visibly superior reconstruction quality in Fig. 4. The third case exhibits artifacts around the neck and hair, while certain regions in 4D-DRESS appear oversmoothed compared with GHG.

Questions for authors

See weaknesses

6 comments

Thank you for the additional experiments. The new LHM++ comparison and the identity-disjoint 4D-DRESS evaluation substantially address my concerns regarding baseline fairness and generalization.

I have one additional concern regarding the LHM++ comparison. Although both methods now use two input views, FACt is trained directly on THuman2.1/4D-DRESS, whereas LHM++ is evaluated using its pretrained model without training on these target datasets. Therefore, the comparison still mixes architectural differences with an in-domain training advantage. If possible, I think a more informative comparison would be to fine-tune LHM++ on the same training split, or alternatively evaluate FACt in a comparable cross-dataset/zero-shot setting. Otherwise, this limitation should be made explicit when interpreting the comparison.

I also still have one question regarding the independent contribution of extended LBS. In the new ablation, when the deformation MLP is disabled, adding extended LBS does not improve performance (PSNR 20.08 →\rightarrow 20.04, LPIPS 0.201 →\rightarrow 0.209). For the additional Standard LBS vs. Extended LBS temporal comparison, is the deformation MLP enabled in both settings? If so, the results may suggest a complementary effect between extended LBS and the deformation MLP rather than an independent gain from extended LBS itself.

Method#Params#Gaussians#ViewsTHuman2.1 PSNR ↑THuman2.1 LPIPS ↓THuman2.1 SSIM ↑4D-DRESS PSNR ↑4D-DRESS LPIPS ↓4D-DRESS SSIM ↑
LHM++459M160K219.080.2000.81117.660.1890.810
LHM++ (Fine-tuned)459M160K219.930.1760.81718.100.1780.812
Ours133M50K220.530.1980.80219.950.1700.817

Thank you for your follow-up and for acknowledging that the additional experiments address much of the concern about baseline fairness and generalization.

To address the remaining training-distribution difference, we fine-tuned LHM++ on the same training split used by FACt and evaluated both methods with two input views. Fine-tuning improves LHM++ on both datasets. FACt retains higher PSNR on THuman2.1 (20.53 vs. 19.93). On the identity-disjoint 4D-DRESS test set, FACt performs better on all three metrics. We will update the comparison and its interpretation accordingly.

For the additional Standard LBS versus Extended LBS temporal comparison, the deformation MLP is disabled in both settings. This comparison therefore measures the effect of extended LBS without an MLP contribution. We agree that, on the 4D-DRESS subset for ablation study, extended LBS alone does not seem to improve reconstruction when the MLP is disabled. The temporal comparison uses the full training data and the same clips as our temporal evaluation. The results show the benefits of extended LBS.

We appreciate again your constructive feedback, which has helped us clarify the evaluation and improve our work.

Regarding LHM++, the fine-tuned results make the comparison more informative. However, I would still like to understand the fine-tuning protocol in more detail, especially which parameters are updated, the optimization settings/training budget, and whether the original LHM++ losses are reproduced. Since the comparison is now used to support the effectiveness of FACt under matched training data, these details are important for assessing fairness.

Regarding extended LBS, I appreciate the clarification that the deformation MLP is disabled in both the Standard-LBS and Extended-LBS temporal comparisons. This shows that extended LBS can provide benefits on the full temporal training setting. However, the gain is still relatively modest, and the earlier ablation on the subset shows almost no improvement when extended LBS is introduced without the MLP. More importantly, the authors also acknowledge that extending the skeleton with clothing joints is established prior work. Therefore, I am still somewhat unclear about the main technical novelty here. Is the key contribution the feed-forward prediction of cloth joints and their pose-dependent motions from sparse-view observations? If so, I suggest making this distinction more explicit in the paper.

Reviewer_fXefOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

FACt reconstructs an animatable clothed Gaussian avatar from sparse RGB views with calibrated cameras and known or estimated SMPL-X parameters. It initializes separate body and garment Gaussians using SMPL-X and category templates prepared offline from DeepFashion3D. A point transformer aggregates locally sampled Sapiens image features through deformable attention and visibility-aware view fusion. Learned cloth joints extend the body skeleton, and distance-dependent skinning blends body and cloth motion before a residual deformation MLP adds finer detail. The paper evaluates novel-view reconstruction on THuman2.1 and 4D-DRESS, fixed-source and per-frame pose sequences, and qualitative pose transfer, with architectural and joint-setting ablations.

Critical strengths
  1. The cloth-joint formulation provides a useful structured model for garment regions that standard body skinning handles poorly. Body-linked joint transforms, local garment support regions, and a continuous body-to-cloth blend offer a more inspectable deformation mechanism than pointwise offsets alone. The paper explains the rest-normalized transforms and initialization in detail, and the rendered examples show plausible loose sleeves, skirts, and coats. This is a consequential contribution toward constructing controllable avatars without per-subject optimization.
  2. The reconstruction results show a meaningful practical benefit with a compact trainable reconstruction network. Table 1 reports the best PSNR and LPIPS among the evaluated methods on both datasets: 20.53/0.198 on THuman2.1 and 19.61/0.166 on 4D-DRESS. The THuman2.1 split holds out identities, providing useful evidence beyond reconstruction of training subjects. Table 4 also supports the value of visibility-aware fusion: two-view PSNR increases from 22.22 to 22.73 with fusion. The appendices provide extensive visual results, initialization details, and an efficiency comparison, making the design relatively clear to assess.
Critical weaknesses
  1. The comparisons do not cleanly establish the claimed generalization and method-level advantage in the dynamic loose-clothing setting. Appendix B.3 states that the 4D-DRESS training split contains all 32 identities and all 61 outfits, while testing holds out takes of largely the same people and outfits. Appendix B.5 uses released baseline checkpoints without adapting them to this training distribution; LHM also receives one view while FACt receives two. This is a valid held-out-sequence experiment, but it does not establish unseen-identity/outfit generalization on dynamic clothing or isolate architecture from training data and input coverage. The identity-disjoint static THuman2.1 result partly addresses generalization, but cannot settle the central dynamic claim.
  2. The specific benefit of learned cloth joints for animation is not isolated sufficiently, and the fixed-source results are mixed. In Table 3, the cloth template and extended LBS are always switched on together, so the gain from 19.50 to 21.03 PSNR cannot be assigned specifically to the joint model rather than the additional garment geometry prior. The residual MLP itself contributes large gains. In the more relevant fixed-source animation setting, Table 2 improves PSNR over LHM (17.95 versus 17.19), but has worse LPIPS (0.210 versus 0.199) and SSIM (0.773 versus 0.791). Thus the results support a tradeoff, rather than consistently better animation fidelity. A template-matched joint ablation evaluated in this setting would materially strengthen the principal contribution.
Questions for authors

The questions below request clarification or evidence for the present claims.

  1. Can you separate the effect of matched training data and source-view coverage from the architectural change, using an available strong baseline trained or adapted under the same conditions? Are there existing dynamic-clothing results on identities or outfits excluded from training? Please distinguish these from the take-level 4D-DRESS split, which the appendix already specifies.
  2. With the same cloth templates, Gaussian budget, image-fusion network, residual MLP, and training schedule, what is the effect of replacing standard LBS with the learned cloth-joint extension? In particular, do fixed-source animation LPIPS/SSIM and garment-region errors improve? This would clarify the joint model's contribution despite the mixed Table 2 results.
  3. How sensitive is the pipeline to the garment-type classifier (App. B.2) and to garments outside the 13 templates (open jackets, layered outfits)?
Reviewer_hc1AOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

It is a paper on feed-forward animatable clothed human reconstruction from sparse-view images, based on LHM. FACt combines two ideas: deformable attention like GeoLRM to obtain local per-gaussian image features from foundation models (e.g. Sapiens), and feed-forward prediction of cloth joints within a two-layer hierarchical body+cloth gaussian design based on these aggregated features. Body and cloth gaussians are initialized from the SMPL-X surface and from DeepFashion3D clothing templates respectively. Gaussian queries are passed through a Point Transformer backbone which applies deformable attention (similar to deformable DETR) to attend to Sapiens features. There is also some multi-view fusion. This part of the architecture ultimately predicts 3DGS parameters. To support deformations due to cloth, a set of cloth joint anchors is uniformly sampled on top of the SMPL-X surface and associated with its nearby Gaussian features. The features of those nearby Gaussians is aggregated and used to feed-forward predict a displacement along the smpl-x surface of the cloth joint wrt. its anchor point on the SMPL-X surface. The body and cloth joints form a two-level LBS. In contrast, LHM uses joint global attention, inputs a single view so there's no cross-view fusion problem, and there is no cloth deformation or LBS hierarchy.

Critical strengths

The paper achieves strong PSNR results on thuman2.1 and 4d-dress in Table 1.

The deformable MLP and two-layer cloth LBS deformation is shown to be effective in ablation studies in Table 3.

The visual results are less blurry compared to LHM which cannot model cloth deformation effectively.

Critical weaknesses

The deformable attention actually does not seem to help much when compared to global attention baseline (17.51 => 17.67 PSNR in Table 3). The main improvement seems to come from a deformable MLP and two-layer cloth LBS deformation, both of which contribute complementary ~1-1.5 PSNR gains.

Questions for authors

Is deformable attention + cross-view fusion required for the cloth template and deformation MLP to work? Maybe a potential simple architecture could be to keep the global attention mechanism of LHM and only introduce the two-layer cloth deformation modeling.

As shown in Table A3, adding more joints also does not seem to significantly help, is there a lower bound for how many joints are needed to maintain similar performance?

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

Dear respected PCs, SACs, ACs, and Reviewers,

We thank all reviewers for their valuable and constructive feedback. Reviewer MAdA recognizes animatable clothed human reconstruction from sparse-view images as a meaningful and challenging problem. All three reviewers acknowledge our meaningful and promising experimental results, highlighting the quantitative reconstruction results on two datasets (hc1A, fXef, MAdA), clear visual results on cloth deformation (hc1A), effective ablation studies (hc1A), and extensive experimental details (fXef). Regarding our method, reviewer MAdA recognizes the well-structured pipeline and the coherence between deformable feature fusion and cloth-joint-based deformation. Reviewer fXef regards our work as a consequential contribution toward controllable avatar reconstruction without per-subject optimization, and reviewer hc1A acknowledges the effectiveness of our extended LBS design for cloth motion modeling.

We have carefully addressed all reviewer concerns and revised the paper with additional clarifications and experiments. We summarize the main concerns and our responses below.

Generalizability and baseline fairness (fXef, MAdA): We retrained FACt on a strict identity-disjoint 4D-DRESS split, holding out six identities and all their associated outfits and sequences. We also fine-tuned GIGA, NoPo-Avatar, and GHG on the same training split and added reconstruction comparisons with LHM++ using the same two source views as FACt. FACt achieves the highest PSNR on the revised 4D-DRESS test set (19.95) and THuman test set (20.53), with competitive or comparable LPIPS and SSIM over five SOTA methods. Notably, we achieve such performance under relatively small trainable parameters and Gaussian budget. These experiments strengthen the evidence for unseen-identity generalization and reduce training-distribution and input-view disparities. We modify Tab. 1 and update Fig. 4 and A4 to reflect performance under the new 4D-DRESS test set. We add Tab. A2 and A5 to reflect performance comparison with LHM++.

Isolated contribution of extended LBS (fXef, MAdA): We expanded the ablations to separate extended LBS from the cloth-template prior and deformation MLP. With the template and MLP fixed, extended LBS improves PSNR from 20.05 to 21.03, LPIPS from 0.208 to 0.200, and SSIM from 0.781 to 0.816. The ablations also reveal complementary effects among additional geometric prior, extended LBS and the deformation MLP. The expanded ablations appear in Tab. 3 and Tab. A4. Furthermore, we additionally compare standard and extended LBS on dynamic pose sequences to assess their animation behavior in Tab. 4, Fig. 7 and Fig. A.

Contribution of deformable attention (hc1A): We evaluated the suggested global-attention alternative with the same cloth template, extended LBS, and deformation MLP. The global-attention settings with other cloth modeling modules only achieve PSNR 18.76, while deformable attention with cross-view fusion improves PSNR from 18.76 to 21.03. This clarifies that our deformable attention is coherent with extended LBS to achieve the best performance. The added ablations appear in Tab. 3 and Tab. A4.

Sensitivity to joint count and template selection (hc1A, fXef): Additional experiments identify 100 cloth joints as the smallest tested configuration that retains the performance of the 200-joint model. We also evaluate misclassified-template and layered-outfit subsets (including challenging cases such as open jackets and layered dresses) against LHM and LHM++, and compare our automatic template selection with manual correction. These results characterize the tolerance to template mismatch. We add discussion in App. E and Tab. A6, A7, A8.

Novelty of cloth-joint-based extended LBS (MAdA): We clarify how our extended LBS differs from existing virtual-bone-based methods such as GART and ToMiE: (1) it supports feed-forward cloth deformation rather than requiring gradient-based per-subject optimization, and (2) it uses an explicit learnable hierarchical kinematic chain to model motion propagation from body joints to cloth joints. We modify relevant parts in the Introduction and Related Work to distinguish our contribution from extended LBS.

We thank all the reviewers and the Area Chair again for their time and valuable feedback, which have greatly strengthened the scientific validity and clarity of our paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.