acceptodds
Under review as a conference paper at ICLR 2027

An Anatomy-Informed 20D Representation for Text-Conditioned 3D Hand Pose Generation

Abstract

3D hand pose is widely used in sign language, human-computer interaction, and virtual avatars. Its representation determines how finger articulation is measured, controlled, and generated. 3D hand poses are commonly represented using MANO, which drives hand meshes with a 45D vector of local 3D joint rotations. This representation uses more dimensions than the hand's anatomical degrees of freedom, and its components do not directly represent finger flexion and spread angles. In this paper, we introduce AHand20, an anatomy-informed 20D hand pose representation guided by joint structure and degrees of freedom, with angles defined according to how each joint moves. AHand20 supports conversion to and from MANO and enables pose augmentation by varying finger flexion and spread while maintaining anatomical validity. Using the augmented poses, we construct a dataset containing 15,400 MANO poses and 30,800 descriptions across 154 handshapes, with each description linked to multiple pose variations. Trained on this dataset, a text-conditioned generator predicts 20D hand poses from descriptions of unseen handshapes and achieves lower joint position and rotation errors than existing methods. AHand20 also covers the main joint motions and converts to and from MANO with low error. Beyond pose generation, the representation identifies implausible poses in existing datasets, extending its application to hand pose data filtering.

Primary area:
generative models
Keywords:
3D Hand Pose Representation, Text-to-Pose Generation, Pose Augmentation

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_9jF3OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces AHand20, an anatomy-informed 20D hand pose representation. While MANO parameterizes each joint with a general 3-axis rotation, AHand20 describes the hand with per-joint anatomical angles, such as flexion and spread, that follow the degrees of freedom of each joint. AHand20 can be converted to and from MANO, which enables controllable pose augmentation in the anatomical space and allows existing MANO-based pipelines to be reused for rendering. Using this augmentation, the paper constructs a pose-text dataset of 154 handshapes and trains a text-conditioned generator that decodes through AHand20. The proposed method, built upon PoseScript, outperforms the adapted baselines, which suggests that an anatomy-informed intermediate representation is useful. The representation is also used to identify implausible poses in existing datasets, suggesting the utility of the proposed anatomy-informed representation.

Critical strengths

[S1] A well-motivated, interpretable, and practical framework. The proposed 20D anatomical representation is well-motivated and practical, addressing MANO’s limitations in generating anatomically plausible hand poses while facilitating more effective data augmentation to improve generalization. In particular, AHand20 enhances interpretability by explicitly modeling joint constraints and articulations based on hand anatomy, enabling more intuitive and controlled pose modifications. Overall, the work is clearly motivated, and its straightforward yet effective design offers practical value for hand pose representation and generation.

Critical weaknesses

[W1] Limited novelty beyond prior reduced-DoF representations and PoseScript. Representing a hand or body pose with a reduced set of anatomically meaningful DoFs is a well-established and straightforward idea [1-3]. The conversion between the 20D space and MANO enables practical data augmentation, but no design specifically tailored to this task beyond prior work is significant. In particular, the only difference from PoseScript is an intermediate 20D representation space inserted before the 45D hand pose is generated. Therefore, although the proposed intermediate representation is shown to be useful and practical compared with the benchmarked methods, the work offers limited substantive methodological or architectural advancement.

[W2] Baselines are adapted from body-pose methods, and their adaptations are not fair or not clearly described.

  • All baselines in Table 1 were originally designed for body pose rather than hand pose. Also, other diffusion-based text-conditioned pose/hand-pose generation methods should be included, such as a static variant of HandMDM [4] and the text-to-hand-parameter stage of HanDiffuser [5]. This diffusion-based family is missing from the current comparison.
  • PoseFix is a pose-editing method. How it is converted to a generation task, including the choice of the fixed source pose, should be described.
  • UniPose's LLM backbone is replaced with DistilBERT, which is far weaker than the original (LlaVA-1.6V). In contrast, ChatPose keeps LLaVA-v1.5-13B. The reason for this inconsistency should be justified, and the fine-tuning strategy for ChatPose (full fine-tuning or LoRA) should be specified.
  • AHand20 is applied only to the PoseScript-style model. Applying the same 20D output space to the other baselines would show whether the gain generalizes across architectures.

References

[1] K. Yao and A. Billard, “An inverse optimization approach to understand human acquisition of kinematic coordination in bimanual fine manipulation tasks,” Biological Cybernetics, vol. 114, pp. 63–82, 2020. https://doi.org/10.1007/s00422-019-00814-9.

[2] X. Zhou, Q. Wan, W. Zhang, X. Xue, and Y. Wei, “Model-Based Deep Hand Pose Estimation,” in Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), 2016, pp. 2421–2427. https://arxiv.org/abs/1606.06854.

[3] Y. Xia, X. Zhou, E. Vouga, Q. Huang, and G. Pavlakos, “Reconstructing Humans with a Biomechanically Accurate Skeleton,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025, pp. 5355–5365. https://arxiv.org/abs/2503.21751.

[4] L. Bensabath, M. Petrovich, and G. Varol, “Text-Driven 3D Hand Motion Generation from Sign Language Data,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026. https://arxiv.org/abs/2508.15902.

[5] S. Narasimhaswamy, U. Bhattacharya, X. Chen, I. Dasgupta, S. Mitra, and M. Hoai, “HanDiffuser: Text-to-Image Generation With Realistic Hand Appearances,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 2468–2479. https://arxiv.org/abs/2403.01693.

Questions for authors

[Q1] How are the 20D targets annotated? Sec. 4.1 and Appendix B.1 only state that the targets are "manually specified" under the guidance of 2D source images. Since all reference poses, augmented variants, and test ground truths derive from these targets, please clarify:

  • How the 20 continuous angles are determined for each handshape (e.g., discrete levels, per-state templates, or iterative fit-and-render tuning).
  • The number of annotators, their expertise, and whether inter-annotator consistency was assessed.

[Q2] What are the limitations and future directions? Please include and discuss limitations of the proposed work. For example, extending the concept of AHand20 to the full human body would likewise require additional joint-specific anatomical definitions. This may limit the scope and applicability of the method in more general settings.

5 comments
AuthorsOpenReview ↗

Response to Reviewer 9jF3 (Part 1 / 3)

Thank you for examining our work in detail and for outlining how the contribution and experimental comparisons could be made clearer. Your assessment describes AHand20 as a well-motivated, interpretable, and practical framework, with particular strengths in intuitive pose control and anatomically informed augmentation.

Your comments highlight the need to clarify our contribution, strengthen the baseline comparisons, and explain the reference-pose annotation process. We have added experiments and revised the relevant descriptions, together with a discussion of limitations and future directions.


Q1: Limited novelty beyond prior reduced-DoF representations and PoseScript. Representing a hand or body pose with a reduced set of anatomically meaningful DoFs is a well-established and straightforward idea [1–3]. The conversion between the 20D space and MANO enables practical data augmentation, but no design specifically tailored to this task beyond prior work is significant. In particular, the only difference from PoseScript is an intermediate 20D representation space inserted before the 45D hand pose is generated. Therefore, although the proposed intermediate representation is shown to be useful and practical compared with the benchmarked methods, the work offers limited substantive methodological or architectural advancement.

A1: In the original paper, developing a new generator architecture is neither the purpose nor a claimed contribution. We use a PoseScript-based generator only to evaluate AHand20.

The cited works differ from ours in their skeletal models and research objectives. Zhou et al. [2] and HSMR [3] use libhand and SKEL, respectively, while AHand20 is defined for MANO. Yao and Billard [1] use a kinematic hand model to study coordination in bimanual manipulation, without developing a representation for pose control.

References

[1] Yao and Billard. An Inverse Optimization Approach to Understand Human Acquisition of Kinematic Coordination in Bimanual Fine Manipulation Tasks. Biological Cybernetics, 2020.

[2] Zhou et al. Model-Based Deep Hand Pose Estimation. IJCAI, 2016.

[3] Xia et al. Reconstructing Humans with a Biomechanically Accurate Skeleton. CVPR, 2025.


Q2: All baselines in Table 1 were originally designed for body pose rather than hand pose. Also, other diffusion-based text-conditioned pose/hand-pose generation methods should be included, such as a static variant of HandMDM [4] and the text-to-hand-parameter stage of HanDiffuser [5]. This diffusion-based family is missing from the current comparison.

A2: We thank the reviewer for recommending these hand-focused diffusion methods. Including them broadens the comparison and strengthens the experimental evaluation. We additionally evaluated a static adaptation of HandMDM [4] and the text-to-hand-parameter stage of HanDiffuser [5]. The results are shown below.

R4-Table1: Comparison with hand-focused diffusion baselines adapted to static poses. HanDiffuser uses a paper-based implementation of its T2H stage. Both diffusion methods use frozen DistilBERT. Position errors are in millimeters, rotation error is in degrees, and R@K is in percent.

MethodMPJPE ↓MPVPE ↓Rotation ↓R@1 ↑R@5 ↑FID ↓
HandMDM14.7714.7714.3714.3718.4818.4838.2138.2181.55\mathbf{81.55}84.0184.01
HanDiffuser17.3717.3716.8816.8821.6921.6921.5221.5256.5056.50103.15103.15
Ours10.19\mathbf{10.19}9.90\mathbf{9.90}12.90\mathbf{12.90}42.03\mathbf{42.03}81.5481.5471.34\mathbf{71.34}

Our method achieves lower geometric errors and FID than HandMDM and HanDiffuser, with the highest R@1 and comparable R@5 to HandMDM.

References

[4] Bensabath, Petrovich, and Varol. Text-Driven 3D Hand Motion Generation from Sign Language Data. CVPR, 2026.

[5] Narasimhaswamy et al. HanDiffuser: Text-to-Image Generation With Realistic Hand Appearances. CVPR, 2024.

AuthorsOpenReview ↗

Response to Reviewer 9jF3 (Part 2 / 3)

Q3: PoseFix is a pose-editing method. How it is converted to a generation task, including the choice of the fixed source pose, should be described.

A3: Appendix D.1 (line 1393) of the original paper describes the PoseFix adaptation: we fix the source pose and replace editing instructions with handshape descriptions. Specifically, the source uses all-zero MANO pose parameters during both training and inference.


Q4: UniPose's LLM backbone is replaced with DistilBERT, which is far weaker than the original (LlaVA-1.6V). In contrast, ChatPose keeps LLaVA-v1.5-13B. The reason for this inconsistency should be justified, and the fine-tuning strategy for ChatPose (full fine-tuning or LoRA) should be specified.

A4: We thank the reviewer for raising this concern. It is important to clarify the role of the text encoder for a fair and transparent comparison. We additionally evaluated UniPose with its native LLaVA-1.6-Mistral-7B backbone and ChatPose with frozen DistilBERT. We also combined both LLaVA encoders with our VAE, AHand20 decoder, and learned MANO projector.

R4-Table2: Text encoder comparison for UniPose, ChatPose, and Ours. Position errors are in millimeters, rotation error is in degrees, and R@K is in percent.

MethodText encoderMPJPE ↓MPVPE ↓Rotation ↓R@1 ↑R@5 ↑FID ↓
UniPoseDistilBERT23.1523.1522.3822.3830.8530.8510.0910.0918.9718.97189.68189.68
UniPoseLLaVA-1.6-Mistral-7B10.0910.099.879.8714.1814.1844.3344.3377.0577.0568.27‾\underline{68.27}
ChatPoseDistilBERT19.4119.4119.0819.0831.8631.868.638.6325.1625.16185.67185.67
ChatPoseLLaVA-1.5-13B10.5510.5510.5810.5829.9629.9647.76\mathbf{47.76}85.5485.5498.8398.83
OursDistilBERT10.1910.199.909.9012.9012.9042.0342.0381.5481.5471.3471.34
OursLLaVA-1.6-Mistral-7B8.18\mathbf{8.18}7.99\mathbf{7.99}11.18\mathbf{11.18}47.76\mathbf{47.76}87.83\mathbf{87.83}58.91\mathbf{58.91}
OursLLaVA-1.5-13B9.06‾\underline{9.06}8.85‾\underline{8.85}11.47‾\underline{11.47}46.56‾\underline{46.56}85.83‾\underline{85.83}71.4471.44

Both UniPose and ChatPose perform better with their LLaVA backbones than with frozen DistilBERT. Our framework also benefits from these stronger text encoders, improving all three geometric metrics over the original DistilBERT-based model. These results suggest that both large text encoders and AHand20 benefit text-conditioned pose generation. Larger text encoders require more computational resources for training, while AHand20 adds little computational overhead.

ChatPose uses LoRA rather than full fine-tuning: rank 128, alpha 256, and dropout 0.05, together with a trainable 15-joint output head. LoRA targets the attention projections and MLP projections. We train for five epochs at learning rate 4×10−54\times10^{-5} and select the checkpoint using validation data.

AuthorsOpenReview ↗

Response to Reviewer 9jF3 (Part 3 / 3)

Q5: AHand20 is applied only to the PoseScript-style model. Applying the same 20D output space to the other baselines would show whether the gain generalizes across architectures.

A5: We thank the reviewer for suggesting this experiment to test whether AHand20's benefits extend beyond the PoseScript-style generator. We selected DPoser [1], MAA [2], PoseD-Flow [3], and T2P [4] to cover diffusion, flow matching, and autoregressive generation. We integrated AHand20 outputs, our learned MANO projector, and geometric losses into each method.

R4-Table3: Geometric errors before and after adding AHand20 with learned MANO projection and geometric supervision.

ArchitectureOriginal MPJPE ↓ (mm)+ AHand20 MPJPE ↓ (mm)Original rotation ↓ (°)+ AHand20 rotation ↓ (°)
DPoser13.6313.6313.61\mathbf{13.61}17.3617.3616.95\mathbf{16.95}
PoseD-Flow15.0815.0813.11\mathbf{13.11}19.8619.8616.31\mathbf{16.31}
T2P16.5516.5514.84\mathbf{14.84}21.0921.0919.53\mathbf{19.53}
MAA13.2013.209.97\mathbf{9.97}17.8617.8612.89\mathbf{12.89}

These methods benefit from our AHand20-based framework, suggesting that the gains generalize across architectures. The experiments are now included in Sec. 5.3 of the revised manuscript.

References

[1] Lu et al. DPoser-X: Diffusion Model as Robust 3D Whole-body Human Pose Prior. ICCV, 2025.

[2] Azadi et al. Make-An-Animation: Large-Scale Text-conditional 3D Human Motion Generation. ICCV, 2023.

[3] Nadar, Foti, and Birdal. PoseD-Flow: Versatile and Guided Flow Matching Model of Human Pose. CVPR, 2026.

[4] Bonnet et al. From Text to Pose to Image: Improving Diffusion Model Control and Quality. arXiv:2411.12872, 2024.


Q6: How are the 20D targets annotated? Sec. 4.1 and Appendix B.1 only state that the targets are "manually specified" under the guidance of 2D source images. Since all reference poses, augmented variants, and test ground truths derive from these targets, please clarify: how the 20 continuous angles are determined for each handshape (e.g., discrete levels, per-state templates, or iterative fit-and-render tuning); the number of annotators, their expertise, and whether inter-annotator consistency was assessed.

A6: The annotation was carried out collaboratively by the authors, who have research experience in 3D hand modeling and pose analysis. We used iterative fit-and-render tuning to determine the 20D targets. For each reference image, we adjusted the anatomical angles, fitted the corresponding MANO pose, and compared the rendered handshape with the reference. This process is now described in Appendix B.1 of the revised manuscript (p. 20).


Q7: What are the limitations and future directions? Please include and discuss limitations of the proposed work. For example, extending the concept of AHand20 to the full human body would likewise require additional joint-specific anatomical definitions. This may limit the scope and applicability of the method in more general settings.

A7: AHand20 currently focuses on hand articulation. Extending this approach to full-body models such as SMPL-X [5] could enable interpretable control of whole-body poses. This would require additional joint-specific anatomical definitions. Another direction is to incorporate AHand20 as an anatomical prior into hand pose estimation to reduce anatomically implausible predictions and improve 3D reconstruction accuracy. We have added this discussion in Sec. 6, "Limitations and Discussion," of the revised manuscript (p. 10).

References

[5] Pavlakos et al. Expressive Body Capture: 3D Hands, Face, and Body from a Single Image. CVPR, 2019.


We are grateful for the opportunity to clarify our contribution and strengthen the experimental comparisons. If these additions address your concerns, we hope you will consider updating your rating. Thank you for your time and feedback.

Reviewer_AWDbOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces AHand20, an anatomy-informed 20-dimensional representation for 3D articulated hand poses, to improve the interpretability and anatomical controllability of the widely used 45D MANO representation. By explicitly modeling joint-specific degrees of freedom, including finger flexion and spread, AHand20 enables anatomically informed pose manipulation, conversion to and from MANO, and detection of implausible poses. Building on this representation, the authors construct a text-annotated dataset comprising about 15K pose variations, 30K descriptions, and 154 hand shapes, with many-to-many text–pose associations. They further propose a VAE-based text-conditioned pose generation method evaluated on a hand-shape-disjoint split. Experiments show competitive performance, achieving the best results on geometric error metrics and FID, and the second-best retrieval performance.

Critical strengths
  • The paper addresses an important challenge in hand pose representation by explicitly accounting for hand anatomy and biomechanical constraints, which are critical for generating plausible hand poses in recent hand modeling and interaction applications.
  • The paper moves beyond generic dimensionality reduction by designing a 20D representation aligned with hand biomechanics. Specifically, it models each finger as a kinematic chain with distinct DOFs: 2 DOFs for MCP/CMC (flexion/spread) and 1 DOF each for PIP/DIP/IP (flexion). By defining these angles relative to a palm reference frame, the representation directly encodes intuitive anatomical movements preventing implausible poses that may appear in unconstrained MANO parameterizations.
  • The authors provide a fitting framework to convert between their proposed representation and MANO. The loss function used for fitting explicitly accounts for biomechanical constraints, ensuring that the 20D targets map to physically valid 3D meshes.
  • Building on their AHand20 representation, the authors introduce a high-quality dataset with 30K fine-grained descriptions across 154 hand shapes.
  • By using a many-to-many structure, they effectively decouple hand shape from pose articulation. Their proposed VAE-based generator leverages this data to achieve sota results, particularly in rotation error, outperforming existings methods that rely directly on MANO parameters. Beyond the dataset, AHand20 provides practical tools for the community:
  1. Implausible pose detection by filtering anatomically invalid poses in existing datasets.
  2. Controlled augmentation for semantically meaningful data augmentation (e.g., varying flexion while preserving shape) with guaranteed anatomical correctness.
  3. A low-error bidirectional conversion between AHand20 and MANO, ensuring compatibility with existing pipelines.
Critical weaknesses
  • Given that anatomical correctness is the paper’s main selling point, it is surprising that there are no quantitative metrics to prove it. The experiments rely heavily on geometric errors (MPJPE, rotation error) but lack specific measures for joint limit violations or physiological plausibility.
  • The related work overlooks relevant hand datasets with fine-grained text descriptions, including HandX [1], BOTH57M [3], HandMDM [2]. It also lacks discussion of biomechanically informed methods such as MS-MANO [4] and AG-MAE [5]. The paper should better explain how AHand20 differs from these works.
  • Given the diversity of textual descriptions and the many-to-many text–pose associations, the model should demonstrate its ability to generate diverse poses consistent with the same description, rather than defaulting to safe or average poses. However, no diversity or multimodality metrics are reported.
  • Although AHand20 is the central contribution of the paper, its impact is not sufficiently isolated or demonstrated experimentally. The evaluation does not clearly establish how much the proposed representation itself improves anatomical validity, controllability, or generation quality compared with MANO or alternative representations.
  • The dataset is relatively small compared with modern hand generation benchmarks. Moreover, the scaling experiments show diminishing returns: increasing pose variations from 10 to 25 per hand shape yields a larger improvement than increasing them from 25 to 100, with a similar trend for textual descriptions (10 to 40 versus 40 to 200). This raises questions about the effective diversity of the dataset and whether adding more variations provides sufficient new information. [1] Zhang, Zimu, et al. "HandX: Scaling Bimanual Motion and Interaction Generation." CVPR 2026 [2] Bensabath, Léore, Mathis Petrovich, and Gül Varol. "Text-driven 3d hand motion generation from sign language data." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. CVPR 2026. [3] Zhang, Wenqian, et al. "Both2hands: Inferring 3d hands from both text prompts and body dynamics." 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2024. [4] Xie, P., Xu, W., Tang, T., Yu, Z., & Lu, C. (2024, June). Ms-mano: Enabling hand pose tracking with biomechanical constraints. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 2382-2392). IEEE. [5] Ikne, O., Allaert, B., & Wannous, H. (2025, March). AG-MAE: Anatomically Guided Spatio-Temporal Masked Auto-Encoder for Online Hand Gesture Recognition. In 2025 International Conference on 3D Vision (3DV) (pp. 1406-1415). IEEE.
Questions for authors
  • Why did the authors choose a VAE-based architecture over alternative generative approaches, such as diffusion models, autoregressive models, or flow matching? Would these alternatives better capture the multimodality and diversity of text-conditioned hand poses?
  • Could the authors compare the proposed method against a baseline that directly predicts the 20D AHand20 representation without the MANO projector and its associated losses? This would help isolate the contribution of the projection and clarify whether the improvements come primarily from the representation or the training framework.
  • While AHand20 explicitly models finger flexion and spread, does it also prevent interpenetration between fingers or between fingers and the palm? If not, how frequently do such cases occur, and how is this limitation addressed?
  • ChatPose appears to be the closest baseline to the proposed method. Since both methods use different text encoders, how much of the performance gap can be attributed to the proposed representation and generation method rather than the text encoder?
7 comments
AuthorsOpenReview ↗

Response to Reviewer AWDb (Part 1 / 4)

We appreciate the time you devoted to this review and the detailed suggestions for strengthening the paper. Your comments highlight several aspects central to our work: anatomical interpretability, compatibility with MANO, fine-grained descriptions with many-to-many pose–text associations, and practical tools for controlled augmentation and pose inspection.

Your comments raise important questions about anatomical validity, generation diversity, and the contribution of AHand20 across different generation settings. To address these concerns, we have conducted additional experiments and expanded the relevant discussions in the revised manuscript.


Q1: Given that anatomical correctness is the paper's main selling point, it is surprising that there are no quantitative metrics to prove it. The experiments rely heavily on geometric errors (MPJPE, rotation error) but lack specific measures for joint limit violations or physiological plausibility.

A1: The original paper already provides quantitative measures for assessing joint-limit violations and physiological plausibility in Appendix D.3 (Appendix D.5 in the revised manuscript). These measures depend on the reference ranges and angle definitions. They quantify angular violations, deviations from hinge planes, and thumb-axis residuals. Violations are measured in degrees below the lower bound or above the upper bound, with zero violation within the range. When specific ranges are required, bounds can be imposed directly on AHand20's explicit anatomical angles.


Q2: The related work overlooks relevant hand datasets with fine-grained text descriptions, including HandX [1], BOTH57M [3], HandMDM [2]. It also lacks discussion of biomechanically informed methods such as MS-MANO [4] and AG-MAE [5]. The paper should better explain how AHand20 differs from these works.

A2: The cited works address related topics, but their focus differs from hand-pose representation and generation. HandX and BOTH2Hands/BOTH57M are already cited in Appendix B (line 902) and Sec. 2.2 (line 107) of the original paper, respectively. Both focus on hand motion sequences, not individual hand poses. HandMDM [1] also generates hand and body motion sequences. MS-MANO [2] integrates a musculoskeletal system with MANO for simulation-based pose refinement, rather than defining a compact anatomical pose representation. Like the biomechanical priors discussed in Sec. 2.1, AG-MAE [3] uses anatomical knowledge as training constraints to guide feature learning for gesture recognition. AHand20 instead defines interpretable pose parameters for control and generation.

Response references

[1] Bensabath, Petrovich, and Varol. Text-Driven 3D Hand Motion Generation from Sign Language Data. CVPR, 2026.

[2] Xie et al. MS-MANO: Enabling Hand Pose Tracking with Biomechanical Constraints. CVPR, 2024.

[3] Ikne, Allaert, and Wannous. AG-MAE: Anatomically Guided Spatio-Temporal Masked Auto-Encoder for Online Hand Gesture Recognition. 3DV, 2025.


Q3: Given the diversity of textual descriptions and the many-to-many text–pose associations, the model should demonstrate its ability to generate diverse poses consistent with the same description, rather than defaulting to safe or average poses. However, no diversity or multimodality metrics are reported.

A3: We additionally measured Diversity across descriptions and Multimodality within each description in the evaluator's pose feature space.

R3-Table1: Diversity and Multimodality in the evaluator's pose feature space. Bold and underlined values are the closest and second-closest to the reference, respectively.

MethodDiversityMultimodality
PoseScript15.18\mathbf{15.18}5.21‾\underline{5.21}
PoseFix9.499.497.227.22
UniPose7.807.80—
ChatPose12.6412.64—
Ours15.11‾\underline{15.11}2.20\mathbf{2.20}
Reference variations15.1715.173.573.57

Multimodality is not reported for ChatPose and UniPose because they use deterministic pose decoding in our benchmark. Ours performs competitively on both metrics, achieving the second-closest Diversity and the closest Multimodality to the reference among the compared methods.

AuthorsOpenReview ↗

Response to Reviewer AWDb (Part 2 / 4)

Q4: Although AHand20 is the central contribution of the paper, its impact is not sufficiently isolated or demonstrated experimentally. The evaluation does not clearly establish how much the proposed representation itself improves anatomical validity, controllability, or generation quality compared with MANO or alternative representations. Could the authors compare the proposed method against a baseline that directly predicts the 20D AHand20 representation without the MANO projector and its associated losses? This would help isolate the contribution of the projection and clarify whether the improvements come primarily from the representation or the training framework.

A4: We thank the reviewer for suggesting this comparison, which helps distinguish the contribution of AHand20 from that of the learned MANO projector and its associated supervision. We additionally trained a model using only 20D reconstruction and KL losses, without a learned MANO projector or MANO-space supervision. Its predictions are converted to MANO for evaluation.

R3-Table2: Geometric accuracy of baseline methods, AHand20 trained with 20D-only supervision and conversion, and the full model. Position errors are in millimeters and rotation error is in degrees.

MethodMPJPE ↓MPVPE ↓Rotation ↓
PoseScript12.0412.0411.6711.6715.4515.45
PoseFix22.9622.9622.3322.3328.8528.85
UniPose23.1523.1522.3822.3830.8530.85
ChatPose10.55‾\underline{10.55}10.58‾\underline{10.58}29.9629.96
AHand20 (20D-only)11.9411.9411.6011.6014.19‾\underline{14.19}
AHand20 (Full)10.19\mathbf{10.19}9.90\mathbf{9.90}12.90\mathbf{12.90}

Even without the learned projector, AHand20 achieves competitive geometric accuracy, ranking third in positional errors and second in rotation error among the compared configurations. These results support the effectiveness of the compact anatomical representation. The full model further improves all three geometric metrics.


Q5: The dataset is relatively small compared with modern hand generation benchmarks. Moreover, the scaling experiments show diminishing returns: increasing pose variations from 10 to 25 per hand shape yields a larger improvement than increasing them from 25 to 100, with a similar trend for textual descriptions (10 to 40 versus 40 to 200). This raises questions about the effective diversity of the dataset and whether adding more variations provides sufficient new information.

A5: Although our dataset is relatively small, we have not identified an openly available dataset that closely matches our setting of individual 3D hand poses paired with fine-grained descriptions. We would be happy to evaluate on a suitable dataset suggested by the reviewer. Constructing such a dataset at scale is challenging, as it requires clearly distinguishable handshapes paired with accurate, fine-grained textual descriptions. We are considering how to expand both the handshape inventory and the pose–text pairs to provide the research community with a larger, more diverse dataset and benchmark.

AuthorsOpenReview ↗

Response to Reviewer AWDb (Part 3 / 4)

Q6: Why did the authors choose a VAE-based architecture over alternative generative approaches, such as diffusion models, autoregressive models, or flow matching? Would these alternatives better capture the multimodality and diversity of text-conditioned hand poses?

A6: We thank the reviewer for suggesting a broader evaluation across generative architectures. We additionally evaluated DPoser-hand [1] and Make-An-Animation (MAA) [2] for diffusion, PoseD-Flow [3] for flow matching, and T2P [4] for autoregressive generation. We retained their architectures and used AHand20 outputs with our learned MANO projector and geometric losses.

R3-Table3: Generation with and without AHand20. Bold indicates better results within each architecture. Diversity and Multimodality are compared by closeness to the reference. MPJPE is in millimeters and rotation error is in degrees.

ArchitectureMPJPE ↓Rotation ↓DiversityMultimodality
VAE12.0412.0415.4515.4515.18\mathbf{15.18}5.215.21
VAE + AHand2010.19\mathbf{10.19}12.90\mathbf{12.90}15.1115.112.20\mathbf{2.20}
DPoser13.6313.6317.3617.3615.3315.337.83\mathbf{7.83}
DPoser + AHand2013.61\mathbf{13.61}16.95\mathbf{16.95}15.30\mathbf{15.30}7.83\mathbf{7.83}
MAA13.2013.2017.8617.8615.3215.329.359.35
MAA + AHand209.97\mathbf{9.97}12.89\mathbf{12.89}15.06\mathbf{15.06}3.71\mathbf{3.71}
PoseD-Flow15.0815.0819.8619.8615.4815.486.626.62
PoseD-Flow + AHand2013.11\mathbf{13.11}16.31\mathbf{16.31}15.06\mathbf{15.06}2.62\mathbf{2.62}
T2P16.5516.5521.0921.0915.3215.3210.2710.27
T2P + AHand2014.84\mathbf{14.84}19.53\mathbf{19.53}15.26\mathbf{15.26}5.43\mathbf{5.43}
Reference poses——15.1715.173.573.57

Adding AHand20 improves geometric accuracy across all evaluated architectures and brings Diversity and Multimodality closer to the reference. Alternative architectures can also improve results: MAA + AHand20 achieves lower geometric errors and a Multimodality score closer to the reference than VAE + AHand20. These results support the effectiveness of AHand20 and its learned MANO interface across different generative architectures. The comparisons are included in Sec. 5.3 of the revised manuscript.

References

[1] Lu et al. DPoser-X: Diffusion Model as Robust 3D Whole-body Human Pose Prior. ICCV, 2025.

[2] Azadi et al. Make-An-Animation: Large-Scale Text-conditional 3D Human Motion Generation. ICCV, 2023.

[3] Nadar, Foti, and Birdal. PoseD-Flow: Versatile and Guided Flow Matching Model of Human Pose. CVPR, 2026.

[4] Bonnet et al. From Text to Pose to Image: Improving Diffusion Model Control and Quality. arXiv:2411.12872, 2024.

Reviewer_BkfxOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

In this paper, the authors propose an anatomy-informed 20D hand pose representation. Based on this representation, they introduce a conversion pipeline between AHand20 and MANO, construct a pose-text dataset, and develop a text-conditioned hand pose generation model.

Critical strengths
  1. The idea and definition of AHand20’s coordinate system are sound and intuitive. They align with everyday observations of how human hands move, and the reconstruction results show that this definition can naturally reconstruct 3D hand poses.

  2. The paper presents a complete system design. The AHand20 representation is not an isolated component; it includes a set of conversion rules to and from the widely used MANO representation, making it easier for users to adopt.

  3. The authors create a new dataset that could benefit future research in this area.

Critical weaknesses
  1. Most results are evaluated on a dataset generated using the AHand20 representation, although there is no data leakage according to author’s setting, the evaluation may still favor the proposed representation.

  2. Although the proposed method achieves the best results on the geometric error metrics, the authors provide limited discussion of its retrieval performance.

Questions for authors
  1. Have the authors evaluated their representation on another dataset that was not generated using AHand20? Since the authors have already developed conversions between AHand20 and MANO, such an evaluation should be feasible.

  2. Could the authors discuss the retrieval performance in more detail? Could the lower retrieval performance of the proposed method indicate that the generated poses for different handshapes are less distinguishable from one another?

3 comments
AuthorsOpenReview ↗

Response to Reviewer Bkfx

We sincerely thank the reviewer for the thoughtful comments and positive assessment. Your recognition of AHand20's intuitive anatomical design and its ability to reconstruct natural hand poses is encouraging. We also thank you for recognizing the practical value of MANO conversion and the dataset's potential to support future research.

Your comments raise important questions about evaluation beyond our constructed dataset and whether generated handshapes can be clearly distinguished. To address these questions, we have added generation examples using external MANO poses and expanded the retrieval analysis with language-backbone comparisons and qualitative examples in the revised manuscript.


Q1: Most results are evaluated on a dataset generated using the AHand20 representation, although there is no data leakage according to author's setting, the evaluation may still favor the proposed representation. Have the authors evaluated their representation on another dataset that was not generated using AHand20? Since the authors have already developed conversions between AHand20 and MANO, such an evaluation should be feasible.

A1: We examine generation without additional training using hand poses with different finger states from FreiHAND [1] and DexYCB [2] as external references. Ours achieves lower MPJPE than PoseScript and more closely reproduces the reference finger states, although some fine details are not fully reproduced. Text descriptions may leave aspects of a hand pose unspecified, and the remaining differences mainly concern fine details of finger articulation and spatial relations. These examples are presented in Appendix D.2 of the revised manuscript (Figure 16, p. 31).

References

[1] Zimmermann et al. FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Images. ICCV, 2019.

[2] Chao et al. DexYCB: A Benchmark for Capturing Hand Grasping of Objects. CVPR, 2021.


Q2: Although the proposed method achieves the best results on the geometric error metrics, the authors provide limited discussion of its retrieval performance. Could the authors discuss the retrieval performance in more detail? Could the lower retrieval performance of the proposed method indicate that the generated poses for different handshapes are less distinguishable from one another?

A2: The original paper (Sec. 5.1, line 405) already notes that ChatPose's larger language backbone may contribute to stronger text–pose alignment. We further evaluated Ours with the same LLaVA-1.5-13B backbone used by ChatPose.

R2-Table1: Generation and retrieval results with different language backbones. Position errors are in millimeters, rotation error is in degrees, and R@K is in percent.

MethodText encoderMPJPE ↓MPVPE ↓Rotation ↓R@1 ↑R@5 ↑FID ↓
ChatPoseLLaVA-1.5-13B10.5510.5510.5810.5829.9629.9647.76\mathbf{47.76}85.54‾\underline{85.54}98.8398.83
OursDistilBERT10.19‾\underline{10.19}9.90‾\underline{9.90}12.90‾\underline{12.90}42.0342.0381.5481.5471.34\mathbf{71.34}
OursLLaVA-1.5-13B9.06\mathbf{9.06}8.85\mathbf{8.85}11.47\mathbf{11.47}46.56‾\underline{46.56}85.83\mathbf{85.83}71.44‾\underline{71.44}

With LLaVA-1.5-13B, Ours achieves retrieval recall comparable to ChatPose, supporting the role of the language backbone in the retrieval gap.

Appendix D.3 of the revised manuscript (Figure 18, p. 33) shows retrieval examples under the three settings. In cases where ChatPose retrieves the correct handshape, Ours retrieves a handshape with a similar overall pose but confuses the involved finger or its joint states, failing to fully match the input description. Using LLaVA-1.5-13B corrects some of these errors. These cases suggest that strong text understanding is also important for text-conditioned pose generation.


We hope our response has addressed your concerns. If there are any additional concerns or comments that we may have missed, we would be grateful for your further feedback to help us improve our work. Thank you again for your time and thoughtful comments.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer,

Thank you again for your time and effort in reviewing our paper.

As the discussion period is nearing its end, we would be grateful if you could review our rebuttal and revised manuscript and let us know whether they address your concerns. If there are any remaining questions or points that need further clarification, we would be happy to address them.

We appreciate your feedback and the opportunity to improve our work.

Best regards,
AHand20 Authors

AuthorsOpenReview ↗

Follow-up Before Discussion Closes

Dear Reviewer,

We sincerely appreciate the time you have devoted to reviewing our work. As the discussion period closes in a few hours, we would be grateful to receive your feedback on our rebuttal and revised manuscript.

Best regards,
AHand20 Authors

Reviewer_oqw9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces AHand20, a 20D hand pose representation with joint-specific anatomical angles and conversion to and from MANO. The authors use the representation for pose augmentation to construct a pose-text dataset spanning 154 handshapes and train a text-conditioned pose generator. Experiments report lower joint, vertex, and rotation errors on unseen handshapes than the compared methods, alongside evaluations of joint-motion coverage, MANO conversion, and identification of implausible poses.

Critical strengths

Interpretable articulation controls and structured pose augmentation. The joint-specific definitions provide explicit flexion and spread controls, with a separate treatment of thumb motion. The augmentation pipeline checks articulation, contact, separation, and self-penetration to preserve handshape identity, making the representation useful for controlled pose construction. The ablations demonstrate improved generation accuracy. With enriched text held fixed, AHand20 reduces MPJPE from 12.04 to 10.19 mm and rotation error from 15.45 to 12.90 degrees in Table 2(a). These results substantiate its practical benefit on the constructed benchmark. The appendix details data augmentation and generator training.

Critical weaknesses

Limited validation of conversion fidelity. Table 3(b) evaluates round-trip conversion only on the 154 reference poses constructed through AHand20 itself. Full conversion errors on independently obtained MANO poses are not reported, leaving pose preservation beyond the constructed reference set unvalidated. Insufficient specification of the evaluation model. Appendix D.1 does not specify the architecture, training procedure, or data splits of the evaluator used for R@K and FID. This limits reproducibility and leaves its exposure to test handshapes unclear.

Questions for authors

Conversion fidelity. What evidence supports preservation of articulation and mesh geometry when independently obtained MANO poses undergo a full round trip through AHand20? Evaluator setup. What architecture, training procedure, and data splits were used for the R@K and FID evaluator? Did its training include test handshapes?

5 comments
AuthorsOpenReview ↗

Response to Reviewer oqw9 (Part 1 / 2)

We really thank the reviewer for the careful evaluation and constructive feedback. We appreciate your recognition of AHand20's interpretable articulation controls, handshape-preserving augmentation, and practical benefits demonstrated by the ablation results.

We agree that validating conversion fidelity on independent poses and clearly specifying the evaluator are important. To address these concerns, we have conducted additional experiments on independent MANO poses and expanded the evaluator description in the revised manuscript.


Q1: Limited validation of conversion fidelity. Table 3(b) evaluates round-trip conversion only on the 154 reference poses constructed through AHand20 itself. Full conversion errors on independently obtained MANO poses are not reported, leaving pose preservation beyond the constructed reference set unvalidated. What evidence supports preservation of articulation and mesh geometry when independently obtained MANO poses undergo a full round trip through AHand20?

A1: We evaluated conversion fidelity on independently obtained MANO poses, performing full MANO → AHand20 → MANO round trips on 2,000 randomly sampled poses from each of FreiHAND and DexYCB.

R1-Table1: MANO → AHand20 → MANO conversion errors. Self-constructed values are from Table 3(b) in the original paper. Mean and 95th-percentile (P95) errors are reported.

DatasetStatistic20D error (°)MPJPE (mm)MPVPE (mm)Rotation (°)
Self-constructedMean0.2950.2950.5180.5180.4940.4940.6170.617
Self-constructedP950.6380.6381.0841.0841.0291.0291.4421.442
FreiHANDMean0.5850.5853.2973.2973.8593.8597.6697.669
FreiHANDP951.6691.6694.9034.9035.3475.34711.74711.747
DexYCBMean0.4970.4972.8422.8422.9812.9814.9344.934
DexYCBP951.3701.3704.3234.3234.4474.4477.0717.071

The results show small anatomical-angle and geometric errors, although errors are higher than on our constructed dataset. It should be noted that independent MANO annotations may contain anatomically implausible rotations, contributing to these differences. Round-trip errors therefore reflect both AHand20's joint-motion constraints and conversion accuracy. For example, converting FreiHAND poses through AHand20 reduces mean PIP/DIP out-of-plane deviations from 9.81°/11.13° to 0.006°/0.035° (Appendix A.5 of the revised manuscript, p. 19, line 973).

To further assess preservation of articulation and mesh geometry, we visualize four independently obtained MANO poses with their AHand20 reconstructions. The reconstructions show almost no visible differences from the original MANO meshes, with MPJPE of 2.636–4.902 mm and MPVPE of 2.687–5.066 mm across these four examples. These visualizations are included in Appendix A.5 of the revised manuscript (Figure 7, p. 19).

AuthorsOpenReview ↗

Response to Reviewer oqw9 (Part 2 / 2)

Q2: Insufficient specification of the evaluation model. Appendix D.1 does not specify the architecture, training procedure, or data splits of the evaluator used for R@K and FID. This limits reproducibility and leaves its exposure to test handshapes unclear. What architecture, training procedure, and data splits were used for the R@K and FID evaluator? Did its training include test handshapes?

A2: In the original submission, evaluator training included the generator's test handshapes. Training a separate evaluator on the full dataset, including the generator's held-out data, has been used for retrieval and FID evaluation in previous motion-generation work [1]. The evaluator does not participate in generator training or inference.

To address the reviewer's concern, we retrained our evaluator using only the generator's training handshapes. Each included handshape contributes 80/10/10 poses and 120/40/40 descriptions to the retrained evaluator's training/validation/test splits, respectively. The results are shown below.

R1-Table2: Retrieval and FID with an evaluator trained only on the generator's training handshapes. All generation methods use DistilBERT as their text encoder.

MethodR@1 ↑R@5 ↑FID ↓
PoseScript38.49‾\underline{38.49}75.14‾\underline{75.14}51.17‾\underline{51.17}
PoseFix3.643.6419.1819.18155.17155.17
UniPose8.008.0021.8421.84165.58165.58
ChatPose10.3210.3237.6037.60174.89174.89
Ours44.57\mathbf{44.57}80.56\mathbf{80.56}43.42\mathbf{43.42}

With DistilBERT used by all five generation methods, Ours achieves the highest R@1 and R@5 and the lowest FID under both evaluator training splits. These results support our comparative conclusions across evaluator training splits.

Retrieval recall and FID use a separate learned pose–text evaluation space [2]. Two MLP encoders map 90D MANO rotation features through 90→512→512→128 and frozen DistilBERT features through 768→512→128. Training uses a symmetric contrastive loss with same-handshape positives and AdamW with a learning rate of 3 × 10⁻⁴ for 4,000 updates. The evaluator setup is now specified in Appendix D.1 of the revised manuscript.

References

[1] Rempe et al. Kimodo: Scaling Controllable Human Motion Generation. 2026, Sec. 6.1.

[2] Guo et al. Generating Diverse and Natural 3D Human Motions From Text. CVPR, 2022.


We hope the additional experiments and clarifications address your concerns. We would be grateful if you could reconsider your rating in light of this new evidence. Thank you once again for your time and effort.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer,

Thank you again for your time and effort in reviewing our paper.

As the discussion period is nearing its end, we would be grateful if you could review our rebuttal and revised manuscript and let us know whether they address your concerns. If there are any remaining questions or points that need further clarification, we would be happy to address them.

We appreciate your feedback and the opportunity to improve our work.

Best regards,
AHand20 Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

For clarity, we refer to Reviewers oqw9, Bkfx, AWDb, and 9jF3 as R1, R2, R3, and R4, respectively.

We sincerely thank all reviewers for their thoughtful and constructive feedback. We are encouraged by their recognition of the key contributions and strengths of our work.

We have conducted additional experiments and expanded the relevant discussions in the revised manuscript. Below, we summarize our core contributions and the main updates made during the rebuttal period.

Core Contributions of Our Work

  • Interpretable hand pose representation. AHand20 defines joint-specific anatomical angles for finger flexion and spread, providing direct controls for hand articulation within a compact 20D representation.
  • MANO compatibility and practical applications. Conversion between AHand20 and MANO supports controlled pose augmentation and inspection of potentially implausible poses while retaining compatibility with existing hand-mesh pipelines.
  • Pose–text data and generation. We construct a dataset spanning 154 handshapes, 15,400 poses, and 30,800 descriptions, and evaluate AHand20 through text-conditioned hand pose generation.

Updates to Experimental Results

  • Evaluation on independent MANO poses — Appendices A.5 and D.2 (R1, R2). We added round-trip conversion experiments on 2,000 independently obtained poses from each of FreiHAND and DexYCB, reconstruction visualizations, and external-reference generation examples comparing PoseScript and Ours.
  • Comparisons across language backbones and generator architectures — Table 1 and Section 5.3 (R2–R4). We added comparisons under matched language backbones and evaluated diffusion, flow-matching, and autoregressive architectures with and without our AHand20-based framework. These experiments demonstrate its benefits beyond the original language backbone and VAE-based generator.
  • Additional evaluation and ablations — Appendix D.4 and individual responses (R1, R3, R4). We added evaluator training-split controls, direct 20D prediction without the learned MANO projector, hand-focused diffusion comparisons, and Diversity and Multimodality measurements.

Updates to Explanations and Discussions

  • Evaluator details — Appendix D.1 (R1). We specify the evaluator architecture, training procedure, and data splits.
  • Related work — Section 2, Appendix B, and individual responses (R3, R4). We expand comparisons with related hand-motion datasets and biomechanically informed methods, clarifying their differences from our work.
  • Retrieval analysis — Appendix D.3 (R2). We analyze retrieval errors through examples and examine how a stronger language backbone affects text–pose matching.
  • Limitations and future directions — Section 6 (R4). We discuss extending anatomical pose controls to full-body models and using AHand20 as an anatomical prior for hand pose estimation.

Revisions in the manuscript are highlighted in blue for ease of reference. We hope these additions and clarifications address the reviewers’ concerns and help the Area Chair assess our work. Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.