acceptodds
Under review as a conference paper at ICLR 2027

BranchTokens: Branch-Centric Tokenization and Test-Time Augmentation for Skeleton Generation

Abstract

Automatic skeleton generation involves predicting both joint positions and skeletal connectivity. However, existing approaches struggle to encode branch structures into token sequences and do not use test-time computation effectively. We address these choices within BranchTokens, a unified autoregressive framework. First, we introduce branch-centric tokenization, a branch-aware representation that places structurally related elements next to each other and encodes connectivity directly in the sequence. Compared with standard BFS-style serialization, this representation yields more compact sequences. Second, we introduce view-augmented generation, a test-time augmentation procedure that applies axis-aligned rotations to the input mesh, maps all predictions back to a common frame, and selects the final skeleton based on mesh coverage and consistency among predictions from different views. Experiments show that our method achieves better skeleton prediction accuracy than state-of-the-art methods. In particular, our method reduces the CD-J2B error by 16.9% on the Articulation-XL2.0 dataset compared to the strongest baseline, Auto-Connect. Qualitative results on in-the-wild meshes further demonstrate generalization across diverse inputs.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
automatic rigging

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_Xh44OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

BranchTokens studies skeleton serialization and inference-time candidate selection for mesh-conditioned autoregressive skeleton generation. Its branch-centric tokenizer applies BFS to a series-reduced tree, restores degree-two nodes within contiguous branches, and adds explicit branch delimiters; siblings are ordered by distance. At inference, it decodes six axis-aligned rotations, maps predictions back, and selects one skeleton using mesh coverage and joint-set consensus. On Articulation-XL2.0, CD-J2B is 1.687, versus 2.030 for the reported Auto-Connect baseline and 1.770 without TTA. Evaluation includes ModelsResource, diverse-pose, structural metrics, and a separately trained Anymate model. The backbone and next-token training objective remain standard.

Critical strengths

1.We presented BranchTokens, a skeleton generation framework built around two design choices:branch-centric tokenization and view-augmented generation. The author has proved the effectiveness of the method through numerous experiments.

2.The theoretical intuition of this paper is excellent, and the design of BCT is highly elegant, effectively introducing graph-theoretic properties into sequence generation. However, there are notable flaws in experimental rigor (specifically, the discrepancy in data size), and the TTA design appears somewhat cumbersome.

Critical weaknesses

1.This is the most critical flaw. The authors must clarify the discrepancy in dataset size (57k vs. 46k) in the rebuttal or revised manuscript to ensure a fair comparison.

2.As shown in Table 10, TTA has a nearly 6-fold increase in computational cost. The authors need to justify whether this significant computational overhead is warranted.

Questions for authors

1.The paper argues that Auto-Connect's BFS traversal "treats all nodes equally" and fails to capture distinct structural roles (junction, leaf, connection). However, Branch-Centric Tokenization (BCT) essentially only reorders the token sequence without altering the underlying model architecture. The authors should explicitly clarify in the Introduction: Why does this seemingly simple sequence reordering yield such significant performance gains?

2.The rationale behind the specific design choices in BCT needs further justification. Specifically, compared to a standard Depth-First Search (DFS) traversal—which also keeps branch nodes contiguous—what are the essential advantages of the proposed concatenation method and the introduction of the boundary token [E3] in preserving topological connectivity? What problems can they solve?

3.Due to the inconsistent orientation of the grids in the dataset, this paper employs six axial rotations for enhancement. However, random rotation augmentation has already been widely used during the training phase (as mentioned in Section 5.1 Implementation). Since the model has already encountered random rotations during training, does fixing the six viewing angles for rotation during testing really compensate for the orientation ambiguity?

4.Equation (6) employs negative exponential decay to calculate the distance from the grid surface points to the predicted bone segments. This measurement method is actually very similar to Chamfer Distance (CD). In this paper, CD is used as the evaluation metric, and a measurement highly correlated with the evaluation metric is adopted as the basis for selecting TTA. Is there any irrationality in this?

5.Table 10 shows that inference time increases from 1.0s without TTA to 5.8s with TTA. This represents a nearly 6-fold increase in computational cost for less than a 10% improvement in CD metrics (e.g., J2J decreases from 2.438 to 2.364). The authors are suggested to discuss the practical application scenarios and efficiency trade-offs of this method in the Introduction or Conclusion.

6.In Section 5, the authors describe an Articulation-XL2.0 dataset containing 57k samples, including the diverse-pose training split. However, the original paper of Auto-Connect reports 46k training samples in its Section 4.1 [1]. Please clarify the discrepancy between the reported sample sizes or add a comparison of absolute alignment with Auto-Connect on the 46k data.

7.The experiments cover Articulation-XL2.0, ModelsResource, the diverse-pose split, and Anymate (in the Appendix). However, the generalization capability in "in-the-wild" real-world scenarios is insufficiently supported, as it relies solely on qualitative demonstrations provided in the Supplementary Website.

Reviewer_3eY1OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents BranchTokens, an autoregressive skeleton generation framework with branch-centric tokenization and test-time view augmentation. The proposed representation give more compact sequences. Experiments show the proposed framework achieves the state-of-the-art performance on skeleton prediction task.

Critical strengths
  1. The paper is well written and presents a clearly motivated approach.
  2. The core observation behind BCT is compelling: simplifying the skeleton tree by eliminating degree‑two intermediate connection nodes. By explicitly building on the intrinsic tree and branch structure of skeletons, BCT enjoys strong, task‑aligned motivation. Rather than proposing a heavy new network backbone, the method introduces focused, lightweight modifications that are straightforward to interpret.
Critical weaknesses
  1. While the proposed method achieves better geometric and structural skeleton metrics, it remains unclear whether these improvements translate to improved skinning quality or more robust mesh deformation during animation. The paper would be strengthened by evaluating the predicted skeletons within a controlled downstream pipeline for skinning and motion retargeting.
  2. The ablation study fails to isolate individual components of BCT. Its tokenization alters branch adjacency, sequence length, degree‑two node handling, and the [E3] delimiter jointly. It remains unclear whether gains stem from branch-aware modeling, shorter sequences, or boundary tokens. Separate ablations are recommended, including comparisons between BFS and branch-centric ordering at matched sequence lengths.
Questions for authors

See Weakness

Reviewer_DEhAOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a new tokenization method for autoregressive skeleton generation from a static mesh, based on Auto-Connect. It first serializes the series-reduced tree and then inserts the intermediate joints, so that joints on the same branch stay together. It also uses distance-aware traversal to make the sequence more consistent under rotations. During inference, the model predicts skeletons from six rotated inputs and selects one based on mesh coverage and agreement between the joint sets. The paper shows improvements on Articulation-XL2.0, ModelsResource, and diverse poses, with additional structural evaluation and an Anymate experiment in the appendix.

Critical strengths
  1. Branch-centric tokenization. The proposed representation makes sense to me. Keeping the joints on one branch together should make the sequence easier to learn. The results in Table 2 show that BCT improves all three distances, and DAT further improves the results. Fig. 4 also shows that the sequences become shorter. I appreciate that these changes are evaluated with the same backbone.

  2. Experimental results. The method without TTA already improves the directly comparable baselines in Table 1, and adding TTA gives further improvements on ModelsResource and diverse poses. The appendix also provides TED and Graph2Vec results, a separately trained model on Anymate, and comparisons with beam search and stochastic sampling. These experiments are useful to understand the method beyond the distance scores on one dataset.

Critical weaknesses
  1. The baseline numbers are a little bit confusing to me. Auto-Connect in Table 1 gives 2.572/2.030/1.682, while the model without BCT and DAT in Table 2 gives 2.543/1.864/1.582. Thus, the baseline in the ablation study already has a considerable improvement in J2B and B2B. I am not sure where this difference comes from. It could be different checkpoints or training settings, but these details are needed to understand the actual contribution of the proposed tokenization. It would be better to clarify which comparison keeps all the other settings the same.

  2. The improvement from TTA needs more comparison w.r.t computation. According to Table 10, TTA increases the inference time from 1.0 s to 5.8 s. In Table 3, the six-candidate beam search already gets 2.377 in J2J, which is close to 2.364 of the proposed method. The bone distances have a larger improvement, which is good. However, using six candidates does not necessarily take the same time for the two methods. I would prefer to see the comparison at similar runtime to understand whether the proposed view augmentation is a better use of the additional computation.

Questions for authors
  1. What causes the difference between Auto-Connect in Table 1 and the model without BCT/DAT in Table 2? Please specify the checkpoints, training, post-training, and decoding settings. Are all the variants in Table 2 trained with the same settings and seeds?

  2. How long does the six-candidate beam search take compared with the six-view augmentation, on the same hardware and with the same timing scope? It would be helpful to compare the existing variants at similar runtime.

  3. Since Auto-Connect is the closest baseline, would it be possible to include its TED and Graph2Vec results in Table 4? If these results are not available, please explain the reason.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.