- Rating
- 3(+2)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
SetMimic learns motion tracking from humanoid references imperfectly retargeted from internet videos, estimating how reliable each part of the reference is and tracking reliable components tightly while letting uncertain ones vary within a tube, with the final reference chosen among sampled candidates through short policy rollouts. It is evaluated on dynamic routines on the AgiBot X2 against BeyondMimic with GMR and NMR references and generalist trackers such as BFMTrack and SONIC, with qualitative hardware results.
Critical strengths
- Learning from noisy video references is a relevant problem, and relaxing tracking where the reference is unreliable is a simple and intuitive idea. The ablations consistently support each component on the physical metrics.
Critical weaknesses
Overall, I don't think the main claims are supported by the current evaluation.
- What if the motion is just more difficult at some points, rather than corrupted? SetMimic seems to loosen tracking in those parts too, since velocity spikes and joint-limit proximity also flag highly dynamic segments. And since Clean MPJPE only covers reliable components, the evaluation can't even tell the two cases apart. This leaves the claim that SetMimic preserves the intended motion while improving physical execution without evidence, as the only fidelity metric excludes exactly the parts the method changes.
- The comparisons are hard to read due to some inconsistent writing. It is not clear which retargeter SetMimic uses (contradicting sentences mention both GMR and NMR in the text), and the generalist trackers were not trained for the X2, with no explanation of how they were adapted. No variances are reported either, and the Clean MPJPE differences are very close to NMR + BeyondMimic, so it's hard to judge which differences in Table 1 are meaningful.
Questions for authors
- Could you report MPJPE on all components, including the most dynamic segments, to show that SetMimic preserves the difficult parts of the motion rather than smoothing them out?
- It is not clear whether SetMimic uses NMR or GMR references in Table 1. Sec. 4.1 says NMR, lines 354 and 413 say GMR, and its Table 1 row is identical to the GMR-based Full SetMimic in Table 2. This matters for the claim that the gains go beyond retargeting. Could you report SetMimic with both?
- Could you provide more details on how SONIC was retrained for the X2 embodiment? On which datasets? SONIC is trained on the Unitree G1, while the paper states that the pretrained models were retained. The same applies to BFMTrack.
- Line 357 states "any embodiment-specific adaptations must be documented before reporting results". Is this a leftover from a previous draft?
- Hardware results are qualitative. The video is not synchronized and doesn't highlight what the reader should look at, so from the current presentation I couldn't clearly tell whether the stumbling could be fixed with better tuning of the baselines. How were the baselines tuned on the X2, and could you report a quantitative hardware measure (e.g., stumbles per motion over repeated trials)?
- SetMimic also ends up committing to a single selected reference, so a constrained retargeter (e.g., KDMR or OmniRetarget) could in principle achieve the same improvements. How does SetMimic compare against such a pipeline?
- Where do the 2-second candidate rollouts start, and how many rollouts are run per candidate?
- Termination conditions for motion tracking pipelines like BeyondMimic are usually restrictive. How do you make sure they are not triggered when relaxing tracking to the tube?