- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SCHA-React to improve online reaction generation by utilizing the information from important historical interactions, while keeping limited context for inference. The methodology first learns VQ-VAE and basic reactive strategies based on flow matching, then refines through historic events that satisfy temporal causal constraints. The work constructs human-verified event annotations on DuoBox, and conducts experiments on Reactive, Two-character, and long-term generation with ablation study. Compared to Ready-to-React, FID improves by 15.5%, 15.8% and 30.1%.
Critical strengths
- Training stage utilizes historic information while keeping clean interface. This methodology refines the model by using selected historic events during training, while keeping inference practical enough to stay away from long history.
- Experimental benefits cover multiple scales including reactive, two-character, and long-term generation with persuasive ablation study. There are large improvements in quantitative experiments, especially reducing FID from 36.76 to 25.69 for long-term generation.
Critical weaknesses
- While the experiments follow the baseline paper Ready-to-React to be conducted only on DuoBox, this setting seems to be area-specific. It remains unclear whether this methodology can work in other interactive scenes.
Questions for authors
- Could you explain how this methodology would work in other scenes?
2 comments
Thank you for your feedback. I will keep my score.
Thank you for asking whether SCHA-React applies beyond boxing. We added experiments on DD100 Cha-cha and Rumba and InterHuman Taekwondo. The results show gains in cooperative dance and conditioned Taekwondo reactions, while identifying a challenging setting for autonomous Taekwondo interaction.
1. Adaptation and experimental settings (App. D.6).
We train models separately for each class using 22-joint paired motion at 30 FPS, without text or music. Recording-level train/validation/test splits are 13/2/4 (Cha-cha), 12/1/3 (Rumba), and 36/5/17 (Taekwondo). Within each class, Ready-to-React and SCHA-React share a VQ-VAE trained for 40,000 updates. All compared policies receive 80,000 updates; ours uses 40,000 for general Flow Matching and 40,000 for historical refinement.
We retain the same refinement procedure using completed events annotated in each class’s training recordings. Selected events must end before the current training window. At inference, the historical pathway is disabled; the policy uses bounded recent motion and maintained latent buffers. In Reactive evaluation, the model responds to observed ground-truth partner motion. In Two-character evaluation, both roles are generated recursively from four-frame prefixes. These experiments evaluate adaptation through class-specific training.
2. Added results and manuscript locations.
Rumba results appear in Sec. 4.5, Table 4 and Fig. 4; Cha-cha results are in App. J.1, Table 19; Taekwondo results are in App. J.2, Table 20. App. J.1, Figs. 7-10 provide dance visualizations. The table reports per-clip FID as Ready-to-React → SCHA-React (lower is better):
For both dance styles, SCHA-React achieves lower per-frame, per-transition and per-clip FIDs than Ready-to-React, ReGenNet and Intention Guidance in both tasks. In all four dance settings, our Jitter values are also closer to real-data values than those of Ready-to-React.
We additionally report frozen-encoder InterCLIP-FID (Sec. 4.1; App. D.5). Relative to Ready-to-React, it improves for Cha-cha (Reactive: 98.73 → 65.59; Two-character: 121.90 → 80.20) and Rumba Reactive (85.28 → 55.72), but worsens for Rumba Two-character (60.33 → 105.09). Sec. 4.5 explicitly reports this disagreement between evaluation spaces.
3. Taekwondo limitation and diagnostic analysis (App. J.2, Fig. 11).
On Taekwondo, Reactive InterCLIP-FID improves from 44.64 to 37.94. However, in Two-character generation, our per-frame/per-transition/per-clip FIDs are slightly higher than Ready-to-React’s (3.35/7.29/54.02 versus 3.05/6.80/53.78), as is InterCLIP-FID (86.69 versus 79.58). We analyzed these weaker autonomous results using three diagnostics:
These observations suggest that sparser eligible history and weaker recurrence may limit historical refinement, while autonomous generation under-represents active interaction states on Taekwondo.
4. Discussion and acknowledged limits (Sec. 5).
We explicitly acknowledge that autonomous generation can enter low-activity states and that lower local-feature FID over long rollouts does not establish long-range response organization. Sec. 5 identifies compact historical-state retention and multi-step closed-loop training with order-sensitive interaction objectives as future directions for sustained responses and recovery. App. D.6 also notes that the small dance test sets, especially the three Rumba recordings, limit conclusions from small FID differences.