- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper argues that failure-based adaptive reset sampling (as in BeyondMimic and SONIC) gives difficult temporal bins far less of the PPO rollout buffer than their reset probabilities suggest. Attempts that start in hard bins terminate early, while long episodes in easy bins keep their environments busy. The authors formalize this as . They then propose ARA, which reserves up to environments for short-horizon rehearsal of high-failure bins once a competence gate opens. On 59 motions (19 hard AMASS clips plus LAFAN1), ARA reaches 59/59 acquired motions. It also lets weak blending () beat strong blending () on MPJPE at 8192 environments.
Critical strengths
- The diagnosis is practical and clearly stated: reset priority sample share. Fig. 2 shows rollout-to-reset ratios as low as ~0.055 for persistently failing bins.
- ARA is simple and plug-in. It separates hard-bin acquisition from broad refinement and so eases a real trade-off set by .
- The training details (observations, rewards, DR, PPO settings) are thorough enough to aid reproduction.
Critical weaknesses
- Weak and possibly unnecessary empirical gains. With 16384 environments, plain A.S. already acquires 59/59 motions, and ARA changes S.R. by only 0.1 (Table 1). The benefit therefore shows up only in a single compute-limited setting. On held-out BONES-SEED (Table 2), ARA gives no improvement and is slightly worse at . The "better tracking than strong A.S." claim mostly reflects the effect of , not ARA itself.
- No ablation of six new hyperparameters. , EMA rate, are all fixed; Sec. 5 concedes sensitivity at fewer environments and an acquisition/forgetting oscillation, but neither is quantified. For a method whose only contribution is an allocation rule, this is the core missing experiment.
- Contribution is thin. Eq. 2 is a one-line renewal-reward identity, and the survival-bias effect it describes is already noted in GMT ("limited practice received by difficult portions of long motions") and analyzed in Rhythm (RSS 2026). Framing it as a discovered phenomenon oversells the analysis.
Questions for authors
- Does ARA degrade motions the policy has already mastered?
- Why not correct by expected episode length directly from Eq. 2? How does that compare to ARA?
Thank you for the thoughtful review.
Weakness 1. Weak and possibly unnecessary empirical gains.
ARA consistently acquired all 59 target motions across the environment counts and sampling strengths in Table 1. A.S. also acquired all 59 at 16,384 environments. At the same count and α=0.9, expanding the training set by roughly 13,000 AMASS clips gave 47/59 for A.S. and 59/59 for A.S.+ARA.
Acquisition and tracking on the 59 target motions after training with additional AMASS motions
Weaker blending contributes to the tracking advantage in Figure 3. At fixed weak blending, ARA recovered the missing motions at a small tracking cost (Table 1). Held-out BONES-SEED performance was broadly similar, with slightly higher tracking errors for A.S.+ARA under weak blending (Table 2).
Weakness 2. No ablation of six new hyperparameters.
We varied the per-bin cap Dmax and total budget CARA, which bound rehearsal capacity. At a cap of 300, ARA acquired all 59 motions at every budget tested. At a budget of 1,000, a cap of 100 left one motion unacquired. Caps of 300 and 500 gave full acquisition, with slightly higher tracking errors at 500.
Sensitivity to rehearsal budget and per-bin cap
In our training experience, final performance is less sensitive to the other four parameters near their Table 7 defaults, held fixed here. The gate (τc=0.99) lets the background sampler learn most of the corpus before rehearsal. The failure threshold (τf=0.5) shifts practice away from improving bins. With fewer environments, lowering it can help retention by delaying release (Section 5). A minimum duration of 0.1 s prevents immediate recycling near a bin's end. An EMA rate of 0.1 smooths brief failure spikes while responding to sustained changes.
Weakness 3. Contribution is thin.
We agree that prior work recognized the need to practice difficult motions and will clarify this in the introduction. We quantify the gap between intended and realized practice under adaptive sampling: a high-priority region can receive a much smaller share of rollout steps than its reset probability suggests. Equation 2 explains this relationship, Figure 2 measures it, and ARA reserves environments for repeated local practice.
Question 1. Does ARA degrade motions the policy has already mastered?
In Figure 3(a), A.S.+ARA's acquired-motion count fell by one around 32,000 iterations and recovered at the next evaluation. A.S. alone showed several temporary drops. From about 52,000 through 90,000 iterations, all 59 motions exceeded 50% success at every A.S.+ARA evaluation. Individual success rates could still vary above that threshold.
Question 2. Why not correct by expected episode length directly from Eq. 2? How does that compare to ARA?
We interpret the proposal as dividing each bin's adaptive reset probability by the mean episode duration from that bin, then renormalizing. This favors both early failures and easy bins near a clip's end. Dividing by the mean fraction of available duration completed would address the latter. Available duration is the shorter of the remaining clip length and the episode horizon.
The reset-to-rollout gap can still persist: a higher reset probability does not guarantee a corresponding share of local rollout steps (Eq. 2). The correction may provide enough practice in some settings and too little in others, as ordinary adaptive sampling did in Table 1. ARA reserves environments for repeated practice in selected regions. We have not evaluated the proposed correction.