- Rating
- 2
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies "silent failures" in low-precision robot policy training: the loss curve and overflow monitors all read normal, yet the action output has already collapsed. The paper identifies three failure modes, localizes the root cause to a self-reinforcing loop in the backward activation-gradient chain, where weak signals are swallowed by rounding and then amplified by Adam, and proposes a 50-step twin probe for diagnosis and the CURE routing protocol for repair, validated on a matrix spanning 0.8M ACT to 7B VLA models.
Critical strengths
- Valuable motivation: silent action-level failure is a real and overlooked problem, and attributing the posterior collapse of CVAE policies to the numerical layer, with recovery by precision means alone, is a meaningful observation.
- Careful mechanism design: the item-by-item exclusions and the "zero-mean noise vs. equal-power toward-zero bias" control experiment are clean, and the rescue-band prediction is falsifiable and confirmed.
- Honest reporting: the limitations section explicitly acknowledges the absence of real-robot validation, probe false positives, the lock's domain boundary, and a single harness per 7B model.
Critical weaknesses
- Severely insufficient sample sizes: most conclusions rest on 1–3 seeds per arm, and the authors themselves acknowledge that lottery outcomes change with the harness; the statistical reliability of the key conclusions is inadequate.
- Weak probe validation: only 7 cells, with a bootstrap 95% CI of [0.38, 1.00] for the 0.86 rank correlation, nearly including zero; the prospective routing validation covers only 2 cells.
- Anecdotal evidence at scale: the full quantitative picture comes almost entirely from the 0.8M ACT; at 7B, CogACT has only 2 seeds on a single harness, the OpenVLA-OFT fp32 reference itself went NaN (making that cell uninterpretable), and the method misfires on GR00T.
- Limited behavioral evidence: rollout success is resolvable on only one cell, the primary metric µdiv admits only directional readings below the floor, and the real-hardware fp8 validation covers a single configuration, at anecdotal level.
- Weak necessity of the repairs: the authors themselves show that four one-line changes (Kahan, ε, β2, clipping) are equally effective, so the incremental advantage of LockedAdamW and UPH is unsupported.
Questions for authors
- Can the core cells be expanded to sufficient seeds (≥5–10) and re-verified on two harnesses?
- Can the conservative collapse on CogACT be reproduced on more large models and seeds?
- Can the confidence interval of the probe correlation be tightened with added samples?
- Can the incremental gains of LockedAdamW/UPH over one-line fixes and global stochastic rounding be quantified?
Note: the motivation and mechanism analysis of this paper are valuable, but the current experimental reliability is insufficient. The score may be raised if the authors provide validation with sufficient seeds and large-scale models across adequate samples, and tighten the probe statistics.