- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies learning a single secondary-task coefficient on top of a supplied robot priority controller. The controller computes primary and secondary joint-command channels, and a PPO policy selects . Fault reports, kinematics, gravity compensation, and the task hierarchy are given. In Isaac Sim, a global logit bound establishes that, under deterministic maximum-probability deployment, the six originally selected actors always output. Removing an initial full-retention bias and retraining changes deployment behavior, but does not meet the control-benefit criteria. A simple margin rule has fewer failures, no fewer recoveries, and lower cumulative secondary error than every neutral-initialization fit. A separate intervention fixes network weights and compares sampling with maximum-probability deployment, finding some improvements accompanied by additional primary violations. Further diagnostic studies investigate limited scheduling opportunities, and two archived trials from a separate Panda policy illustrate scalar execution.
Critical strengths
-
The constant-deployment diagnosis is stronger than a finite collection of observed trajectories. Section 5 and Appendix C establish a positive global logit-gap bound for each of the six selected original actors under the stated activation and arithmetic conditions. This verifies that their deterministic decisions cannot depend on observations, even though training changes weights and probabilities. It provides concrete evidence for the distinction between changes to a network and changes to its deployed decisions.
-
The interventions and negative results are reported with appropriate limits. The paper distinguishes changing initialization and retraining from changing only the deployment rule of fixed weights. Tables 2 and 3 retain all fitting seeds and report violations separately from recovery and secondary error. The authors also distinguish independent scenarios from masks and action streams, and avoid claiming a physical safety certificate or general RL impossibility. These choices make the limited empirical conclusions easier to assess.
Critical weaknesses
-
The expected value of the learning scheme remains insufficiently supported. Sections 2–3 explain that torque limits, inertia, and transients during faults and the restoration of actuator effectiveness can cause secondary commands to affect the primary task, motivating state-dependent adjustment of the secondary coefficient. The paper aims to improve control performance through learning and treats an advantage over existing feedback rules as the test of the learning scheme's value. However, in Table 2, the margin rule is no worse than all six neutral-initialization PPO fits on the three aggregate metrics of failures, recoveries, and cumulative secondary error, and the learning scheme does not meet the prespecified control-benefit criteria. The paper does not sufficiently explain which important decision difficulties in the chosen tasks are hard for simple feedback rules to handle. An unsuccessful scheme can still be worth studying, but the importance of the chosen problem and why this negative result deserves other researchers' attention require clearer justification.
-
The new insights and value of the failure diagnosis for other work remain unclear. Section 10 states, "We do not claim initialization sensitivity itself as new." The main deployment bound follows directly from bounded activations and output-layer parameters and establishes a property of particular selected models. The original case uses an initial full-retention bias of 4, an output-layer initialization gain of 0.01, a fixed training budget, and update-12 checkpoints; neutral initialization changes deployment behavior but still does not meet the prespecified control-benefit criteria. Appendix C.1 also shows that untrained neutral actors already produce 20–21 action categories on the same validation observations visited by the selected policies. These findings support diagnosis of the current system, but do not sufficiently identify diagnostic principles that other researchers can apply beyond existing knowledge of initialization sensitivity and training/deployment differences, or establish the conditions and evidence supporting such use. The ideal retiming, finite-option, hidden-budget, gradient, residual, and probe analyses provide local boundaries or screening conclusions, but their connection to the main feedback experiment's remaining performance gap is still unclear. The main simulation also lacks corresponding coefficient and primary/secondary-error trajectories, while the hardware traces concern a separate policy; Appendix B lists baseline parameters but does not fully define the margin signal, slope estimate, or coefficient update rule. These gaps make it difficult to understand the failure behavior, reproduce the central comparison, and assess which lessons can inform other work.
Questions for authors
- Expected value of the learning scheme: The paper treats an advantage over feedback rules as the test of learning's value. Which specific decision difficulties in the chosen tasks make learned coefficient adjustment worth investigating, and what advantages is it expected to offer over simple feedback rules? Are there explicit objectives beyond improved control performance, and how does the existing evidence support them?