- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
NBF-PAS trains neural control barrier functions without external safety oracles. It backtracks along failed rollouts, searching alternative actions for recoverable full-state "anchors" guided by low-dimensional safety geometry. Anchors and recovery trajectories supervise barrier fitting and policy fine-tuning, improving boundaries and collision avoidance on Dubins, Segway, quadrotor, and Hopper systems.
Critical strengths
The paper takes on a real bottleneck in neural CBF learning: existing methods depend on expert demonstrations, safe policies, or hand-designed reward signals. NBF-PAS gets its supervision from the system's own counterfactual recoveries. Backtracking from failures to full states where some alternative action still avoids collision gives anchors that fix both where the zero level set sits and which way its normal points.
R_norm shows baselines with low collision rates still have poorly oriented normals (Dubins 9.9% vs. 3.6%; quadrotor 22.3% vs. 0.0%). it directly supports the claim that anchors fix what sign and feasibility losses leave underdetermined.
Critical weaknesses
There is only one baseline, can compare with safe RL methods? The idea is good but evaluation is too thin: one baseline, no variance reporting, and only single-obstacle setups. Safety gains partly come from conservatism (100% falls on Hopper before retraining).
The paper has a strong assumption, requires resettable counterfactual rollouts plus a hand-specified projection k and margin g, which effectively means knowing the obstacle geometry.
Questions for authors
There is only one baseline, can compare with safe RL methods?
Weakness 1-1 (CPO baseline with variance reporting):
We added a new CPO baseline initialized from scratch and from nominal controller weights. We did not include safe RL methods because our proposed NBF-PAS focuses on learning a neural barrier that captures a more correct landscape compared to conventional training, instead of establishing our superiority over all safe RL methods.
While safe RL methods are powerful tools in collision avoidance tasks, if the solution is too narrow and the goal reward is far, RL methods maybe sensitive to seeds if exploration cannot find the narrow solution. If RL must update the policy given an existing nominal one, it may get trapped at negotiating between staying close to the nominal policy and improving long-term discounted reward. Segway collision avoidance is such a challenging task that successful passage requires coordinated tilting and forward motion. An easy local minimum is just to stand still in front of the obstacle. We use this as the example to show why our learned neural barrier can help in Appendix D.4 with four seeds.
Among four seeds (40-43), only seed 41 and 42 that initialized from scratch initialization are able to improve with 2M training steps. Collision rates are extremely high when CPO is initialized with the nominal controller that is not trained with collision avoidance.
Weakness 1-2 (Multiple-obstacles Setting):
We added a two-obstacle Segway scenario (Appendix D.5) to show that our neural barrier trained with one obstacle environment can be directly used as the auxiliary loss in PPO training for two-obstacle environments. The two obstacles have the same height and radius as the training obstacle but different horizontal positions. The barrier network Bθ is kept frozen throughout; only the control policy is adapted. Since Because the Segway dynamics are invariant to horizontal translation, we evaluate the same network in coordinates relative to each obstacle:
Bi(x)=Bθ(Tix),Tix=(p−pi,ϕ,p˙,ϕ˙),Bmulti(x)=mini=1,2Bi(x)
The policy is updated through
Lchange=Ex∼Dactive[[−ΔBmulti−Δt,γ(Bmulti(x)−ℓ)]+2]
Warm-started single-obstacle controller, we are able to obtain 98.13%±1.41 success rate with only 1.5%±1.64 collision rate across four seeds.
Weakness 1-3 (Falling):
We would like to clarify that the falling of Hopper does not falsify the learning of neural barrier itself. Before policy adaptation, the barrier filter modifies the nominal action to satisfy the learned obstacle-avoidance constraint while minimizing action deviation. The criterion does not preserve gait stability. Even a small action modification may drive the Hopper into a state where nominal policy has never seen and knows how to react. The pre-retraining result therefore shows that in practice, direct filtering alone is insufficient for successful Hopper control, and this is why in practice most paper also formally proposes to iterate on the policy update like PNCBF. The way we propose to train a CBF can actually replace how CBF is learned while combining the policy update section.
Weakness 2:
We do assume access to geometric safety specification and a resettable simulator that allows restoration of sampled states and transition queries under alternative actions. For neural CBF learning, sign supervision on safe and unsafe states must exist to help place the learned nonnegative set within the physically admissible region as explained in Appendix A. Our projection κ and margin g provide inexpensive geometric labels without requiring expert annotations from a given safe controller. This access supports both counterfactual rollouts for boundary supervision and the local action-affine dynamics approximation used in policy refinement. However, we do not require analytical dynamics derivatives.
Question:
We added CPO as an additional baseline in Appendix D.4 to show why our learned neural barrier can help with four seeds.