W1. Limited theoretical novelty
Proposition 3 appears to follow directly from standard split-conformal rank guarantees and a weighted union bound over complete policies. The error allocation and rank-feasibility conditions are useful, but a substantial theoretical advance is unclear. Proposition 1 is direct monotonicity; Proposition 2 aggregates candidate and fallback risks; Propositions 6-7 apply standard convex geometry without yielding a gate-learning algorithm. Propositions 4-5 clarify important limitations through relatively elementary arguments.
The differences from LTT [1], CRC [2], and CPC [3] are appropriately acknowledged. However, a distinct certification target alone does not establish novelty; the paper should explain what new capability or understanding it enables.
W2. Limited general insight from one author-constructed gate
The gate combines established components rather than reproducing a representative published method. Its failure may therefore reflect implementation, training, or calibration choices rather than a broader limitation of safety gating. V4 establishes failure against the chosen mixer, but does not distinguish critic error, state-distribution shift, fallback inadequacy, or calibration-target mismatch. V5 cannot isolate timing causality, while V6 provides no clear component-level explanation.
Negative findings are valuable, but the paper does not sufficiently identify when and why this failure occurs or which other designs the lesson applies to. This limits its significance as an audit.
W3. Narrow evaluation and unclear positioning against existing offline RL
The principal evaluation covers only PointGoal1, PointButton1, and PointPush1. The 54 checkpoint pairs strengthen replication across seeds, not generality across dynamics, difficulty, or dataset quality. Exhaustive benchmark coverage is unnecessary, but broader representativeness is insufficiently justified; earlier experiments with uninformative completion outcomes do not resolve this concern.
Because the proposal is BC-All, the study also leaves unclear the gate's incremental value for stronger offline RL policies such as CQL or IQL, and its position relative to CAPS [4]. Representative comparisons or a general failure mechanism would strengthen the contribution; state-of-the-art performance is not required.
W4. Weak connection between theory and public evidence
Deployment exchangeability and fallback-risk bounds are not established publicly, and trajectory qualification is retrospective. The experiments therefore do not demonstrate useful operation of the theoretical procedure under realistic conditions.
The empty qualified set is also unsurprising given violation rates far above the target level; even policy-by-policy qualification fails. This supports reasonable refusal, but does not show an advantage over existing certification approaches. Although these limitations are acknowledged, the theory and experiments do not sufficiently reinforce one another.
W5. Incomplete performance reporting in the principal comparisons
Raw return and mean cost are reported in Section 6 and Appendix H, Table 7. However, the principal V4/V5 tables emphasize completion and violation differences without jointly presenting return, mean cost, and uncertainty. These metrics are complementary. Consistent reporting, including DSRL normalized scores where applicable, would clarify the performance-safety trade-off. This is a secondary concern that compounds W3.
References cited in Weaknesses
- [1] Angelopoulos, A. N., Bates, S., Candès, E. J., Jordan, M. I., and Lei, L. "Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control." The Annals of Applied Statistics, 2025.
- [2] Angelopoulos, A. N., Bates, S., Fisch, A., Lei, L., and Schuster, T. "Conformal Risk Control." ICLR, 2024.
- [3] Prinster, D., Fannjiang, C., Park, J. W., Cho, K., Liu, A., Saria, S., and Stanton, S. "Conformal Policy Control." ICML, 2026.
- [4] Chemingui, Y., Deshwal, A., Wei, H., Fern, A., and Doppa, J. "Constraint-Adaptive Policy Switching for Offline Safe Reinforcement Learning." AAAI, 2025.
Thank you for the detailed clarifications. However, my main concerns remain unresolved. In particular, the implementation replaces the computed conformal threshold with a quantile targeting a 20% rejection rate, leaving a substantive gap between the theoretical specification and the evaluated gate. I therefore keep my recommendation of weak rejection.