- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper learns a Lyapunov certificate V and a drift predictor F from passive expert demonstrations of an unknown control-affine system, then deploys a CLF-QP built from them. A one-sided warning score (expert margin deficit plus the positive part of the drift error along grad V) provably exceeds a tight threshold whenever the deployed QP violates certificate decrease (Lemma 3). Under an encounter-coverage assumption (Assumption 4), deployment risk is at most the logged warning loss divided by nu_enc (Theorem 1). The paper gives parametric, scenario-based, and RKHS bounds on that loss, and a minimax lower bound of order d/(nu N) on a constructed family (Theorem 4). Experiments on a pendulum and a cart-pole with linear saturated experts compare against behavior cloning, MSE identification plus QP, and neural baselines. Since nu_enc is never established, the reported deployment bound is Q = 1 throughout.
Critical strengths
-
The directional warning and detection lemma are a clean, useful idea: only model errors that can hide a certificate increase are penalized, and the threshold is tight in the scalar case. The statistical side is careful (simultaneous bounds allowing same-batch selection, correct scenario dimension p+1, certified numerical optimizer transfer). I reproduced the scenario bounds in Table 6 exactly.
-
The paper is transparent. It reports Q = 1 instead of an optimistic number, gives counterexamples to favorable coverage (Table 7), proves the coverage dependence is unavoidable for this architecture, and uses a frozen protocol with interval verification of the final pendulum controllers.
Critical weaknesses
-
The deployment guarantee depends entirely on Assumption 4, which can't be verified from passive data and is never instantiated. Given Lemma 3, Theorem 1 follows in a few lines. The authors' own estimates (Appendix G.6) put the coverage ratio around 0.07-0.08. Even if that value were certified, the best source bound at N = 256 (0.1635) would still give Q = 1, and by my calculation Q <= 0.1 would need roughly 6,000 episodes on the pendulum. The "sharp" result (Theorem 4) holds on a constructed family with uninformative expert trajectories, one terminal record per episode, and a fixed QP architecture, where the authors note u = -3x stabilizes every member. The title overstates this.
-
The experiments don't show when the method is needed. The experts are linear and behavior cloning recovers them almost exactly. BC-ridge and NLC-on-model have zero violations in every cell. At N = 128 the scenario learner has 13/384 (pendulum) and 144/384 (cart-pole) violations, worse on cart-pole than R39-ERM (21/384) and MSE-ID+QP (115/384). On cart-pole, BC also has lower effort and faster settling (Table 11).
Questions for authors
- Proposition 1 gives if a warning sublevel set is forward invariant. Since you already run interval verification (Appendix G.10), can you certify this for the selected pendulum pair and report a non-vacuous ?
- Is "OURS-ERM" in Table 1 the same as "R39-ERM"? What does R39 refer to?
- The matching upper bound in Theorem 4 comes from a bit-guessing selector. Does Algorithm 1 achieve the same rate on that family? Since the class there is a continuum, Theorem 3 does not apply directly.
- Why does the scenario learner do much worse than R39-ERM on cart-pole at ( vs. violations out of )?
- Nonlinear experts or higher-dimensional systems, where cloning is not trivially exact, would show where the approach pays off.
- Direct testing needs only failure-free rollouts for a bound (Appendix G.7). When is the passive route preferable to a small amount of on-policy testing?
3 comments
General Comment (2/2)
Three additions to the revised manuscript concern the designed expert resets (rKsi Q5, ZbmT Q1 and W1, gP8d Q1) and a system above four states (gP8d Q5, rKsi W5). All were specified before their data.
How much designed logging is needed (Appendix H.15, Figure 14). The designed laws of the paper draw each expert reset from a probe law with probability one half: near the origin for the pendulums, and on an enlarged set for cart-pole. We varied this probability over 0, 0.025, 0.1, 0.25, 0.5, 0.75 and 1 on the four benchmarks, with 12 new batches per cell and deployment starts from the original law. At , Algorithm 1 at zero onset had 467 and 112 detections in 49,152 cart-pole starts at and 0.025. It had none in 196,608 starts across the four benchmarks at every share from 0.1 to 1. At , detections vanished from with the slow nonlinear expert and from 0.025 with the fast one. On cart-pole at they were fewest at 0.25 and 0.5 (72 and 65 of 12,288) and rose to 6,398 at , where no reset comes from the deployment law. MSE-ID+QP had detections on 27.4% to 100% of starts in every cell. With zero-mean derivative noise added, the noise level at which most starts have detections was the same at all seven shares.
What the design certifies. These counts are measurements. The reset coefficient of Appendix C.3 is at most on the pendulums and on cart-pole, so the passive report on the physical benchmarks remains one, as stated. Measured with simulator labels, the probe law gave a positive logged loss for each of the 878 fitted pairs with detections on at least 1% of starts. The ratio of that loss to the mean path score was below the oscillator's 0.916 for 607 of 965 pairs. The design thus exposes each such controller in the logs, without a uniform coefficient.
A six-state benchmark (Appendix H.15, Figure 15). We repeated the logging and expert comparison on a planar quadrotor with six states, two inputs and an LQR expert (, , 12 batches per cell). Under the uniform reset law, Algorithm 1 at zero onset had detections from 1,749 of 12,288 starts at , 115 of 49,152 at and 3 of 49,152 at . MSE-ID+QP had detections from 84% to 97% of starts, and ridge and neural cloning of the linear expert had none in any cell. Every trajectory of every method settled, so the detections concern the prescribed rate. The probe laws did not lower the counts at or 4096. At share one half and , the near-origin law gave 22 detections and the enlarged law 30,785, with slack above in every fit, so the report was one. This is one system above four states. It does not measure scaling with dimension.
From a sampled check to a proof on cart-pole (Appendix G.8). The manuscript reported a sampled check of one zero-onset cart-pole fit under enlarged logging. An interval branch-and-bound now covers the whole certificate sublevel set containing the reset set, for all 12 such fits at . Each fit is verified: the normalized decrease residual is at most on the set, with 15.9 to 49.7 million accepted boxes per fit and 17.2 CPU-hours in total. Hence decreases at the prescribed rate along every model trajectory from the reset set, for all time, for each of the 12 controllers. The same program verifies the three pendulum fits of Appendix G.8 and returns a counterexample for a cart-pole fit with measured violations. As for the pendulum, this is a verification with the plant model after fitting, and the passive report is unchanged.
These runs used a second machine. Its check against 48 rows of Table 1 reproduces every Algorithm 1 count and differs in one MSE-ID+QP count, so the share study is reported from its own rows.
Response to —— gP8d —— on Certification and Sample Complexity (1/1)
Thank you for reproducing the scenario bounds. Your guide is in the supplement of the uploaded full paper, just after the references. Links under each question and concern jump to the highlighted passages.
Q1. Can the pendulum sublevel be certified? Yes. For each of the three frozen fits, intervals enclose every potentially active face derivative under the clipped QP control. This proves invariance of the warning sublevel at within an invariant ellipsoid containing all resets. Since every episode logs its reset, Proposition 1 gives coverage one for that fit. We checked the analytic expert actions and derivatives at all resets used to train these fits. Lemma 7 then gives a 95% deployment bound below 2.874% per fit under independent draws from the benchmark’s finite-precision reset law. Proposition 10 separately proves all-time decay throughout the continuous reset disk. For the 12 zero-onset cart-pole fits under enlarged logging, a new interval proof gives all-time decrease. These checks use the model after fitting. The passive deployment report is one (Appendix G.8).
For the separate study with , and zero effective slack, certifying coverage at 0.08 or 0.07 would first bring the exact 95% scenario report to 10% or less at 5,452 or 6,232 episodes, respectively. The observed ratios alone cannot certify those coverage values (Appendix G.5).
Q2. Method names. Yes. OURS-ERM and R39-ERM denote the same estimator. R39 was an internal identifier. The estimator averages, over episodes, the largest squared one-sided warning in each episode.
Q3. Does Algorithm 1 attain the minimax order? Yes. Proposition 14 uses the single certificate from the continuum, so Theorem 3 applies with and . A probe revealing a nonzero bit forces its fitted coefficient up. Only unseen active bits can contribute to risk, yielding expected risk at most . Theorem 4 gives the matching lower order for the declared class and QP. The construction uses one terminal record per episode. The unrestricted policy stabilizes this family. Proposition 17 separately establishes non-identifiability even with complete expert paths.
Q4. Why 144 versus 21 detections? The positive training onset allows warnings up to even at zero slack. At zero objective, the average-loss fit forces every logged warning to zero. On fresh cart-pole expert episodes at , warnings exceeded on 7.1% to 9.5% of episodes at positive onset and 0.35% to 1.87% at zero onset. Lowering the onset changed deployment detections from 896 to 345 in the logging and expert comparison, and from 498 to 670 in its independent replication, out of 12,288 in each study. The onset determines which warnings the fit penalizes. Its empirical effect depends on the batch (Appendices H.3 and H.11).
Q5. Nonlinear experts and baselines. With the slow nonlinear expert, ridge and neural cloning both settled from every start. Under uniform logging at , their diagnostic certificates missed the rate on 100% and 8.4% of starts, respectively. Zero-onset Algorithm 1 had zero detections. With the fast expert, both cloners passed the rate test. In the four-state cart-pole comparison, NLC-on-model had 0/384 detections at , and ridge cloning used less effort and settled faster. On a new six-state quadrotor with an LQR expert, both cloners had no detection and zero-onset Algorithm 1 had at best 3 in 49,152 starts, at 4,096 episodes (Section 5, Appendices G.7 and H.15).
Q6. When use the passive route? With a justified coverage bound, the report is available before the replacement’s first plant trial. When representative independent trials of a frozen controller are feasible, 59 event-free trials give a one-sided 95% upper limit of 4.95% if each trial measures the target event. With only expert data and no coverage evidence, Proposition 11 still bounds violations at logged states.
General comment (1/2)
The uploaded full paper includes a linked guide for each reviewer in the supplement material pdf immediately after the references. The links under each question and concern jump to the passages highlighted for that item. Footer links take you back to the guide.
The reviewers ask what a batch of expert demonstrations can establish about a controller that will take different actions. The risk we bound is the probability that a fresh start leads to any violation of the prescribed Lyapunov decrease over a fixed horizon. On a trajectory that meets this requirement, the certificate decays at least exponentially through the stated horizon. We give a worked offline certificate, a model-assisted proof for the fitted pendulum controllers, and matched tests of decrease and terminal settling.
A worked certificate before the first replacement rollout. The terminal-record coverage example uses an oscillator with unknown drift coefficients. The learner knows their ranges and the reset rule. The expert and replacement follow different trajectories, but radial ordering makes the expert’s terminal record cover the warning along the replacement path. With 32 episodes, this gives a 95% deployment bound of 13.985%. All 32 test starts lie outside the inner disk. The oscillator figure shows why the entire 4% of nominal risk lies inside it. The structural information matters because two plants can generate identical complete expert paths while every candidate controller in the class fails on at least one plant (Proposition 17, Sections 3 and 5, Appendix G.9).
Where the statistical rate is sharp. For a fixed certificate library and affine predictor class, the sufficient episode budget depends on the requested risk, confidence and coverage. On the constructed realizable family, Algorithm 1 attains the minimax order of expected deployment risk in the episode count, predictor dimension and coverage, even though uniform drift-estimation error remains bounded away from zero (Theorem 4 and Proposition 14).
What the benchmarks establish. With a training margin and designed logging at 1,024 episodes, Algorithm 1 had zero detected decrease violations in 196,608 pendulum and cart-pole deployments across four instances. Full-vector identification followed by the same QP had detections on 46% to 100% of starts. A second study on new batches had zero detections for every share of designed resets from 10% to 100% (Appendix H.15). With the slow nonlinear pendulum expert under uniform logging, both cloners settled from every start. Ridge and neural cloning nevertheless missed the prescribed decrease on 100% and 8.4% of starts, respectively (Section 5). Under zero-mean derivative noise, detections of Algorithm 1 began only at median noise-to-signal ratios of 37.8% and 55.5%, and new Proposition 19 covers sign-symmetric noise of any size (Appendix H.15). On a new six-state quadrotor with a linear expert, both cloners had no detection and Algorithm 1 had at best 3 in 49,152 starts, at 4,096 episodes (follow-up comment).
The passive deployment report is one on both benchmarks. For three frozen pendulum fits, the optimizer checks and a model-based invariant-set proof give a 95% deployment bound below 2.874% per fit under independent draws from the benchmark’s finite-precision reset law. A separate interval proof establishes all-time decrease on the full reset disk. For three cart-pole fits on independently reconstructed batches, treating their records as exact gives a bound below 0.04550 on mean logged warning loss. The exact 0.011 margin proved for the enlarged cart-pole reset law concerns the expert. For the 12 zero-onset cart-pole fits under that law, an interval proof on the model now gives all-time decrease from every reset (Appendices G.8 and H.14).
We retained the submitted title for review. If accepted, we will use Sharp Sample Complexity for Certifying Lyapunov Decrease in Offline Control of Unknown Systems for the camera-ready version.