acceptodds
Under review as a conference paper at ICLR 2027

From Model Coverage to Control Value: Auditing Few-Shot Safety Filters under Topology Shift

Abstract

Safety filters are often judged by model error, yet a tighter model can be useless or uncertified: its error premise may fail, its reserve may empty the feasible set, or its rows may be inactive at the deployed decision. We introduce an optimizer-level, four-stage audit that separates model-premise validity, certificate availability, decision relevance, and control value. Under shared convex non-target constraints and a strictly convex objective, a single-solve certificate proves that every candidate constraint model whose feasible set contains a reference optimizer with strict target-row slack returns the same unique action; a computable perturbation radius extends the screen to a local model family. This certificate requires verified optimality and a shared problem, not voltage feasibility alone. We instantiate the audit for few-shot voltage shielding of electric-vehicle charging under topology shift using three complete target sensitivity queries. On three frozen feeders, interpolation upper-covers all 60/60 unqueried interior matrices and reduces final upper-bound overstatement by 38.74–61.96% relative to a same-budget baseline. The tighter model nevertheless changes no action in 34,560 comparisons: one of 15 base-feasible target–pressure layers fails zero-action voltage membership, and all remaining deployed actions have strictly inactive voltage rows. A prospectively frozen IEEE 8500 stress test covers only 3/20 matrices and therefore stops at the first gate; a state-binned reserve improves transfer coverage in 0/8 development comparisons and is not confirmed. The audit converts null and failed-transfer results into falsifiable diagnoses without treating finite-grid coverage as a population, arbitrary-topology, or nonlinear-AC guarantee.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
safety filters, feasibility shields, topology shift, certificate availability, decision relevance, electric-vehicle charging

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_jRHuOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper looks at a convex voltage safety filter for EV charging on a feeder the filter has not seen before. Given three measured sensitivity matrices from the new feeder, the authors interpolate between them and add a small calibrated margin ("InterpBox"). They compare this against error boxes built around a topology-based nominal model, and frame the evaluation as a four-stage audit: is the error model valid, is the certificate available, does the constraint matter for the decision, and does it improve control. On three SMART-DS feeders, all methods pass static coverage, InterpBox is tighter (by a post hoc metric), and closed-loop actions are identical across methods in every one of 34,560 slots. An external IEEE 8500 test fails coverage, and a state-dependent reserve fails to transfer.

Critical strengths

The empirical practice seems carefully designed. Protocols were frozen before the confirmation runs, the held-out feeders were selected before any outcomes were seen, and negative results are reported honestly instead of being buried. The point that "zero violations" can simply mean "the controller refused to charge" is a good one, and more safety papers should report availability and service alongside violation counts.

Critical weaknesses

Novelty is limited. The theoretical results are standard. Theorem 1 follows in one line from P ≥ 0, and Proposition 1 is the familiar fact that an inactive constraint at the optimum of a convex program can be dropped without changing the solution. The four-stage audit is a reasonable way to organize an evaluation, but the core idea that model accuracy is not the same as decision value comes straight from the predict-then-optimize literature, as the authors themselves note. Interpolating measured matrices instead of centering on a coarse nominal model feels close to self-evident. There is also little learning anywhere in the pipeline (the proposal is a max-charge heuristic), which makes the fit with ICLR questionable.

The main experiment can't answer its own question. The central question is whether a tighter model changes decisions, but the voltage constraints never bind in the confirmation runs. The minimum slack is over 300 times the active-constraint tolerance, and service is 94.75% for every method, so capacity and deadlines are what limit charging. In that regime a null result is guaranteed, and it says nothing about whether tightness matters. The authors' own activation check (App. H) shows that the development feeders rarely produce binding states either, so this was foreseeable.

The out-of-distribution evidence goes against the method. On IEEE 8500, InterpBox covers only 3/20 matrices, worse than both nominal-centered baselines (8/20 and 15/20). The margin is calibrated at 1.025 times the worst residual on four development feeders, which leaves almost no safety buffer, so this fragility is not surprising. Coverage is also only checked at 20 discrete points, with no smoothness argument for what happens between them.

The certificate isn't actually used. Because the stored results lack optimality evidence, Proposition 1 can't certify the identical actions in the main study. The separate synthetic test certifies 25 of 160 cases and gives no speedup.

The setup seems narrow. All confirmation feeders come from one synthetic source and use 8 stations, constant background load per day, and no grid-state feedback. I'm also not convinced by the information assumption. Measuring a full matrix over 1,600+ phases realistically requires a detailed model of the target feeder, and if you have that model, you could just recompute sensitivities online or solve the OPF directly.

The paper is hard to read in general. It reads more like a lab notebook than a paper, with protocol versions v0.3 through v0.12, withdrawn interpretations, and heavy hedging in nearly every sentence. The main takeaways are hard to find.

Questions for authors
  1. How would the three full sensitivity matrices be obtained in practice, and why not compute them online if a feeder model is available?
  2. Why not design the confirmation so that voltage constraints actually bind (heavier load, more stations, weaker feeders)?
  3. Given the IEEE 8500 result, what supports recommending interpolation over the nominal-centered boxes?
  4. What does Proposition 1 add beyond standard results on inactive constraints?
Reviewer_uK7GOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a four-stage audit for safety filters, separating model-premise validity, certificate availability, decision relevance, and control value. It provides a conditional decision-invariance certificate and a local perturbation bound, and instantiates the audit using three-query interpolation of voltage sensitivities for EV charging. On three held-out feeders, interpolation reduces coefficient overstatement but changes no charging actions or service outcomes. A frozen IEEE 8500 test fails coefficient coverage, and proposed reserve refinements fail development transfer gates.

Critical strengths

The paper carefully distinguishes coefficient coverage, conditional safety, certificate availability, and delivered service. Its transparent reporting of null effects and failed transfer is valuable: it avoids treating tighter models or zero observed violations as evidence of useful certified control.

Critical weaknesses

Limited technical novelty. The technical components are linear interpolation with empirical margins, convex projection, a constraint-inactivity argument, and a perturbation bound obtained through Hölder’s inequality. The paper does not sufficiently establish what substantive capability the four-stage audit adds beyond standard model validation, feasibility checks, active-constraint analysis, and downstream decision evaluation. The framework is sensible, but its incremental methodological contribution appears limited.

Narrow validation and insufficient broader significance. The main confirmation study uses a fixed heuristic charging proposal, constant within-day background loads, and three synthetic feeders from one data source. It observes no action differences, but cannot apply its central optimizer-level certificate to the retained application outputs. The separate forward implementation certifies only 25/160 candidate comparisons and demonstrates no runtime advantage. Consequently, the paper establishes neither substantial practical utility of the certificate nor broadly informative findings for learning-based safety filters. Its connection to machine learning is primarily motivational rather than demonstrated through a learning method or evaluation of learned controllers.

Questions for authors

Methodological novelty: What substantive diagnostic capability does the four-stage audit provide beyond combining standard error-bound validation, feasibility testing, constraint-inactivity analysis, and paired control evaluation?

Certificate validation: Given the missing shared-problem and optimality evidence in the main application, what results establish that the proposed certificate provides practical value beyond retrospective action comparison? How should its contribution be assessed relative to the direct shared-box baseline in Appendix A.4?

Broader relevance: Which findings are specific to learning-based safety filters rather than generic convex optimization with approximate constraints? What evidence supports relevance beyond the evaluated heuristic EV-charging setup?

Reviewer_XHZuOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This reviewer read up to Section 3, including the abstract, and could not identify what research problem(s) this paper address.

Critical strengths

Not identifiable

Critical weaknesses

Not identifiable

Questions for authors

None

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.