acceptodds
Under review as a conference paper at ICLR 2027

Verification-Guided Safety-Repair of Learned Controllers

Abstract

Formal verification can certify that a learned controller is safe. But what should be done when verification fails? Simply suppressing actions that lead to unsafe next states is insufficient because the dangerous action that ultimately caused the safety violation might have occurred many steps earlier. Retraining modifies the policy globally, thereby potentially introducing new failures. Instead, we introduce ReachECO, a verification-guided repair procedure that exploits the information of failed multi-step reachability analysis for synthesizing explicit constraints on controller behavior. For Differentiable Weightless Controllers, the repairs compile into exact circuit patches that leave the controller unchanged everywhere outside the targeted policy cells. We prove soundness and termination of the repair procedure for ReachECO, as well as, assuming exhaustive continuation search, completeness relative to the reachability abstraction. Empirically, we demonstrate ReachECO’s ability to successfully repair otherwise unsafe controllers in three continuous-control benchmarks: ACC, Docking2D and Car2D

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
formal verification, controller repair, closed-loop reachability, learned controllers, safety games, differentiable weightless controllers

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_ABasOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper looks at how to repair a learned controller when closed-loop reachability verification fails. The idea is that the unsafe action often happens several steps before the violation, so the method backtracks through the reachable sets, solves small look-ahead games to find safe actions earlier on, and then solves all the resulting constraints together while staying close to the original policy. For weightless (lookup-table) controllers the fix becomes an exact circuit patch that leaves the rest of the controller untouched. The authors prove soundness, and termination and completeness under some simplifying assumptions, and show on three benchmarks (ACC, Docking2D, Car2D) that uncertifiable controllers can be repaired and then certified.

Critical strengths

The paper addresses a real gap. Most verification work stops once a controller fails, and here the failed analysis is used to actually fix the controller so that it passes. The baselines it compares to either cannot be certified afterwards or change most of the policy.

I also like the underlying idea of local repair. Since only the parts of the policy that lead to failures get changed, the cost should depend on how unsafe the policy is rather than on the size of the state space, and the rest of what the policy learned is kept. If this holds up, it would be a real advantage over building a safe controller from scratch.

Critical weaknesses

My main concern is that the advantage over existing synthesis methods is not shown. Because the controller is essentially a lookup table over a grid, one could also synthesize a safe controller on the same grid with standard abstraction-based tools (e.g., SCOTS [1], PESSOA [2]), or with lazy variants [3, 4], and then pick the safe action closest to the original policy. The paper does not compare against this, so it is hard to tell what the backtracking and local solving actually buy.

The experiments also do not really test the setting where the method should shine. The systems are small and mostly linear, the strongest results use a losing set that is known in closed form, and the only nonlinear example is a single small case without a baseline. Some of the baselines also start from a different controller, which makes the comparison less clean.

[1] M. Rungger and M. Zamani. SCOTS: A Tool for the Synthesis of Symbolic Controllers. HSCC 2016.
[2] M. Mazo Jr., A. Davitian, and P. Tabuada. PESSOA: A Tool for Embedded Controller Synthesis. CAV 2010.
[3] K. Hsu, R. Majumdar, K. Mallik, and A.-K. Schmuck. Lazy Abstraction-Based Control for Safety Specifications. CDC 2018.
[4] K. Hsu, R. Majumdar, K. Mallik, and A.-K. Schmuck. Lazy Abstraction-Based Controller Synthesis. ATVA 2019.

Questions for authors

How does the method compare with synthesizing a safe controller on the same grid and projecting to the closest safe action? Where does local repair start to win?

How does the runtime scale with grid resolution and state dimension? A scaling experiment would help a lot.

What happens if you only allow one-step repairs? This would directly support the claim that suppressing actions is not enough. How well does the method do without the known losing set in ACC?

Clarification: the termination and completeness results assume no merging and a fixed losing set, while the experiments use both. Could you comment on how often runs timed out or failed in practice?

Reviewer_pmSpOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks what to do when closed-loop reachability fails to certify a learned controller. It proposes ReachECO. ReachECO alternates forward reachability with a backward search for safe multi-step continuations. When an intervention at the failing step is too late, it backtracks to earlier steps and extends the required horizon. Successful searches are stored as game-DAG constraints. CP-SAT then finds one controller that satisfies all constraints and stays close to the original. For Differentiable Weightless Controllers (DWCs), the repair compiles into an exact LUT patch. The patch leaves all unrepaired policy cells unchanged. The authors prove soundness, termination, and completeness relative to the reachability abstraction. Experiments cover ACC, Docking2D, and Car2D. On ACC with the known losing set, ReachECO certifies all five seeds in 2±1 s and changes 8.0% of the policy. A verification-guided supervised fine-tuning baseline changes 96.9% (Table 2).

Critical strengths
  1. The repair procedure itself has formal guarantees. Most controller repair work relies on fine-tuning and only verifies the result afterwards. ReachECO derives explicit constraints from failed reachability and proves soundness of the full loop. This is the main reason I take the paper seriously. It brings the constraint-based view of static network repair to the multi-step closed-loop setting.

  2. The repairs are local and exact. For DWCs, the patch changes behavior only in targeted policy cells. On Docking2D, the hardest seed needs 3,326 modified cells, under 10^-3% of all cells (Table 3). Locality matters for safety-critical deployment. An engineer can inspect exactly what changed.

Critical weaknesses
  1. The termination and completeness guarantees do not cover the experimental implementation as presented and the empirical comparison is narrow. Theorems 2 and 3 assume a fixed losing set L = U and no merging. However, the implementation uses bounding-box merging, some experiments expand the losing set online, and Docking2D uses a greedy restart. The experiments also use restricted continuation backends, whereas Theorem 3 requires exhaustive search over the original action alphabet. Section 2.4 acknowledges these restrictions, and soundness is a separate guarantee that applies to the full procedure. My concern is therefore specifically that termination and completeness are emphasized in the abstract and Section 4 without being established for the implementation used in the experiments.

I suggest either extending the proofs to cover the relevant implementation choices or making the scope of these two guarantees explicit wherever they are highlighted.

Only ACC, a 2D linear system with an analytic losing set, has head-to-head repair baselines. Docking2D compares only against a naive patch that the authors themselves call a lower bound. Car2D has no baseline at all. Within Table 2, several rows are not apples-to-apples. The NCubeV rows use a ReLU network that starts safer than four of the five DWC seeds (Table 5). The supervised baselines use a fixed full-braking target for failing samples, alongside safe samples retaining the original actions. The reported 96.9% change rate therefore reflects this particular repair setup; it does not establish that such widespread changes are unavoidable for fine-tuning-based repair.

  1. Several published repair methods are missing and the positioning also needs work. ISAR [1] and causal repair [2] are cited but not evaluated. Two more are not cited at all. Gangopadhyay and Dasgupta [3] apply targeted updates at failure points. Tappler et al. [4] repair DRL policies with fuzz-testing data. These are relevant candidates for comparison in the unsafe-set-only setting of Table 2, where STLGym is currently the only baseline. Their applicability needs to be assessed individually: for example, [4] uses a DQfD-based repair algorithm and would require adaptation to the controllers used here. The paper should explain which methods can be compared under the same setup and evaluate compatible alternatives.

ReachECO’s backward search toward the losing-set boundary is close in spirit to the boundary states of search-based RL testing [5]. There, the states where a depth-first search backtracks mark the border between recoverable and unrecoverable situations. I do not think this undermines the contribution. ReachECO works over sets of states, turns rescues into composable constraints, and gives formal guarantees. The work in [5] does none of this. But the paper should say clearly that backtracking to the recoverable boundary is not new by itself. Finally, Appendix B describes the repaired controller as a statically compiled shield. Then a natural baseline is abstraction-based safety synthesis on the same grid, e.g., SCOTS [6]. Docking2D and Car2D are only 4D, so I would like to understand whether such a comparison is computationally feasible.

The head-to-head repair comparisons are currently limited to selected adaptations of existing methods on one low-dimensional benchmark. Without broader comparisons, I cannot tell where ReachECO actually pays off.

[1] Pengyuan Lu, Matthew Cleaveland, Oleg Sokolsky, Insup Lee, and Ivan Ruchkin. Repairing Neural Network-Based Control Policies with Safety Preservation. Research Directions: Cyber-Physical Systems, 2025.

[2] Pengyuan Lu, Ivan Ruchkin, Matthew Cleaveland, Oleg Sokolsky, and Insup Lee. Causal Repair of Learning-Enabled Cyber-Physical Systems. In IEEE International Conference on Assured Autonomy (ICAA), 2023.

[3] Briti Gangopadhyay and Pallab Dasgupta. Counterexample Guided RL Policy Refinement Using Bayesian Optimization. In Advances in Neural Information Processing Systems (NeurIPS), 2021.

[4] Martin Tappler, Andrea Pferscher, Bernhard K. Aichernig, and Bettina Könighofer. Learning and Repair of Deep Reinforcement Learning Policies from Fuzz-Testing Data. In Proceedings of the 46th IEEE/ACM International Conference on Software Engineering (ICSE), 2024.

[5] Martin Tappler, Filip Cano Córdoba, Bernhard K. Aichernig, and Bettina Könighofer. Search-Based Testing of Reinforcement Learning. In Proceedings of the 31st International Joint Conference on Artificial Intelligence (IJCAI), pp. 503–510, 2022.

[6] Matthias Rungger and Majid Zamani. SCOTS: A Tool for the Synthesis of Symbolic Controllers. In Proceedings of the 19th International Conference on Hybrid Systems: Computation and Control (HSCC), 2016.

Questions for authors
  1. Which of Theorems 1–3 applies to each experimental configuration? Does any configuration satisfy all assumptions of Theorem 3?

  2. How large are the joint-realization problems, and how much runtime does JOINT consume? Please report problem sizes and how often the 60 s limit is reached. Does this limit apply only to optimization, or can the solver stop without finding an existing feasible repair?

  3. How does locality in input space translate into hardware cost? The Docking2D seed 3 patch is about twice the original controller’s LUT6 count, and the ACC patches add roughly 50–80%. Please discuss this trade-off.

  4. Is abstraction-based safety synthesis on the same grid, such as SCOTS [6], a feasible baseline for Docking2D and Car2D? If not, what prevents the comparison?

  5. Is the runtime improvement from online losing-set learning consistent across seeds? Please report per-seed runtimes and paired differences for the two settings in Table 1.

Reviewer_DtnhOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes ReachECO, a method for repairing learned controllers that fail formal safety verification. Formal verification can prove a controller is safe by computing all states the system might reach. But when verification fails, the usual fixes are weak: blocking only the immediately dangerous action misses problems that began many steps earlier, and retraining the whole network can introduce new failures elsewhere.

ReachECO instead uses the information from a failed multi-step reachability check. It runs verification forward; when it finds regions where the controller's action is unsafe, it searches backward for alternative actions that keep the system safe for the next several steps. If no safe option exists at that point, it backs up one step earlier and requires a longer safe horizon. Each successful search produces an explicit repair constraint. Because different failures can demand conflicting actions in the same state region, ReachECO solves all constraints together with a CP-SAT solver, preferring changes closest to the original policy. It also grows a "losing set" of states from which failure is unavoidable.

The method targets abstraction-based controllers with finite state partitions, instantiated on Differentiable Weightless Controllers (DWCs), which are trainable by gradient descent but have exact Boolean circuit forms. Repairs become small exact circuit patches — a few hundred extra lookup tables — leaving the controller untouched outside the repaired cells.

Contributions : (1) constraint-based repair brought to the closed-loop, multi-step setting; (2) proofs of soundness, termination, and completeness relative to the verifier's abstraction (with exhaustive search); (3) exact circuit patch compilation for DWCs.

Validation covers three continuous-control benchmarks: Adaptive Cruise Control, Docking2D, and Car2D. ReachECO certified all controllers, including infinite-horizon safety on ACC. Repairs were highly localized (under 10⁻³% of policy cells on Docking2D; 42 cells on Car2D), and mean reward actually improved on ACC. Compared to supervised fine-tuning and STL-Gym baselines, ReachECO was faster and changed behavior less, and was the only method yielding certificates without knowing the true losing set.

Critical strengths

1. Constraint-based repair lifted to the closed-loop setting.

ReachECO derives repair constraints from failed multi-step reachability rather than single-step counterexamples, correctly handling danger established many steps before violation. Lazy backtracking with incrementally growing safe horizons is principled and avoids the intractable exact losing-set computation that blocks local methods.

2. Formal guarantees on the repair procedure itself.

Soundness, termination, and abstraction-relative completeness (Theorems 1–3) are proved, unlike fine-tuning-based repair offering no progress guarantees. The anchored-constraint discipline driving conflict resolution is what makes termination provable, addressing a genuine gap in verification-guided repair.

3. Exact, minimal, verifiable circuit patches for DWCs.

Repairs compile to bit-exact LUT/BRAM overlays, so the patched controller stays inside the verifier's supported class — precisely where REASSURE-style patches break certification. Empirically striking: <10⁻³% of cells modified, ~223 LUT6s, 8% policy change, versus 96.9% for supervised repair.

Critical weaknesses

1. Guarantees are relative to a coarse, possibly unsound abstraction.

Certificates inherit over-approximation error, and the GPU-specific Post/Tube operators are only informally asserted sound (Appendix C); a bug there voids Theorem 1. Worse, all repair-side reasoning uses under-approximated losing sets, so UNKNOWN carries no impossibility content — the completeness claim is narrow and mostly aspirational.

2. Scale and generality remain unestablished.

Only ≤3-dimensional benchmarks, 3–5 seeds; the largest ACC setting already demands depth-11+ continuation games, infeasible for the general backend — forcing reliance on single-action braking or oracle L*. Theorems 2–3 hold only for fixed L=U without merging, the very abstractions the implementation depends on. Scalability shall be considered for practical use.

Questions for authors

1. Soundness of the concrete Post/Tube operators (clarification).

Theorem 1 presumes sound over-approximations. Please report whether the GPU Post and inter-sample Tube computations were validated against an independent tool or verified analytically, and whether certificates survive tighter, general tube constructions (e.g., Taylor models). If soundness rests on unaudited code, how much of your acceptance case remains?

2. Gap between proved and implemented guarantees (clarification).

Theorems 2–3 assume fixed L=U and no merging, yet experiments rely on losing-set expansion and merging. Does the implemented configuration admit any guarantee beyond Theorem 1, and how often do merging-induced spurious conflicts or lost continuations trigger UNKNOWN?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.