acceptodds
Under review as a conference paper at ICLR 2027

Certify, Then Act: Formal Safety Certificates for Cooperative LLM Agents in Stochastic Environments

Abstract

Large language model (LLM) agents are increasingly asked to act, to move robots, edit live systems, and coordinate with other agents, rather than merely to answer questions. When deploying these systems in safety critical settings, average-case competence is not enough: a single low-probability unsafe action can be costly or irreversible. We propose a protocol whereby LLM agents can obtain and respect formal safety certificates in stochastic, cooperative, multi-agent environments. Specifically, we introduce the certify-then-act protocol in which independent LLM agents negotiate a joint contract, certify it using a sound probabilistic model checker that computes the probability of the contract leading to an unsafe state, and execute it only once that probability falls below a pre-specified threshold. This process is then repeated whenever random disturbances void the certificate. We instantiate the protocol in two stochastic cooperative domains, (a windy Gridworld and a collapsing Blocksworld), spanning six scenarios, and run a full factorial study over certification regime, communication, and base model. We find that (i) communication is highly beneficial; given communication it is enforcement rather than availability of the verifier that buys safety, with unsafe terminations falling from 17.3% to 0.0%; (ii) there is a trade off between guarantees of safety and task completion within a given budget; and (iii) models show a clean capability gradient. Certification comes with extra costs in additional actions and tokens, however, without these extra resources the enforced agents time out maintaining safety rather than fail unsafely. Additionally, we show heterogeneous pairs typically match or exceed their members' individual averages. We believe our results indicate equipping LLM agents with model checking tools presents an essential step toward deploying AI in safety critical Multi-Agent Systems.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
LLM Agents, Agentic Safety, Formal Methods, multi-agent systems

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_mAoAOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a certify-then-act protocol for cooperative LLM agents in stochastic environments. Agents negotiate a finite sequence of joint actions, obtain a probabilistic safety certificate from a model checker, and execute the contract through an enforcement layer. Following a disruption, they negotiate and certify a replacement contract. The evaluation covers a few scenarios and models, with additional experiments on heterogeneous teams, larger budgets and increasing agent counts. The main finding is that enforced certification substantially reduces unsafe terminations, while increasing timeouts under a fixed budget.

Critical strengths

Separating LLM planning from formal verification and runtime enforcement provides a clear way to constrain unreliable action proposals. Joint contracts address interactions that individual agents cannot assess independently.

Comparing absent, optional and enforced verification helps distinguish access to a verifier from an obligation to respect its verdict. The homogeneous study covers a large number of episodes, and the heterogeneous experiments broaden the evaluation beyond identical agents.

Distinguishing unsafe failures from timeouts is valuable and honest. The budget ablation also provides evidence that some completion losses can be recovered by allowing more resources. The authors explicitly acknowledge the assumptions of an accurate environment model, perfect information and cooperative agents.

Critical weaknesses

My main concern is the scalability of the approach. The main evaluation uses six small scenarios with hand-built models. This is reasonable for an initial study, but it leaves substantial uncertainty about larger, more complex problems and time-sensitive operation. Moreover, increasing the number of agents also increases the number of available grippers, which can make task execution easier. The experiment needs to separate additional execution capacity from increasing problem complexity and verification costs.

Related to the above, when dealing with large and complex problems, how to abstract the problem into a manageable model and how to ensure that the abstraction is faithful and useful are critical problems that this paper does not seem to address.

Probabilistic shielding and multi-agent shielding already establish closely related methods for constraining actions using formal models (previous work cited by the paper). The main methodological contribution appears to be integrating these ideas with LLM-negotiated joint contracts. The experiments compare verification regimes within the proposed harness, without establishing the advantage of contracts over a simpler enforced shield.

The paper often correctly limits its guarantee to the duration of a bound contract. Repeatedly accepting contracts with violation probability does not, however, imply an episode-level violation probability. For example, ten segments with independent failure probabilities of 0.05 would produce an overall failure probability of approximately 0.40. CTSR establishes that executed actions were certified; its definition alone does not establish a single risk bound for the complete episode. The paper should state the resulting guarantee across successive contracts and distinguish this explicitly from certification coverage.

Each model–domain–mode cell contains only 15 episodes, drawn from three scenarios. The large reduction in unsafe outcomes is persuasive within the benchmark, but the precise model ordering and heterogeneous-team advantages are less secure.

Questions for authors

What guarantee holds over an entire episode containing multiple certified contracts? Is there an episode-level risk budget?

Does the checked Markov chain include the actual disruption-triggered cancellation and waiting behaviour?

How does multi-step contract certification compare with enforced one-step joint-action certification in safety, completion and verification cost?

What are the model sizes and verification latencies, and how do these change with contract horizon and agent count? How do the proposed method scale to large and complex models and problems?

In the six-agent experiment, how much of the observed performance comes from additional grippers rather than scalable coordination?

6 comments
AuthorsOpenReview ↗

Verification latency

Verification latency - Response to R3 (mAoA) We agree that the cost of verification is important for assessing practicality, and we will run a controlled scalability study of our verifier and share the results in a follow-up post. This study will use the same Storm-based model checker that the agents call at run time. Each candidate joint contract will be compiled into a discrete-time Markov chain that captures the stochastic wind dynamics and the other agents' moves, and Storm will compute the probability of an obstacle hit or collision within the contract horizon. We will time this check on randomly generated grid worlds with 15% obstacle density, varying (i) grid size from 5×5 to 30×30, (ii) contract length from 4 to 20 steps, and (iii) the number of agents. For each setting we will check randomly sampled contracts across several seeds and report median wall-clock time, along with the size of the generated model and a breakdown into model construction and model checking. Because Storm uses sparse representations, we will report latency and model size separately, without assuming one predicts the other. We will also time the checks on our benchmark scenarios at their actual horizons, so that the results can be read directly against the settings used in our experiments. We will include the full results as tables in a follow-up post and in the final version of the paper.

AuthorsOpenReview ↗

Separating execution capacity from coordination cost in the multi-agent experiment

Separating execution capacity from coordination cost in the multi-agent experiment - Response to R3 (mAoA) We thank the reviewer for raising this issue. We agree that for k≤4k \leq 4, additional agents add both execution capacity and coordination burden, so the improvement in TSR and CTSR from k=2k=2 to k=4k=4 cannot be attributed to scalable coordination alone. However, the 4_rotate task was designed so that the k=5k=5 and k=6k=6 settings separate the two effects (Appendix C). Only four legal moves exist at the first step, and the task has a deterministic optimum of 4 joint steps for every k≥4k \geq 4. Agents beyond the fourth therefore cannot contribute any parallelism: they can only wait or contend for blocks other agents need. Going from k=4k=4 to k=6k=6 thus holds useful execution capacity fixed while increasing the number of agents that must negotiate, agree on a byte-identical kk-wide contract, and be covered by each certificate. This isolates coordination and verification overhead.

In this range, adding agents provides no benefit, and the cost of coordination is visible:

  • Under enforced certification, GLM 5.2 maintains a CTSR of 1.0 at k=5k=5 and k=6k=6, while DeepSeek V4 Flash drops from 1.0 to 0.8 (Figure 6).
  • Game steps do not fall further beyond k=4k=4, while model turns continue to rise (Figure 7), and total input tokens grow from ~2.2M at k=2k=2 to ~6.6M at k=6k=6 (Appendix C.1).
  • Under optional certification, DeepSeek V4 Flash's TSR falls as kk increases beyond 3, because uncoordinated agents act before a contract is agreed (Appendix C).

So the protocol's safety and success hold up once extra agents stop adding capacity, and the overhead of more agents appears in inference cost rather than in unsafe outcomes. We note that each point is computed over five episodes, so the drop from 1.0 to 0.8 corresponds to a single episode; we read these results as evidence that certified success does not collapse with additional agents.

AuthorsOpenReview ↗

Does the checked Markov chain include cancellation and waiting?

Does the checked Markov chain include cancellation and waiting? - Response to R3 (mAoA) No. The checker verifies the chain induced by executing the full contract (Definition 5). This is sound: release on disruption only truncates the contract, and a prefix's violation probability is at most that of the full contract. After a release, agents wait until a new certified contract is bound, and wait is deterministic and never disrupted. Without communication, agents still executing their own contracts were certified against any behavior of the others, including waiting. The per-contract and episode-level bounds therefore cover cancellation and waiting.

Reviewer_hq8COpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes the certify-then-act protocol for decentralized, cooperative multi-agent LLM systems operating in stochastic environments. Agents negotiate multi-step joint action contracts, verify them against an avoid-only safety specification using an exact probabilistic model checker, and execute them via auto-play until stochastic disruptions trigger renegotiation. Across two custom benchmarks (Gridworld and Blocksworld), the authors show that strict runtime enforcement, rather than optional tool availability, is necessary to eliminate catastrophic failures (reducing unsafe terminations from 17.3% to 0.0%), trading execution speed for safe timeouts.

Critical strengths
  • Effectively pairs natural-language decentralized planning with sound probabilistic model checking (PCTL), amortizing coordination costs per contract rather than per step.
  • Demonstrates conclusively that granting LLMs optional access to a verifier fails to prevent crashes (due to unconstrained post-disruption improvisation), whereas hard harness-level enforcement reliably guarantees safety.
  • Shows that pairing strong and weak models yields performance close to the stronger agent (as more capable models naturally lead contract proposal) and demonstrates that enforced certification scales to k=6k=6 agents.
  • The dual-domain benchmarks (windy Gridworld, collapsing Blocksworld) isolate coupled multi-agent safety, stochastic contract invalidation, and exact model-checking tractability without visual perception confounds.
Critical weaknesses
  • While the paper critiques single-step runtime shields for horizon myopia and synchronization overhead, it does not evaluate an empirical H=1H=1 baseline using the same STORM verifier to quantify the exact completion and efficiency benefits of multi-step contracts.
  • The safety certificate λ\lambda bounds the violation probability strictly over the horizon of a single contract segment, meaning cumulative episode risk over MM chained segments scales up to MλM\lambda.
  • Several system prompts contain copy-paste discrepancies that contradict the actual mode (e.g., instructing agents to "re-certify" in off modes, or to "rebind" in optional ×\times none where the tool is withheld).
Questions for authors
  • How does the multi-step contract protocol compare empirically against an enforced H=1H=1 single-step probabilistic shield in terms of success rate, timeouts, and token overhead?
  • Did the contradictory prompt instructions in the ablation baselines (e.g., advising unassisted agents in off to re-certify) degrade the baseline agents' performance?
  • In heterogeneous pairings, how much of the contract authorship dominance was driven by the fixed first-seat turn order versus model capability?
3 comments
AuthorsOpenReview ↗

Seat order vs capability in contract authorship

Seat order vs capability in contract authorship - Response to R2 (hq8C) We thank the reviewer for this question. Table 19 already provides partial evidence that capability, not seat order, drives authorship. In the DeepSeek V4 Flash + GLM 5.2 pairing, GLM 5.2 holds the second seat, which should favor DeepSeek, yet GLM 5.2 authors 74% of bound contracts under enforcement, with an interval clear of 50%. In the Sonnet + Mistral pairing, however, the stronger model also holds the first seat, so the two effects are confounded there. To separate them directly, we are rerunning the heterogeneous pairings with the seat order swapped, on the corrected harness described in our response on prompt discrepancies. We will report the results, including authorship shares under both seat orders, in a follow-up comment once they are complete.

We thank the reviewer for their careful reading and constructive comments. Most of the points raised are shared with other reviewers, so we have addressed them in general comments above:

  • Single-step shield baseline (W1, Q1): see Comparison with single-step shielding. We are running an enforced H=1H=1 ablation and will report the results in a follow-up comment.
  • Episode-level risk (W2): see Episode-level guarantees, where we state the resulting bound, P(unsafe)≤1−(1−λ)n≤nλP(\text{unsafe}) \leq 1-(1-\lambda)^n \leq n\lambda, and show how the protocol can be extended to a fixed episode-level budget.
  • Prompt discrepancies (W3, Q2): see Prompt discrepancies and statistical power. We have fixed these at their source and rerun the full homogeneous study; the main findings are unchanged.

Our response to the question on seat order versus capability in contract authorship (Q3) is above. We are running one of heterogeneous pairings with the seat order swapped and will report the results in a follow-up comment.

AuthorsOpenReview ↗

Authorship experiment results

We reran the hetero experiment with sonnet and mistral with both models in the first seat. The results are summarized below.

Authorship by seat: share of bound contracts first proposed by Sonnet 5.5 (vs Mistral Small), with the 95% Confident interval shown in square brackets. Authorship is the first agent to put forward an exact contract that was later bound.There are 120 episodes per seating, and CIs are over episodes.

SeatingStronger (seat)offoptionalenforced
Sonnet + MistralSonnet (first)92% [86–96], n=12291% [86–96], n=15395% [92–99], n=155
Mistral + SonnetSonnet (second)76% [67–84], n=11985% [79–91], n=14783% [77–88], n=155

Moving Sonnet to the second seat lowers its authorship by 7 to 16 percentage points, but it still writes the large majority of contracts from either seat. For this pairing model strength is a lot more important to authorship than seat order but both have effects. If the reviewer would find it interesting we can run another reversed pairing also.

Reviewer_zPeMOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors propose Certify-Then-Act, a protocol for safe collaboration among multiple LLM agents, where safety is defined as avoiding predefined unsafe states, such as collisions between agents or with obstacles. In this protocol, agents first communicate to negotiate a joint plan, referred to as a contract, represented as a list of lists of action strings. The contract is then submitted to a probabilistic model checker, which assumes an accurate mathematical representation of the environment and certifies the contract if the probability of reaching an unsafe state does not exceed a predefined threshold. Once certified, all agents commit to the same contract, and its execution is enforced by the runtime framework. If an unexpected environmental disruption, such as wind displacing an agent, invalidates the contract during execution, the agents must renegotiate, certify, and commit to a new contract before continuing.

The authors evaluate their approach in two simulated environments, Gridworld and Blocksworld, where agents must navigate around obstacles or rearrange blocks while avoiding collisions and unexpected environmental disruptions. The experiments cover six scenarios and five LLMs, comparing configurations with and without agent communication and three levels of certification: disabled, optional, and enforced. The results show that with agent communication, enforced certification reduces unsafe outcomes from 17.3% without certification to 0%.

Critical strengths

Interesting application of probabilistic model checking to multi-agent coordination. The authors integrate probabilistic model checking into a protocol where LLM agents negotiate joint plans, check their safety, and execute them under enforced constraints.

Evaluation with both homogeneous and heterogeneous agents.

Investigation of the safety–utility trade-off and systematic evaluation of different configuration of their Certify-Then-Act framework.

Critical weaknesses

Limited evaluation scenarios. The evaluation primarily compares different configurations of the proposed framework across two relatively simple simulated environments.

Questions for authors

Enforced certification reduces task completion rates by a noticeable chunk, although increasing the execution budget mitigates this effect. While the authors attribute this primarily to additional negotiation and verification overhead, it would also be interesting to examine whether the certification process leads agents to generate overly conservative or suboptimal plans, further contributing to timeouts.

3 comments
AuthorsOpenReview ↗

Does certification lead to overly conservative plans?

Does certification lead to overly conservative plans? - Response to R1 (zPeM) We thank the reviewer for this question. The additional timeouts under enforcement could come from two sources: more conservative plans, which would show up as shorter contracts or more movement steps, and negotiation and verification overhead, which shows up as non-movement steps. Our results allow us to separate the two. Averaged over the five models with communication:

  • Trajectories do differ, but certified contracts are not overly conservative. Certification does change the routes agents take, particularly in Gridworld, where enforced agents tend to keep wider margins from obstacles and from each other (Figure 1). However, this costs little: bound contracts average 3.7 steps under enforced against 4.9 under off in Gridworld, and 4.5 against 5.4 in Blocksworld, about one step shorter; against optional the difference is about half a step. Total movement per episode also changes little (about 3 extra steps in Gridworld, about 1 fewer in Blocksworld).
  • Overhead dominates. In Gridworld, total game steps nearly double (19.9 to 38.5) and model turns rise from 23.7 to 63.1. Of the roughly 19 additional game steps, only about 3 are movement; the remainder are turns spent negotiating, model checking and committing.

The step-budget ablation also corroborates this: increasing the budget to 2$\times$ and 4$\times$ raises enforced certified success for DeepSeek V4 Flash, while unsafe outcomes across the ablation stay essentially unchanged (38, 39 and 41 of 220). While certification changes the trajectories agents take, it does not make them overly conservative; the additional timeouts are driven mainly by deliberation overhead, which can be reduced, for example, by a leader-proposes-and-others-ratify scheme (Appendix C.1) or simply by reducing the cost of these operations on the total step budget. We will add these metrics to the revised paper.

We thank the reviewer for their positive assessment and helpful comments. Several of the points raised are shared with other reviewers, so we have addressed them in general comments above:

  • Limited evaluation scenarios: see Scale of the evaluation and Scalability and obtaining faithful abstractions. Since submission we have added two harder scenarios and rerun the full study with 20 episodes per cell (see Prompt discrepancies and statistical power).
  • Comparing only configurations of our own framework: we are running an enforced single-step shield baseline; see Comparison with single-step shielding.

Our response to the reviewer's question on whether certification leads to overly conservative plans is above. In short, certified contracts are only modestly shorter, and the additional timeouts come mainly from deliberation overhead rather than from conservative plans.

Reviewer_zPeMOpenReview ↗

Response

I thank the authors for their efforts to address my comments. They all make sense to me.

Comments on the paper

7 comments
AuthorsOpenReview ↗

Episode-level guarantees

Episode-level guarantees - Response to R2 (hq8C) and R3 (mAoA) We thank both reviewers for raising this point. We agree that the protocol as presented does not provide an a priori episode-level bound on the probability of an unsafe outcome. We did not intend to claim one. Our guarantees are stated per contract throughout the paper (e.g. Section 1, the Section 6.1 takeaway, Definition 7), and Appendix D.3 notes explicitly that "the guarantee holds per contract, not per episode." We will make this distinction prominent in the main text, and we will qualify the sentence in Section 7 that could be read as an episode-level claim.

What the protocol does guarantee. Under enforced certification the protocol gives two guarantees:

  1. A per-contract guarantee. Each bound contract cc has violation probability V(s,c)≤λV(s,c) \leq \lambda from the state ss at which it was bound (Definition 7).
  2. An action-level guarantee. Every non-wait action executed in the episode belongs to a contract that was certified before it was bound (Definitions 11-12). CTSR measures this certification coverage. As R3 correctly notes, coverage is distinct from an episode-level risk bound.

The resulting episode-level bound. Each certificate is computed from the actual state at binding time. By the Markov property, the conditional safety probability of each segment, given the history, is therefore at least 1−λ1-\lambda. A truncated segment (released early by a disruption) is no riskier than the full contract. Under enforced certification no agent moves between contracts except by wait, which is deterministic and never disrupted. Hence, for an episode that binds nn contracts,

P(unsafe)≤1−(1−λ)n≤nλ.P(\text{unsafe}) \leq 1-(1-\lambda)^n \leq n\lambda.

For a random number of contracts NN, a union-bound argument gives P(unsafe)≤λ E[N]P(\text{unsafe}) \leq \lambda \, \mathbb{E}[N]. This matches R3's example (ten segments at λ=0.05\lambda = 0.05 giving ≈0.40\approx 0.40). We will add this derivation to the paper and report the empirical number of contracts per episode alongside CTSR.

Why do we not fix an episode-level budget a priori? In a stochastic environment, the number of contracts needed to complete a task is not known in advance. It depends on how many disruptions occur, and each disruption forces a re-certification. So no a priori allocation of an episode-level risk budget ε\varepsilon can guarantee that completion stays feasible under that budget. In unfavorable realizations the budget runs out before the goal is reached, and the agents must then stop safely rather than continue. The risk needed to complete a task is a property of the realized trajectory, not something that can be bounded beforehand. For this reason we chose a fixed per-contract threshold and report episode-level risk empirically: with communication, unsafe terminations fall to 0.5% (1/200) under enforcement, and every unsafe episode under enforcement occurred under a certified contract, i.e. within the per-contract threshold.

How to obtain an episode-level guarantee. The protocol extends naturally to an explicit episode-level bound ε\varepsilon, at the cost of possibly halting before completion:

  • Bounded number of contracts. Cap the episode at Nmax⁡N_{\max} contracts and certify each against λ=ε/Nmax⁡\lambda = \varepsilon / N_{\max} (or the slightly less conservative λ=1−(1−ε)1/Nmax⁡\lambda = 1-(1-\varepsilon)^{1/N_{\max}}). The bound above then gives P(unsafe)≤εP(\text{unsafe}) \leq \varepsilon.
  • Convergent threshold schedule. Remove the cap by certifying the kk-th contract against a threshold λk\lambda_k from a convergent series with ∑kλk≤ε\sum_{k} \lambda_k \leq \varepsilon, for example λk=ε 2−k\lambda_k = \varepsilon \, 2^{-k} or λk=6ε/(π2k2)\lambda_k = 6\varepsilon / (\pi^2 k^2). The union bound gives P(unsafe)≤∑kλk≤εP(\text{unsafe}) \leq \sum_{k} \lambda_k \leq \varepsilon for any number of contracts. Later contracts must be safer, so long or heavily disrupted episodes become progressively more conservative.

Both variants need no change to the verifier, only to the threshold passed to it. A refinement of either is to charge each contract its actual certified risk V(s,c)V(s,c) rather than λ\lambda, which spends less budget whenever contracts certify well below threshold. We will add this discussion as a new subsection and, budget permitting, an experiment with the bounded-contract variant.

AuthorsOpenReview ↗

Comparison with single-step shielding

Comparison with single-step shielding - Response to R2 (hq8C) and R3 (mAoA) We thank both reviewers for this suggestion. We agree that the paper currently argues for multi-step contracts over per-step shielding only conceptually (Section 2), without an empirical comparison against an enforced single-step shield using the same verifier. To address this, we are running an H=1H=1 ablation of our protocol and will report the results in a follow-up comment once they are complete. In this ablation we will rerun the enforced certification regime with contracts restricted to a single joint step. Every executed joint action must therefore be certified by the same STORM verifier at the same thresholds before it runs, which makes this an enforced one-step probabilistic shield. The harness, prompts, negotiation channel and verifier are held fixed, so the comparison varies only the contract horizon. This isolates the benefit of multi-step contracts from the benefit of having a sound verifier in the loop. We will report TSR, CTSR, unsafe-termination and timeout rates, game steps, model calls, tokens and cost, together with the number of verifier calls.

AuthorsOpenReview ↗

Scalability and obtaining faithful abstractions

Scalability and obtaining faithful abstractions - Response to R1 (zPeM) and R3 (mAoA) We agree that our approach requires a formal model of the environment, and that obtaining a faithful, tractable abstraction for large and complex problems is a central challenge. However, this limitation is not specific to our protocol. It is shared by every model-based safety method, including probabilistic shielding (Jansen et al., 2020), multi-agent shielding (ElSayed-Aly et al., 2021; Melcer et al., 2022) and the guaranteed-safe AI framework more broadly (Dalrymple et al., 2024), all of which assume a world model and a formal specification. In this work we deliberately hold the model fixed, so that we can study a question those methods leave open: whether independent LLM agents can negotiate, certify and act on such guarantees. How to obtain these models is an active and largely orthogonal line of research, and our protocol is designed to benefit directly from progress in it:

  • Autoformalization. LLMs are increasingly used to translate natural-language descriptions into formal artifacts that trusted solvers then check, including temporal-logic specifications (Chen et al., 2023; Fuggitti & Chakraborti, 2023) and planning models (Liu et al., 2023; Wu et al., 2022).
  • Learned models. Environment models can instead be estimated from execution traces, as ProbGuard (Wang et al., 2026b) does for LLM agents. Where transition probabilities are only known up to confidence intervals, interval MDPs yield sound but more conservative certificates (Appendix D.2).

Two features of our protocol make it well suited to these approaches. First, the verifier only checks a proposed contract over a short horizon HH. It never solves the full MDP, so a formal model is needed only for the states that contract can reach, which could be built on demand rather than for the entire domain up front (Appendix D.1). Second, only the safety specification needs to be formalized. Goals are given to the agents in natural language and never checked by the verifier. Safety constraints such as "do not collide" tend to be few, stable and shared across tasks, so formalizing them is far cheaper than formalizing every task. We will make these points more prominent in the main text.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.