acceptodds
Under review as a conference paper at ICLR 2027

AI Harness: Certification under Proposal-Conditioned Information for Foundation-Model Agents

Abstract

Foundation-model agents are often modeled as policies over an observed state. In deployed systems, however, a runtime may intervene only after the model has emitted a semantic proposal, making the proposal both an action candidate and a decision-time observation generated by a history-conditioned process. We show that collapsing this structure into a state-only proposal envelope can preserve proposal coverage while destroying certifiability. In a finite robust interface, the viability kernel of the collapsed model is contained in the physical projection of the history-augmented kernel, and the collapse is lossless exactly when every proposal-conditioned collapsed fiber retains a common robust-safe intervention. This gap can be maximal even with constant-size proposal and history alphabets. The same common-action condition yields a dual result: observing the current proposal can restore robust feasibility when it separates latent modes requiring incompatible interventions. We extend these one-step results over time using exact finite beliefs and standard safety and reachability fixed points, separating indefinite operational viability from finite worst-case verified progress. Controlled model-in-the-loop tests reproduce the predicted obstructions when telemetry or effect verification is removed or intervention authority is restricted. Thus, our contribution is not a new fixed-point calculus, but a characterization of when proposal–history correlation at the model–tool boundary is necessary for certification.

Primary area:
learning theory
Keywords:
Foundation-model agents, runtime assurance, formal verification, viability theory, proposal-conditioned control, partial observability

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_Y3pTOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

(I think) the paper puts forward theoretical considerations that I would place close to work in abstraction to establish whether for every proposal made by a model there is an intervention that's robust with respect to uncertainty. In a nutshell the key point put forward is the limitations implied by a state-only harness.

I appreciate in general the line of research here pursued and I think there is merit in these observations. However I have concerns in terms of:

  • The clarity of the contribution made;
  • Some of the experimental results (see below)
Critical strengths

Analysing the limits of state-based harnesses in history dependent models and traces is I think novel and potentially significant. The machinery put forward seems to provide a good leverage to understand the resulting properties.

Critical weaknesses

I have concerns about the formulation and clarity of the results. These concerns start from the abstract. I would encourage the authors to be much more clear in terms of what is being claimed and avoid negative sentences eg in the abstract to denote their contributions.

While I appreciate the general direction of research, it seems to me that at a higher level the paper states that for history-dependent runs a state-based harness inevitably will be insufficient in some key properties. This is consistent, even if not nearly directly following from, results in computation semantics of history-dependent processes and results in run-time monitoring including automata theory (which is unconnected to LLMs but related in this context)

Experimental results. Appendix B. It seems to me that full and no effect verification execute the same operations against the same environment trace until their first no-commit. Therefore the number of full-harness retries (in their terminology) should be fewer than the number of no effect verification retries that fail * number of retries. But in the table (page 24) this is not the case. What am I getting wrong?

My concerns above are conditioned by limited expertise on the subject matter.

Questions for authors

Appendix B. It seems to me that full and no effect verification execute the same operations against the same environment trace until their first no-commit. Therefore the number of full-harness retries (in their terminology) should be fewer than the number of no effect verification retries that fail * number of retries. But in the table (page 24) this is not the case. What am I getting wrong?

Reviewer_vgLAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The authors formalize how history-conditioned model proposals affect robust safety certification at the model–tool boundary, deriving conditions under which collapsing proposal–history correlations preserves or reduces the set of certifiably safe states. They extend the analysis to multi-step safety and reachability using finite beliefs and validate the predicted limitations through controlled model-in-the-loop tests.

Critical strengths
  • AI agent harnesses are an increasingly popular topic, referring to the software infrastructure and scaffolding that surround an LLM and enable it to translate its reasoning into reliable real-world actions.
  • The paper provides a theoretical formulation of the proposed method with several theorems and lemmas that can be generally used in AI agents.
Critical weaknesses
  • The paper provides insufficient background and related work to establish the state of the art in this area and clarify how its key contribution differs from prior work. The authors should more clearly position the paper’s contribution relative to existing approaches.
  • The evaluation is limited in scope, with only a small number of experimental results reported.
Questions for authors
  • How does the proposed method compare with state-of-the-art approaches in this domain? Please clarify its key differences and provide empirical evidence to support its claimed advantages.
Reviewer_sHa3OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper seeks to answer the question: Can we make AI agents operate safely even when the underlying LLM itself is probabilistic and unreliable. It proposes a way to control the harness so that we can provide mathematical guarantees for safety. The harness can do this by having a controller that verifies the proposed actions and decides whether to execute, modify, delay, or block the actions. In this work, the importance of memory of past actions and responses is studied: if the harness forgets whether some update was tried before, it might lose the ability to make safe choices. It also argues that the harness must see the complete proposed action by the AI. The full proposal can reveal a lot of information about the current state as understood by the system. Also the distinction between safety and task completion is also explored: we can trivially make an agentic system safe by having a harness that blocks every proposed action by AI, but it does nothing to make progress towards goals. The proposed framework first lists every possible situation the system can encounter. Then unsafe situations where any intervention by the harness cannot result in a safe state is removed. It results in a subset of situations where safety is possible but not yet guaranteed. Then another layer of refinement is applied to find the situations from which the harness can safely reach its goal and the rest are dropped. Experimentation performed on controlled database tasks showing that the framework allows for safe completion of tasks.

Critical strengths

Proposes a framework for safely reasoning about the controller/harness we put around an AI agent. This problem is an important one considering the fast evolving capabilities of frontier models and the challenge is controlling them from executing harmful actions. Formally establishes why the harness needs memory of previous actions and the full action proposal from LLM

Critical weaknesses

One of the main claims of the paper that memory is required is not clearly tested in the experiments. Several ablations are reported on removing telemetry, routing authority, and verification but memory related ablations appear not to be established The assumptions are strong and difficult to make in a practical scenario. The proposed framework requires the harness to know about all the proposals AI might make, the available interventions, and the possible set of outcomes. For a controlled experiment, we can define these but in an open ended problem it may be difficult Many sections of the paper appear to be AI generated and not written in a human digestible way.

Questions for authors

Justify the assumptions of the manuscirpt.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.