acceptodds
Under review as a conference paper at ICLR 2027

Trust the Monitor, Not the Model: Epistemic Capability Control for LLM Agents

Abstract

As LLM agents turn uncertain claims into memory, tool use, and external actions, hypotheses can silently acquire operational authority while uncertainty labels re- main entrusted to the same fallible models. We propose Epistemic Capability Con- trol (ECC), a structured runtime-authorization framework that separates semantic formalisation from deterministic verification. Natural-language policies and raw workflow context must first be mapped onto a common structured representation. Given these structured representations, ECC explicitly represents provenance, sensitivity, capability, epistemic status, evidence, support relations, and promo- tion history. A deterministic reference monitor mediates consequential actions, prevents ordinary model transformations from increasing authority, and permits authority increases only through admissible trusted promotions such as indepen- dent verification or human approval. ECC’s guarantees are therefore conditional on the structured policy and runtime state supplied to the monitor, rather than on the correctness of semantic interpretation itself. Our theory establishes non- expansion and structural-authorization invariants, shows that epistemic soundness is impossible from untrusted labels alone, and bounds residual over-authorization through requirement-relative threshold-crossing events. ECC thereby identifies a central reliability boundary: semantic formalisation remains learning-dependent, whereas formal authorization can be deterministic, reproducible, and auditable once the relevant state has been structured. On 480 held-out agent traces, ECC- Ref achieves 85.8% authorization accuracy with no decision flips under unsup- ported authority perturbations, while the learned formalisation interface exhibits a 26.25% authorization-relevant error rate.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM agents, agent safety, runtime governance, deterministic authorization, formal verification

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_FYpJOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a run-time authorization mechanism called Epistemic Capability Control (ECC). ECC separates aspects such as provenance and authority from evidence strength. An action is authorized if it is structurally eligible and if one complete witness satisfies every applicable epistemic requirement. The evaluation tests conformance to these rules on constructed structured inputs.

Critical strengths
  • The theoretical foundations are strong and clearly motivated. ECC makes explicit how permissions are granted (e.g., requiring one witness to satisfy all epistemic conditions), and proposition 1 (page 6) provides a clear reference.
  • The evaluation is appropriate and (even if only in appendices) it shows that it can replicate existing approaches like OPA/Rego and Cedar.
Critical weaknesses
  • There are some very strong underlying assumptions that should be clearly motivated. In particular, ECC assumes valid support coverage, sufficient-witness declarations, initial labels, action attributes, etc. While the paper acknowledges some of these assumptions, practical usefulness remains uncertain without a concrete account of who supplies these required inputs.

  • It is not clear to me what are the benefits over OPA/Rego or Cedar, especially considering the experimental results in the appendix. The paper should include a discussion of this, probably in section 2. There are also some presentation issues, with notation used but only defined in appendix.

Questions for authors

In the case of very large witnesses families, is there a mechanism to avoid the explicit enumeration?

Reviewer_fsgyOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Epistemic Capability Control, a formalism for enforcing formal protocols to monitor and authorize agent actions. It proves a few routine properties of the formalism such as soundness and non-expansion of dependency epistemic status. It applies this formalism on synthetic datasets to study its behavior and compare with some baseline methods including RF, MLP, LLM judge. These results shows when implementation and formalization can be done reliable, the monitor is reliable.

Critical strengths
  • The high-level insight is valuable, that formal methods can be useful in monitoring and authorizing agent actions (this is not clearly novel), but the formalism (when deciphered) looks new to me and very expressive
Critical weaknesses
  • I find the formalism unnecessarily complicated and not presented in a natural way. There are many mathematical notations used without clear purpose. Decompositions of entities into many components (e.g. eq. 1, eq. 6) are defined without explaining what each component is and how they can obtained. They can be useful in guiding operations, but when presented as-is, they make the formalism rather inaccessible, and more complex than it really is. Such complexity are not justified by theory, since they are useful in deriving any theoretical results. I would rather keep much of the decompositions informal than directly embeded into the formalism. Another example is Table 6, the meaning of each rank in each dimension can only be inferred from their literal meaining. If so neither humans nor LLMs can operationalize them consistently.
  • Using formal methods in agent action monitoring and authorization is not novel. The theoretical results are direct consequences enforced by constructions, hence I believe these are not the core contribution. The framework would be significant only if it provides useful guidance in actual implementation of the monitoring system. But I find it hard to perceive how the empirical results in the paper can sufficiently justify such usefulness. There are many degrees of freedom when applying the algorithm in operation, and much of the efficacy depends on these actual implementations and formalizations. While I am sympathetic to the formal method in general and understand the empirical evaluation for such papers can be difficult, I believe more can be done than is available. Table 20 as far as I can understanding provides the most informative comparison and shows the challenge of formalization itself when the correct formal input to ECC is not given. The modest performance shown in the middle row is a serious issue to me and is how the end-to-end procedure should improve on. The experiments in the main text, in contrast, are much less informative, which is made worse by the lack of implementation detail (configurations, hyperparameters) of the baseline methods. They are essentially verifying what are already constructed to hold.
Questions for authors
  • Could you please provide a glossary of the mathematical notations, clearly specifying their type signatures and intended meaning?
  • Will the codebase and generated datasets be made available at some point?
  • Could you please explain why Table 20's "ECC with reference inputs" does not have perfect scores? Are the datasets imperfect or the rules (possibly deliberately) fallible?
Reviewer_TxcWOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Epistemic Capability Control, a runtime authorization monitor for LLM agents. It checks whether an action has a complete supporting justification that meets policy requirements. It also restricts how evidence and authority can change during claim reuse and trusted updates. Experiments mainly evaluate authorization decisions on structured inputs.

Critical strengths

1.The complete-witness rule clearly explains why combining incomplete justifications can lead to incorrect decisions.

2.Typed updates usefully distinguish new evidence from approval.

  1. The paper states its assumptions explicitly and separates monitor errors from input formalization errors.x
Critical weaknesses
  1. The method relies on correct labels, support relations, and trusted updates. Obtaining these inputs is a major practical challenge that the monitor itself does not solve. 2, Performance drops substantially when structured inputs are extracted from raw text. The low legitimate-allow recall raises concerns about practical usefulness.
  2. The main comparisons emphasize learned models that approximate rules, while ECC executes the rules directly. These results provide limited evidence of an advantage over equivalent symbolic policy systems.
  3. Most experiments evaluate structured authorization decisions rather than live agent execution. They provide limited evidence about task completion, recovery, and enforcement in realistic workflows.
Questions for authors
  1. What are the main causes of the low legitimate-allow recall after formalization in Table 20?
  2. Beyond direct rule execution, what is the main practical advantage of ECC over an equivalent symbolic policy engine?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.