acceptodds
Under review as a conference paper at ICLR 2027

Beyond Trust: Proof-Carrying Generation and Independent Release Assurance for Multi-Agent LLMs

Abstract

Multi-agent LLM systems act across trust boundaries, yet downstream consumers cannot independently verify *why* an output was released or how often released outputs fail. We introduce **PCG-MAS**, a proof-carrying release-assurance framework with two certificates. A *DecisionCertificate* binds each release to a consumer-derived acceptance scope, exact execution and policy obligations, semantic checks, and replay metadata, so a producer that curates what the checker sees cannot force acceptance. An *EpochRiskCertificate* binds the complete committed release stream to an independent risk-limiting audit. Across seven models and eight factual and agentic datasets, we test controlled-coverage acceptance, exhaustive curation attacks, log-bypass attacks, weighted auditing, replay, and cost. Relative to frozen primary comparators, the worst-case released-failure difference under selective- view curation is −3.1 pp and the natural released-failure difference at matched coverage is −1.3 pp; the governed population records 0 unlogged consequential releases, and the pooled epoch audit reaches CERTIFY with a 36.7% label saving over uniform auditing.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Large Language Model Agents, Multi-Agent Systems, Proof-Carrying Generation, Agentic AI Safety, Runtime Verification, Replayable Execution, Grounded Generation, Evidence Provenance, Tool-Using Agents, Verifiable AI

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Ge38OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper aims to present release-assurance framework for multi-agent LLM systems dubbed PCG-MAS, which ams to provide assurance to a downstream users about the system’s outputs. They evaluate seven model families on eight datasets spanning factual reasoning, web interaction, function calling, and adversarial agents. The authors claim improvement on several metrics for multi-agent systems.

Critical strengths

None

Critical weaknesses

The paper has a main drawback in its disjointedness between the narrative, background and explaining results.

The formalization of the paper has multiple weaknesses. Some of the key terms are not defined, or are loosely coupled with other terms in the main text.

First, the paper does not precisely define the “release-assurance framework” which is the key topic of this paper. While a reader can discern what it may mean from the narrative, this point should be fully clear.

Figure 1 is fully out of place. It cannot serve as a summary of the framework and the contribution. Most of the terms in Figure 1 are not defined, or are defined only later in the paper. As such, a reader cannot get a proper understanding of the results. This figure should be presented only in the Evaluation section.

“Release stream” is not defined before use in the introduction. Same with “committed epochs”. The scope of the audit is not clear until Section 3, and even there the definitions are never explicitly connected.

Evaluation is not providing clear insights, just numbers. The evaluation scope in the introduction is just a laundry list of models and settings that were evaluated, without a clear insight of the results. Similarly, the contributions are failing to clearly state the novelty of this work for the broader ML community.

The definitions in Section 2 also suffer from “impatient formalization”, which fails to describe all the terms in the formulas before they are being used. Let us take the definitions 1 and 2 for example

D1:

  • ‘Typed’ in the typed graph is not clearly defined
  • Evidence nodes carry information that is not defined, neither the functions h and H or the function x have been defined at this time.
  • Other terms, E, \tau, attracted are not being defined.

D2:

  • The details of the consumer contract are not given
  • consumer scope is not properly defined in D3 (it just refers to the opaque Derive_\Omega term).
  • Producer witness and all remaining terms are either not defined at all or are defined much later in the text.

In either case, the missing parts of definitions make the remaining formalization difficult to understand and appreciate.

The evaluation is similarly difficult to comprehend. It just lists various statistics without putting them in the context. Each RQ is description contains only a 2-3 sentences, that neither explain the setup, the form of the plots/tables nor the insights. The choice of benchmarks and the metrics are also missing a clear justification.

Finally, the related work is missing from the paper and fully relegated to the appendix. This also makes it hard to understand the novelty of this paper compared to the state-of-the-art.

These sections confirm the underlying issue of the paper: it aims to put together too many concepts and results, that it fails to explain any of them clearly. As a result, the paper is not self contained.

Overall, the paper is not ready for publication at this time.

Questions for authors

What is a precise working definition of release-assurance framework used in this paper?

Several plots have the labels “Derived”, “Direct”, and “Protocol”. What is the meaning of these labels and how they differ?

Can you provide the justification why these benchmarks were selected?

Reviewer_14EXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PCG-MAS, a release-assurance framework for multi-agent LLM systems. It introduces two certificates: (1) a DecisionCertificate records the evidence scope, execution checks, policy requirements, and replay information behind an individual release; (2) an EpochRiskCertificate supports a statistical statement about the failure rate of a committed release population through independent auditing. The consumer determines the admissible evidence scope, preventing a producer from obtaining acceptance simply by hiding unfavorable evidence. The evaluation covers seven model families and eight datasets, including curation attacks, release-log bypass attempts and auditing costs.

Critical strengths

S1. The integration of consumer-controlled acceptance and independent population auditing is a distinctive contribution compared with existing works. Earlier work provides proof-carrying LLM certificates and risk-limiting statistical audits separately. This paper connects them through a committed release stream, addressing both what justified an individual release and what can be established about the released population. This is a useful architectural contribution beyond adding another factuality or policy checker.

S2. The provided formal analysis and experimental design help to address relevant failure boundaries. The selective-view argument explains why checking only producer-selected evidence can conflict with sensitivity to refutation, and the consumer-derived scope provides a clear response within the declared threat model. The evaluation is also extensive, including matched-coverage comparisons, component ablations, bypass attacks, cross-host recomputation, and deliberate violations of audit assumptions.

Critical weaknesses

W1. The reported certification outcomes need reconciliation with the reported failure rates. Table 3 reports CERTIFY for all four task-family epochs and the pooled epoch at a 5% failure threshold. However, Table 30 reports 6.7% released failure for both PCG-MAS audit configurations, using the same pooled audit-label counts. Figure 1 also reports a natural released-failure risk of 8.3%. It can be explained by the fact that different populations, weighting schemes or endpoints would lead to some differences, but still their relationship is unclear. A risk-limiting audit can occasionally certify an above-threshold population, so this is not by itself a contradiction of the theorem. Nevertheless, the paper still needs to establish whether these are successful certifications or observed false certifications.

W2. The central audit guarantee is stated more precisely than the implemented audit is specified. In Theorem 4.1, it is assumed that the sampling process and sequential test already constitute a valid level-delta test. Its proof then correctly invokes that validity property. Appendix E.8 describes the audit records and required conditions, but does not clearly identify the exact test statistic or confidence sequence, its update rule, and the construction used for sampling. Since independent population assurance is a primary contribution, these details are in need of further assess on whether the implemented procedure satisfies the theorem’s premise.

Questions for authors

Q1. What are the realized failure count, population size, and failure definition for each epoch in Table 3? Please explain how its certifications at a 5% threshold relate to the 6.7% failure rate in Table 30 and the 8.3% natural risk in Figure 1.

Q2. Which exact sequential test or confidence-sequence construction is implemented? Please specify its update equation, stopping rule, weighting procedure, and the result establishing validity under the actual conditional sampling law.

If these questions especially Q2 (as it decides how solid the certification goes in practice) can be properly resolved, I'd like to raise to a weak accept at least.

Reviewer_8voUOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PCG-MAS, a framework for independently verifying and auditing outputs from multi-agent LLM systems. It combines consumer-defined evidence scopes and per-output release certificates with sampling-based audits of the released population. Experiments across eight benchmarks and seven generator models report reduced vulnerability to selective evidence presentation, lower failure rates among accepted outputs, and lower audit labeling costs.

Critical strengths
  1. Consumer-defined evidence scopes and independent release checks address the risk of agents presenting only favorable evidence.
  2. Experiments across eight benchmarks and seven generator models report lower failure rates at comparable acceptance rates and reduced audit labeling costs.
Critical weaknesses
  1. The main-text organization needs improvement. Related work is briefly addressed in the introduction and largely deferred to the appendix, while the conclusion is embedded in the experimental section. Dedicated sections would improve readability.
  2. The construction of acceptance contracts and evidence scopes is insufficiently specified. It is unclear whether they are manually designed or LLM-generated, and how their completeness and validity are assessed. Because these definitions determine what the system accepts, their quality is central to the claimed assurance.
  3. The choice of an 80% target coverage is insufficiently justified. Results at different coverage levels, ideally risk–coverage curves, are needed to assess whether the reported gains persist beyond this operating point. This limits assessment of the practical trade-off between reliability and task completion.
  4. The semantic acceptance checker's model and checkpoint are not explicitly identified in the paper, leaving a key experimental component insufficiently specified. This limits reproducibility and makes it difficult to assess how much the results depend on the chosen checker.
Questions for authors
  1. How are acceptance contracts and evidence scopes constructed for each task? Are they manually specified or LLM-generated?
  2. Beyond the reported scope-removal ablation, how much do different consumer-defined evidence scopes affect the results?
  3. What motivates the 80% target coverage? Could you provide risk–coverage curves to show whether the reported gains persist at other coverage levels?
  4. Which exact NLI model and checkpoint/revision are used for semantic acceptance checks? Is the same checker used across all generator models and datasets?

Comments on the paper

1 comment
Area_Chair_5S7KOpenReview ↗

Last 24 hours of the reviewer discussion period

Dear Authors and Reviewers

Just a reminder we are soon to be entering the last 24 hours of the reviewer discussion period.

Authors: If you have not yet responded to the reviews and intend to please do so ASAP. Please note reviewers will most likely not have time to respond to any late rebuttal comments so please make sure any comments are clear and not in need of further clarification.

Reviewers: If the authors have responded to your reviews and you have not yet responded please do so, even if it is a one line response acknowledging you have read the rebuttal.

Thanks for all your time, and I hope you all learnt something interesting during the process.

Kind regards,

Your ICLR Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.