acceptodds
Under review as a conference paper at ICLR 2027

TargetGuard: Freshness-Certified Target Integrity for State-Changing LLM Actions

Abstract

An API call can succeed while modifying the wrong entity. TargetGuard makes this distinction explicit through a target-integrity benchmark contract and a freshness-certified selective execution gate. Given a proposed action and a public typed relation, the gate admits a unique evidenced target only after revalidating its relation, pre-state, and executor-version certificate; otherwise it abstains. We establish conditional target integrity when the relation links candidates to the requested entity and the executor protects validation and mutation atomically. Observationally indistinguishable targets require abstention or additional evidence. Our primary comparison uses 192 controlled Git and SQLite cases. Raw dispatch records 96 wrong-target writes. Both TargetGuard and an independently implemented typed revalidation reference record none, retain all 96 uniquely identifiable operations, and abstain on all 96 ambiguous cases, without additional LLM calls. Both reject all 96 actions made stale before revalidation in a separate stratum. These tests measure stale-state detection, not protection against concurrent changes between validation and mutation. Forty-two application replay cases and an eight-case resolver boundary audit test adapter behavior. A controlled, oracle-assisted 12-pair intervention isolates target substitution under fixed model responses. A separate 1,800-case analytical suite compares policy decisions, not persisted writes. The result is a reusable contract for measuring correct-target execution and the valid work retained by execution policies. It does not establish an advantage over typed revalidation or estimate deployment failure prevalence.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Large Language Model Agents, Tool Use, Agent Safety, Runtime Verification, Selective Execution, Abstention, Entity Resolution, Benchmarks

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_CtQbOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies a failure mode in LLM systems where an entity mutating operation, such as delete(file_n) can be syntactically correct, execute without error, but be semantically wrong (it should have modified file_m). To address this they define TargetGuard, a selective execution algorithm for this failure mode. TargetGuard will judge the LLM action given additional inputs (typed target, runtime visible relation) and determine whether the action can be applied, or must be withheld (abstain). In the case that the entity was wrong, but the additional inputs uniquely determine the entity, the action is repaired.

TargetGuard is evaluated on a synthetic 192 case dataset involving Git and SQL operations, and correctly identifies the unique and ambiguous cases. 12 tasks from AppWorld are evaluated and demonstrate that the framework can correctly substitute entity ID's in API calls with oracle defined intended ID's.

Critical strengths

None

Critical weaknesses
  • The novelty and significance of the contribution are limited. TargetGuard is a small framework that given a sufficiently rich adapter input, applies basic relation operations to determine uniqueness of an entity. The adapter and runtime visible relation must be constructed by the consumer, often requiring intent inference that is close in complexity to solving the task. For instance an LLM action from the task "delete the file Dave edited last week" requires parsing the natural language into constraints that describe file editors and edit time. Thus the success of the system is primarily dependent on the adapter implementation or semantic inference, and therefore most of the design and implementation complexity of the system is deferred to the consumer.

  • The evaluation of TargetGuard focuses on correctness of the mechanism when integrated into a small set of fixtures. These are closer to unit or integration tests for the implementation, rather than an evaluation of the merit of such a system in an LLM-based system. The evaluation does not establish that TargetGuard can be used to improve the effectiveness or correctness of an existing live LLM system.

  • The presentation of the paper is overly complex and includes many esoteric terms. The paper presents theorems involving elementary proofs that do not add substance to the work. For example Theorem 2, that states that an incorrect choice will be made if identical resolver inputs can be constructed for different target entities, and the resolver does not abstain. This is intended to motivate abstention in TargetGuard, however abstention in TargetGuard applies when there are multiple candidate entities produced by the adapter. Therefore the correctness burden is entirely placed on the client producing inputs to the resolver that accurately reflect the ambiguity. In other words, if the ambiguity in the entity choice is not made explicit, TargetGuard will be incorrect. The theorem adds little to the paper, instead, the space would be better served by a fully worked example. The example in the introduction is too brief, and does not adequately illustrate what a 'public relation' would look like in a real setting. Given a worked example, it would be clearer that constructing the constraints and input to the resolver can require significant target identification logic.

Questions for authors

The abstract states: "Both TargetGuard and an independently implemented typed revalidation reference record none …"

Could you elaborate on this. How is the other implementation different? Does this not undermine the novelty of TargeGuard if another system behaves the same.

1 comment

Thank you for putting the adapter boundary at the center of the review. We agree that TargetGuard does not infer arbitrary intent from natural language. The adapter must expose a relation that connects the requested target to candidates; constructing that relation can require application-specific logic and may itself involve substantial semantic work. Our claim therefore applies only when an adapter can supply a sound relation under the stated contract. We will make this dependency and its limits central to the paper.

We also agree that the current controlled Git/SQLite evaluation demonstrates behavior on specified fixtures, not improved performance in a live, general-purpose LLM agent. We will describe it as a controlled evaluation and remove any implication that it establishes deployment prevalence or broad real-world effectiveness.

The typed revalidation reference achieves the same reported outcomes, which limits the algorithmic novelty claim. We will say so directly and position the contribution around the target-integrity measurement contract and the comparison of execution policies, rather than a demonstrated advantage over typed revalidation.

Your point about abstention is also important: in the evaluated setup, abstention prevents a mutation but leaves the requested task unfinished. We will state that explicitly. We will also expand the worked example so readers can see what information the adapter supplies and what TargetGuard itself decides.

Reviewer_7AifOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper reports an experimental study of the target-integrity failure in a state-changing LLM call against a wrong entity. It proposes a selective execution rule that admits an action only when a runtime-visible typed relation identifies a single evidenced candidate and otherwise refuses the action before any mutation. The experimental results show that the rule prevents wrong-target writes on the tested cases while retaining the work of the identifiable ones.

Critical strengths
  • A series of conditions under which a wrong-target write is prevented before any state change, based on a public relation available at runtime.
Critical weaknesses
  • The methodology is not presented systematically, but intertwined with the implementation and evaluations throughout the paper. The theorems read more like a workflow and are proved in a straightforward way, without carefully defined context.
  • The evaluations are not presented systematically, and the results are reported repeatedly, e.g., the 192-case comparison result. Moreover, this primary result rests on a synthesized dataset, and it is not clear how the work would generalize to real deployments.
  • A systematic comparison with the related work the paper itself discusses is missing, e.g., the closest conceptual priors [1,2]. The references are also cited in an unusual way.
Questions for authors
  1. How much of the paper is actually written or edited by the authors?
  2. Please use the (id_a, id_b) case as a running example to clarify the methodological design of this paper.
  3. What makes a wrong-target write occur at all when the claimed conditions hold, esp. across separate worlds?
  4. A non-dispatch outcome in this work is terminal. Where a wrong-target mutation is prevented, how is the intended mutation to proceed? A refusal without a continuation leaves the underlying task undone.
  5. In Table 1, it appears that the typed revalidation reference is good enough. How would TargetGuard outperform it?
2 comments

Thank you for the careful reading. We agree that the method, implementation, and evaluation are currently too intertwined. We will reorganize the presentation around one worked case, using the (ida,idb)(id_a,id_b) example, and distinguish the task specification, runtime-visible relation, policy decision, executor outcome, and post-state score.

On the question of when a wrong-target write can occur, the guarantee is conditional: it requires the adapter’s relation to be sound and sufficiently complete for the requested target, and it requires validation and mutation to be protected by the stated execution mechanism. Uniqueness alone does not establish that a candidate matches the user’s intent. We will make these assumptions explicit and avoid presenting the result as an open-world guarantee.

You are also right that abstention is terminal in the evaluated setup. It prevents that action from mutating state, but does not finish the underlying task. We have not evaluated a clarification or retry policy, so we will state that limitation rather than imply that abstention preserves task completion.

The typed revalidation reference matches TargetGuard on the reported cases. We will present this as a reference-policy comparison and will not claim that TargetGuard outperforms it.

Thank you for asking. I am the sole author. I used LLM tools for literature discovery, research ideation and execution support, drafting some sections, and language editing, as disclosed in the submission. I did not track a meaningful percentage of text attributable to these tools because their assistance was iterative and mixed with my own drafting and revision. I reviewed the final manuscript and take responsibility for its claims, analysis, and conclusions.

Reviewer_AFWwOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper addresses the "target-integrity" failure mode in agentic tool-calling, where tool call mutates the wrong entity due to alias collisions or stale state. The proposed solution resolves candidate entities using types and provenance and then binds the disambiguated entity for the purpose of the tool call. Empirical evaluation shows the elimination of wrong-target writes while retaining 100% utility.

Critical strengths
  • The problem is well explained and seems meaningful and relevant.

  • The evaluation shows solid results of reducing wrong-target writes while maintaining task success rate.

Critical weaknesses
  • The theoretical novelty seems minimal, in spite of three pages of (semi-)formal notation. In particular, I'd like to see a comparison with (pre-LLM) work on edit automata for run-time security policies from Ligatti et al in 2005.

  • The formalism that is provided is often imprecise, padded with natural language definitions, or simply unclear. Similar the terminology is confusion and seems to change from section to section.

    • The following terms seem to refer to the same mechanism: "freshness-certified selective execution gate", "selective execution algorithm", "portable contract", "selective execution rule", "benchmark reference implementation", "execution-side target-only projection", "target-only intervention", "identity-preserving target substitution", "atomic projection", "selective target-integrity resolver", and "partial action transformer", among others. It is confusing!
    • The concept of "provenance" is never defined and thus it is unclear to me what it means. Is it the immediate source of a data item (e.g., from user prompt, from memory, from previous tool call)? Or does it refer to entities (user, external web sites, etc.) that are of various trust levels?
    • The type system from which types are drawn is not defined.
    • Definition 1 describes valid(O)\mathrm{valid}(O) as a predicate checking whether OO is "present" and "well formed" but does not define what those terms mean.
    • Even the term "public typed relation" is not defined anywhere though it features prominently throughout. What are the entities the relation is between? What makes the relation "public" or "non-public"?
  • Notation is also ambiguous, with simple symbols such as RR being used for different things at different times. Also, writing O(∣O∣)O(|O|) to mean asymptotic complexity in the size of a set OO is a poor choice.

  • The description of the "benchmark instantiation" talks about a "primary causal track" but it is unclear what this means and where the causal information comes from.

  • In Section 5, what do WonW^\mathrm{on} and WoffW^\mathrm{off} represent?

  • Since the proposed technique and typed revalidation achieve the same level of accuracy, it would be useful to provide an overhead comparison between the two.

Questions for authors

N/A

1 comment

Thank you for identifying where the paper became difficult to follow. We agree that the number of names used for closely related mechanisms obscures the central idea. Our intended contribution is narrower than that terminology suggests: TargetGuard checks a proposed state-changing action against an adapter-provided typed relation and either permits the uniquely supported target or abstains. We will use one term for this rule throughout and define the relation, types, provenance, and well-formedness conditions explicitly. We also agree that the connection to runtime enforcement and edit automata, including the work of Ligatti et al. (2005), should be discussed directly. Our intended distinction is that TargetGuard addresses target identity under an explicit relation contract; it does not introduce a general runtime policy language. We will clarify this distinction and narrow the theoretical contribution accordingly.

Finally, the current results do not show an accuracy advantage over typed revalidation. We will state this plainly. The paper’s claim is a measurable contract and comparison of target-integrity outcomes, not superiority over that reference. We do not have a complete, comparable overhead measurement to add, so we will not claim a cost advantage.

Comments on the paper

1 comment

Thank you for the feedback. I have uploaded a revised version of the paper. The revision focuses on making the scope and method easier to follow; it adds no new experiments and changes no reported results.

The introduction now uses one running example to distinguish API success, task completion, and correct-target execution. I also define the adapter-provided relation, entity type, provenance, and the adapter’s soundness and completeness obligations. The text now states explicitly that a unique candidate alone does not establish user intent, and that abstention prevents the mutation but leaves the task unfinished unless a separate recovery policy is used; such a policy is not evaluated here.

I have standardized the terminology to distinguish the TargetGuard execution policy, the target-integrity evaluation contract, and the fixed-response target-only intervention. I also clarified that the 12-pair intervention is a controlled mechanism check, not evidence of deployment prevalence or learned generalization. Finally, I added a discussion of edit automata and narrowed the distinction between that runtime-enforcement prior and our target-specific contract.

The revised paper continues to report the observed agreement between TargetGuard and the typed revalidation reference; it does not claim superiority over that baseline.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.