acceptodds
Under review as a conference paper at ICLR 2027

TRACE-MAS: Provenance Guided Selective Recovery for Multi-agent Incidents

Abstract

In multi-agent systems, a message that seems safe locally can become harmful after it enters shared memory, reaches another agent, or drives a tool action. Current runtime controls flag a risky record or stop a run, leaving the downstream state requiring repair unspecified. We present TRACE-MAS, a runtime controller for selective recovery. At each checkpoint, it builds a typed event graph over observed messages, state changes, and tool interactions, then uses that graph to locate a likely source, estimate the affected state, and choose an allowable action through a shared action-permission interface. We evaluate TRACE-MAS on TRACE-Bench, which covers resource competition, collective deliberation, shared memory, and tool transfer, together with incomplete runtime records, public interface adapters, and a bargaining game interface. The evaluation pairs TRACE-MAS with a matched Direct-risk control that keeps the same alerts, permitted actions, recovery templates, and input allowance. TRACE-MAS additionally receives the graph and source and scope output. Paired replay tests the selected action on the same trajectories under true root, decoy, global, and no intervention branches. The matched resource competition comparison lowers Harm by 12.0% and ASR by 49.0% while raising Utility by 13.0% relative to Direct-risk. Supplementary evaluations examine noisier provenance and alternative ranking rules. Our findings show that typed dependencies and selective actions can repair affected state while preserving unrelated work.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Multi-Agent system safety,agentic workflows,runtime recovery,typed provenance graph

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_AmNROpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper tackles a problem that is easy to state but rarely handled head-on: in a multi-agent system a message that looks locally safe can turn harmful once it lands in shared memory, reaches another agent, or drives a tool call. Most existing runtime controls either flag a single suspicious record or halt the whole run, which leaves the already-propagated downstream state in limbo. TRACE-MAS reframes this as a selective recovery decision — once a risky state has crossed agent boundaries, figure out where to repair and how much.

Mechanically, at each checkpoint the controller builds a typed event graph over observed messages, reads/writes, and tool interactions, with time, actor, and causal (read/write, propagation, tool) edges. It scores each event for "direct risk" from observable signals (trust, authority, handoff content, requested tool actions, runtime alerts), picks source candidates as risky nodes whose dependency predecessors are not risky, adds a bounded downstream-continuation term so sources with real propagation outrank dead-end alerts, keeps the top-K roots, estimates the affected state as the recorded dependency descendants of those roots, and finally selects a single bounded action (invalidate a memory write, revert, quarantine a channel/agent, block or revoke a handoff, etc.) from a fixed per-setting permission map. If there is no credible source or no compatible permitted action, it abstains.

Evaluation is on TRACE-Bench, a four-setting suite (resource competition, collective deliberation, shared memory, tool transfer) built on public seeds (MultiAgentBench, MMLU-Pro/GSM8K/FEVER, HotpotQA, AgentPoison, AgentDojo, ASB). The key comparison is against a matched "Direct-risk" control that receives the same alerts, permitted actions, recovery templates, and input length — everything except the typed graph and the affected-state estimate — plus a Full-trace reference and an Oracle-root localization ceiling. Metrics are residual Harm, retained Utility, incident success rate (ASR), and a scope-weighted Cost. The headline result: in resource competition TRACE-MAS lowers Harm by 12 points and ASR by 49 points while raising Utility by 13 points relative to Direct-risk, with the same ordering in shared memory, consistent direction across four target models (GPT-5.6, Claude Opus 4.8, DeepSeek-V4-Flash, Qwen3.8-Max), open repeated games, trace-integrity stress tests, and public-interface adapters. A paired counterfactual replay links localization quality to recovery success (true-root repair 96%, the selected plan 82%, decoy repair ~43–45%).

Contributions, as I read them: (1) formulating propagated MAS incidents as a runtime selective-recovery problem; (2) the TRACE-Bench matched-control benchmark; (3) evidence that typed dependency information actually buys you something, via matched ablations, counterfactual replay, and incomplete-record stress tests.

Critical strengths

The matched-control design. The comparison that carries the paper is TRACE-MAS vs. a Direct-risk controller that is identical in signals, actions, templates, and input budget, differing only in access to the typed graph and affected-state estimate. This is the difference between "our system does well" and "the specific component we are advocating is what does the work," and it is why I take the large resource-competition and shared-memory gains seriously. The paired counterfactual replay reinforces this: repairing the true root succeeds 96% of the time while repairing a decoy of matched size recovers under half, so localization quality — not just taking some action — is doing the lifting.

The problem is real and under-served. "A record already crossed agent boundaries; now what do we fix?" is a question that prevention-focused and access-control-focused prior work mostly sidesteps, and the bounded-scope framing (repair the affected state, leave unrelated work running, abstain when evidence is thin) is a sensible and practically relevant target. The breadth of the evaluation (four interaction structures, four models, repeated games, deployment adapters) makes the behavior look robust in direction even where magnitudes are noisy.

Critical weaknesses

Dependence on clean provenance, weakest exactly where reality is messiest. The whole pipeline assumes stable event IDs and sufficiently complete typed dependency edges, and the paper shows the method falls off most under implicit, within-context information reuse — which is how a lot of real agent influence actually travels. The stress tests are a good-faith attempt to probe this, but they are synthetic perturbations of an otherwise clean record; they don't establish how often real deployments will supply edges good enough for the gains to survive. This matters because it bounds who can actually adopt the method.

Much of the evidence lives inside a self-constructed benchmark. The incident predicates, action maps, and cost schedule are all authored by the same team, and the central comparison runs inside their own runtime. The matched controls and public-interface adapters mitigate the "home-field" worry, but they don't eliminate the possibility that TRACE-Bench rewards precisely the structural reasoning TRACE-MAS performs. Combined with the small per-cell n on the secondary analyses, this means some of the broader "robust across everything" claims rest on thinner ground than the two strong settings.

Questions for authors
  1. In the implicit-influence regime, is there any realistic way to recover the missing edges (e.g., from attention or retrieval logs), or is the honest answer that TRACE-MAS simply should not be deployed where the runtime cannot emit explicit dependency edges? A clear statement of the deployment precondition would help.
  2. For the secondary slices (target-model, OOD/topology), which specific cell-level differences do you consider statistically distinguishable given the n=30 intervals? I am not asking for more experiments — just a clearer read on which claims the current data actually supports.
  3. How sensitive are the results to the fixed direct-risk weights and the λ/κ/K/τ settings chosen before evaluation? The structural-sensitivity appendix sweeps these on local prefixes with no competing candidates; a sentence on robustness under competing risk scores would close the loop.
  4. The abstention behavior is appealing — can you report how often TRACE-MAS abstains in each setting, and whether abstentions concentrate in the harder/noisier cases (which would be the desired behavior)?
8 comments
AuthorsOpenReview ↗

Response to Reviewer AmNR(Part 1 / 2)

We sincerely appreciate your insightful and professional comments. We are encouraged by your recognition of the selective-recovery problem and your careful assessment of the matched-control design. Your emphasis on separating mechanism evidence from deployment applicability is particularly helpful. Below, we clarify the provenance requirements, the role of the benchmark, and the interpretation of the secondary analyses.

Q1: Does dependence on clean provenance restrict practical adoption?

A1: The deployment requirement is an instrumented runtime with stable identifiers, recorded consumption links, and compatible recovery operations. Appendix B.3 examines both degraded records and a fixed unstructured-log extraction interface. Under the latter, TRACE-MAS achieves 73.3% scope F1 and 15.0% Harm, compared with 21.7% Harm for Direct-risk. This shows the method can operate beyond the clean structured interface tested in the main comparison. The perturbation studies characterize dependence on record quality; they do not estimate how frequently production systems provide suitable logs.

Q2: Could the self-constructed benchmark favor TRACE-MAS, and do the secondary studies justify broad robustness claims?

A2: The benchmark separates two questions. The matched comparison tests whether dependency-based targeting helps within a declared runtime: Direct-risk shares the alerts, action permissions, recovery templates, and input allowance, while graph-aware controls additionally share the graph. The bargaining and public-interface studies examine additional task organizations; they provide complementary transfer evidence. For example, the AgentPoison interface reports 11.7% Harm for TRACE-MAS and 18.3% for Full-trace. Our conclusion concerns selective recovery under these evaluated interfaces. Cost is the declared action-and-scope score, and the secondary studies describe behavior under specific shifts rather than universal robustness.

Q3: Can retrieval or attention logs recover implicit-influence edges?
(Original Question 1)

A3: Retrieval and tool logs can recover explicit delivery or consumption links when they record which item was supplied to which operation. Merely retrieving an item does not establish that it influenced the answer, and attention weights are not validated recovery dependencies in this work. Where important influence remains entirely within an unlogged model context, graph traversal cannot reconstruct that missing information. TRACE-MAS should therefore be used for recovery scopes supported by the runtime's recorded dependencies; the implicit-influence experiment measures this boundary directly.

AuthorsOpenReview ↗

Response to Reviewer AmNR(Part 2 / 2)

Q4: Which target-model and OOD differences are statistically distinguishable at n=30?
(Original Question 2)

A4: The individual paired 95% intervals identify Harm differences excluding zero in resource competition for GPT-5.6, Claude Opus 4.8, and Qwen3.8-Max, and in shared memory for GPT-5.6, DeepSeek-V4-Flash, and Qwen3.8-Max. These conclusions use the corresponding paired contrasts, not Table A6's group-level half-widths. We interpret the smaller deliberation and tool-transfer Harm differences descriptively. Deliberation ASR is tied across models; the OOD study also includes endpoint ties in two- and five-agent deliberation and memory injection after the checkpoint. We make no additional cell-specific OOD significance claims here.

Q5: How sensitive is source selection when risk candidates compete?
(Original Question 3)

A5: The parameters have distinct roles: direct-risk weights govern local evidence, λ and κ bound the contribution of downstream exposure, and K and τ control candidate retention. For a fixed eligible candidate set, ranking remains unchanged while score perturbations stay below the pairwise margins; the retained set additionally depends on the top-K boundary and relative threshold. This explains conditional stability under competing scores without assuming universal insensitivity. Appendix D.2 demonstrates the isolated-candidate case, and all reported evaluations use the same fixed configuration. The multiple-root study complements this by testing recovery with several sources, rather than serving as a parameter sweep.

Q6: How often does TRACE-MAS abstain, and does abstention concentrate in harder cases?
(Original Question 4)

A6: The reported abstention rates are 8.3% in resource competition, collective deliberation, and shared memory, and 5.0% in tool transfer. The implicit-influence/noisy-segmentation condition records 12.5%, a higher observed rate than the standard settings. This describes a frequency difference, rather than proving that every abstention is safe. We distinguish no supported candidate, no compatible action, and parsing failure when interpreting non-execution; benign non-intervention is a separate quantity. The policy may also retain a permitted model nomination when no structural action is selected, so structural-policy abstention and final non-execution must remain distinct.

AuthorsOpenReview ↗

Response to Reviewer AmNR

Thank you for the careful follow-up questions. We added the following blue-marked clarifications:

  • Section 4.2, Building the Provenance Graph: Explicitly states the deployment prerequisites and the boundary of the method: TRACE-MAS uses stable identifiers, typed recorded dependencies, trust metadata, and compatible recovery actions, and does not infer unlogged within-context influence from attention weights or temporal proximity.
  • Appendix D.1, Sample Retention and Uncertainty: Clarifies that Table A6 reports group-level half-widths, whereas the reported model conclusions use paired cell-level intervals. The text identifies the resource-competition and shared-memory model cells with Harm intervals excluding zero and treats smaller deliberation and tool-transfer contrasts descriptively.
  • Appendix D.2, Structural Sensitivity: Clarifies the separate roles of direct-risk weights, λ\lambda, κ\kappa, KK, and τ\tau, and states that the observed sweep is conditional on isolated candidates rather than a competing-candidate sensitivity experiment.
  • Appendix B.1, Runtime Schema: Reports the observed abstention rates of 8.3% for resource competition, collective deliberation, and shared memory, 5.0% for tool transfer, and 12.5% for the combined implicit-influence/noisy-segmentation condition. Structural-policy abstention is distinguished from final non-execution.
Reviewer_ybxHOpenReview ↗
Rating
1
Confidence
3(−1)
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The authors present a system that can help recover from runtime faults in multi-agent deployments. The system acts as a controller that tracks system state as well as events. Once an adverse effect is detected, the framework tries to detect the likely provenance and then rolls back system state. Across a suite of benchmarks, the authors report reduction in harm, increase in resource utilization.

Critical strengths

The problem is highly relevant.

Critical weaknesses

I think the strongest weakness of the paper is its writing. I'm very sorry, but for lack of a better word, I would describe it as sloppy. The notation is inconsistent and careless. I strongly suspect that parts of the presentation are mere decoration. 3.1 defines an event eie_i as a tuple of "producing agent, channel, ... , resulting state transition". It also says, "The controller receives the observed prefix of this sequence. The environment withholds ..., and counterfactual outcome". What is the observed prefix? How can the environment know the counterfactual outcome? 3.2 then defines a graph. Curiously "nodes represent events and materialized state objects". I find it very surprising that here events, which explicitly record a state transition, are used as nodes. And states are also nodes. Edges seem to simply record a temporal relation? I don't think the concept of an edge is well defined. 3.3 introduces the concept of episodes, 4.1 the concept of checkpoints. Neither is clearly defined. It seems that checkpoints simply happen at the end of an episode? 4.2 suddenly revisits the concept of edges, now as "dependency edges". The definition of a dependency edge seems similar to the entries in the event tuple, with one important disclaimer "these edges cross agent boundaries when state moves through shared memory ...". Let me take a step back here. The entire problem statement is exceptionally poorly defined. It seems to introduce redundant concepts in a very confusing, overlapping way. Yet it fails to touch upon the core of the problem studied here: Shared memory, shared state, communication and coordination in a multi-agent system. To me, the writing defects are so gregarious that for this alone I would vote for clear rejection.

An additional critical weakness is the poor positioning with respect to related works. SagaLLM(2025), ALAS(2025), and Atomix(2026) all seem very closely related. Atomix in particular discusses a multi-agent setting where effects are coordinated and targeted rollbacks (compensations) are possible. Without a careful discussion of this rich body of existing work, the present manuscript should not be published.

The benchmarking is poorly explained. Analyzing faults in multi-agent systems is admittedly complicated. But the chosen datasets that underpin the TRACE-Bench methodology don't clearly show where/how faults would be introduced (MMLU-Pro, GSM8K, ...). A careful discussion of the fault model, ideally as part of the revised problem statement which pays appropriate attention to the shared state / communication, would be necessary.

Questions for authors

Why did you not discuss SagaLLM, ALAS, Atomix?

14 comments
AuthorsOpenReview ↗

Response to Reviewer ybxH

Thank you for recognizing the relevance of runtime recovery in multi-agent systems. We address the terminology, fault protocol, and related-work concerns directly below.

Q1: Are the runtime concepts redundant or undefined, and do graph edges merely encode temporal order?

A1: Sections 3.1–3.2 and 4.2 already distinguish runtime records, state objects, and edge semantics; Appendix A.1 specifies the checkpoints. Section 3.2 explicitly separates contextual time/actor edges from the dependency edges used for recovery, so interpreting all edges as temporal does not reflect the formulation.

TermMeaning and role
EpisodeOne task execution under a fixed scenario, seed, and condition.
CheckpointAn intervention boundary within that execution, followed by continued execution and outcome evaluation.
Observed prefixEvents recorded up to the checkpoint; “events observed so far” denotes this same information boundary.
Event / resulting state transitionAn operation record and its effect on state.
Materialized state objectThe information an operation reads or writes; it can subsequently be consumed by another agent.
Time / actor edgesExecution context, excluded from recovery traversal.
Dependency edgesRecorded read/write, message-propagation, or tool-consumption relations, traversed to estimate affected state.

Shared state is information available across agents; memory stores it, communication transfers it, and coordination combines agent contributions into a joint outcome. When Agent A writes a note that Agent B consumes, the operation and stored note have distinct identities, connected by recorded consumption. Recording a transition therefore does not make event and state nodes redundant.

Q2: How are faults introduced using question datasets, and how are counterfactual outcomes obtained?

A2: MMLU-Pro and GSM8K supply task content; TRACE-Bench supplies the fault protocol. Appendix A.1 specifies unverified priority signals before allocation, incorrect authority recommendations before aggregation, untrusted memory notes before verification, and untrusted tool outputs before execution. Incident success is evaluated after intervention, not inferred from exposure alone.

Appendix F.2 computes counterfactual outcomes offline by replaying the same stored episode under five intervention branches with deterministic transitions and no additional LLM calls. These outcomes and gold labels are unavailable to the online controller; the environment does not predict future LLM responses.

Q3: Do SagaLLM, ALAS, and Atomix already address the contribution of TRACE-MAS?

A3: SagaLLM studies validated planning with Saga-style compensation, ALAS disruption-aware planning with local compensation, and Atomix transactional effect coordination and settlement. TRACE-MAS focuses on selecting risky sources and affected state after cross-agent propagation, tested under matched signals, actions, and recovery templates. Shared compensation mechanisms do not make these contributions equivalent.

Our existing discussion emphasizes the security and provenance dimensions directly studied here: memory poisoning and governance (AgentPoison, GateMem), tool-output injection (AgentDojo), malicious propagation (SafeFlow), and trace diagnosis (AgentTrace). The three suggested works complement this positioning through planning and transactional recovery.

ICLR allows updating the PDF during the review period. I would encourage the authors to rework the problem statement. Once my comments regarding definitions, notations, etc are addressed, I'm willing to take another look at the PDF.

AuthorsOpenReview ↗

Response to Reviewer ybxH

Thank you for your willingness to take another look at the manuscript. We have reworked the problem statement and clarified the definitions and notation. The revised passages are highlighted in blue. The changes are located as follows:

  • Section 3.1 (Runtime Setting): Explicitly defines an episode, a checkpoint, and the observed prefix; explains each event field and distinguishes recorded state changes from future outcomes. It also clarifies shared state, communication, and coordination.
  • Section 3.2 (Typed Provenance Graph): Explains the distinct roles of event and state-object nodes, defines dependency direction and consumption semantics, and separates contextual time/actor edges from recovery dependencies. A shared-memory example illustrates the cross-agent relationship.
  • Section 4.3 (Estimating the Source and Affected State): Clarifies the immediate-predecessor and reachable-descendant notation.
  • Appendices A.1 and F.2: Clarify the fault-exposure protocol and intervention boundaries, and distinguish online observations from offline counterfactual replay.
  • Section 2 (Related Work): Adds a discussion of SagaLLM, ALAS, and Atomix and their relationship to the source-and-scope selection problem studied here.

We appreciate the opportunity for you to reassess the manuscript with these clarifications.

Reviewer_99pkOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies runtime recovery in LLM multi-agent systems after a tainted message has spread through shared memory, messages, or tool handoffs. The system build a typed provenance graph from events emitted by the runtime. Time and actor edges are kept as context and the read/write propagation and the tool edges are treated as dependencies.The paper also introduces Trace-Bench, which covers resource competition, shared memory, and tool transfer, and compare against setting specific defenses, an LLM based full trace controller, and a direct risk control matched on alerts, actions, templates, and input length.

Critical strengths
  1. I think the framing of a repair scope after propagation is useful and not explored, and the control design does a good job of isolating the claimed mechanism.
  2. All weights and parameters are clearly specified, and the authors evaluate against conditions that should break the algorithm, (e.g. edge dropout, trust forgery, etc)
Critical weaknesses
  1. The algorithm is fairly simple, and the technical contribution is weak. The propagation paths are all single paths. The runtime gives the controller exactly the seven relation types that make up those paths. It's also similar to algorithms from the systems literature, such as Taser, SOSP 2006 and Retro, OSDI 2010).

  2. There are several inconsistencies in the results (listed in questions.)

Questions for authors
  1. Under the F.2 replay rule, how can the Decoy branch differ from None in Table A9? If partial-target repairs have an effect, is the rule stated correctly?
  2. Table A6 gives one half-width per result group. Are these per-cell paired bootstrap intervals? Which differences in Table 1, Table 2, and Fig. 3 exclude zero?
  3. B.1 says the policy derives roots and targets and substitutes the compatible action. What fraction of executed decisions follow the controller model’s own nomination? What explains the cross-model differences in Fig. 3 if structure and actions are fixed?
  4. Table 4(a) shows TRACE-MAS intervening in 33% (resource competition) and 40% (collective deliberation) of benign episodes. Its Utility also exceeds No defense in every benign suite. What triggers these interventions? Does part of the attack-setting gain come from actions such as “require evidence” that improve task accuracy whether or not an incident occurred?
7 comments
AuthorsOpenReview ↗

Response to Reviewer 99pk(Part 1 / 2)

We sincerely appreciate your insightful and professional comments. We are encouraged by your recognition of our core idea: identifying a repair scope after an incident has propagated across agents. We are also grateful for your positive assessment of the matched-control design, the explicit specification of weights and parameters, and the evaluation under adverse conditions such as edge dropout and trust forgery. Your recognition of these aspects encourages us to further develop this line of research.

We have carefully considered your feedback on the technical contribution and the interpretation of the experimental results. Below, we clarify the relationship to prior recovery systems, highlight the branching and multiple-root evaluations in Appendix B.3 and Table A4, and explain the replay protocol, paired statistical analysis, model–policy interaction, and benign interventions.

To address each concern directly, we provide point-by-point responses below.

Q1: The algorithm is simple and resembles Taser and Retro. What is the technical contribution?

A1: Thank you for the question. TRACE-MAS connects source selection, affected-state estimation, and action-compatible recovery at a multi-agent checkpoint. Taser and Retro establish dependency-based recovery for system operations; our contribution concerns selecting sources from observable agent-level risk signals and translating their propagation into a bounded repair decision. Table 1 isolates this mechanism under shared signals, actions, and templates: resource-competition Harm is 11.0% for TRACE-MAS, versus 23.0% for Direct-risk, 16.0% without scope estimation, and 27.0% without localization. These controls demonstrate why source and scope selection matter even when recovery operations are already available.

Q2: Are the propagation paths restricted to single chains, with the answer supplied by seven runtime relation types?

A2: We would like to clarify that Appendix B.3 includes longer and branching paths, and Table A4 includes multiple roots. The branching evaluation reports 86.7% root recall and 80.0% scope F1. The seven relation types record producer–consumer relationships, such as retrieval-to-memory; they do not disclose incident roots or affected-state labels. The graph-aware controls also receive the same graph and permissions, distinguishing access to structure from how it is used.

Q3: Under F.2, why does Decoy differ from None? Does partial-target repair have an effect?
(Original Question 1)

A3: Thank you for highlighting this distinction. The server replay applies local state updates to the selected targets. Complete coverage governs restoration of the true incident-affected state; an incorrectly targeted action can still change local state without achieving that restoration. Decoy therefore executes a nonempty intervention on incorrect targets, whereas None leaves the state unchanged. Their Harm, Utility, and recovery-success outcomes can differ because these metrics are evaluated on the resulting task state. This explains why Decoy differs from None yet remains below Chosen and True root in recovery success. We will clarify F.2 to distinguish local state updates from complete incident restoration.

AuthorsOpenReview ↗

Response to Reviewer 99pk(Part 2 / 2)

Q4: Are Table A6’s intervals cell-specific, and which comparisons exclude zero?
(Original Question 2)

A4: Thank you for asking us to clarify the uncertainty reporting. Table A6 summarizes uncertainty by result group; the conclusions below use the individual paired 95% intervals, calculated from within-seed differences with seed-level bootstrap resampling.

  • Table 1: In standard resource competition, Harm, Utility, and ASR differences versus both No defense and Full-trace exclude zero; Cost versus Full-trace does not. In the resource-competition mechanism block, Harm, Utility, and ASR versus Direct-risk exclude zero, while Cost does not.
  • Table 2: Harm and Utility versus Direct-risk exclude zero; ASR and Cost do not.
  • Figure 3: The identified Harm contrasts excluding zero are resource competition with GPT-5.6, Claude Opus 4.8, and Qwen3.8-Max, and shared memory with GPT-5.6, DeepSeek-V4-Flash, and Qwen3.8-Max. We retain a descriptive interpretation for the smaller deliberation and tool-transfer Harm differences rather than claim significance for every cell.

Q5: How often does execution follow the model’s nomination, and why do models differ under a fixed policy?
(Original Question 3)

A5: Nonempty action nominations occur at 91.7% of all checkpoints. Of those nonempty nominations, 83.3% are accepted and executed, while 8.3% are replaced by a policy action. Abstention is reported separately at 8.3% of all checkpoints. Thus, 83.3% is the nomination acceptance rate, conditional on a nonempty nomination; it is not the agreement rate conditional on execution. Action acceptance also differs from agreement over the target scope. A compatible policy action can replace a differing nomination under the shared one-action budget. Cross-model differences arise because the selected model generates both task-agent behavior and controller nominations, changing the trajectory while the structural rule remains fixed.

Q6: What triggers benign interventions, and could “require evidence” improve tasks without an incident?
(Original Question 4)

A6: Benign intervention can arise from observable risk signals or a permitted model nomination; the controller never receives the hidden attack label. Evidence verification and independent voting can also improve ordinary decisions, so intervention does not imply an attack was detected. This is why Direct-risk shares the same signals, actions, and recovery templates. Comparison with No defense measures the overall operational benefit; matched controls and localization/scope ablations test the additional value of provenance-guided targeting.

AuthorsOpenReview ↗

Response to Reviewer 99pk

Thank you for the detailed questions. We made the following targeted clarifications in the blue-marked manuscript:

  • Section 2, Related Work: Added Taser and Retro with their selective-recovery focus and clarified how TRACE-MAS differs by selecting a risky source and affected scope across agent messages, shared memory, and tool handoffs.
  • Section 4.2, Building the Provenance Graph: Made the deployment contract explicit: stable event identifiers, typed recorded dependencies, observable trust metadata, and a compatible recovery interface are prerequisites. The text also states that unlogged within-context influence is not inferred from attention weights or temporal proximity.
  • Appendix F.2, Counterfactual Replay: Clarified that the five branches replay the same stored prefix without additional LLM calls; local updates are applied to branch targets, while complete root-plus-affected coverage determines full restoration. This explains why an incorrect nonempty Decoy intervention can differ from None without constituting correct recovery.
  • Appendix D.1, Sample Retention and Uncertainty: Distinguished result-group half-widths from cell-level paired intervals and listed the comparisons whose paired intervals exclude zero. Smaller deliberation and tool-transfer differences remain descriptive.
  • Appendix B.1, Runtime Schema: Reported the controller nomination statistics with their denominators: 91.7% nonempty nominations over checkpoints, 83.3% acceptance and 8.3% substitution among nonempty nominations, and 8.3% aggregate abstention over checkpoints. The text separates nomination acceptance from executed-decision agreement.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

Dear Reviewers and Area Chairs,

As the discussion period draws to a close, we sincerely thank all reviewers for the time devoted to evaluating our paper and discussing TRACE-MAS. For clarity, we refer to Reviewers 99pk, ybxH, and AmNR as R1, R2, and R3, respectively.

Below, we summarize the contribution and the principal clarifications established during the discussion.

Core Contribution and Supporting Evidence

TRACE-MAS addresses a concrete runtime decision: identifying which state requires repair after risky information propagates across agents, while preserving unrelated work. Its typed provenance graph connects source localization, affected-state estimation, and action-compatible recovery.

The matched evaluation holds risk signals, permitted actions, recovery templates, and input allowance fixed. In resource competition, TRACE-MAS records 11.0% Harm compared with 23.0% for Direct-risk; removing scope estimation yields 16.0%. These comparisons examine the contribution of dependency structure and repair scope under a common intervention interface.

Paired counterfactual replay further connects targeting to recovery on identical stored trajectories. The selected plan achieves 82.0% recovery success in both evaluated settings, with true-root repair reaching 96.0%.

Main Clarifications During the Discussion

  • Recovery semantics and timing (R1, R2). We explain the checkpoint boundary, the information available before intervention, and the subsequent evaluation of outcomes. Replay executes local state updates under five branches without additional LLM calls. A Decoy intervention changes its selected targets, explaining its difference from None; complete root-and-scope coverage governs full restoration.

  • Definitions, notation, and prior work (R1, R2). The revised paper expands the explanations of events, state objects, typed dependencies, contextual relations, and compatible actions. Our responses also explain how source-and-scope selection relates to existing rollback, compensation, and agent-recovery systems.

  • Deployment requirements and graph interpretation (R1, R3). We specify stable event identifiers, recorded dependencies, trust metadata, and compatible recovery actions. Dependency traversal uses explicit runtime relations. Attention weights are not used to establish recovery dependencies.

  • Extraction and statistical interpretation (R3). The unstructured-log experiment evaluates provenance reconstruction and downstream recovery from textual records containing event identifiers and references. For target-model comparisons, we interpret the reported mean reductions descriptively, without ranking the statistical support for Claude and DeepSeek.

These responses and revisions make the recovery decision, its execution semantics, and the scope of the evidence more explicit. TRACE-MAS’s central contribution remains the integration of observable source selection, affected-state estimation, and selective intervention, evaluated through matched controls, mechanism ablations, and paired replay.

We appreciate the reviewers’ engagement and the Area Chairs’ time and judgment in considering the complete discussion, the revised paper, and its methodological and empirical contributions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.