acceptodds
Under review as a conference paper at ICLR 2027

KARMA: Epistemic Principles for Trustworthy Multi-Agent Collaboration

Abstract

Multi-agent LLM systems amplify collective capability but also amplify errors: without epistemic discipline, fabricated information propagates through agent networks, and agents cannot reliably distinguish credible from unreliable peers. From structural constraints on decentralized evidence processing, a constraint-to-failure-to-principle derivation yields KARMA, implementation-agnostic epistemic principles that govern both what an agent may assert from its own evidence and how it weights peers by track record. We then test whether this specification can be operationalized across two mechanistically distinct substrates. In LLM experiments on 150 multi-layer adversarial scenarios ( runs, 36,000 total judgments across six conditions), KARMA produces large improvements over the no-principle baseline for mid-tier models: to pp (Cohen's –, all ). Without principles, models perform near chance (49.7–53.7%); with KARMA, they approach frontier performance (90–99%). The frontier model achieves 100% under both conditions—the specification adds nothing when intrinsic capability suffices. Generic prompting strategies—caution, chain-of-verification, reputation, role authority—do not substitute for the specification: all fall significantly short of KARMA for non-frontier models (, –). The reputation heuristic is the strongest competitor but collapses on recency-conflict traps where, for two of the three mid-tier models, it falls below the no-principle baseline, precisely where temporal discounting—prioritizing recent outcomes over stale reputation—is required. A companion Bayesian simulation characterizes convergence and scaling behavior under the same specification across 7,000+ runs. The two mechanistically unrelated substrates—natural-language reasoning and an explicit Bayesian reputation mechanism—exhibit structurally aligned behavior (signal extraction, unreliable-source resistance, and capability-dependent activation) while sharing only the KARMA specification; both implementations were constructed by the same authors and therefore demonstrate realizability rather than independent convergence. Frontier models from multiple labs reach KARMA-consistent judgments without explicit principles, while capacity-constrained models incur a prompting-substrate floor—bounding useful operationalization to an intermediate capability regime.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
multi-agent systems, epistemic reasoning, trust, credibility assessment, LLM agents, Bayesian inference, adversarial robustness, specification design

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_HhTyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces KARMA, three rules for deciding which agents and sources to trust. It tests these rules as prompts on synthetic credibility tasks and as a hand-designed Bayesian reputation system. The prompts help several mid-tier models, while the frontier model is already at ceiling.

Critical strengths
  1. The paper reports all the prompts and full details of the simulated data used in the paper. The paper also reports important negative results.
  2. The findings are informative that the prompts help several mid-tier models, while the frontier model is already at ceiling.
Critical weaknesses
  1. The experiment is a stateless source-selection task rather than an interacting multi-agent system. KARMA prompt directly states the decision rules used to construct the benchmark. The 150 scenarios are variants of only 6 logical templates, and the extreme benchmark was constructed after the pilot was used for model selection and difficulty calibration. Thus, the large gains can show that explicit task-specific rules help selected models on scenarios constructed around those rules, but they do not convincingly prove general epistemic robustness or improved multi-agent collaboration.
  2. The powered benchmark lacks direct ablations of P1, P2, and P3. The paper should better isolate the claimed contribution: one thing is to distinguish KARMA’s training-free packaging and operationalization from more direct methods like direct evidence, unreliable recommendations, distributed trust, etc.
Questions for authors

Can the principles be evaluated in an actual multi-agent workflow where credibility judgments can practically affect task delegation, information propagation, final-answer accuracy, or system utility?

Reviewer_RbJROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes KARMA, three epistemic principles for trust judgments in multi-agent LLM systems: Outcome-Grounding, Evidence Scope, and Source Credibility. It implements them through a natural-language prompt and a Bayesian reputation mechanism with adaptive variance. On the 150-scenario adversarial benchmark in Table 2, DeepSeek V3.2 improves from 53.7% to 98.6%, Llama 3.3 from 49.7% to 98.1%, and Haiku 4.5 from 71.7% to 90.1%. Sonnet 4.5 achieves 100% with and without KARMA. The generic prompting baselines B1–B4 perform below KARMA on the three non-frontier models.

Critical strengths
  1. The effects on DeepSeek and Llama are large and robust to the clustering analyses reported in Table 10, including aggregation over six templates. Their reported effect sizes are d = 1.24 and d = 1.00. The use of fictional, deterministically generated scenarios and automated exact-match scoring also makes the benchmark's ground-truth construction inspectable.

  2. The adaptive-variance component has a useful within-simulation ablation. Table 4 reports that removing it changes sybil inflation from −0.06 to +0.10 and increases drift error by about 4.4 times. The authors also explicitly distinguish the realizability demonstrated by their two implementations from independent convergence.

Critical weaknesses
  1. The injection closely matches the benchmark's decision rules, leaving an attribution question. Appendix G.2 explicitly describes topic-specific records, discounting confident or detailed presentation, and recency shifts, while Section 3.1 constructs traps around these distinctions. B2 and B3 cover some of these considerations, but no combined baseline is reported that matches KARMA's scope, recency coverage, length, and specificity. A coverage-matched comparison would help establish what the proposed specification adds beyond explicitly describing relevant decision rules.

  2. Several reporting and scope issues need clarification. The two implementations provide evidence of realizability, but Table 12 does not establish independent convergence or a common quantitative measure across substrates. The main LLM tests already use 150 per-scenario means, and Table 10 reports clustering sensitivity; nevertheless, the blanket p < 10^−9 claim conflicts with Haiku's p = 1.4×10^−8, and its six-template result is p = 0.067. The claim that every scenario is solved either 10/10 or 0/10 also needs reconciliation with Table 13 values such as 96.8% and 0.8%, since 25 strictly binary scenarios would yield multiples of 4%. Finally, the Reproducibility Statement lists maxTokens = 400 whereas the Table 7 note describes 4096 for the extreme benchmark; condition-specific budgets should be stated.

Questions for authors
  1. Can you add a combined prompt baseline matched to KARMA in coverage, length, and specificity, including topic scope and recency?

  2. Which token budget was used for each condition in the pilot and the 150-scenario extreme benchmark? Please reconcile the Reproducibility Statement with Table 7.

  3. Please reconcile the p < 10^−9 headline with Table 10, and explain how the claimed 10/10-or-0/10 scenario behavior produces the non-multiple-of-4 percentages in Table 13. What do the underlying run-level results show?

Reviewer_ZpaXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

KARMA specifies outcome grounding, evidence scope and source credibility for evaluating peer information. It operationalizes these through natural-language instructions and a Bayesian reputation mechanism. A study with four models, six prompt conditions, 150 synthetic credibility scenarios and ten draws per condition reports large gains for three models and a frontier ceiling. Separate pilots identify a negative effect for MiniMax. Simulations examine adversarial routing, density, module ablations and parameter sensitivity. The intended contribution is an implementation-independent epistemic specification.

Critical strengths
  1. The controlled study reports substantial gains of 18.4-48.4 percentage points and informative per-trap behavior, including improvement on a four-layer composition. Scenario-level analysis avoids treating all repeated judgments as independent (pp. 5-7, Tables 1-2; p. 26, Table 13).
  2. Useful boundaries are reported: a frontier ceiling, negative MiniMax pilot, weaker Haiku template-level evidence, collusion degradation and tolerance sensitivity. The paper acknowledges the LLM routing gap and that the two realizations were constructed by the authors (p. 9; Figures 5-6; Tables 7 and 10-12).
Critical weaknesses
  1. The comparisons do not isolate the contribution or joint necessity of the principles. KARMA explicitly instructs topic-specific selection and recent-history override; B3 omits those rules and emphasizes past reliability, while B4 mandates specialist deference. There is no LLM leave-one-principle-out test, so full-prompt versus NP trap differences cannot identify marginal contributions (pp. 3-6; p. 25, Appendix G.2). The result supports a detailed task-aligned instruction package more directly than the stronger specification claim.
  2. The Bayesian mechanism is insufficiently specified to establish a faithful second realization. P1 prioritizes recent outcomes, but Equations 1 and 4 show no explicit forgetting/window step for direct outcomes or credibility. As written, 100 old successes followed by four failures leave the direct mean at 0.962 and full trust at least 0.917 with n0=5, even with zero adjusted testimony. Omitted details could resolve this; it is not evidence that the code is wrong. Routing, outcome visibility and baseline updates also need specification. In addition, the below-5% convergence criterion conflicts with descriptions of 3.7-4.6% terminal routing as nonconvergence (pp. 7-8, 14-15). These issues undermine the central correspondence and sparse-convergence comparison.
Questions for authors
  1. On a limited existing recency/scope subset for one mid-tier model, could you compare a similarly detailed ordinary evidence-evaluation prompt containing the same topic/recency guidance, and remove the corresponding P1 or P2 clause from KARMA? This would address attribution without requiring a new task family or full benchmark expansion.
  2. Please specify direct-outcome and credibility forgetting/window rules and routing/update pseudocode, including outcome visibility and baseline configurations. Show the handling of a long successful history followed by recent failures. Please reconcile the below-5% convergence definition with the 3.7-4.6% nonconvergence statements and give the corrected comparison from retained runs.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.