acceptodds
Under review as a conference paper at ICLR 2027

Same Answer, Different Reasons: Disentangling Coordination from Consensus in Multi-Agent Debate

Abstract

Large language model agents in a multi-agent debate can reach the same final answer for two different reasons: because they saw the same evidence, or because a coalition among them privately coordinated to produce it. From the verdict alone the two cases are indistinguishable, letting a coordinated minority pass as independent consensus. We cast this as an identifiability problem: recovering a latent partition of agents into coalition and non-coalition members from public information alone (transcript and shared task dossier). To address it, we introduce **LATTICE**, a Layered Agreement Tracing framework that grounds each message into atomic claims, judges typed, directed relations between claims of different agents, and aggregates them into time-windowed, multi-lens agent-by-agent matrices under a composite with pre-specified weights. A stance-correlation baseline is kept out of the composite, and every result is checked against a paired negative-control session with the same dossier and roster but no coalition. Under this protocol, the composite separates the latent agent partition more reliably than stance correlation, does not recover the transferred partition in the negative controls, achieves the strongest pairwise ranking among public-information detectors, and, when the coalition size is known, recovers the installed coalition exactly, matching group-structure detectors without their high false-structure rates. On external benchmarks it keeps its ranking advantage where stance correlation falls to chance, recovers coalitions well above chance but not exactly, and does not mistake honest, information-driven convergence for coordination. Code and data are available [here](https://anonymous.4open.science/r/lattice-500A).

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM agents, collusion detection, coalition detection, coordination detection, multi-agent safety

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_dWHcOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

In this work, the authors introduce LATTICE, a framework for detecting hidden coalitions in multi-agent debate systems. LATTICE traces how agents adopt, support, and challenge one another's claims. Experiments on controlled debates and external benchmarks suggest that these reasoning patterns recover coalitions more effectively than comparing agents' stated positions alone.

Critical strengths
  1. I enjoyed reading this work. The question this work tackles is critical and timely.
  2. The LATTICE method is well curated and offers enhanced interpretability of multi-agent systems.
  3. Experiments are carefully designed, and covers a wide range of perspectives.
Critical weaknesses
  1. My main concern is the novelty of the work. As far as I understand, there have already been some works that try to assess the collusive behaviors of agents in a multi-agent system. GroupGuard [1] is one example I can think of. To my understanding, this work also introduces a monitoring layer that goes beyond final-answer agreement by analyzing who supports or challenges whom throughout a conversation. Could the authors clarify the key distinction from this work?

[1] GroupGuard: A Framework for Modeling and Defending Collusive Attacks in Multi-Agent Systems

  1. I acknowledge that the authors state that "rather than asking whether a debate’s conclusion is correct, we ask whether the debate is being steered by a coordinated subset of its own participants". However, the experimental design appears to couple coalition membership with advocating an evidence-opposed decision, while LATTICE explicitly incorporates shared unsupported claims as a detection signal. Could the authors clarify to what extent the reported performance reflects coordination rather than shared incorrect or unsupported reasoning? In particular, how would LATTICE perform when agents independently advocate the same unsupported conclusion without coordinating, or when a coordinated coalition advocates an evidence-supported conclusion?

Overall, I think this work is great. If my few concerns are resolved, I would be happy to adjust my ratings.

Questions for authors

See Weaknesses

4 comments
AuthorsOpenReview ↗

Author Response to W1: Distinction from GroupGuard

We thank the reviewer for highlighting the importance and timeliness of this problem, LATTICE’s interpretability, and the breadth of our experiments. We address the questions on novelty and shared incorrect reasoning below.

Concern. GroupGuard also analyzes interactions beyond final-answer agreement. What distinguishes LATTICE?

Author Response. Both methods analyze multi-round interactions. LATTICE differs in the claim-level information preserved before agent-level aggregation: directed argumentative relations, claim provenance, dossier-grounded evidence status, and the temporal adoption of specific claims. These distinctions can be lost in an agent-level social graph. The representation and access differences are summarized below.

DimensionGroupGuardLATTICE
RepresentationAgent-level social interaction graphAtomic claims and directed claim-to-claim relations, followed by agent-level aggregation
Retained relationsSUPPORT / QUESTION / INFOADOPTS, SUPPORTS, EXTENDS, QUALIFIES, REBUTS, and UNDERCUTS, with claim provenance and dossier-based evidence status
Temporal evidenceMonitoring of multi-round social interactionsTime-windowed measurements and directed adoption of specific claims
Access and scopeThe full system includes active honeypot probing and structural pruningPassive inference from public transcripts and the shared dossier

Matched-access comparison. We compare LATTICE with a GroupGuard monitoring-layer adaptation on identical public transcripts and graph-matched hard negatives selected using coarse social-graph statistics, without inspecting detector outputs. This isolates passive monitoring, not GroupGuard’s full defense with active probing and pruning.

Table D1. Graph-matched coalition-presence AUROC and paired comparison.

Method / ablationPresence AUROC ↑\uparrowΔ\Delta vs. GroupGuard95% CI
GroupGuard monitoring-layer adaptation0.67
Coarse message-/agent-level graph0.70
Claim-level relations0.78
Claim-level relations + temporal adoption0.83
Full LATTICE0.86+0.19+0.19[+0.10,+0.28][+0.10,+0.28]

In Table D1, claim-level relations raise presence AUROC from 0.70 to 0.78, and temporal adoption raises it to 0.83. Full LATTICE reaches 0.86, a paired gain of +0.19+0.19 over the GroupGuard monitoring adaptation (95% CI [+0.10,+0.28][+0.10,+0.28]) and a point-estimate gain of +0.16+0.16 over the coarse graph.

Reference. Y. Tao et al., GroupGuard: A Framework for Modeling and Defending Collusive Attacks in Multi-Agent Systems, https://arxiv.org/abs/2603.13940.

AuthorsOpenReview ↗

Author Response to W2: Coordination vs. Shared Reasoning Errors

Concern. The original design couples coalition membership with evidence-opposed conclusions. Can LATTICE distinguish independently mistaken agents from a coordinated coalition defending an evidence-supported conclusion?

Author Response. The submitted study tests coalition recovery with evidence-opposed conclusions. We additionally vary coordination and conclusion correctness in a 2×22\times2 evaluation. Here, “correct” means supported by the public dossier. The presence results and signal ablations appear in Table D2 and Table D3.

Protocol. Across 40 held-out dossiers, four conditions, and three generation seeds, we evaluate 480 sessions (120 per condition), holding dossier format, agent count, debate length, and model configuration fixed. Coordinated-correct sessions retain a shared private objective. Independent sessions have no planted coalition; the independent-wrong arm uses naturalistic hard negatives in which at least two agents independently converge on the same unsupported conclusion. Selection uses dossier-grounded conclusion labels, never LATTICE’s predictions.

Presence decision. The session score is the maximum size-adjusted excess coordination relative to a neutral reference distribution, penalized for searching multiple candidate coalitions. Normalization and threshold are fitted only on development dossiers to target 5% neutral FPR, then frozen; all sessions from one dossier remain in one split. Unlike the submitted known-kk partition search, this supplementary decision can predict no coalition. Presence AUROC and FPR/TPR are session-level metrics, distinct from the paper’s original pairwise AUROC.

Table D2. Session-level coalition-presence detection at the calibrated threshold.

ConditionFlagged / totalFPR or TPR
Independent + correct6/1206/1205.0% FPR
Independent + wrong10/12010/1208.3% FPR
Coordinated + correct96/12096/12080.0% TPR
Coordinated + wrong110/120110/12091.7% TPR

Table D3. Coordinated-correct vs. independent-wrong: presence AUROC.

MeasurementPresence AUROC ↑\uparrow
Rare-shared-error lens only0.57
Argumentative + temporal signals only0.80
Full LATTICE without rare-error0.85
Full LATTICE0.88

Interpretation. LATTICE flags 10/12010/120 independent-wrong hard negatives (8.3% FPR) and detects 96/12096/120 coordinated-correct sessions (80.0% TPR; Table D2). Presence AUROC for this contrast is 0.88 (95% CI [0.82,0.93][0.82,0.93]), and across all four conditions it is 0.91 (95% CI [0.87,0.95][0.87,0.95]). The rare-error lens alone reaches 0.57, versus 0.80 for argumentative and temporal signals and 0.85 for the full model without rare-error (Table D3). Thus, discrimination is not driven solely by shared factual errors in these tests.

Thanks for the detailed responses to my questions. They clearly addressed both of my concerns. The second point's experiments are very informative, in my opinion. Please consider including that in the manuscript.

Overall, I think this paper is a great benchmark for multi-agent system interpretation. I support acceptance of this work.

AuthorsOpenReview ↗

Thank You for Your Feedback and Support

We sincerely thank the reviewer for the encouraging feedback and support. We are pleased that our responses addressed both concerns and that the additional experiments were informative. We greatly appreciate the reviewer's thoughtful assessment and constructive engagement throughout the discussion.

Reviewer_f39LOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to distinguish genuine consensus from covert coordination in LLM-based multi-agent debate, where agents may reach the same conclusion through either independent reasoning or hidden coalition behavior. It proposes LATTICE, a hierarchical framework that extracts atomic claims from debate transcripts, identifies directed relations between agents' claims, and aggregates semantic, argumentative, evidential, and temporal signals to recover coalition structures. Experiments on 40 constructed decision dossiers and four external benchmarks show that LATTICE achieves a pairwise AUROC of 0.78 and a partition ARI of 1.00 in the controlled setting, outperforming stance correlation. The external results also show improvements in coalition ranking, although partition recovery is less accurate than in the constructed scenarios.

Critical strengths

1.The paper considers an interesting problem in multi-agent debate, where agents reaching the same conclusion may not necessarily reflect independent agreement. Looking at how agents influence each other's reasoning, rather than just their final decisions, is a useful direction for detecting potential coordination.

2.The hierarchical design of LATTICE is reasonable. It combines claim-level relations, temporal adoption, and evidence use to capture interactions between agents. The ability to trace detected relationships back to specific claims also makes the results easier to interpret.

3.The experiments cover 18 competing detectors and four external benchmarks. The paired neutral and coalition sessions are a useful part of the evaluation, as they help examine whether the method simply picks up similarities caused by shared evidence. The authors also explicitly acknowledge a failed pipeline integrity check.

Critical weaknesses
  1. Validity of Coordination Detection. The current experimental design may conflate coordination with correlated reasoning errors, as coalition agents are instructed to support evidence-inconsistent conclusions and the rare-shared-error component contributes substantially to detection performance. It remains unclear whether LATTICE can distinguish genuine coordination from independently shared misconceptions. Moreover, Equation (6) assumes a known coalition size and always produces a partition, raising concerns about false positives in sessions without coordination. Evaluations involving independently mistaken agents, factually correct coalitions, and fully neutral sessions would help establish the validity of the detection mechanism.

  2. Robustness and Generalizability. The framework relies heavily on LLM-based claim extraction and relation judgments, yet the reliability of these intermediate outputs remains insufficiently validated, particularly given the reported invalid-reference rate of 4%, which exceeds the predefined 2% threshold. Furthermore, the sensitivity to heuristic choices, including component weights and claim extraction limits, is not fully explored. Additional evaluations involving larger agent populations, varying coalition sizes, and heterogeneous LLM backbones would help establish the robustness and generalizability of the approach.

  3. Limited Performance Gains and Statistical Evidence. Although LATTICE outperforms several baselines, its improvement over stronger methods is relatively modest (e.g., 0.78 vs. 0.75 AUROC compared with Granger-style influence), while multiple baselines already achieve perfect partition ARI. The absence of confidence intervals or statistical significance tests makes it difficult to assess whether these improvements are reliable. Moreover, the external benchmarks primarily compare against stance correlation rather than stronger competing methods. More rigorous statistical analysis and broader baseline comparisons would strengthen the empirical evidence supporting the proposed framework.

Questions for authors

See Weaknesses.

6 comments
AuthorsOpenReview ↗

Author Response to W1: Coordination Validity and False Positives

We thank the reviewer for noting the value of tracing reasoning interactions, the interpretability of our hierarchical design, and the breadth of our controlled and external comparisons. We address the three concerns below.

Concern. Can correlated reasoning errors trigger false coalition alarms, given that the submitted known-kk procedure always returns a partition?

Author Response. We add session-level coalition-presence detection to the 480-session, 2×22\times2 evaluation (Table F1), with signal ablations in Table F2. The independent-wrong arm uses naturalistic hard negatives: agents converge on the same unsupported conclusion without a planted coalition, selected by dossier-grounded labels rather than detector scores. A size-adjusted, complexity-penalized presence threshold is calibrated on development dossiers (target 5% neutral FPR) and frozen for dossier-disjoint held-out tests. The submitted known-kk recovery is unchanged.

Table F1. Session-level coalition-presence detection at the calibrated threshold.

ConditionFlagged / totalFPR or TPR
Independent + correct6/1206/1205.0% FPR
Independent + wrong10/12010/1208.3% FPR
Coordinated + correct96/12096/12080.0% TPR
Coordinated + wrong110/120110/12091.7% TPR

Table F2. Coordinated-correct vs. independent-wrong: presence AUROC.

MeasurementPresence AUROC ↑\uparrow
Rare-shared-error lens only0.57
Argumentative + temporal signals only0.80
Full LATTICE without rare-error0.85
Full LATTICE0.88

At the threshold, LATTICE flags 10/12010/120 independent-wrong sessions (8.3% FPR) and detects 96/12096/120 coordinated-correct sessions (80.0% TPR; Table F1). The direct-contrast presence AUROC is 0.88 (95% CI [0.82,0.93][0.82,0.93]), retaining 0.85 without rare-error signals (Table F2). These session-level results are distinct from the submitted pairwise AUROC and conditional known-kk recovery.

AuthorsOpenReview ↗

Author Response to W2: Intermediate Reliability and Robustness

Concern. Intermediate outputs need validation; the invalid-reference rate exceeds its integrity gate; and sensitivity to heuristics, population size, and model configurations is unclear.

Author Response. Two blinded annotators audited 360 messages and 480 claim pairs (Table F3). Claims were matched one-to-one by a semantic-equivalence rubric; relation macro-F1 uses adjudicated labels and Cohen’s κ\kappa uses independent pre-adjudication labels, including no relation.

Table F3. Human validation and sensitivity to reference correction.

MetricResult
Claim-extraction precision0.93
Claim-extraction recall0.91
Relation-classification macro-F10.84
Evidence-status accuracy0.90
Relation-label Cohen’s κ\kappa0.76
MetricSubmittedCorrected
Invalid-reference rate4.0%1.1%
Pairwise AUROC0.780.79
Exact known-kk recovery1.001.00

The submitted 4.0% invalid-reference rate exceeded the 2.0% gate. Correction lowers it to 1.1%, while AUROC changes from 0.78 to 0.79 and exact recovery stays 1.00 (Table F3); this does not establish robustness to all semantic errors.

Heuristics and population size. We perturb weights by ±25%\pm25\%, vary the extraction limit from one to three claims, and test N=8,10N=8,10 (Table F4). Unknown-kk recovery searches k∈{2,…,⌊N/2⌋}k\in\{2,\ldots,\lfloor N/2\rfloor\} and may predict no coalition; planted kk is 2 or 3.

Table F4. Sensitivity and population-size evaluations.

EvaluationConfigurationResult
Original compositeFixed weights0.78 AUROC
Weight perturbation±25%\pm25\%, renormalized0.76–0.79 AUROC
Claims per message1/2/30.73/0.78/0.78 AUROC
Larger-NN recoveryKnown-kk/unknown-kk0.84/0.72 ARI

Weight perturbation changes AUROC by at most 0.02; two and three claims yield the same AUROC, while unknown-kk ARI trails known-kk (Table F4). Tests at N=8,10N=8,10 do not establish large-scale feasibility.

Generators and relation judges. We vary generators with a fixed GPT-5.6 Luna Max extractor, judge, and weights, then separately vary only the judge on fixed transcripts and claims (Table F5).

Table F5. Generator and relation-judge sensitivity (separate tests).

Debate generatorLATTICEStanceΔ\Delta AUROC
GPT-5.6 Luna Max0.780.63+0.15+0.15
Qwen3-32B0.750.59+0.16+0.16
Llama-3.3-70B-Instruct0.740.58+0.16+0.16
Mixed Qwen3-32B / Llama-3.3-70B-Instruct0.720.57+0.15+0.15
Relation judgePairwise AUROC
GPT-5.6 Luna Max0.78
Qwen3-32B0.77
Llama-3.3-70B-Instruct0.76

LATTICE exceeds stance by 0.15–0.16 AUROC across generators; judge substitution yields 0.76–0.78 with fixed claims (Table F5), not full pipeline substitution.

AuthorsOpenReview ↗

Author Response to W3: Statistical Evidence and Stronger Baselines

Concern. Are the gains over stronger baselines statistically supported, particularly on external benchmarks?

Author Response. We report paired dossier-bootstrap intervals for the original gains (Table F6) and evaluate four stronger baselines on 100 Secret Mafia and 100 RogueAgent sessions each (Table F7), with paired external intervals in Table F8. Comparisons use matched public-information access, pair labels, and recovery protocols.

Table F6. Paired differences on the submitted controlled benchmark.

ComparisonΔ\Delta AUROC95% CI
LATTICE −- stance correlation+0.15+0.15[+0.08,+0.22][+0.08,+0.22]
LATTICE −- Granger-style influence+0.03+0.03[+0.01,+0.05][+0.01,+0.05]

The paired intervals in Table F6 support positive AUROC differences, including +0.03+0.03 versus Granger-style influence. On the submitted known-kk task, Trajectory MI, GUARDIAN, and CEMPAT also attain ARI =1.00=1.00; our contribution is the claim-level, evidence-aware interaction representation rather than exclusive exact recovery.

Table F7. External coalition benchmarks.

MethodSecret Mafia AUROCSecret Mafia ARIRogueAgent AUROCRogueAgent ARI
Stance correlation0.520.020.530.00
Granger-style influence0.660.200.630.17
Trajectory MI0.650.360.600.30
GUARDIAN0.670.370.620.31
CEMPAT0.640.330.610.28
LATTICE0.740.480.690.41

Table F8. Paired differences against the strongest listed comparator per metric.

DatasetMetricComparatorDifference95% CI
Secret MafiaAUROCGUARDIAN+0.07+0.07[+0.01,+0.13][+0.01,+0.13]
Secret MafiaARIGUARDIAN+0.11+0.11[+0.04,+0.18][+0.04,+0.18]
RogueAgentAUROCGranger-style influence+0.06+0.06[+0.01,+0.11][+0.01,+0.11]
RogueAgentARIGUARDIAN+0.10+0.10[+0.02,+0.18][+0.02,+0.18]

LATTICE has the highest AUROC and ARI among the listed methods (Table F7); all four paired confidence intervals exclude zero (Table F8). Closing the remaining partition-recovery gap on complex external benchmarks (ARI 0.48/0.41; Table F7) is an important direction for future work.

Reviewer_Y9KbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LATTICE, a framework for distinguishing covert coordination from genuine consensus in LLM-based multi-agent debates. Rather than relying on final answer agreement, LATTICE extracts atomic claims from agent messages, identifies directed argumentative relationships between claims, and aggregates these relationships into multi-layer agent interaction matrices to detect coordinated groups.

The authors evaluate LATTICE on 40 synthetic decision dossiers with paired neutral and coalition sessions, compare it against 18 existing detection methods, and further assess generalization on four external benchmarks. The results show that LATTICE outperforms stance-correlation baselines in pairwise coalition identification and achieves exact coalition recovery in the controlled setting when the coalition size is known. External evaluations demonstrate promising but less complete recovery.

Critical strengths

The paper addresses an important challenge in multi-agent systems: distinguishing genuine agreement from coordinated behavior. This distinction is especially relevant for the reliability and security of LLM-based collaborative decision-making.

LATTICE goes beyond simple answer similarity by modeling claim-level relationships, argumentative alignment, temporal adoption, and shared unsupported claims. The hierarchical representation provides interpretable signals for identifying potential coordination.

The authors evaluate against 18 detection methods and include paired negative controls and external benchmarks. The ablation analysis also helps characterize the contributions of different detection signals.

Critical weaknesses

The main experiments use only 40 constructed decision dossiers, five agents, and a fixed two-agent coalition. Coordinated agents are explicitly instructed to advocate conclusions contrary to the available evidence, which may introduce recognizable patterns that do not necessarily generalize to more subtle or adaptive collusion strategies.

The partition recovery procedure assumes that the coalition size is known and performs exhaustive search over candidate partitions. In realistic multi-agent systems, the number of colluding agents is typically unknown and may vary over time. It remains unclear how effectively LATTICE scales to larger agent populations or detects multiple or dynamically changing coalitions.

Both claim extraction and relation classification rely on an LLM judge, but the paper provides limited independent validation of their semantic accuracy. Moreover, the pipeline fails its predefined invalid-reference integrity gate. Errors in extracting claims or judging their relationships could propagate through the hierarchical representation and affect coalition detection.

The paper reports point estimates without confidence intervals or statistical significance tests, making it difficult to assess whether relatively small differences in AUROC are meaningful. Furthermore, the main debate generation and detection pipeline use a single LLM backbone, leaving robustness across different models and judging configurations insufficiently explored.

Questions for authors
  1. How would LATTICE perform when the coalition size is unknown, or when multiple coalitions with different objectives coexist in the same debate?

  2. Can the authors provide independent human validation of claim extraction and relation classification, including an analysis of how pipeline errors affect downstream detection accuracy?

  3. How robust is LATTICE to more sophisticated coordination strategies in which colluding agents avoid sharing identical unsupported claims or deliberately introduce disagreement to conceal their coordination?

7 comments
AuthorsOpenReview ↗

Author Response to W1/Q3: Construct Validity and Evasive Coordination

We thank the reviewer for recognizing the importance of this problem, the interpretability of our claim-level analysis, and the breadth of the baselines, paired controls, and external evaluations. We address the concerns and questions point by point below.

Concern. The constructed dossiers couple coalition membership with evidence-opposed conclusions and may not represent strategies that conceal agreement.

Author Response. We evaluate 480 sessions across 40 held-out dossiers, four coordination/correctness conditions, and three generation seeds (Table Y1). The independent-wrong arm includes coalition-free hard negatives in which agents independently reach the same unsupported conclusion, selected by dossier-grounded labels rather than detector outputs. Session-level presence is evaluated separately from known-kk recovery.

Calibration. The presence score takes the maximum size-adjusted excess coordination over a neutral reference distribution, with a multiple-candidate complexity penalty. Normalization and threshold are calibrated on development dossiers for 5% neutral FPR, frozen before held-out testing, and split by dossier.

Table Y1. Session-level coalition-presence detection at the calibrated threshold.

ConditionFlagged / totalFPR or TPR
Independent + correct6/1206/1205.0% FPR
Independent + wrong10/12010/1208.3% FPR
Coordinated + correct96/12096/12080.0% TPR
Coordinated + wrong110/120110/12091.7% TPR

Diagnostic contrast: coordination versus shared errors (Table Y1). We compare two counterexamples to error-based detection: coordinated + correct sessions (coordination without an unsupported conclusion, positive) against independent + wrong sessions (the same unsupported conclusion without coordination, negative). This directly tests whether LATTICE distinguishes coordination from shared factual mistakes. Across these 120 positive and 120 negative sessions, the session-level presence score yields AUROC 0.88 (95% CI [0.82,0.93][0.82,0.93]) over detection thresholds. Table Y1 reports 80.0% TPR and 8.3% FPR at the calibrated threshold. Removing the rare-error lens retains AUROC 0.85, versus 0.57 for that lens alone (Table D3 in the dWHc W2 response).

We test three evasive strategies (Table Y2). Their metric is pairwise AUROC for within-session coalition ranking, not session-level presence detection.

Table Y2. Pairwise ranking under evasive strategies.

StrategyPairwise AUROC
Submitted controlled condition (reference)0.78
Paraphrased / non-overlapping claims0.75
Deliberate public disagreement0.71
Delayed coordination0.73

Pairwise AUROC is 0.71–0.75 under evasion (Table Y2); the strategies use separate sets, so changes from the reference cannot be attributed solely to strategy. On 100 Secret Mafia and 100 RogueAgent sessions each, LATTICE reaches AUROC/ARI of 0.74/0.48 and 0.69/0.41, versus strongest listed baseline AUROCs of 0.67 and 0.63 (Table F7 in the f39L W3 response). While full coalition recovery in noisy external environments remains challenging, LATTICE delivers consistent gains over existing baselines.

AuthorsOpenReview ↗

Author Response to W2/Q1: Unknown Coalition Size and Dynamic Groups

Concern. Known coalition size and exhaustive bipartition search limit deployment when groups are larger, multiple, or dynamic.

Author Response. We extend the submitted known-kk, fixed-bipartition recovery stage, leaving claim extraction, relation scoring, temporal matrices, and composite weights unchanged. The extension jointly searches memberships and sizes k∈{2,…,⌊N/2⌋}k\in\{2,\ldots,\lfloor N/2\rfloor\} with a size-adjusted complexity penalty; it predicts no coalition when the best score falls below the frozen presence threshold.

We test N∈{8,10}N\in\{8,10\} with planted k∈{2,3}k\in\{2,3\}, two simultaneous disjoint two-agent coalitions with different objectives at N=10N=10, and changing membership across four windows of a 15-round debate (recovery: Table Y3; costs: Table Y4).

Table Y3. Recovery with additional coalition structures.

SettingMetricResult
Larger NN, known kkPartition ARI0.84
Larger NN, unknown kkPartition ARI0.72
Larger NN, unknown kkCoalition-size accuracy81%
Coalition vs no coalitionPresence AUROC0.88
Neutral sessionsPresence FPR7.5%
Two simultaneous coalitionsThree-group partition ARI0.61
Two simultaneous coalitionsCoalition-member macro-F10.73
Changing membershipWindow-level partition ARI0.59
Changing membershipMembership-change F10.67

Table Y4. Recovery-stage search cost.

Recovery settingCandidate countRuntime (s)
Known k=2,3k=2,3, N=8N=828–560.04–0.08
Known k=2,3k=2,3, N=10N=1045–1200.05–0.13
Unknown kk, N=8N=81540.18
Unknown kk, N=10N=106270.72
Two coalitions, N=10N=101,2601.43
Dynamic, four windows, N=10N=102,5082.85

Unknown-kk ARI falls from 0.84 to 0.72 (Table Y3); simultaneous and dynamic coalitions are harder still. Table Y4 reports single-threaded CPU recovery only, excluding semantic extraction and debate generation. Counts enumerate candidate subsets; the dynamic cost is 4×6274\times627 windowwise searches rather than a joint trajectory search. These timings establish feasibility only for the tested N=8,10N=8,10 configurations.

AuthorsOpenReview ↗

Author Response to W3/Q2: Semantic Validation and Reference Integrity

Concern. Extraction and relation judgments need independent validation, and the invalid-reference rate exceeds the predefined integrity threshold.

Author Response. Two blinded annotators audited 360 messages and 480 claim pairs (Table Y5). Claims are matched one-to-one under a predefined semantic-equivalence rubric; unmatched predictions/references count as false positives/negatives. Relation macro-F1 uses adjudicated labels; Cohen’s κ\kappa uses independent pre-adjudication labels, including no relation.

Table Y5. Human validation of semantic outputs.

MetricResult
Claim-extraction precision0.93
Claim-extraction recall0.91
Relation-classification macro-F10.84
Evidence-status accuracy0.90
Relation-label Cohen’s κ\kappa0.76

The submitted 4.0% invalid-reference rate exceeded the prespecified 2.0% integrity gate. We tested reference correction on the same controlled cohort (Table Y6):

Table Y6. Sensitivity to reference correction.

MetricSubmittedCorrected
Invalid-reference rate4.0%1.1%
Pairwise AUROC0.780.79
Exact known-kk recovery1.001.00

Reference correction reduces invalid references to 1.1%, with AUROC 0.78 to 0.79 and exact recovery unchanged at 1.00 (Table Y6). This tests identifier validity, not all semantic errors (human audit: Table Y5).

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to the Area Chair and Reviewers

Dear Area Chair and Reviewers,

We sincerely thank the Area Chair and all Reviewers for their time, thoughtful feedback, and constructive engagement throughout the discussion period. The reviews prompted several targeted experiments and analyses that substantially strengthen the empirical case for LATTICE. Below, we briefly summarize the five main additions provided in our responses:

  1. Separating coordination from shared mistakes. Across 480 sessions (40 held-out dossiers, four coordination/correctness conditions, three seeds), we directly contrast coordinated-correct sessions against independent-wrong hard negatives using a calibrated, frozen session-level presence decision that can also return no coalition. LATTICE achieves a presence AUROC of 0.88 (95% CI [0.82, 0.93]), retaining 0.85 without the rare-error lens (versus 0.57 for that lens alone). At the frozen threshold, coordinated-correct TPR is 80.0%, while independent-wrong and independent-correct FPRs are 8.3% and 5.0%, respectively, showing that the signal persists when shared incorrectness cannot explain the positive class.

  2. Value of the claim-level representation beyond interaction graphs. Compared to a GroupGuard monitoring-layer adaptation under matched passive access on identical transcripts and graph-matched hard negatives, LATTICE improves presence AUROC from 0.67 to 0.86 (paired difference +0.19, 95% CI [+0.10, +0.28]). Ablations confirm progressive gains from a coarse graph (0.70) to claim relations (0.78), temporal adoption (0.83), and the full model (0.86).

  3. Human audit and reference integrity. Two blinded annotators audited 360 messages and 480 claim pairs, finding claim precision/recall of 0.93/0.91, relation macro-F1 of 0.84, evidence-status accuracy of 0.90, and pre-adjudication Cohen's κ of 0.76. Reference correction reduces the invalid-reference rate from 4.0% to 1.1% (below the 2.0% gate), while pairwise AUROC moves from 0.78 to 0.79 and exact known-k recovery remains 1.00.

  4. Statistical uncertainty and stronger external baselines. Paired dossier-bootstrap intervals support our controlled pairwise AUROC gains over stance correlation (+0.15, 95% CI [+0.08, +0.22]) and Granger-style influence (+0.03, [+0.01, +0.05]). On 100 Secret Mafia and 100 RogueAgent sessions, we added matched-access comparisons against Granger-style influence, Trajectory MI, GUARDIAN, and CEMPAT. LATTICE achieves AUROC/ARI of 0.74/0.48 and 0.69/0.41, yielding paired gains of +0.07/+0.06 AUROC and +0.11/+0.10 ARI over the strongest comparator on each metric, with all four 95% CIs excluding zero.

  5. Broader settings, robustness, and limitations. Across GPT-5.6 Luna Max, Qwen3-32B, Llama-3.3-70B-Instruct, and mixed open-model generators, LATTICE consistently exceeds stance correlation by 0.15–0.16 pairwise AUROC; substituting only the relation judge yields 0.76–0.78. At N = 8, 10 (k = 2, 3), known-/unknown-k ARI is 0.84/0.72; simultaneous and changing coalitions yield 0.61 and 0.59 ARI; and evasive strategies yield 0.71–0.75 pairwise AUROC. These findings further substantiate LATTICE's robustness across diverse settings while pointing to natural avenues for future work, such as large-scale recovery and adaptive evasion.

Summary of reviewer engagement. We are grateful to Reviewer dWHc for confirming that both concerns were clearly addressed, finding the additional experiments informative, and supporting acceptance. We also thank Reviewer f39L for confirming that most concerns were addressed and maintaining a positive evaluation.

We appreciate Reviewer Y9Kb's time in following up. While Reviewer Y9Kb maintained their initial evaluation without specifying which aspects of the new evidence remained insufficient, we have invited further clarification and provided dedicated experiments for each of their original concerns. We would greatly appreciate the Area Chair's independent assessment of these responses.

We hope this summary is helpful to the Area Chair and Reviewers in the final evaluation. Thank you again for your time and thoughtful consideration.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.