acceptodds
Under review as a conference paper at ICLR 2027

Probing Multi-Agent Coordination with Multivariate Multifractal Detrended Fluctuation Analysis

Abstract

Understanding how information access shapes language-model team interactions requires diagnostics that capture semantic variation across scales. We introduce a semantic-trajectory diagnostic that applies a multivariate generalization of multifractal detrended fluctuation analysis to message embeddings, quantifying how accumulated variation changes with message window length under different weightings of small and large residual fluctuations. We test its sensitivity to information access by comparing shared and restricted access to prior exchanges in 100 matched pairs of four-agent realizations across four repeated task templates. Four structured reordering controls probe the contribution of message order. Shared history is associated with broader fluctuation-scaling spectra, with the largest differences at orders emphasizing smaller residual fluctuations. This contrast persists across alternative ordering controls and stricter fit requirements, and its direction is consistent across five encoders. These results demonstrate that the diagnostic detects differences in multiscale semantic structure associated with team information access in a consistent direction and that it generalizes across embedding models and robustness checks. By combining a multiscale semantic measure with a controlled history-access comparison, this work provides a methodological foundation for studying language-model team interactions and for identifying semantic failures.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Multi-agent language models, Semantic trajectories, Multivariate multifractal detrended fluctuation analysis, Message embeddings

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_buthOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper applies a trace-based multivariate formulation of MF-DFA to message-embedding trajectories from teams of language-model agents, examining how access to conversation history affects their finite-scale fluctuation structure. The authors generate 100 matched pairs of runs involving four agents and four repeated task templates, comparing shared and local history access. The main analysis uses 83 pairs that satisfy the fit criteria. Shared-history runs have broader spectra, with the largest differences in generalized scaling exponents occurring at negative moment orders. The direction of the contrast persists across structured reordering controls, five embedding models, and stricter fit criteria. A complementary covariance analysis finds that semantic variation is more concentrated in a smaller number of directions under shared history. The results support a consistent signature of the history-access intervention in this corpus. They do not establish the descriptors as measures of coordination quality or detectors of semantic failures, a distinction the authors explicitly acknowledge.

Critical strengths
  1. The history-access contrast is unusually consistent within the experimental setting. Across the 83 retained pairs, the mean shared-minus-local difference in task-block-adjusted spectrum width is +0.20664, with a 95% confidence interval of [0.19855, 0.21438], and the difference is positive in every pair. Requiring all five diagnostics retains 81 pairs and yields a difference of +0.20627. The direction also persists across all five embedding models on the common retained sample (Tables 1 and 2; Section 6). These checks support the conclusion that the main observation does not depend solely on one embedding representation or a permissive fit criterion. The central finding of intervention sensitivity is persuasive within this restricted setting.
  2. The estimator is explicitly specified, and its interpretation is appropriately cautious. The trace-based construction remains computable when the covariance matrices are singular because the windows are short relative to the 768-dimensional embedding space. The paper specifies the moment weighting, reordering controls, and pair-level inference, and reports numerical agreement with explicit detrending (Appendices A and C.1). The authors also acknowledge the weak within-condition associations with task scores and the lack of established validity for failure detection. Their decision not to equate spectrum width directly with reasoning quality makes the contribution easier to assess fairly.
Critical weaknesses
  1. The added diagnostic value beyond distinguishing the intervention conditions is insufficiently established. The demonstrated result is that the statistic distinguishes history-access conditions for one generator and four repeated task templates. Although pooled correlations with task scores are approximately 0.77, within-condition associations are small, and the correlations between paired changes in the three raw spectrum descriptors and changes in score are 0.104, −0.001, and 0.157, with all confidence intervals including zero (Section 5.6; Appendix C.7). Much of the pooled association may therefore reflect separation between the conditions. Phrase-match scores are also not an independently validated measure of coordination quality. Moreover, the six conventional summaries already classify history condition with 95.8% accuracy, compared with 100% for raw spectrum width (Table 11). High sensitivity to the condition contrast alone does not establish what useful diagnostic distinction the spectrum adds. I recognize that the authors’ central claim concerns sensitivity rather than failure prediction, and weak score associations do not invalidate that narrower claim. Nevertheless, assessing the significance of the methodological contribution requires interpretable validation, such as distinguishing evidence integration from repetition.
  2. The influence of the repeated task structure on the finite-scale descriptors is insufficiently calibrated. The analysis covers only 8–48 messages, approximately 0.78 decades, and crosses fixed 16-message task boundaries. History resets between tasks, while the same four sets of questions and evidence are repeated. The structured shuffles establish sensitivity to the specified reorderings, but do not show how much spectrum width or the negative-order contrast can be generated by task-position-dependent means, heterogeneous residual variance, or boundary placement. The five embedding models remeasure the same message sequences, and the stricter fit criteria select a subset of the same corpus. Main inference also conditions on asymmetric fit retention: 84 shared-history runs versus 98 local-history runs. Appendix B.5 documents changes to the reporting policy after inspecting the data. The similar estimate in the additional cohort is reassuring, but its independence from the final analysis policy needs clarification. The authors’ finite-scale framing is appropriate, and I am not asking for proof of asymptotic multifractality. The unresolved issue is what the descriptors respond to under this experimental design. Applying the same analysis, including fit-based selection, to controls matched in length and task structure would directly address this concern.
Questions for authors
  1. Validation of added diagnostic value Beyond distinguishing history conditions, what property of the conversation can be understood from the spectrum? Could the authors demonstrate within-condition associations with independently assessed evidence integration or constraint use, or sensitivity to controlled transcript modifications that match lexical repetition, message length, and task/routing structure? This is a request for construct validation, not necessarily for task-score prediction.
  2. Calibration for finite length and task boundaries To what extent can length-matched controls that preserve the four-template cycle and within-task position structure reproduce the observed widths or condition contrasts? Please apply the same scale grid, detrending, and fit-selection procedure, and assess sensitivity to window alignment relative to the 16-message boundaries. Existing task-block permutations change the arrangement of the tasks themselves and therefore do not resolve this question on their own.
  3. Clarification of reported results Section 5.6 states that the descriptors add no held-out prediction of task score beyond condition and the six conventional summaries. However, Appendix C.8 and Table 11 report history-condition classification. Where are the task-score prediction model, evaluation procedure that keeps matched pairs together, and comparative metrics reported? Please provide these results or revise the statement to reflect the association analyses actually presented. Similarly, please provide the adjusted estimate supporting the statement in Section 6 that task-block adjustment reduces the hidden-channel contrast. These omissions do not demonstrate a computational error, but the claims cannot currently be independently checked from the reported results.
  4. Timing of analysis decisions Which choices concerning scales, moment orders, fit thresholds, and the primary control were fixed before the additional 80 pairs were examined? I appreciate the similar cohort-specific estimates in Table 5, but would like the independence of the additional-cohort evaluation to be made explicit. Please also explain how differential fit retention limits the population of runs to which the inference applies.
1 comment

We thank the reviewer for reviewing the manuscript. The reviewer finds the central result “persuasive” and notes that our claim concerns sensitivity rather than failure prediction. The remaining concerns are construct validation (Q1), calibration against matched controls (Q2), two reporting gaps (Q3), and the timing of analysis decisions (Q4). We ran the Q2 control as specified, corrected both Q3 statements, and documented the Q4 timing.

  1. Q2 / W2: The requested control reproduces less than a fifth of the contrast

The reviewer asked for “length-matched controls that preserve the four-template cycle and within-task position structure,” analyzed with “the same scale grid, detrending, and fit-selection procedure.” We built this control by permuting messages across repetitions of the same template at the same within-task position. It preserves run length, the template schedule, every boundary location, and the embedding distribution at each template position, including position-dependent means and heterogeneous residual variance. We applied the same 8–48 grid, detrending, and R² ≥ 0.95 rule, refit on every draw (§5.4, Table 2, App. C.6).

Width: the control contrast is +0.053 versus the observed +0.284. With the fit rule reapplied to each draw, all 100 pairs are retained and the control contrast is 0.052–0.056. Position-dependent means, variance heterogeneity, and boundary placement together account for less than 20% of the observed contrast. Negative orders: the observed h(−2) difference exceeds the control by 0.135 [0.127, 0.143], about 70% of the observed 0.194; at q = 2 the excess is only 0.014 [0.013, 0.015]. The asymmetry the reviewer identified survives the control. Boundary alignment: across 16 equal-length crops that shift every window relative to the resets, the contrast stays positive: +0.156 to +0.323 on the fixed 83 pairs, and +0.154 to +0.294 with the fit rule reapplied at each offset (86–100 pairs retained).

Under this design, the descriptors therefore respond mainly to dependencies among messages within a conversation, which is the structure history access changes. We agree that 8–48 messages span less than one decade and interpret the spectra only as finite-scale descriptors.

  1. Q1 / W1: Construct validation, integration versus repetition

The reviewer cites “distinguishing evidence integration from repetition” as an example of interpretable validation. We now measure this directly in the expedition task (§5.7, App. C.11). A deterministic rule separates explicit use of the bridge constraint to reject an option, repetition without application, and conflicting recommendations. Among eligible final decisions, explicit use occurred in 1,097 of 1,196 under shared history (92%) and 502 of 725 under local history (69%). The intervention that broadens the spectrum thus increases integration, not only restatement. Restatement within eight messages rose from 28.7% to 55.2%, and from 14.4% to 51.3% for agents outside the initial exchange. Within each condition, h(−2) − h(2) added no held-out predictive value for explicit use (App. C.11), consistent with a sensitivity claim rather than a per-run predictor.

  1. Q3: Both reporting gaps are corrected

Thank you for identifying these. §5.6 now reports only the held-out evaluation we performed: pair-grouped history-condition classification, with 100% accuracy for raw width alone and 95.8% for the six conventional summaries (App. C.9, Table 14). We removed the unsupported task-score prediction statement and the hidden-channel example. Every quantitative claim now maps to a reported table or appendix.

  1. Q4 / W2: Timing and fit retention

The 19 scales (8–48), nine moment orders, R² ≥ 0.95 threshold, and adjustment by the median of 200 unrestricted task-block permutations were fixed before inspecting any reserved-cohort contrast (App. B.5). The later amendment changed how the structured null was used in reporting, not any numerical choice. The reserved cohort (67 pairs, +0.206) and selection cohort (16 pairs, +0.209) agree; we still label the pooled analysis exploratory.

The paired estimate applies to the 83 pairs in which both runs pass. The contrast keeps its sign as the retained set varies: 81 pairs under all five checks (+0.206), 82–90 pairs across encoders, 86–100 pairs across crop offsets, and all 100 pairs in the concentration analysis (App. C.10). Its direction therefore does not depend on which shared-history runs failed the fit. The encoders remeasure the same messages, so we present them as representation robustness checks, not independent replication.

Summary

The contrast survives the exact control requested in Q2, including at the negative orders the reviewer highlighted. The Q1 integration-versus-repetition distinction is now measured, both Q3 statements are corrected, and the Q4 timing is documented. If these address the reviewer’s concerns, we would be grateful if they would reconsider their score.

Reviewer_yqi9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes a method based on multifractal analysis to study communication between LLM agents. The authors analyze message embeddings from four-agent conversations and compare teams with shared and restricted conversation history. They find differences in the resulting semantic trajectories and show that these differences are consistent across several embedding models.

Critical strengths

I think studying how information is shared between LLM agents is an interesting and relevant problem. I also appreciate that the authors use matched experiments and test their results with different embedding models and shuffling controls.

The differences between the two experimental conditions seem quite consistent.

Critical weaknesses

The authors show that their metric can distinguish between shared and restricted history. However, this doesn't necessarily mean that it captures successful coordination or communication failures. In Figure 3, the correlations with task performance look quite strong. However, the authors acknowledge that these correlations are mostly explained by the difference between the two conditions. Within each condition, the correlations are much weaker. I'm therefore not sure what the proposed method tells us about coordination that we couldn't already learn from knowing how much information the agents can access.

The authors use only one generation model and four task templates. Each task is repeated 16 times per run, using the same questions and evidence. I find this quite limiting for a paper proposing a general method for studying multi-agent coordination.

I'm also concerned about concatenating these repeated tasks into one long sequence. The conversation history resets after each task, but the analysis uses windows that can cross task boundaries. Could this influence the observed patterns?

The authors do include several controls, which I appreciate. Still, I would like to see experiments with more diverse tasks and ideally different agent models.

Questions for authors

How do we know that the proposed metric measures coordination rather than just differences in message content between the two history conditions?

Have you tried predicting actual communication failures or task performance with this metric? I think this would make the contribution much stronger.

Why did you use only four repeated tasks? Would the results be similar if agents worked on different tasks instead of repeating the same ones?

1 comment

We thank the reviewer for a careful and constructive review. We read the concerns as three testable claims: the spectrum may (i) reflect only differences in message content, (ii) re-encode the known access configuration, or (iii) be an artifact of windows crossing task resets. We tested each directly.

  1. Message order, not content, carries the contrast (Q1, W1)

If the difference came from what agents say rather than how messages depend on earlier ones, shuffling messages while preserving content would leave it intact. It does not. Permuting messages across repetitions of the same template at the same position preserves the embedding distribution at every position in both conditions, yet reduces the width contrast from +0.284 to +0.053, an 81% reduction (Table 2, App. C.6). Conventional content summaries (message length and variability, embedding variance, adjacent task-centroid similarity) classify condition at 95.8% held-out accuracy, versus 100% for width alone (Table 14). The spectrum thus measures how strongly messages depend on earlier exchanges (effective team memory), which is what the history rule changes.

  1. Information beyond the known configuration (W1, Q2)

We agree that the pooled correlations of about 0.77 in Figure 3 mainly reflect condition separation; §5.3 and App. C.8 now say so. However, the nominal rule does not determine what agents carry forward: compression, truncation, and summarization change it while the configuration still reads “shared history.” In §5.8, with identical shared access and only the compression threshold varied, the spectral peak shifted monotonically (0.707 → 0.720 → 0.726), whereas the configuration predicts no change. With one run per threshold, this is descriptive.

The spectrum also corresponds to observable behavior. In 3,200 expedition conversations (§5.7, Table 3), 55.2% of non-Checker agents restated the bridge constraint within eight messages under shared history versus 28.7% under local history (paired difference 26.5 pp, 95% CI 24.4–28.6); for agents outside the initial exchange, 51.3% versus 14.4%.

We ran the prediction test the reviewer proposed. Adding the spectral descriptor to a held-out model of whether the final speaker applied the constraint did not improve prediction within either condition (App. C.11). We report this negative result because it fixes the scope: the method is a diagnostic of the team’s memory regime, validated by a controlled intervention, fact-tracing, structured nulls, and five encoders, not a failure predictor (§7–8). We ask that it be evaluated on that basis. Failure prediction requires independently labeled failure events, our next step.

  1. Task boundaries cannot produce the contrast (W3, Q3)

Both arms of each pair share identical boundary positions, template order, evidence, and seeds, so a boundary artifact would have to act differently on the two arms. Three checks show it does not:

Shuffling intact conversations among slots of the same template changes which conversations meet at every boundary, yet leaves the contrast unchanged (+0.287 vs. +0.284, Table 2). Against each run’s own block-permutation null, the adjusted contrast is +0.207 [0.199, 0.214], positive in all 83 pairs (Table 1). Across 16 crop offsets that shift every window relative to the resets, the contrast stays positive (+0.156 to +0.323, Table 2).

Template sequence and window placement affect the magnitude, as we report, but never the sign. Resets kept prompts within the token budget; concatenating 64 matched conversations yields 1,024 messages for scaling over 8–48 messages.

  1. Breadth of tasks and models (W2, W4, Q3)

We repeated four fixed templates deliberately: holding questions, evidence, and order fixed makes history access the only manipulated variable, whereas diverse tasks would confound task variance with condition. The result holds across 100 matched pairs, two pre-specified cohorts (0.209 and 0.206), a stricter five-check fit mask (81 pairs, 0.206), and five embedding models, with all four descriptors keeping their direction (Table 5). We also added a different generator on a continuous 512-turn task without resets (§5.8).

Summary

Each weakness now has a direct test: the contrast depends on message order rather than content (Table 2), is not a boundary artifact (Tables 1–2), corresponds to observable fact spread (Table 3), and has a precisely stated scope, including a negative prediction result (App. C.11). If the reviewer agrees that these analyses resolve the concerns behind the rating, we would be grateful if they would reconsider the score. If concerns remain, we would appreciate knowing which analyses would change the assessment so we can run them during the discussion period.

Reviewer_f3G8OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes a diagnostic that is sensitive to shared information access in language-model teams. The diagnostic is based on embeddings of the agent messages, and extends an existing analysis type (MF-DFA). The results show that this method is sensitive to information access, across a number of different embedding models and robustness checks.

Critical strengths

The paper makes a good effort at formalizing the representation of team communication and construct a diagnostic that seems to do what it is supposed to.

Critical weaknesses

The main weakness of this paper is that while they do a good job of rigorously setting up their formalism and analyzing their results, the significance of the work is not properly explained. Even after reading this whole paper, I'm not sure what the practical use-case for this approach would be.

The paper is very dense, so it wasn't the easiest read. This was exacerbated by it being chock-full of LLM-flavored prose, which tends to often be abstruse and convoluted.

Minor nit: There seem to be way too many significant figures in the results.

Questions for authors

No questions for the authors.

5 comments

We sincerely thank the reviewer for recognizing the rigor of our mathematical framework and for highlighting the need to better articulate its practical significance. By following reviewer f3GB suggestion on readability, we revise and comprehensively explain why analyzing multiscale semantic dynamics matters for multi-agent AI systems. Of note, this is a new problem formulation with critical impact on multi-agent AI systems research. Therefore, we have substantially revised the Introduction and Discussion to clarify the motivation, practical applications, and interpretation of our findings. Practical significance and applications. An unsolved fundamental challenge in multi-agent AI is understanding how information-access mechanisms influence collective communication and coordination. Conventional evaluations primarily measure final task performance, but cannot characterize how the semantic organization of agent interactions changes across communication scales. Our framework is the first to address this knowledge gap by providing a quantitative diagnostic of multiscale semantic dynamics. Specifically, this new mathematical framework enables researchers to model, analyze, and compare alternative multi-agent communication architectures, investigate the effects of shared versus restricted conversational history, and identify changes in interaction dynamics that may warrant further investigation for information-propagation or coordination failures. Such measurements could inform the design of more effective and computationally efficient multi-agent systems, although their utility for selecting architectures remains to be validated.

Concrete example and experimental evidence. We revised the Introduction to illustrate the motivation through an expedition-planning scenario in which one agent identifies a critical bridge-weight constraint, yet the final recommendation may overlook this information. This example highlights the distinction between information being available, successfully propagated, and correctly incorporated into collective decisions. Our experiments compare shared-history and local-history communication architectures. Across 83 retained matched pairs, shared-history interactions consistently exhibit broader multifractal spectra, including after adjustment for task-block reordering. This demonstrates that our diagnostic detects systematic differences in multiscale semantic organization associated with information-access mechanisms, with consistent directional results across five embedding models.

Additional validation and interpretation. To better connect the proposed mathematical diagnostic with observable communication behavior, we added an analysis of information propagation (§5.7 and Table 3). Across 3,200 expedition conversations, 55.2% of agents other than the Checker restated the bridge warning within eight messages under shared history, compared with 28.7% under local history. These findings illustrate the practical importance of information-access mechanisms. Importantly, the spectral feature provided little additional held-out predictive value for warning restatement within either condition (Appendix C.11). We therefore distinguish the demonstrated ability to characterize changes in collective semantic dynamics from the separate challenge of predicting specific communication failures.

Improved clarity and presentation. We also substantially reorganized the manuscript to improve accessibility. We moved the experimental design before the mathematical methodology, introduced the analysis through three intuitive steps before presenting the equations, clarified the interpretation of Figure 3 and its correlations, shortened the figure caption, and rounded numerical estimates and confidence intervals in the Results and Tables 1–2.

We hope these revisions address the reviewer's concerns by making clear both what the proposed framework establishes - robust sensitivity to information-access mechanisms - and why this capability provides a useful foundation for evaluating and ultimately improving multi-agent communication architectures. We appreciate the reviewer's constructive feedback, which helped us substantially strengthen the manuscript.

Reviewer_f3G8OpenReview ↗

Acknowledgment

Thank you for your response. All of these are good improvements to the paper, and I think I understand it a little bit better now. However, some of my main concerns still remain. The nice explanation you have given in this author response:

An unsolved fundamental challenge in multi-agent AI is understanding how information-access mechanisms influence collective communication and coordination. Conventional evaluations primarily measure final task performance, but cannot characterize how the semantic organization of agent interactions changes across communication scales. Our framework is the first to address this knowledge gap by providing a quantitative diagnostic of multiscale semantic dynamics.

is nowhere to be found in the actual paper. I think the introduction needs to do a better job of situating the work (you have already done some of this in the author response!).

Questions that come to my mind as a reader:

  • Why is it important to characterize how the semantic organization of agent interactions changes across communication scales?
  • What is the connection between the semantic organization and task performance, if there is any?

Also, I think the introduction would benefit from a less abrupt beginning. The new version now starts with this sentence:

The shortest route crosses a bridge that cannot carry the vehicle.

The bridge, vehicle, and the task of finding the shortest route have not been mentioned prior to this sentence at all, which is a bit jarring for the reader.

These are just a couple of examples of writing issues that pervade the rest of the paper as well, making it hard to understand what the significance and contribution of the paper is.

Thank you for following up and for identifying where the explanation in our response needed to appear in the paper. We revised and expanded the introduction to explain why the organization of communication across scales matters, what the spectrum measures, and how it relates to task performance.

Why examine multiple scales? A fact can shape the next reply and then drop out of the conversation, or it can continue to inform what agents say several exchanges later. That distinction matters when a decision depends on information introduced early. A final task score records the outcome. Our method characterizes how fluctuations in the messages agents actually produce change as we examine longer stretches of the conversation.

How does this relate to task performance? Our method characterizes temporal correlation patterns in the messages agents exchange. These patterns can reflect how long earlier information continues to shape later messages. The spectrum can therefore help identify communication regimes that are more prone to information-retention failures when spectral patterns are evaluated against independently labeled failures across runs.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.