acceptodds
Under review as a conference paper at ICLR 2027

Can LLM Agents Communicate Without Autoregression? Thought Blocks for Latent Multi-Agent Collaboration

Abstract

LLM-based multi-agent systems typically communicate through text, incurring repeated generation and re-encoding costs. Latent communication removes explicit verbalization, but constructing a latent message may still require a sequential latent trajectory. We therefore distinguish message representation from message construction and ask: can an agent directly construct a useful continuation state without executing the trajectory that normally produces it? We formulate this as trajectory-to-state compilation and instantiate it with Thought Blocks, fixed-size blocks of latent key–value (KV) memory constructed in one shot. Thought Blocks map a compact continuation representation and compressed contextual KV memory to parallel latent positions, which are rendered into model-native KV continuation states in a single frozen-backbone forward pass. We train the writer by distilling continuation states reached by an autoregressive latent teacher, whose trajectory is omitted at inference. Across multiple LLM backbones and reasoning and question-answering tasks, Thought Blocks substantially reduce sequential communication computation and latency relative to autoregressive latent communication while maintaining competitive accuracy, achieving a 2.5 times average latency speedup. Compared with text-based multi-agent communication, the efficiency gains are broader still, reducing text-token traffic, autoregressive computation, and latency.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multi Agent Systems;Latent Multi-Agent Collaboration;Non-Autoregressive Generation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_y5kHOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes thought blocks, a mechanism for multi-agent LLM to reduce latency. Instead of generating sequential intermediate latent states, this paper proposes to build fixed-size KVC continuation states. This is done by distilling continuation states from a teacher's sequential rollout. Across experiments, the methods reports 2.5x latecy improvement over sequential latent communication.

Critical strengths
  1. The method seems practically implementable.
  2. Experiments show substantial improvements over baseline.
Critical weaknesses
  1. insufficient comparisons against closely related SOTA. For example, "Cache-to-Cache: Direct Semantic Communication Between Large Language Models" and "Parallel Continuous Chain-of-Thought with Jacobi Iteration".

  2. all experiments are done in one fixed controlled setting, so wider applicability is still in question. For example, iterative collaboration (may benchmark with "Sanzaru: Benchmarking Protocol Formation under Epistemic & Communication Asymmetry"), different communication graphs, or even heterogeneous agents will be very informative on wider applicability.

Questions for authors

See weaknesses

Reviewer_TaM9OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces non-autoregressive latent communication, separating how an inter-agent message is represented from how it is constructed. Its proposed Thought Blocks directly generate fixed-size KV continuation memory from a layer-wise continuation seed and compressed context bank. Learned slots expand the seed into parallel virtual positions, which a frozen backbone renders into KV states in one forward pass. Training distills states reached by an autoregressive latent teacher, whose trajectory is removed at inference.

Across four backbones and eleven reasoning, question-answering, and reading-comprehension configurations, Thought Blocks largely preserve accuracy while reducing communication passes, autoregressive steps, latency, and decoding-time KV load relative to text and autoregressive latent communication. Ablations support the importance of learned block content, both state components, and the chosen block size.

Critical strengths

1/ Clear and consequential problem formulation. The paper distinguishes latent-message representation from latent-message construction and introduces trajectory-to-state compilation. This reframing exposes sequential construction as a remaining bottleneck in latent communication and suggests a broadly useful direction beyond the specific architecture proposed.

2/ Convincing efficiency validation with targeted ablations. Across four backbones and eleven benchmark configurations, Thought Blocks substantially reduce autoregressive steps, latency, and KV-cache load while largely preserving accuracy. The ablations in Tables 3–5 show that learned block content and both components of the factorized sender state are necessary, strengthening the claim that the gains come from the proposed mechanism rather than simply adding latent positions.

Critical weaknesses

1/ The training data are insufficiently specified. Appendix B.1 refers only to an unidentified training corpus D, while Appendix B.5 gives optimization settings without reporting the corpus source, size, task composition, or overlap controls for Beacon and writer training. Because the method requires learned, backbone-specific components, this omission prevents assessment of contamination and generalization and makes the results difficult to reproduce.

2/ The evaluation does not establish an advantage over simpler direct latent-state transfer. Section 4.1 compares primarily against Text-MAS and an autoregressive Latent-MAS baseline, despite Table 1 discussing non-autoregressive activation/KV-transfer methods such as AC, KVCOMM, C2C, LCF, and StateBridge. The reported speedup over Latent-MAS is expected when twelve sequential communication passes are replaced by three parallel passes; without comparison to efficient direct-transfer alternatives, it remains unclear whether trajectory-to-state compilation provides benefits beyond avoiding autoregression.

Questions for authors

1/ What corpora, sample counts, and task mixtures were used for Beacon training and trajectory-to-state distillation for each backbone? What decontamination procedures ensured no overlap with the eleven evaluation benchmarks?

2/ Appendix B.5 describes a 12-step teacher rollout, while Table 2 reports 12 total Latent-MAS communication passes across three inter-agent transfers. Is the 12-step horizon applied per message or per complete episode, and are the teacher and Latent-MAS trajectory budgets matched?

3/ Under the same four-agent setup, how does Thought Blocks compare with directly transferring or compressing the already-materialized sender KV state? This comparison would materially determine whether trajectory-to-state compilation adds value beyond simply avoiding autoregressive rollout.

Suggestions for future work: None; these questions concern the validity and interpretation of the paper’s current claims.

1 comment

Dear Authors,

Thank you for you efforts! Your response has fully addressed my concerns and questions. I will update my score to 3: Weak Acceptance.

Reviewer_sMcmOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Thought Blocks, a learned writer that replaces autoregressive latent-message construction with one parallel expansion and one frozen-backbone rendering pass. Learnable slots attend to the sender's layer-wise boundary hidden states, and the resulting virtual embeddings are rendered against a Beacon-compressed context bank to produce outgoing KV memory. Training matches KV and hidden states extracted from an autoregressive latent teacher. Experiments use four backbones and eleven benchmark configurations in a four-agent sequential pipeline. The paper reports an average 2.5× speedup over Latent-MAS, with an average accuracy difference of −0.72 points. Final-answer generation remains autoregressive.

Critical strengths
  1. The distinction between message representation and message construction is useful. Avoiding text does not remove the sequential cost of producing latent messages. The proposed implementation addresses that cost directly: block embeddings are available together before rendering, so causal attention within the rendering pass does not require token-by-token generation. Distilling the model-native states consumed by the next agent is a reasonable way to train this interface without updating the underlying language-model weights.

  2. The experiments show a potentially worthwhile tradeoff across several backbones. Table 2 reports communication passes decreasing from 12 to 3, and substantial latency reductions in many settings. The paper also provides full per-dataset results, a block-size sweep, and controls replacing the learned embeddings or removing continuation-state components. On Qwen3-8B, replacing the learned embeddings with zeros reduces dataset-macro accuracy from 79.64 to 73.44. This supports the importance of the constructed embeddings beyond simply allocating additional positions.

Critical weaknesses
  1. The comparisons do not yet isolate the contribution of trajectory-to-state distillation. The student uses a separately trained Beacon mechanism and a compact outgoing block, while the paper does not establish that the Latent-MAS baseline uses the same compressed context, retained KV budget, and extraction rule. Matching roles and backbone weights is insufficient to isolate message construction when the accessible memory also changes. The internal autoregressive teacher is a natural matched baseline, but its downstream accuracy and latency are not separately identified. Table 3's zero/random and R-only/C-only interventions show that the proposed inputs and learned embeddings matter; they do not show how much the post-trajectory teacher targets contribute over distilling a prefill-only state. Please also clarify whether component-removal variants were retrained. A focused comparison on one backbone would substantially strengthen the central compilation claim.

  2. Missing protocol details and heterogeneous outcomes make the headline tradeoff difficult to assess. Appendix B describes the training equations and several hyperparameters, but does not identify the Beacon/writer training corpora, writer-training example count, benchmark splits and evaluation counts, or checkpoint-selection data. It also omits inference sampling/output limits and a repeated timing protocol. These omissions prevent assessing data separation and the reliability of small average accuracy differences. Per-dataset tradeoffs are less uniform than the average suggests: on Qwen3-8B, MedQA-US falls from 77.34% to 71.16% relative to Latent-MAS, while GPQA latency increases from 2.83 to 7.19 seconds and AR steps rise from 16 to 106. Thus, eliminating communication recurrence does not establish a consistent end-to-end advantage or negligible utility loss. These exceptions warrant explanation and paired uncertainty, rather than relying on category averages alone.

Questions for authors
  1. Does the reported Latent-MAS baseline use the same Beacon checkpoint, context-bank construction, and outgoing KV budget as the student? Can the actual compressed autoregressive teacher be evaluated as a matched reference, alongside a prefill-only/no-rollout target control on one representative backbone? Please reconcile Table 2's 12 total communication passes with the per-teacher rollout horizon in Appendix B.5.
  2. What data and example counts are used in each training stage, how are they separated from each benchmark's evaluation split, and what data select the writer checkpoint? Please specify the retained teacher positions and their positional/RoPE correspondence to student block positions, as well as whether training records use teacher- or student-generated incoming messages.
  3. What are the inference decoding settings and timing protocol? Can paired results or uncertainty be provided for the large MedQA-US accuracy loss and the GPQA slowdown, including the changed final-decoding workload?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.