- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes an implicit communication framework called "Multi-Modal Attention" (MMA). This framework enables multiple Large Language Model (LLM) agents to collaboratively generate a shared response while maintaining their respective local contexts and KV caches. The core concept involves exchanging only the hidden states of the current token at selected Transformer layers, rather than exchanging raw text information or sequence-length-dependent KV caches. MMA introduces a "directional gated cross-branch attention mixer" for intermediate communication and employs a "latent-fusion aggregator" to facilitate shared next-token prediction. Throughout this process, the backbone models remain frozen, with training applied exclusively to the communication and aggregation modules.
Critical strengths
- A well-designed, communication-efficient collaborative decoding framework.
This paper addresses a significant limitation in existing "multi-agent communication" methods: the overhead incurred by text-based communication and the growing burden of KV-cache transmission as sequence lengths increase. The proposed framework exchanges only the hidden states at the current timestep, establishing a compact communication interface where the communication load at each step remains independent of the input context length. Furthermore, the shared decoding mechanism eliminates the cumbersome process of generating separate responses for individual agents before aggregating them. This design holds substantial practical value, particularly for distributed, long-context processing scenarios.
- Promising experimental results and comprehensive evaluation.
MMA demonstrates strong performance across multiple benchmarks, showing exceptional potential—especially in long-context retrieval and heterogeneous task scenarios. Stress tests regarding context length reveal that, compared to KV-cache-based approaches, this framework significantly reduces the logical communication load while maintaining high accuracy. Ablation studies and "evidence removal" experiments further validate the importance of cross-branch communication. These results provide compelling evidence that the framework effectively integrates information from multiple branches, rather than relying merely on task-specific adaptations.
Critical weaknesses
- The core efficiency claims lack validation in a realistic distributed environment.
Although "communication efficiency" is a primary motivation for this work, all latency experiments reported in the paper were conducted on a single NVIDIA RTX 5090 GPU. MMA requires frequent synchronization across multiple Transformer layers during each decoding step; when agents are deployed across different GPUs or machines, this could introduce significant communication and synchronization overhead. Consequently, it remains unclear whether the proposed method truly offers the practical efficiency advantages it claims within the distributed multi-agent scenarios for which it is intended.
- The experimental comparisons fail to adequately isolate the specific gains attributable to the proposed communication mechanism.
MMA introduces a substantial number of trainable parameters (approximately 147 million for the Qwen3-1.7B model and 230 million for the Qwen3-4B model), whereas some baseline methods do not require task-specific training on a comparable scale. Furthermore, the architecture is primarily composed of existing components, and the performance improvements over the strongest baselines are relatively modest.
Questions for authors
-
Distributed Efficiency. Have the authors evaluated the deployment of branched MMA models across different GPUs or machines? Could they provide data on end-to-end latency, throughput, and synchronization overhead within a realistic inter-device communication environment?
-
Fairness of Baseline Comparisons. To what extent can the performance gains of MMA be attributed specifically to the cyclic communication between branches, rather than to additional trainable parameters or task-specific supervision? Could the authors provide a comparison against a baseline that is more closely matched in terms of model capacity?
Open letter to Reviewers cWfB, aWmT, j3u8 and the Area Chair: respond before the discussion closes
Dear Reviewers and Area Chair,
We would like to reminder that the author reviewer discussion ends on Tomorrow. All three ratings are still "2: Weak rejection," and we haven't received responses from the reviewers yet. The Reviewer Guide states: "It is crucial that you are actively engaged during this phase." We look forward to the reviewers' discussion.
Also, we would like to summarize major points for each review report:
Reviewer cWfB. Your review made two critical weakness for rejection: 1. figure fonts (ironically); 2. novelty relative to CALM and C2C and asked no questions. We answered on Nov 6. The fonts are fixed. C2C was already a baseline in the submission. We confirm that MMA beats it on every benchmark with significant less payload and more efficiency. We hope the current version is now acceptable.
Reviewer aWmT. You listed two weaknesses: no real distributed evaluation and no capacity-matched baseline. On Nov 8 we delivered both, a two-machine deployment and a parameter-matched CALM (145M vs. 147M), plus a cross-benchmark transfer study. We are happy that you recognize the framework as "well-designed" and the evaluation "comprehensive." With both weaknesses answered, we hope you agree that now the paper is more acceptable.
Reviewer j3u8. Thank you for engaging on Nov 9 and confirming that the cross-family concern is resolved. We answered both follow-up questions the same day, including the exact concurrent-serving experiment you requested. Your further check is appreciated.
Given that we have successfully addressed the raised concerns, we hope the reviewers could re-evaluate the paper and provide updated comments and decision.
To the Area Chair: We are concerned that we may not have enough time to respond to any remaining concerns, given that reviewers haven't responded to our response yet, while Nov 18 is the deadline. We are wondering whether a reminder message can be sent to the reviewers today (Nov 17).
The Authors of Submission 34781