- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes TFlow (Thought Flow), a weight-space communication framework for multi-agent LLM collaboration that replaces conventional natural-language message passing with instance-specific parameter perturbations. Specifically, frozen sender agents extract hidden-state representations of the input, which are transformed into LoRA perturbations by a trainable parameter generator. These perturbations are fused and temporarily injected into a frozen receiver to guide its generation without expanding the textual context. The framework is trained with a next-token prediction loss and a diversity regularizer. Experiments across five reasoning, knowledge, and coding benchmarks show that TFlow consistently improves accuracy over a single-agent baseline while substantially reducing token consumption and inference latency compared to a text-based multi-agent system, albeit with generally lower accuracy.
Critical strengths
(1) The paper revisits multi-agent collaboration from the perspective of communication interfaces, shifting information exchange from explicit text or hidden-state transfer toward dynamic modulation of the receiver's computation. This perspective is conceptually interesting and offers a promising direction for exploring deeper forms of inter-model collaboration. (2) The methodological design is well aligned with the underlying motivation. By translating sender representations into instance-conditioned parameter perturbations, the framework enables dynamic information injection without modifying the frozen backbone or expanding the receiver's textual context, offering a practical approach to efficient collaboration. (3) Beyond evaluating downstream task performance, the paper investigates the role of dynamic weight generation through static adaptation, mismatched perturbation injection, and representation similarity analyses. These experiments provide useful empirical insights into instance-conditioned parameter modulation, extending the evaluation beyond conventional performance comparisons.
Critical weaknesses
(1) The paper frames instance-conditioned weight perturbations as a multi-agent communication mechanism, yet the sender agents merely perform forward passes on the same input under different role prompts. The distinction between this process and conventional conditional parameter adaptation remains unclear, as does the independent or complementary information contributed by different agents. Consequently, the conceptual foundation of the proposed multi-agent collaboration paradigm is insufficiently established. (2) The method largely builds upon existing techniques in hypernetworks, dynamic LoRA, and structured parameter generation, while its substantive technical novelty remains unclear. Moreover, several design choices lack sufficient justification. For instance, the diversity regularizer encourages orthogonal perturbations across samples without considering that semantically similar tasks may benefit from shared update directions, leaving its underlying rationale and necessity insufficiently established. (3) The experimental evidence does not adequately support the paper's central claims. Existing ablations primarily demonstrate the effectiveness of dynamic parameter adaptation without isolating the contribution of multi-agent communication. In particular, a single-agent dynamic LoRA baseline is missing, and the supervision conditions differ between TFlow and its main baselines. Furthermore, the absence of direct comparisons with existing latent communication methods leaves the distinctive advantages of the proposed mechanism unclear.
Questions for authors
(1) What evidence demonstrates that TFlow benefits from genuine multi-agent communication rather than conventional query-conditioned parameter adaptation? In particular, how would it compare with a similarly trained parameter generator conditioned on a single agent's or the receiver's hidden states? Do different sender roles contribute meaningful complementary information? (2) How can the benefits of the proposed communication mechanism be disentangled from those of additional supervised training? Given that TFlow's parameter generator is trained while the main baselines remain frozen, would the conclusions hold under comparable supervision? Can the authors also provide direct comparisons with existing latent-space communication methods? (3) What constitutes the key technical advancement over existing hypernetwork-based and dynamic LoRA generation approaches, particularly HY-WU? Beyond changing the conditioning signal, which architectural components are essential to the observed improvements, and what motivates their specific design? (4) Although the paper reports end-to-end inference latency, can the authors further break down the computational costs of sender forwarding, parameter generation, and dynamic LoRA injection? How do the reported efficiency advantages hold up when considering memory consumption and batched inference throughput?