acceptodds
Under review as a conference paper at ICLR 2027

Federation over Text: Insight Sharing for Multi-Agent Reasoning

Abstract

Modern agents specialize in varying domains while there is no clear approach combining different domain skills. We propose a federated learning-like framework, *Federation over Text* (FoT), that enables multiple clients solving different tasks to collectively generate a shared library of metacognitive insights by iteratively federating their local reasoning processes without sharing actual problem instances. Instead of federation over gradients (e.g., as in distributed training), FoT operates at the **semantic level** without any gradient optimization or supervision signal. At each round, client LLM agents independently apply arbitrary local reasoning and self-improvement procedures to their own tasks and share the resulting reasoning traces with a central server. The server then aggregates, distills, and consolidates knowledge across tasks and domains into a shared insight library, which can be reused by current and future agents to improve their reasoning. Experiments show that FoT improves reasoning effectiveness and efficiency across real-world daily tasks, cross-domain collaboration, and research insight discovery, achieving an average performance gain of 11.9 absolute percentage points while reducing client-generated task-completion tokens by 5.5%.

Primary area:
optimization
Keywords:
Federated Learning, Multi-Agent System, Distributed System, Recursive Self-Improvement

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_u1PeOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

FoT builds a shared insight library by extracting procedural knowledge from agents’ local reasoning traces and aggregating it through an LLM. The library is iteratively updated and reused across tasks and models. Experiments evaluate task performance, transfer, and research-insight discovery.

Critical strengths

The reported improvements on held-out tasks and the transfer of insights from weaker to stronger models suggest that sharing distilled experience has practical value. Comparisons with alternative aggregation methods provide some support for the proposed design.

Critical weaknesses

The quality of the figures and writing is poor. Figures 1 and 2 look like rough AI-generated sketches. I have no objection to using AI to create figures, but these figures have too many visual and presentation flaws to be acceptable in their current form. Many sentences also have a formulaic, AI-like style, leaving the manuscript insufficiently polished.

The resulting insight library seems little different from detailed instructions that a knowledgeable human could write, and may be less useful than a carefully prepared guide or system prompt. The paper does not convincingly establish what additional value comes from producing this guidance through multi-agent experience sharing, rather than directly articulating effective problem-solving strategies.

Questions for authors

What does FoT offer beyond a detailed guide or system prompt? Does it capture knowledge that such guidance would miss, or primarily automate its construction?

Reviewer_eRGxOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes Federation over Text (FoT), a framework for sharing reasoning experiences across agents without parameter updates or external supervision. Agents independently solve tasks and summarize their reasoning experiences, while a central LLM-based aggregator organizes these traces into a shared library of reusable insights. Experiments on daily agent tasks, cross-domain reasoning benchmarks, and research insight discovery show improvements over isolated reasoning and existing experience-sharing methods.

Critical strengths
  1. Inspired by the client–server paradigm of federated learning, FoT enables agents to iteratively share and aggregate reasoning experiences in textual form, building a reusable insight library across tasks and domains without updating model parameters.
  2. The many-to-many aggregation mechanism goes beyond directly pooling or retrieving past trajectories, enabling higher-level abstraction of experiences from different agents.
  3. The improvements on held-out tasks and cross-domain benchmarks, together with the aggregation ablations, provide encouraging evidence for the usefulness of shared insights.
Critical weaknesses
  1. Both local reasoning traces and server-side aggregation rely heavily on LLM self-reflection and judgment, without external validation. The paper does not systematically evaluate the correctness of generated insights or how erroneous experiences affect the shared library.
  2. The HLE experiments use a base model with very low task accuracy, meaning that most source trajectories correspond to incorrect answers. While the resulting library improves downstream performance, it remains unclear whether useful insights are genuinely extracted from these unsuccessful trajectories or mainly generated from the aggregator's existing knowledge.
  3. While Section 6 examines library size, agent participation, and aggregation frequency, it remains unclear how FoT handles knowledge redundancy, conflicts, and forgetting during long-term continual updates.
Questions for authors
  1. How is the correctness of generated insights assessed? Could the authors provide independent or expert evaluations of the clustering results and insight quality, including the rate of incorrect or misleading insights?
  2. Given the low HLE accuracy, how much do incorrect reasoning trajectories contribute to downstream improvements? Could the authors compare libraries constructed from correct-only, incorrect-only, and mixed trajectories?
Reviewer_WFi2OpenReview ↗
Rating
3(+2)
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes FoT, where agents share abstracted reasoning traces and a server consolidates them into reusable insights. Experiments include task performance, cross-domain and cross-model transfer, and research insight discovery.

Critical strengths
  • An useful, modular framework that supports different local reasoning methods without parameter updates/mandatory external supervision.
  • ⁠Broad empirical evaluation with relevant baselines and aggregation ablations. Held-out improvements are substantial.
  • ⁠Cross-model transfer and compact shared libraries strengthen the practical value.
Critical weaknesses
  • The central contribution needs sharper isolation. Existing ablations support semantic aggregation, but do not fully distinguish the benefits of many-to-many cross-domain synthesis from well-designed centralized memory consolidation.
  • ⁠Cross-domain transfer is not established causally. Performance on related benchmarks demonstrates distribution transfer, but stronger evidence would come from excluding the target domain when constructing the library. Agent-reported insight usage provides limited causal evidence.
  • ⁠Research discovery is evaluated retrospectively. Matching insights into existing papers measures conceptual coverage rather than the ability to generate useful new research.
  • The fonts in the tables are so tiny, hard to read.
Questions for authors

Does FoT retain its advantage against budget-matched centralized consolidation and when all source tasks from the target domain are removed?

Comments on the paper

1 comment
Reviewer_WFi2OpenReview ↗

Incorrect Assesment

Dear PC, I would like to inform you that this is a completely incorrect assessment. My review is fully 100% written by me, not even the grammar or spelling correction were done by any tool/LLM. Being 100% certain, I request you to reject this claim and re-consider the AI-detection tool you are using. Best

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.