acceptodds
Under review as a conference paper at ICLR 2027

ReM-MoA: Reasoning Memory Sustains Mixture-of-Agents Scaling

Abstract

Mixture-of-Agents (MoA) architectures improve inference-time scaling by organizing multiple LLM agents into layered reasoning pipelines. However, existing MoA variants struggle to sustain gains as depth increases, exhibiting degradation, early plateauing, or saturation on most benchmarks. We propose ReM-MoA, a memory-augmented MoA framework that sustains scaling through two mechanisms: (1) a Ranked Reasoning Memory that persistently stores and ranks reasoning traces from all layers using a comparative Reviewer Agent, and (2) a Curated Diversified Memory Routing scheme that exposes different agents to distinct combinations of successful and failed traces, preserving exploration diversity while propagating high-quality reasoning. We further introduce an optional multi-domain Reviewer distillation pipeline that improves ranking quality through frontier-model supervision. ReM-MoA consistently outperforms prior MoA variants under both depth and width scaling on five reasoning benchmarks spanning math, formal logic, code, knowledge, and commonsense. As depth grows, ReM-MoA widens the lead and gains nearly three times as much accuracy per additional token as any baseline, establishing structured cross-layer reasoning memory as a key missing mechanism for scalable MoA inference.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multi-agent system, Mixture-of-agents

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_kDhzOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes ReM-MoA, which stores the outputs of all previous layers in a shared memory, ranked by an LLM Reviewer. Each agent in the next layer gets a different set of high and low scoring traces from this memory, which aims to improve both the quality and the diversity of the MoA system. It demonstrates good performance comparing other MoA variants while scaling on depth, width and token usage across 5 reasoning benchmarks; in the same time, the diversity converges more slowly than other methods. It is a smart and lightweight idea that fit well on MoA system.

Critical strengths
  1. It provides a lightweight method to improve the diversity of MoA, and it is easy to integrate into existing MoA systems. It perform good on both quality and diversity. At L=9, it reaches 72.8% average accuracy, compared with 68.3% for the best baseline. It also preserves diversity better and works with 7B to 32B proposer pools. Such good performance does not depend much on the Reviewer’s ability; an off-the-shelf 7B model already works well. Therefore, I think this idea is smart and can be widely applied in similar settings.
  2. It has clear ablation to demonstrate that the three core components are important, which make it more convincing. At L=9, removing cross-layer access lowers the average accuracy from 72.8% to 68.7% (−4.12), removing routing diversification lowers it to 70.2% (−2.66), and removing the rationales lowers it to 71.2% (−1.60).
Critical weaknesses
  1. Limited novelty compared with VeriMoA. VeriMoA already keeps a global record for all previous outputs with quality scores and give the top-n to the next layer. So the main adds on in this paper is making agents in one layer have different input and include low score traces as negative examples, which is not validated well as shown in the next point.
  2. The mixed routing is not compared with the Top-n only design. VeriMoA gives all agents the same top-n traces, while ReM-MoA mixes top and bottom traces and gives different agents different routes. This mixed design is the main difference from VeriMoA, but the paper never compares it with simply using Top-n for all agents. The only routing ablation compares different routes against one shared route, and it does not say which shared route is used. Without a Top-n only baseline, we cannot tell whether the low-scoring traces actually help, or how much ReM-MoA gains over a general version of VeriMoA.
Questions for authors
  1. Can you compare the mixed routing with a Top-n only version, where all agents share the same top-n traces (a general version of VeriMoA)? Also, which shared route is used in the “routing diversification” ablation?
  2. How much does reviewer contribute to the performance. How does ReM-MoA perform with a Reviewer that gives random scores? Does the ranking quality really matter? The Reviewer scores traces within each layer, but Top/Bot are selected across all layers. Are scores from different layers comparable?
  3. Does the method still help with newer models, harder benchmarks, or compared with a single stronger model at the same cost?
5 comments
AuthorsOpenReview ↗

Response to Reviewer kDhz (Part 1/3)

We thank the reviewer for the constructive comments. The new results use the setting of the paper: N=3, the five benchmarks MATH, MMLU-redux, Formal Logic, CRUX and HellaSwag, and the main pool, with Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct and Qwen2.5-Coder-7B-Instruct as proposers and Qwen2.5-7B-Instruct as aggregator and off-the-shelf Reviewer. Each table below states its own benchmarks, models, metric and source.


W1. Novelty relative to VeriMoA.

We agree with the reviewer's description of VeriMoA: it keeps a global cache of earlier outputs with quality scores and gives the top-n to the next layer. Our related-work sentence "No prior MoA variant maintains a structured cross-layer memory ..." (lines 095–096) is too strong, and we have revised it to credit VeriMoA’s global cache. ReM-MoA differs in two ways. First, the quality signal needs no domain verifier: VeriMoA scores candidates by simulating hardware designs against testbenches and generates diversity through multiple domain-specific paths (direct HDL generation, C++ and Python), while our Reviewer is a general-purpose LLM that scores traces comparatively and writes a rationale. The Reviewer can also be strengthened without any verifier: the optional distilled Reviewer raises accuracy from 72.8 to 75.2 at L=9, with gains on all five benchmarks, including the three outside its distillation domains (Table 1). Second, reads are differentiated: instead of the same top-n for every agent, each agent reads a different mix of high- and low-scoring entries.

Beyond the architecture, the paper contributes a controlled characterization of depth scaling in existing MoA variants (our first contribution): under one shared pool and up to L=9, Standard MoA degrades, RMoA plateaus, and AttentionMoA saturates or plateaus on four of five benchmarks (Table 1). ReM-MoA is designed against these failure modes jointly, and its components are ablated separately (Table 1 and the variants below). VeriMoA already reports error accumulation in Standard MoA for HDL generation. What our study adds is that, on general reasoning tasks where such domain-specific paths are unavailable, a quality-ranked cache alone does not preserve diversity, which the Top-n-only comparison below quantifies.


W2 and Q1. Comparison with a Top-n-only design and which route the ablation uses.

We added two variants. Each agent receives two references, as in ReM-MoA. In Shared-Top and Distinct-Top, ties in score are broken once per layer so that all agents see the same ranking. In ReM-MoA, ties are broken independently for each agent.

  • Shared-Top: all agents receive the same Top-2 entries of the global pool. This is the closest analogue of VeriMoA's retrieval rule (its Eq. 4–6) in our setting. Since no simulator exists for these tasks, the scores come from our Reviewer. The variant also keeps our rationales and aggregator, so it is a favourable version of a general VeriMoA.
  • Distinct-Top: agents receive different high-scoring sets, {Top-1, Top-2}, {Top-1, Top-3} and {Top-2, Top-3}, and no low-scoring entries.

Table K1. Routing variants. Accuracy (%), mean over the five benchmarks and five seeds, N=3. Vendi is the Vendi Score of the three proposer outputs at L=9, averaged over the five benchmarks (1 for identical outputs, 3 for orthogonal outputs), as in Table 12. Sources: ReM-MoA from Tables 8 and 12, Shared-Top and Distinct-Top from new runs.

VariantL=1L=3L=5L=7L=9Vendi at L=9
Shared-Top (Top-n only)62.365.967.768.468.91.18
Distinct-Top62.366.969.470.470.81.65
ReM-MoA62.367.970.872.372.81.85

ReM-MoA gains 3.9 points over the Top-n-only design at L=9, and the gap widens with depth (it is 2.0 points at L=3). Shared-Top also loses output diversity fastest. Adding low-scoring references and further reducing overlap between agents' reference sets together contribute the remaining 2.0 points (Distinct-Top → ReM-MoA).

On the existing ablation: in "− routing diversification" every agent uses the Contrastive route, Top-1 ∪ Bot-1, with ties broken independently per agent. The description at lines 413–414 ("one shared reference set") was imprecise and has been corrected.

AuthorsOpenReview ↗

Response to Reviewer kDhz (Part 2/3)

Q2. Contribution of the Reviewer, random scores, ranking quality, cross-layer comparability.

Random scores. We replaced every score with a uniform random integer in 1–10, with no Reviewer call and no rationales.

Table K2. Random-Score ablation. Accuracy (%), mean over the five benchmarks and five seeds, N=3. Sources: ReM-MoA from Table 8, − rationale from Table 11, Random-Score from new runs.

VariantL=1L=3L=5L=7L=9
ReM-MoA62.367.970.872.372.8
− rationale61.967.269.670.871.2
Random-Score61.164.866.767.868.1

The Reviewer contributes 4.7 points at L=9: 3.1 from the quality of the ranking (71.2 vs. 68.1) and 1.6 from the rationales (72.8 vs. 71.2). Ranking quality therefore matters, and it matters in both directions: the distilled Reviewer, whose scores separate correct from incorrect traces better (AUROC 0.84 vs. 0.78), raises accuracy further to 75.2. At the same time, the system still scales with depth under random scores (61.1 to 68.1), because agents keep reading different entries from all earlier layers. In particular, 12.6 of the 17.3-point gain over Standard MoA comes from cross-layer memory and diversified routing alone. This agrees with the reviewer's observation that an off-the-shelf 7B Reviewer is sufficient.

Cross-layer comparability. We re-scored the 27 traces of an L=9 run in random triples drawn from three different layers (main pool, seed 0, 926 problems, three groupings, both Reviewers). Per-layer scores correlate with cross-layer scores at Spearman 0.83, close to the 0.86 between two cross-layer groupings (0.88 and 0.91 for the distilled Reviewer). The Top-2 entries selected from the 27 traces are correct 71.5% of the time under per-layer scores and 70.8% under cross-layer scores. The AUROC for separating correct from incorrect traces is 0.782 for same-layer pairs and 0.775 for cross-layer pairs. Scores are thus approximately comparable across layers. We have reported this analysis and noted the small residual dependence on the candidate set in the limitations paragraph.

AuthorsOpenReview ↗

Response to Reviewer kDhz (Part 3/3)

Q3. Newer models, harder benchmarks, and a single stronger model at the same cost.

Newer models and a harder benchmark. We ran MMLU-Pro (1,400 questions, 100 per domain) with the main pool and with three 8B models released in the first half of 2025, each serving as proposer, Reviewer and aggregator: Qwen3-8B (thinking disabled), DeepSeek-R1-Distill-Llama-8B and DeepSeek-R1-0528-Qwen3-8B.

Table K3. MMLU-Pro with the main pool and three 8B models. Accuracy (%) on a fixed 1,400-question subset of MMLU-Pro (100 questions from each of the 14 domains), mean±std over five seeds, N=3. Methods: Standard MoA, AttentionMoA and ReM-MoA with the off-the-shelf Reviewer. All numbers are new.

PoolMethodL=1L=3L=5L=7L=9
Main (heterogeneous)Std MoA48.4±0.850.1±0.649.0±0.847.7±0.646.4±0.7
AttMoA49.5±0.552.0±0.353.5±0.454.0±0.854.2±0.3
ReM-MoA49.9±0.454.0±0.456.8±0.457.9±0.558.1±0.4
3×Qwen3-8BStd MoA52.4±0.754.1±0.853.0±0.852.1±0.851.0±0.8
AttMoA53.0±0.755.8±0.857.1±0.857.4±0.957.9±0.4
ReM-MoA53.4±0.657.5±0.860.6±0.562.1±0.663.0±0.6
3×R1-Distill-Llama-8BStd MoA51.0±0.652.5±0.951.4±0.450.7±0.549.6±0.9
AttMoA51.4±0.653.4±0.655.1±0.955.4±0.955.9±0.7
ReM-MoA52.0±0.955.5±0.458.2±0.859.7±0.660.6±0.4
3×R1-0528-Qwen3-8BStd MoA54.2±0.455.4±0.954.7±0.453.4±0.452.3±0.5
AttMoA54.6±0.557.1±0.558.4±0.759.1±0.759.3±0.8
ReM-MoA55.1±0.359.0±0.762.1±0.763.4±0.964.3±0.5

ReM-MoA leads AttentionMoA by 3.9–5.1 points at L=9 across all four pools. Standard MoA degrades on all four. The three single-model ReM-MoA curves sustain growth through L=9, while the main pool plateaus at L=7–9 (+0.2). The R1-0528-Qwen3-8B pool starts highest but the ReM-MoA gain over Standard MoA (+12.0) remains comparable to the main pool (+11.7).

A single stronger model. The numbers come from the L=1 proposer traces of our 32B pool (Appendix C.2: Qwen2.5-32B-Instruct, Qwen2.5-Coder-32B-Instruct and DeepSeek-R1-Distill-Qwen-32B), on the five benchmarks with five seeds. One call to a single 32B model reaches 70.2–72.6, depending on the model. A single 32B model thus already matches ReM-MoA over the 7–8B pool at L=9 (72.8) at a fraction of the token cost, so when a stronger model can be deployed, it is the more cost-effective choice. ReM-MoA is complementary to model scale: on the same 32B pool, ReM-MoA at L=1 (74.8, five LLM calls rather than one) already exceeds the single-model range, at L=3 it reaches 80.5, and at L=9 it reaches 87.5, still leading AttentionMoA by 4.5 points (87.5 vs. 83.0 in Table 10). Standard MoA degrades on the 32B pool as well (Table 10), confirming that the depth-scaling pattern holds across model scales.

Reviewer_AcygOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ReM-MoA, a memory-augmented Mixture-of-Agents framework designed to sustain performance gains as depth and width increase. It introduces two core mechanisms: (1) a Ranked Reasoning Memory that stores reasoning traces from all previous layers together with scores and rationales assigned by a comparative Reviewer Agent; and (2) Curated Diversified Memory Routing that provides different agents with distinct reference , preserving quality information while maintaining exploration diversity.

Critical strengths
  1. ReM-MoA combines Ranked Reasoning Memory with Curated Diversified Memory Routing to preserve useful reasoning across layers while maintaining exploration diversity.
  2. Experiments on five reasoning benchmarks show consistent improvements over relevant MoA baselines.
  3. The authors also conduct experiments indicating that the performance gains are not solely attributable to additional compute or proposer scale.
Critical weaknesses
  1. Although the same Reviewer model and rubric are used across layers, each evaluation only sees the current layer’s traces. Scores may therefore depend on the local candidate set, yet they are directly compared across layers for Top/Bottom retrieval. The paper does not verify this cross-layer comparability.
  2. The ablation studies examine the effect of removing Reviewer rationales but do not directly assess the contribution of quality ranking.
  3. The motivation for Top/Bottom/Contrastive routing is reasonable, but alternative routing strategies are not evaluated. In particular, it remains unclear whether low-scoring references provide additional benefits over giving each agent a different set of high-scoring references.
Questions for authors
  1. Could you clarify how the shared reference set is constructed in the routing ablation? It is unclear whether it uses Top, Bottom, Contrastive, or another selection, which may change reference quality alongside input diversity.
  2. Since the final aggregator also receives historical reasoning traces with quality scores, do the gains from cross-layer access come from better proposer reasoning, better final aggregation, or a combination of both?
  3. The compute comparison is based on total token consumption. Is this sufficient to conclude that the gains are not due to additional computation?
5 comments
AuthorsOpenReview ↗

Response to Reviewer Acyg (Part 1/3)

We thank the reviewer for the careful and constructive comments. The new results use the setting of the paper: N=3, the five benchmarks MATH, MMLU-redux, Formal Logic, CRUX and HellaSwag, and the main pool, with Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct and Qwen2.5-Coder-7B-Instruct as proposers and Qwen2.5-7B-Instruct as aggregator and off-the-shelf Reviewer. Each table below states its own benchmarks, models, metric and source.


W1. Cross-layer comparability of Reviewer scores.

We verified this in two ways.

(a) Re-scoring with a different candidate set. For 926 problems (200 each from MATH, MMLU-redux, CRUX and HellaSwag, and all 126 Formal Logic problems), we took the 27 traces of an L=9 run and randomly regrouped them into triples whose members come from three different layers. The Reviewer then scored each triple with the unchanged prompt. We repeated this for three random groupings and for both Reviewers.

Table A1. Re-scoring under cross-layer candidate sets. Data: 926 problems. For each problem, the 27 proposer traces of an L=9, N=3 run (seed 0) with the main pool. Reviewers: the off-the-shelf Reviewer (Qwen2.5-7B-Instruct) and the distilled Reviewer (a LoRA fine-tune of the same model), each re-scoring the traces of its own run. Metrics: Spearman correlation of scores within a problem, averaged over problems. Top-2 correct and Bot-2 incorrect are the shares of the two highest-scored and the two lowest-scored traces among the 27 that are correct and incorrect, respectively. Cross-layer scores are averaged over the three groupings. All numbers are new.

ReviewerSpearman, per-layer vs. cross-layer scoresSpearman, between two cross-layer groupingsTop-2 correct: per-layer / cross-layerBot-2 incorrect: per-layer / cross-layer
Off-the-shelf0.830.8671.5% / 70.8%56.2% / 55.5%
Distilled0.880.9177.3% / 76.8%61.8% / 61.2%

The original per-layer scores agree with cross-layer scores almost as well as two cross-layer groupings agree with each other (0.831 vs. 0.858, bootstrap 95% CI [0.82, 0.84] vs. [0.85, 0.87]). The Top-2 and Bot-2 entries retrieved under either scoring regime have similar quality (differences below 1 point), though the specific entries retrieved may differ.

(b) Discrimination within and across layers. From the Reviewer scores logged in the L=9 runs of ReM-MoA (main pool, five benchmarks, five seeds, one set of runs per Reviewer), we computed the AUROC of the score for separating correct from incorrect traces of the same problem. It is 0.782 for pairs from the same layer and 0.775 for pairs from different layers (distilled Reviewer: 0.841 and 0.836).

In short, scores do depend slightly on the local candidate set, as the reviewer anticipated, but the effect is small and the retrieved references remain of similar quality. We have added both analyses to the appendix and stated in the limitations paragraph that scores are approximately, not exactly, comparable across layers.


W2. Direct assessment of quality ranking.

We added a Random-Score variant: every score is replaced by a uniform random integer in 1–10 and the Reviewer is not called. Routing, prompt format and the aggregator are those of the existing "− rationale" ablation, so the two variants differ only in whether the scores carry quality information. We note that the prompt still labels traces as high- or low-scoring, so the 3.1-point contribution includes both the quality of the routing and any effect of the label itself. A label-free control would isolate the routing effect.

Table A2. Random-Score ablation. Accuracy (%), mean over the five benchmarks and five seeds, N=3. Sources: ReM-MoA from Table 8, − rationale from Table 11, Random-Score from new runs.

VariantL=1L=3L=5L=7L=9
ReM-MoA62.367.970.872.372.8
− rationale61.967.269.670.871.2
Random-Score61.164.866.767.868.1

Quality ranking contributes 3.1 points at L=9 (71.2 vs. 68.1), about twice the contribution of the rationales (1.6), and its contribution grows with depth (0.8 points at L=1). Ranking quality also matters in the other direction: the distilled Reviewer separates correct from incorrect traces better (AUROC 0.84 vs. 0.78) and raises accuracy from 72.8 to 75.2. We have added this control to the appendix and refer to it in Sec. 5.4.

AuthorsOpenReview ↗

Response to Reviewer Acyg (Part 2/3)

W3. Alternative routing: do low-scoring references add value?

We added two routing variants. Each agent still receives two references, and everything else is unchanged. In Shared-Top and Distinct-Top, ties in score are broken once per layer so that all agents see the same ranking. In ReM-MoA, ties are broken independently for each agent (as now stated in the revised Sec. 4.2).

  • Shared-Top: all agents receive the same Top-2 set.
  • Distinct-Top: agents receive different high-scoring sets, {Top-1, Top-2}, {Top-1, Top-3} and {Top-2, Top-3}, and no low-scoring references.

Table A3. Routing variants. Accuracy (%), mean over the five benchmarks and five seeds, N=3. Vendi is the Vendi Score of the three proposer outputs at L=9, averaged over the five benchmarks (1 for identical outputs, 3 for orthogonal outputs), as in Table 12. Sources: ReM-MoA from Tables 8 and 12, Shared-Top and Distinct-Top from new runs.

VariantL=1L=3L=5L=7L=9Vendi at L=9
Shared-Top62.365.967.768.468.91.18
Distinct-Top62.366.969.470.470.81.65
ReM-MoA (Eq. 7)62.367.970.872.372.81.85

Giving agents different high-scoring references already helps (+1.9 over Shared-Top at L=9). Adding low-scoring references further improves accuracy (+2.0 over Distinct-Top) and maintains higher output diversity (Vendi 1.85 vs. 1.65). We note that the Distinct-Top → ReM-MoA step changes both the presence of low-scoring traces and the degree of overlap between agents' reference sets, so the 2.0 points reflect both factors.


Q1. Construction of the reference set in the routing ablation.

In "− routing diversification", every agent uses the same route, the Contrastive route Top-1 ∪ Bot-1 over the global pool, instead of one route per family. Ties in score are broken uniformly at random and independently for each agent, as in the full method, so the agents receive identical references only when the top and bottom scores are unique. Our wording at lines 413–414 ("one shared reference set") and the attribution at lines 423–424 were therefore imprecise. We have corrected both and stated the tie-breaking rule in Sec. 4.2. The reviewer's concern is also valid: this ablation changes the composition of the references (it drops the Top-2 and Bot-2 routes) together with their diversity. Shared-Top and Distinct-Top in W3 were added to separate the two factors.

AuthorsOpenReview ↗

Response to Reviewer Acyg (Part 3/3)

Q2. Do the cross-layer gains come from proposers or from the aggregator?

Mostly from the proposers. We first correct a description: "− cross-layer access" restricts only the proposers' reads, and the aggregator still reads the full memory. Lines 412–413 say "every memory read" and have been corrected. To isolate the aggregator, we replayed only the aggregator on the saved memories, with its reference sets restricted to the last layer.

Table A4. Cross-layer access on the proposer side and on the aggregator side. Accuracy (%) at L=9, mean over the five benchmarks and five seeds, N=3. Left column: from Table 1 (ReM-MoA and − cross-layer access). Right column: new, only the aggregator is re-run, on the saved memories of the corresponding left-column runs.

Proposers readAggregator reads full memoryAggregator reads last layer only
All earlier layers72.872.2
Previous layer only68.768.2

Restricting the proposers costs 4.0–4.1 points at L=9, and restricting the aggregator costs 0.5–0.6. The cross-layer gain thus comes mainly from better proposer reasoning, with a small additional gain at aggregation. Direct evidence for the proposer improvement appears in Table T5 of our response to Reviewer trwh: individual trace accuracy rises from 49.2% at layer 1 to 59.3% at layer 9, consistent with the 10-point system-accuracy gain.


Q3. Is total token count sufficient?

Not by itself, so we now report the cost in more detail (per query, L=9).

Table A5. Cost per query at L=9. Mean over the five benchmarks and five seeds, N=3. Tokens, in thousands, count all LLM calls of a query. Sources: total tokens and accuracy from Table 8. The input/output split is newly reported from the logs.

MethodInput tokens (k)Output tokens (k)Total (k)Accuracy
Standard MoA48.816.465.255.5
RMoA70.523.293.764.7
AttentionMoA149.049.2198.268.3
ReM-MoA65.821.887.672.8

Three observations follow. (i) ReM-MoA uses fewer input tokens and fewer output tokens than RMoA and AttentionMoA, so its lead over them does not depend on how the two are weighted. (ii) Against self-consistency, we re-matched the SC budget with output tokens weighted 3× or 4× relative to input tokens. The best SC (over Qwen2.5-Coder-7B, five-benchmark mean) then reaches 63.0 and 62.9 at the L=9 budget, against 72.8 for ReM-MoA, so the gap remains large (9.8–9.9 points vs. 9.3 under total tokens). (iii) Latency is where ReM-MoA does not win. Standard MoA at L=9 makes 3L+1=28 LLM calls in L+1=10 serial stages (the N=3 proposers within a layer run in parallel). ReM-MoA makes 4L+1=37 calls in 2L+1=19 serial stages (each layer runs proposers then Reviewer sequentially). Self-consistency samples run fully in parallel. We have added this table and stated the latency cost in the new limitations paragraph.

Reviewer_trwhOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes the Reasoning Memory Sustains Mixture-of-Agents (ReM-MoA) framework, which stacks multiple layers of parallel proposer agents and a final review agent, along with a memory mechanism. Unlike previous work that faces a scaling collapse issue, ReM-MoA uses each pair of proposer and reviewer agent responses to build a ranked reasoning memory pool. With dedicated Routing functions, subsequent agents receive distinct combinations of traces from the memory pool. RoM-MoA has demonstrated strong scaling performance across math, formal logic, code, knowledge, and commonsense benchmarks.

Critical strengths
  1. Multi-agent interaction scaling is a timely, trending topic that this work aims to improve.
  2. The paper is well written and generally easy to follow.
  3. This paper formally characterizes and discusses scaling failure in existing MoA systems.
  4. The proposed ReM-MoA introduces an additional reasoning memory that is intuitive and makes sense. There is clear evidence that this method scales well in both the number of proposer agents and the depth.
Critical weaknesses
  1. The three mechanisms driving the scaling-collapse claim make sense. However, I suggest adding additional examples in the appendix to make this more explicit.
  2. While the experiments are comprehensive, the model selection is relatively fixed. Serving multiple LLMs is complex and expensive. Does ReM-MoA work effectively under the same model setting (i.e., all agents are using a single model)?
  3. In line 368, the claim that “The advantage does not come from extra tokens” seems unsupported by Figure 3(c). Many methods show a clear positive correlation.
  4. The paper does not discuss the quality of the memory obtained. What is the distribution of the integer score the reviewer agent obtains? What type of reasoning memory is picked more often at different depths?
Questions for authors

See Weakness

5 comments
AuthorsOpenReview ↗

Response to Reviewer trwh (Part 1/3)

We thank the reviewer for the constructive comments. The new results use the setting of the paper: N=3, the five benchmarks MATH, MMLU-redux, Formal Logic, CRUX and HellaSwag, and the main pool, with Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct and Qwen2.5-Coder-7B-Instruct as proposers and Qwen2.5-7B-Instruct as aggregator and off-the-shelf Reviewer. Each table below states its own benchmarks, models, metric and source.


W1. More examples of the three mechanisms.

We have added one new worked example per mechanism to Appendix F, alongside the two existing cases (Tables 14–15) which already illustrate error accumulation and diversity collapse. To show that these patterns are not isolated, we counted their frequency among the 2,026 problems of the five benchmarks (500 + 400 + 126 + 500 + 500) at L=9.

Table T1. Frequency of the three failure patterns. Each pattern is counted on the L=9, N=3 runs of the named method with the main pool, over the five benchmarks. All numbers are new.

MechanismPatternShare of problems
Error accumulation (Standard MoA)A layer with a correct majority is followed by a layer with a wrong majority, and the final answer is wrong15.4%
Loss of early reasoning (RMoA)A correct trace exists in layers 1–3, the final answer is wrong, and ReM-MoA answers the same problem correctly9.8%
Diversity collapse (AttentionMoA)A correct minority exists early, then all agents converge to the same wrong answer8.1%

We identified one representative problem per mechanism from MATH-500, verified in the L=9 traces (seed 0):

  • Error accumulation: a GCD problem (Level 4, answer 21) where the correct majority at layer 1 gradually gives way to an incorrect factoring approach at deeper layers in Standard MoA.
  • Loss of early reasoning: a geometric-sequence problem (Level 4, answer 243/625) where a correct derivation at layer 1 is not retained in later layers of RMoA, and fresh arithmetic errors are introduced.
  • Diversity collapse: a summation identity (Level 3, answer 11) where the correct minority answer (using N² = 121) is suppressed as AttentionMoA's agents converge on a common misconception (confusing the sum with the index of the N-th odd integer).

The full per-layer traces appear in the revised Appendix F.


W2. Single-model setting.

We ran three single-model pools in which one model serves as all three proposers, the Reviewer and the aggregator (a single deployed model): Qwen2.5-7B-Instruct, Qwen2.5-Coder-7B-Instruct and Llama-3.1-8B-Instruct. With a single model, Standard MoA corresponds to the Self-MoA setting (Li et al., 2025).

Table T2. Single-model pools. Accuracy (%), mean over the five benchmarks and five seeds, N=3. Methods: Standard MoA and ReM-MoA. SC at the L=9 token budget from Table 8 (matched to the main pool): Qwen2.5-7B 62.3, Qwen2.5-Coder-7B 63.5, Llama-3.1-8B 57.0. All numbers except SC are new.

PoolMethodL=1L=3L=5L=7L=9
3×Qwen2.5-7BStd MoA59.560.259.057.656.4
ReM-MoA59.963.866.367.868.5
3×Qwen2.5-Coder-7BStd MoA57.558.457.456.355.5
ReM-MoA57.961.864.265.966.7
3×Llama-3.1-8BStd MoA52.553.452.651.651.0
ReM-MoA53.056.458.760.060.8

In all three single-model pools, ReM-MoA sustains scaling through L=9 while Standard MoA degrades, with L=9 accuracy 9.8–12.1 points above the corresponding L=9 Standard MoA. For reference, the MMLU-Pro evaluation in our response to Reviewer kDhz (Table K3) also shows the single-model pools under AttentionMoA, where ReM-MoA leads AttentionMoA by 4.7–5.1 points at L=9 on that benchmark. In our setting, the best single-model pool (3×Qwen2.5-7B, 68.5) is comparable to the heterogeneous pool's AttentionMoA (68.3, Table 8), though the heterogeneous pool with ReM-MoA (72.8) is higher by 4.3 points.

AuthorsOpenReview ↗

Response to Reviewer trwh (Part 2/3)

W3. The claim at line 368 and Figure 3(c).

We agree. Accuracy rises with tokens for most methods, and the heading overstated the point. What the data support is that ReM-MoA is more accurate at a matched token budget. We have retitled the paragraph and also revised the similar claim at lines 406–408 ("so the advantage of ReM-MoA does not rely on additional compute"). Table T3 makes this comparison.

Table T3. Accuracy at matched depth. All numbers are from Table 8 of the paper: accuracy (%), mean over the five benchmarks and five seeds, N=3, main pool. Best SC is the compute-matched self-consistency over Qwen2.5-Coder-7B at the ReM-MoA token budget of that row.

DepthReM-MoAStandard MoARMoAAttentionMoABest SC
L=367.962.364.465.762.5
L=570.860.365.067.463.0
L=772.357.864.968.063.3
L=972.855.564.768.363.5

From L=3 to L=9, ReM-MoA gains 4.9 points, against 2.6 for AttentionMoA, 1.0 for Best SC and 0.3 for RMoA, while Standard MoA loses 6.8. At every depth in the table, ReM-MoA leads the strongest baseline (AttentionMoA) by 2.2–4.5 points, and the lead widens with depth. The comparison also holds in tokens (Table 8): at the same depth, ReM-MoA uses fewer tokens than RMoA and AttentionMoA, and Best SC runs at the ReM-MoA budget. Standard MoA uses fewer tokens at the same depth, but its accuracy peaks at 62.3 (L=3, 19.8k tokens), which ReM-MoA matches at L=1 with 7.5k tokens. We have retitled the paragraph "ReM-MoA is more accurate at matched token budgets" and added this table.

AuthorsOpenReview ↗

Response to Reviewer trwh (Part 3/3)

W4. Quality of the memory.

Distribution of the scores.

Table T4. Score distribution. Percentage of traces receiving each score, broken down by correctness, and the fraction of traces at each score that are correct. Data: main pool, off-the-shelf Reviewer (Qwen2.5-7B-Instruct), L=9 run, five seeds, all nine layers pooled (273,510 traces).

ScoreCorrect traces (%)Incorrect traces (%)P(correct given score)
10.96.314.8%
22.512.120.6%
34.615.128.0%
47.316.036.8%
510.415.046.8%
613.512.857.1%
716.09.967.1%
817.26.976.0%
916.34.183.4%
1011.41.789.2%

Correct traces score an average of 6.9 and incorrect traces 4.7. P(correct given score) increases monotonically from 14.8% at score 1 to 89.2% at score 10, confirming that the score is informative. The strongest discrimination is at the extremes: scores 1–2 are correct only 15–21% of the time, while scores 9–10 are correct 83–89% of the time. This table is included in the revised appendix.

What is picked at different depths. By design the mix of routes is fixed: at every layer the three agents read Top-1 ∪ Bot-1, Top-2 and Bot-2, that is, three high-scoring and three low-scoring reads. What changes with depth is where these entries come from, how reliable they are, and how accurate the proposer traces themselves become.

Table T5. Per-layer trace accuracy, Top-1/Bot-1 quality and provenance. Data: main pool, off-the-shelf Reviewer (Qwen2.5-7B-Instruct), L=9 run, five benchmarks, five seeds (accuracy is the five-benchmark average). Trace correct: fraction of correct traces among the three proposer outputs at that layer. Pool correct: fraction of correct traces in the cumulative pool available to the reader. Top-1 correct and Bot-1 incorrect: conditioned on the pool containing both correct and incorrect traces. From layer l−1: fraction of Top-1/Bot-1 entries drawn from the most recent layer (uniform chance = 1/(l−1)). All numbers are new.

Layer lTrace correct (this layer)Pool correct (all prior)Top-1 correctBot-1 incorrectTop-1 from layer l−1Bot-1 from layer l−1
149.2%—————
354.3%50.7%82.6%77.4%59.3%44.7%
557.0%52.9%84.3%74.6%38.5%20.3%
758.5%54.4%83.7%73.8%28.1%14.6%
959.3%55.5%84.9%72.1%22.8%10.5%

Three observations. (i) Individual trace accuracy increases with depth (49.2% at layer 1 to 59.3% at layer 9) because proposers read increasingly useful memory. The 10-point improvement is consistent with the system-accuracy gain (62.3 to 72.8) and with the proposer-side attribution in Table A4 of our response to Reviewer Acyg. (ii) In mixed pools, Top-1 is correct 83–85% of the time and Bot-1 is incorrect 72–77% of the time, confirming that high scores reliably identify correct traces and low scores identify incorrect traces. (iii) Top-1 is drawn from the most recent layer more often than the uniform chance of 1/(l−1) (e.g., 22.8% vs. 12.5% at layer 9), yet the majority of Top-1 entries still come from non-adjacent layers (61.5%, 71.9% and 77.2% at layers 5, 7 and 9), i.e., layers that previous-layer-only access would miss. This is consistent with the 4.1-point drop when cross-layer access is removed (Table A4). Bot-1 is biased toward earlier layers (below the uniform chance from the most recent layer).

On the 926-problem sample described in our response to Reviewer Acyg (Table A1), the Top-2 entries among the 27 traces of a problem are correct 71.5% of the time and the Bot-2 entries are incorrect 56.2% of the time, confirming the same pattern at a different granularity.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.