- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents CERA-MoA, a Mixture-of-Agents framework where the router and agents are trained in the same reinforcement learning process. The router estimates how familiar each agent is with a query before generation, and uses a cumulative-threshold rule to decide how many agents to activate. The selected agents are then updated on the queries they receive, and their rewards are used to update the router. In this way, routing changes as the agents improve, and different agents can become specialized in different types of tasks. The paper evaluates CERA-MoA on shared-backbone and heterogeneous agent pools across math, code, instruction following, and general reasoning. The results show better ID and OOD performance than static routing and fixed-workflow training baselines, lower generation cost than fixed Top-2 routing, and different specialization patterns across agents.
Critical strengths
Strength 1: The paper studies an important problem by training the router and agents together instead of treating routing and agent post-training as separate stages.
Strength 2: The familiarity-based router can select agents before generation and avoids running every agent or using an external judge for routing.
Strength 3: The experiments cover both shared-backbone and heterogeneous agent pools, with results on ID, OOD, routing efficiency, and agent specialization.
Critical weaknesses
Weakness 1: The paper claims that the router tracks changes in agent capability during training, but the current experiments do not show this clearly. Since only selected agents are updated on a query, agents that are routed more often to a certain type of query also receive more training on that type. Their rewards may then improve, which can further increase their familiarity scores through Eq. (15). This creates a feedback loop between routing and training, so it is unclear how much of the observed routing behavior comes from tracking capability changes and how much comes from reinforcing earlier routing decisions.
Weakness 2: Each agent has its own predictor–target pair, and the predictor heads are updated with different numbers of queries during training. The paper then compares the resulting familiarity scores across agents as if they were on the same scale. It is unclear whether a score of 0.7 for one agent means the same thing as 0.7 for another agent, especially when some agents receive far more updates than others. Since these scores are used for both ranking and the cumulative threshold in Eq. (6).
Weakness 3: The paper reports one training run with seed 42 for each setting. This is a notable limitation for CERA-MoA because the routing decisions affect which data each agent sees, and those updates then affect later routing decisions. Small differences early in training can therefore lead to different specialization patterns and final results. The paper does not show whether the reported gains or the specialization in Figure 3 are stable across runs.
Questions for authors
Question 1: Can you directly test whether the familiarity score tracks changes in agent capability over time? For example, if an agent’s performance changes while its routing frequency is held fixed, does the router update its familiarity score accordingly?
Question 2: Are the familiarity scores calibrated across agents? Can you show that the scores remain comparable when the predictor heads have very different update counts, for example by reporting per-agent score distributions or calibration against actual rewards?
Question 3: Do the main performance gains and the specialization patterns in Figure 3 remain consistent across different random seeds? Please report results from multiple runs for the main setting.
3 comments
Thank you for provide the rebuttal. My concerns have been addressed. I'll keep my score.
Response to Reviewer t7A8
Thank you again for the positive assessment and constructive comments
Follow-up on the additional seeds
The experiments in the manuscript are seed 42. Seeds 41 and 43 are two further training runs of the same Qwen3-4B pool (four agents, the default recipe). Averages are the unweighted means used in Tables 1 and 2.
In-distribution
Seed 41 is 0.7 below the reported ID average; seed 43 is 0.5 below. Across the three seeds the average stays in 62.5–63.2, and the mean is 62.8.
Out-of-distribution
Seed 41 is 2.3 below the reported OOD average; seed 43 is 0.1 above. The three-seed mean is 72.1, and the range is 70.5–72.9.
All three seeds stay above the strongest Qwen3-4B baseline, AT-GRPO (61.0 / 71.5), on ID; the three-seed mean is 62.8 / 72.1. On OOD, seeds 42 and 43 remain above AT-GRPO. Seed 41 is slightly lower because OlympiadBench is slightly lower.
The gap is a trade between hard math and code. In seed 41 the hard-math agent receives fewer of those queries, so MATH, DAPO, and OlympiadBench are lower, while the code agent is slightly stronger. Seed 43 goes the other way: hard math is higher, and the code agent is slightly weaker. The seed only changes which specialist is stronger; across all three seeds the routed averages stay at the level reported in the maniscript.