- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes MOMO for finding a single best solution to a verifiable code-optimization problem. Several language models first perform independent RLVR-based refinement. At each round boundary, the best states are transferred to the other models, which retain their own trained weights while resetting search visitation statistics and optimizer state. The motivation is that different model families favor different optimization techniques and can improve each other's solutions after individual runs stall. Experiments cover two mathematical construction problems, circuit placement, and qubit routing. MOMO reports new best results on three tasks, while AC1 improves without reaching the previous best. The paper also studies state transfer versus prompt summaries and retained versus fresh weights.
Critical strengths
-
The method is simple and the resulting discoveries are concrete. In particular, the new Erdős minimum-overlap construction is a substantive output even though the numerical improvement is small. MOMO improves over the better individual model in all three reported seeds on this problem. The results on mathematical constructions and hardware optimization make the approach more interesting than a demonstration on only one objective.
-
The paper provides useful evidence beyond final scores. The AC1 trajectory shows Muse moving Gemma's construction to finer grids and Qwen subsequently polishing it; the routing analysis traces changes to decay and front-layer attraction. These examples make model complementarity plausible. Table 2 also directly tests two handoff choices, and the appendices explain the evaluators, budgets, generated programs, and baseline protocols in considerable detail.
Critical weaknesses
-
The central fixed-budget comparison remains incomplete. Figure 5 aligns MOMO and individual runs at 25 steps along a lineage, but MOMO uses 37 aggregate model-steps, versus 25 for each individual run. Taking the best of both independent 25-step runs would instead cost 50 steps, so this is not evidence that MOMO necessarily uses more total compute than an independent ensemble. Rather, the plot does not resolve which strategy is better at a common aggregate budget. A continued single-model run and an independent ensemble under the same total generation/evaluation budget would materially strengthen the main claim. A continuation with the same state pruning and search/optimizer resets would also help distinguish changing models from changing the search conditions at a round boundary. This matters because the proposed goal is the best solution within a fixed search budget.
-
The experiments do not sufficiently isolate the contribution of heterogeneous, continuously trained models. TTT-Discover already combines online RL with a search buffer, and AlphaEvolve already exploits complementary models within evaluated program search (both are acknowledged). MOMO's particular combination is distinct, but its value should be demonstrated against a same-family population with separate seeds/adapters and the same state exchanges. Table 2 shows that retaining weights helps relative to reinitializing them; however, the fresh-weight recipients also undergo RL, so this does not isolate the benefit of continued weight updates over a frozen recipient. The retained-weight gains are relatively modest and have no reported uncertainty. The technique counts and detailed trajectories are informative, but do not substitute for these controls. I therefore remain uncertain whether the proposed combination contributes enough beyond strong search and handoff to support acceptance.
Questions for authors
-
Can you report best-result curves against aggregate candidate evaluations and generated tokens, including every branch and round, together with total training and evaluator compute? At that budget, how does MOMO compare with longer single-model refinement and an independent ensemble? Please clarify which branches in Figure 6 are included in the search that produced the reported results.
-
Can you compare with a same-family population that exchanges states identically, and with heterogeneous recipients whose weights are frozen at their round-1 checkpoints during subsequent rounds? Keeping the evaluator, refinement objective, handoff schedule, and search resets matched would distinguish model-family diversity from ordinary population search and continued learning.
-
For Table 2, are the starting donor states, recipient budgets, and seeds matched between the fresh and retained conditions, and how variable are these gains? A self-continuation control applying the same top-state selection and resets would also be useful. The three-seed Erdős result is encouraging; how stable are the handoff gains on the other tasks, especially routing, where the final reported total is only 21 SWAPs below Hyra? These are requests to assess the current claims rather than suggestions to broaden the task scope.
4 comments
I did the review myself and received feedback from the author.
First, thanks for the additional experiments. The matched-step comparisons and the two additional routing seeds address an important part of my original concern, which show that MOMO can improve on extending a single-model run under the tested budgets.
I still share the concern raised by another reviewer about how best to spend a fixed search budget. In both Erdős seeds reported for the restart comparison, the best of three short Qwen runs slightly outperforms MOMO. The one-step adapter comparison is also useful, but it does not establish whether cross-family handoff improves on same-family exchange over multiple rounds, or whether recipients benefit from continuing to update their weights after handoff.
Some updates:
- "Whether recipients benefit from continuing to update their weights after handoff" - We have two experiments for this.
First a proxy experiment. We tested running historical adapters on the step 12 state-tree, and measuring the improvement rate relative to the step 12 adapter. Values represent percentage changes relative to the step-12 adapter (0% baseline).
| Model | Metric | Base | Step 4 | Step 8 | Step 12 |
|---|---|---|---|---|---|
| Qwen | Improvement Rate | -59% | -50% | -46% | 0% |
| Qwen | Validity | -43% | -31% | -11% | 0% |
| Muse | Improvement Rate | -60% | -53% | -29% | 0% |
| Muse | Validity | -3% | +4% | -9% | 0% |
We see that for Qwen, training helps improve the improvement rate and crucially its validity rate, because most programs starting out are invalid (code doesn't compile, syntax errors, timeout errors, etc.). In contrast, Muse starts out with a very good validity rate. Over the course of training from step 0 -> 12, we see that both models' improvement rate increases monotonically.
Second, at the hand-off position we compared on two seeds whether pure inference from step 12 - 20 could outweigh continued weight updates on the recipient model.
| Seed | Donor @12 | Recipient @20 (with training) | Recipient @20 (without training) |
|---|---|---|---|
| Seed 1 | 7.59 | 6.97 | 7.04 |
| Seed 2 | 18.96 | 18.90 | 18.91 |
| Seed 3 | 13.95 | 12.68 | 13.20 |
In 2/3 seeds we see that training enables a larger gain to the step 12 split point, especially on seed 3.
-
"Same-family exchange over multiple rounds vs cross-family": We provided the 1-step adapter experiment as a proxy. We are still running seeded experiments for this and will add them as they come.
-
"Best of 3 short-runs conclusion": We agree, which is why we have continued MOMO experiments, pairing Qwen with stronger models that we can train on Tinker, like Inkling2Small a large MoE model with 17B activated, to understand whether using stronger models, even with pure inference, might be able to better demonstrate the effects of MOMO on the Erdos problem. We trained Inkling2small with 5 steps of training on R1, and also tested a no-train version that runs MOMO R2 from step 12 -> 20. Both versions outperform best of 3 draws.
| Seed | Best of 3 Qwen (12/12/13) | MOMO@37 (12/8/5) (Qwen/Muse) | New MOMO@32 (12/8) (Qwen/Inkling2small, no-train) | New MOMO@32 (12/8) (Qwen/Inkling2small, 5-step R1 train) |
|---|---|---|---|---|
| 1 | 6.89 | 6.93 | 6.804 | 6.682 |
| 3 | 12.28 | 12.32 | 12.24 | 12.08 |
Thank you for the follow-up. The trained-versus-frozen recipient comparison directly addresses part of my concern about continued learning after handoff. The trained recipients perform better in all three reported seeds, although the gains vary substantially.
The Inkling2Small results are encouraging, but this comparison changes both the recipient model and the search strategy. Without an independent-search or no-handoff baseline using the same models and a comparable total budget, it remains unclear how much of the improvement comes from handoff rather than access to a stronger recipient. Model-step counts alone do not establish comparable cost across these configurations.
Rebuttal
Thank you for the constructive feedback and interest in our paper! We address your questions below:
Compute comparison. At a matched budget of 37 model steps on the Erdos problem (~19k candidates), MOMO falls between standalone Qwen and Muse training in generated tokens, grader-hours, and training FLOPs.
Qwen's generation length increases over training, leading to higher token consumption per candidate. Grader-hours also increase as generated programs increasingly exhaust the 1,000-second evaluation time limit. Qwen's validity rate improves substantially (57% → 87%), whereas Muse starts with a much higher validity rate (86% → 91%). Here, validity is the fraction of generated programs that pass compilation and timeout checks. We expect an independent ensemble matched for the number of rollouts to exhibit similar aggregate compute characteristics to MOMO.
At this matched budget, we find the MOMO exceeds the performance of single-model training across seeds (represented as distance to a target C_5 bound of 0.38085, in units of 1e-6, lower is better):
Same-family population vs heterogenous recipients. We ran the same-family control. On a Qwen tree at step 12, we sampled one step with the tree’s own adapter, another Qwen seed’s adapter at step 12, Muse’s adapter at step 12, and the base model (same 16 parents, 512 children each, same grader). All trained adapters far exceed the base model (improvement rate 0.29–0.42 vs 0.10). We found that the average improvement rate of using another Qwen seed's adapter versus Muse's adapter was statistically indistinguishable (Qwen was 0.36e-6 while Muse was 0.32e-6), though each was larger than that of the tree's own adapter. This indicates that mixing seeds for a model can be a useful strategy for introducing variance.
Self Continuation Control. We tested this by running a variant of our experiment, whereby for Qwen at seeds 1-3 on the Erdos problem at step 12, we created two clones of the adapter in two forked threads. Then we reset the optimizer state, visitation count, and set the seed of the fork 1 to (source seed)*3+1 and fork 2 to (source seed)*3+2. We compared the performance at step 20 across three seeds as shown below. In 2/3 seeds, simply resetting the optimizer state of a run is worse than letting it continue.
Handoff procedure detail For Table 2, we drew these results by allowing the same donor states, recipient budget (8 steps of training), and all training conditions between all arms done on Muse and Gemma. We only ran one seed per model and used cross-model agreement to justify our design choice.
Stability in Qubit Routing For qubit routing, we tested two additional seeds as seen below. As also mentioned in Appendix D, we used a larger compute budget, because a) qubit routing is a more challenging problem than Erdos, as the model has to optimize a singly policy across three quantum topologies b) the models are still rapidly improving at step 12. We use 20 steps for round 1, 9 for round 2, and 6 for round 3 (totaling 55 between Gemma and Muse). After step 20, we see that Gemma and Muse models taper off performance, and running them to step 55 only reduces SWAPS by an average of 0.2% and 4.3% for Gemma and Muse, respectively. However running MOMO, reduces SWAPS from the R1 model, Gemma, by 0.6% on average, which is 3x that of running Gemma longer.
Our world record run came from seed 1, and our claim is not that we consistently achieve it, but that MOMO's procedure consistently gives better gains than single-model refinement.