acceptodds
Under review as a conference paper at ICLR 2027

Modular Norm RandOpt: Population-Efficient Ensembling through Architecture-Aware Perturbations

Abstract

RandOpt samples weight-perturbed language models and ensembles top-ranked candidates through plurality voting, but its global perturbation scale ignores heterogeneous module geometry. We propose Modular Norm RandOpt, an architecture-aware sampling method using module-wise natural norms and calibrated scales while preserving selection and voting. It outperforms RandOpt using fewer candidates on Countdown and at least fewer on GSM8K, with corresponding wall-clock savings. Evaluations across seven tasks and three Qwen scales (B–B) show higher mean accuracy than RandOpt on Countdown, GSM8K, and MATH-500 at every scale. The gains extend to Llama 3.2 B and Gemma 3 B on Countdown and GSM8K. On Qwen2.5-1.5B, our ensembles also achieve higher mean accuracy than iterative baselines on both tasks at comparable main-run evaluation budgets. On GSM8K, a tail-density diagnostic implies only a – candidate reduction, while most ensemble improvement is associated with more favorable correct-expert support. These results highlight perturbation geometry as a key design choice for population-efficient, gradient-free search around pretrained models.

Primary area:
foundation or frontier models, including LLMs
Keywords:
gradient-free post-training, population-based search, weight perturbations, modular norm, language model ensembling

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_9wrEOpenReview ↗
Rating
3
Confidence
4(+1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes MN RandOpt, which is a meaningful improvement over RandOpt to me. Following the RandOpt, the proposed method does not require gradient computation or backpropagation as well. The main contribution is about how to sample candidates in weight space, and to build a stronger ensemble with fewer candidates. Overall, I find the main claim reasonable, and the extensive experiments/analyses may make the paper very interesting to people in the post-training community. However, I still have some concerns about the design, the intuition supporting some of their claims, and the completeness of the evaluation.

Critical strengths

I find three aspects of the paper particularly strong.

  1. The authors argue that different parameter types should use different natural norms rather than being treated uniformly. They introduce a module-mass allocation scheme and determine perturbation scales based on both this allocation and a sensitivity correction. Furthermore, the proposed Jacobian-based calibration is a reasonable way to account for differences in functional sensitivity across modules.

  2. Beyond downstream task performance, the paper examines the effects of population size, model scale, and model family. It also includes runtime comparisons, ablations, and comparisons with conventional iterative zeroth-order optimization methods.

  3. The paper is well written and provides detailed hyperparameters, making the results relatively easy to reproduce.

Critical weaknesses
  1. The choice of norms seems remain heuristic. While I agree that different parameter types have different structural properties, the specific choice of norm for each module still appears to be a manual design decision. The ablation results provide some empirical support, but the paper does not establish mathmatically why these choices should be optimal or whether alternative norm assignments could work equally well or better.

  2. Inference cost may become a practical concern. I think one of the main obstacles to deploying RandOpt is the inference overhead of ensembling multiple models. Although the paper demonstrates substantial reductions in searching time, the final inference still requires running multiple selected models.

  3. Limited evidence at large-scales. The RandOpt literature observes that larger models tend to have more high-performing regions in weight space. The Qwen2.5 used in this paper cover only 0.5B to 3B parameters. Although the paper also evaluates the OLMo 3 7B, the cross-family experiments are limited to two tasks. More systematic evaluation on larger models would help establish whether the proposed method continues to provide benefits as model scale increases.

Questions for authors
  1. Recent works such as [1], aim to consolidate rewarded perturbations into a single model, avoiding the need to run multiple models at inference time. What advantages does MN RandOpt offer over such approaches?

  2. In Figure 2, MN RandOpt appears to use 100 candidates, while RandOpt uses 300, with an ensemble size of only 10. It looks a little bit unfair as it does not establish whether its sampling strategy is better under the same candidate budget. How does RandOpt perform with N=100 and K=100? Such a comparison would help isolate the effect of the proposed sampling geometry from the effect of population size.

  3. Given the availability of Qwen 3.8, why do the authors employ Qwen 2.5 and Llama 3.2? Is this choice motivated by reproducibility, architectural compatibility, or other considerations?

[1] Consolidating Rewarded Perturbations for LLM Post-Training

Typos: Please adjust the img size of Fig. 1.

2 comments

I thank the authors for clarifying my concerns. I will maintain my positive score and have increased my confidence.

Dear Reviewer 9wrE,

Thank you very much for your positive assessment and for taking the time to consider our responses.

We appreciate your constructive feedback and will incorporate the clarifications and suggested improvements into the revised manuscript.

Thank you again for your thoughtful review and encouragement.

Reviewer_Wnb9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Modular Norm RandOpt that replaces the isotropic sampling of RandOpt with architecture-aware perturbations. Each perturbation is normalized by module-wise norm and scaled by mass allocation and sensitivity correction while selection and voting procedures are kept unchanged. The authors report that it requires fewer candidates than RandOpt and outperforms RandOpt in most settings.

Critical strengths
  • The method modifies only the sampling method while keeping selection and voting fixed which makes the comparison straightforward.
  • The improvement is noticeable and persists at a large population size which suggests that the choice of sampling distribution can meaningfully affect ensemble quality.
Critical weaknesses
  • The evidence is narrower than the claims. The clear gains are concentrated in GSM8K while improvements in other settings are small. Also, since MN and RandOpt barely improve with N, results are better read as a consistent accuracy gain at equal N rather than a measured reduction.
  • It is unclear which components of the method drive the improvement. The method combines three components yet ablations do not cleanly separate their contributions which makes it hard to assess the claim that the modular-norm geometry itself is responsible for the gains.
Questions for authors
  • Could the authors add results for N < 25 on GSM8K to show where MN RandOpt stops matching RandOpt at N=300?
  • Please clarify the "No sensitivity" and "No recursive scaling" variants in Table 3(c). What exactly is changed in each?
  • Could the authors extend the ablations to GSM8K and clarify how each component's contribution is isolated from others?
3 comments
Reviewer_Wnb9OpenReview ↗

Reviewer response

Thank you for extensive experiments and detailed response. It does address some weaknesses and questions yet several remain.

q1 result are still flat in both methods across N which makes gain constant at same N not a reduction in number of candidates needed. I suggest reframing this contribution. Also sensitivity correction in q3 table makes little difference on gsm8k (noise range) so its contribution is still not well supported.

So I will keep my original score for now.

AuthorsOpenReview ↗

Follow-up: Reframing the Main Contribution and Sensitivity Ablation

Dear Reviewer Wnb9,

Thank you for your thoughtful follow-up. We agree with both points and will revise the manuscript accordingly.

1. Population scaling and the main contribution.

We agree that the relatively flat population-scaling curves are better interpreted as evidence of consistent accuracy improvements at matched candidate budgets, rather than a general reduction in the number of candidates required.

We will therefore reframe our main contribution around improved ensemble accuracy at matched N, rather than emphasizing candidate-reduction factors.

After independently tuning both methods on GSM8K, MN-RandOpt achieves higher mean accuracy at all tested matched population sizes (N=25, 100, 300), with gains of +5.20, +2.93, and +2.40 pp, respectively.

We will revise the abstract, main text, and figures accordingly. Previously reported population-reduction factors will be presented only as observations under specific calibration settings, rather than general efficiency guarantees.

2. Contribution of sensitivity correction.

We agree that sensitivity correction provides only a marginal improvement on GSM8K (80.34% vs. 80.01%, +0.33 pp), which does not establish its necessity on this task.

However, under the revised common evaluation protocol on Countdown, removing sensitivity correction reduces mean accuracy from 38.00% to 34.13% (−3.87 pp).

These results suggest that the contribution of sensitivity correction is task-dependent: it shows a larger observed benefit on Countdown but little additional improvement on GSM8K. Given the limited number of seeds, we do not claim that this difference establishes a statistically robust benefit across tasks.

We will clarify this distinction and avoid claiming that all three components are uniformly necessary. The revised ablations provide stronger empirical support for module-specific natural norms and architecture-based mass allocation.

Thank you again for your constructive feedback. We believe these revisions will better align our claims with the experimental evidence.

Dear Reviewer Wnb9,

Thank you again for your thoughtful feedback throughout the discussion.

We have carefully considered your remaining concerns and will revise the manuscript to emphasize matched-budget accuracy improvements rather than general candidate-reduction claims. We will also clarify the task-dependent contribution of sensitivity correction and avoid overstating its necessity.

We hope these revisions address your main concerns. Given the additional experiments and clarifications, we would be grateful if you could reconsider your overall assessment.

Thank you again for your time and constructive feedback.

Reviewer_E55FOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors modify RandOpt's candidate distribution using module-specific norms, architecture-based allocation, and calibrated sensitivity corrections, while retaining candidate selection and plurality voting. The paper reports comparable or better ensembles with 3× fewer candidates on Countdown and at least 12× fewer on GSM8K under a shared calibration protocol. Experiments examine task and model transfer, runtime, iterative baselines, and the relationship between individual candidate quality and ensemble accuracy.

Critical strengths
  1. The main comparisons appear to control important sources of variation, 6he population reductions translate into measured end-to-end speedups of 2.48× on Countdown and 3.49× on GSM8K. I consider this a fair basis for comparing the calibration task.

  2. The empirical gains extend beyond one task or model, with mean accuracy improving on Countdown, GSM8K, and MATH-500 at all experimented Qwen scales, with additional positive outcomes on Llama and Gemma. From my point of view, reusing the Countdown calibration across tasks provides a meaningful test of transfer.

Critical weaknesses
  1. The experiments do not sufficiently isolate the value of the particular modular construction. The method changes both perturbation allocation and functional displacement. Selecting both scales on Countdown is fair for transfer, but I believe it leaves open whether the large GSM8K gain persists against a task-tuned or functionally matched isotropic baseline. Functional matching is performed for other model families, but not for the main Qwen comparison. The component ablations also change perturbation magnitude: the Frobenius radius uses different units without a documented selection procedure, and module-restricted controls are not renormalized. In my view, these controls therefore do not establish which structural choices are necessary and need to be readdressed in a revision.

  2. The precision of the headline efficiency claims is unclear; for example, the 3× and ≥12× figures compare particular observed points over three population seeds; paired uncertainty for these contrasts is not reported. Transfer is also uneven outside the main reasoning tasks. I believe the results support useful gains under the tested calibration, but do not yet establish a robust accuracy-matched population advantage across settings.

  3. The iterative comparison leaves the strongest alternative ensemble untested. The budget counts final ensemble evaluations, but compares 25 retained models with one optimized model. At K=1, Modular Norm RandOpt loses substantially to ES? I think a budget-matched ensemble of iterative runs or checkpoints would clarify whether population search is a better use of compute when both approaches can exploit ensembling.

Questions for authors
  • Does the main GSM8K advantage persist after independent task-specific scale tuning or matching output KL and hidden-state displacement?
  • How was the Frobenius radius selected, and do the component gains persist with separately calibrated perturbation magnitudes?
  • Some presentation issues that you could take a look at: Figure one is too wide in my opinion, while the highlighting of best results across the tables could be enhanced. I found it difficult to distinguish bold from non-bold outcomes. Maybe a color box highlighting these values could make it easier to follow?

Comments on the paper

2 comments
AuthorsOpenReview ↗

Correction to the Countdown Ablation Baseline

We would like to correct the Countdown Full MN entry in our additional ablation results.

The correct result under the revised common evaluation protocol (1,500 held-out examples; seeds 42–44) is 38.00 ± 3.07%, rather than 37.80 ± 0.87%, which was inadvertently carried over from the original 500-example evaluation.

The corrected seed-wise accuracies are 34.47%, 40.07%, and 39.47%.

All other reported ablation results remain unchanged. Full MN continues to achieve the highest mean accuracy among the evaluated variants on both Countdown and GSM8K.

We apologize for the inconsistency and will use the corrected value in the revised manuscript.

AuthorsOpenReview ↗

Summary of Manuscript Revisions

Dear Area Chair and Reviewers,

Thank you for your constructive feedback throughout the discussion. We have uploaded a revised manuscript incorporating the main clarifications and additional experimental results.

The key revisions are:

  1. Reframed the main claims: We now emphasize ensemble accuracy improvements at matched candidate budgets, while restricting the observed population-reduction comparisons to their specific calibration settings.

  2. Added GSM8K-specific scale tuning: Section 5.2 reports independently tuned comparisons of RandOpt and MN RandOpt under the same tuning budget, with protocol details in Appendix B.3.

  3. Updated component ablations: We replaced the previous Countdown ablations with consistently evaluated controls using independently selected radii, added GSM8K ablations, and clarified the differences between mass-allocation and norm controls.

  4. Expanded uncertainty reporting: Appendix C includes smaller-population GSM8K results and paired confidence intervals for the main cross-budget and matched-budget comparisons.

  5. Clarified scope and limitations: We revised the iterative-baseline comparisons, mechanistic interpretation, figures, and limitations to reflect the discussion.

We appreciate the reviewers' suggestions, which have helped us strengthen the experimental comparisons and better align the manuscript's claims with the evidence.

Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.