- Rating
- 3
- Confidence
- 4(+1)
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes MN RandOpt, which is a meaningful improvement over RandOpt to me. Following the RandOpt, the proposed method does not require gradient computation or backpropagation as well. The main contribution is about how to sample candidates in weight space, and to build a stronger ensemble with fewer candidates. Overall, I find the main claim reasonable, and the extensive experiments/analyses may make the paper very interesting to people in the post-training community. However, I still have some concerns about the design, the intuition supporting some of their claims, and the completeness of the evaluation.
Critical strengths
I find three aspects of the paper particularly strong.
-
The authors argue that different parameter types should use different natural norms rather than being treated uniformly. They introduce a module-mass allocation scheme and determine perturbation scales based on both this allocation and a sensitivity correction. Furthermore, the proposed Jacobian-based calibration is a reasonable way to account for differences in functional sensitivity across modules.
-
Beyond downstream task performance, the paper examines the effects of population size, model scale, and model family. It also includes runtime comparisons, ablations, and comparisons with conventional iterative zeroth-order optimization methods.
-
The paper is well written and provides detailed hyperparameters, making the results relatively easy to reproduce.
Critical weaknesses
-
The choice of norms seems remain heuristic. While I agree that different parameter types have different structural properties, the specific choice of norm for each module still appears to be a manual design decision. The ablation results provide some empirical support, but the paper does not establish mathmatically why these choices should be optimal or whether alternative norm assignments could work equally well or better.
-
Inference cost may become a practical concern. I think one of the main obstacles to deploying RandOpt is the inference overhead of ensembling multiple models. Although the paper demonstrates substantial reductions in searching time, the final inference still requires running multiple selected models.
-
Limited evidence at large-scales. The RandOpt literature observes that larger models tend to have more high-performing regions in weight space. The Qwen2.5 used in this paper cover only 0.5B to 3B parameters. Although the paper also evaluates the OLMo 3 7B, the cross-family experiments are limited to two tasks. More systematic evaluation on larger models would help establish whether the proposed method continues to provide benefits as model scale increases.
Questions for authors
-
Recent works such as [1], aim to consolidate rewarded perturbations into a single model, avoiding the need to run multiple models at inference time. What advantages does MN RandOpt offer over such approaches?
-
In Figure 2, MN RandOpt appears to use 100 candidates, while RandOpt uses 300, with an ensemble size of only 10. It looks a little bit unfair as it does not establish whether its sampling strategy is better under the same candidate budget. How does RandOpt perform with N=100 and K=100? Such a comparison would help isolate the effect of the proposed sampling geometry from the effect of population size.
-
Given the availability of Qwen 3.8, why do the authors employ Qwen 2.5 and Llama 3.2? Is this choice motivated by reproducibility, architectural compatibility, or other considerations?
[1] Consolidating Rewarded Perturbations for LLM Post-Training
Typos: Please adjust the img size of Fig. 1.
2 comments
Dear Reviewer 9wrE,
Thank you very much for your positive assessment and for taking the time to consider our responses.
We appreciate your constructive feedback and will incorporate the clarifications and suggested improvements into the revised manuscript.
Thank you again for your thoughtful review and encouragement.
I thank the authors for clarifying my concerns. I will maintain my positive score and have increased my confidence.