- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes IMAX to improve reasoning diversity through a pool of learned soft prefixes. The backbone remains frozen. Prefixes are optimized using correctness rewards and a response-level mutual information objective. A classifier predicts the prefix identity from each generated response. The paper evaluates inference-time performance and uses the learned prefixes to guide subsequent full-policy RLVR.
Critical strengths
-
Learning reusable prefixes on a frozen backbone is a practical approach to controlling reasoning diversity. Their use in subsequent RLVR training extends the method beyond inference-time prompting.
-
Some improvements are substantial. On Qwen3-4B-Base, IMAX improves MATH-500 Pass@4 by 11.60 points and Avg@4 by 7.00 points over GRPO-prefix.
Critical weaknesses
-
The novelty over closely related work is not sufficiently established. LTPO [1] already uses policy gradients to optimize continuous input vectors while keeping the LLM frozen. Model Whisper [2] learns continuous input prefixes on a frozen backbone through entropy minimization. Their objectives differ from IMAX, but they are directly relevant to its adaptation mechanism. SoftCoT++ [3] explicitly diversifies continuous thought representations. Random Soft Prompts [4] studies embedding-based diversification during inference and RL rollouts. These works are not discussed. UpSkill is discussed but lacks an empirical comparison. The paper needs clearer positioning and matched comparisons to establish the benefit of its learned prefix pool and InfoMax objective.
-
Unresolved reporting issues weaken the experimental evidence. Tables 1 and 3(a) report very different results for what is described as Qwen3-8B prefix-only IMAX. MATH-500 Avg@4 is 41.00 in Table 1 and 66.80 in Table 3(a). The configuration difference is unclear. Table 6 also reports IFEval Pass@4 as 47.87±0.00, which conflicts with the per-example standard error definition in Eq. (9). Evaluation templates and grading details are incomplete. Aditionally, the appendix shows mixed AIME results that receive little discussion. For example, IMAX reduces 8B AIME24 Pass@4 from 16.67 to 10.00 relative to GRPO-prefix. These issues limit confidence in the claimed robustness. The base-model scores also need clarification. MATH-500 Avg@4 is only 30.00% for Qwen3-4B-Base and 33.65% for Qwen3-8B-Base. The official Qwen3 report gives MATH scores of 54.10% and 60.80%, respectively. These results are not directly comparable because the official evaluation uses full MATH with 4-shot CoT. Still, the large gap warrants an explanation of the evaluation setup.
[1] Thinking on the Fly: Test-Time Reasoning Enhancement via Latent Thought Policy Optimization
[2] Model Whisper: Steering Vectors Unlock Large Language Models' Potential in Test-time
[3] SoftCoT++: Test-Time Scaling with Soft Chain-of-Thought Reasoning
[4] From Noise to Diversity: Random Embedding Injection in LLM Reasoning
Questions for authors
-
What checkpoints and training configurations produced Tables 1 and 3(a)? Please explain the large discrepancy in their prefix-only results.
-
Why are several nonzero, nonperfect Pass@4 scores reported with zero standard error? Please provide corrected uncertainties and clarify whether independent training runs were conducted.
-
Please specify the evaluation prompts, few-shot settings, generation limits, stopping criteria, and answer extraction rules. What are the truncation and parsing failure rates?
-
What evidence establishes an advantage over independently resampled random soft prompts and UpSkill under comparable budgets?
-
Why are AIME results omitted from Table 1? How do the mixed results in Tables 4–6 affect the claimed improvements in reasoning quality and coverage?
-
Can you conduct experiments using more frontier LLMs, such as Qwen3.5, MiniCPM5?
2 comments
Response to the authors
Thanks for your rebuttal. Most of my concern has not been addressed (e.g., Weakness 1, Question 4 & 6). I would like to maintain my rating.
Response to Reviewer UMj5
W2 / Q1: Different Checkpoints in Tables 1 and 3(a)
The two tables evaluate different models. Table 1 uses Qwen3-8B-Base, as specified in Section 5.1, whereas Table 3(a) uses the post-trained Qwen3-8B, as stated in its caption. Due to space constraints, Table 1 abbreviates the model names and omits the “-Instruct”/“-Base” suffixes, which made this distinction unclear. Thus, 67.20/41.00 and 76.40/66.80 are not results from the same checkpoint. We will restore the full model names in Table 1 and clarify the model used in Table 3(a).
W2 / Q2: Corrected IFEval Standard Errors
We thank the reviewer for pointing out this error. The
±0.00entries are reporting errors, not measured zero variance. These late IFEval runs were not merged into the main uncertainty computation, so missing values defaulted to0.00. Using Eq. (9) over 541 examples, the corrected Pass@4 standard errors are: Table 4—GRPO-full2.15, DAPO-prefix2.15, DAPO+IMAX2.15; Table 5—GRPO-full2.13, MERCI-prefix2.15, DAPO+IMAX2.14; Table 6—GRPO-full2.15. We will update these entries in the next revision.W2 / Q3: Evaluation Protocol and Comparison with Official Qwen Results
Our results and the official Qwen numbers use different evaluation protocols. We use MATH-500 with four sampled responses, a 2,048-token limit, temperature 0.7, top-k 20, and top-p 0.8 (Table 10), whereas the cited Qwen results use full MATH with 4-shot CoT. All methods in our paper use the same decoding budget.
W2 / Q5: Mixed AIME Results
The prefix-only AIME estimates are high variance and are not used as central evidence. AIME has few problems, and our compute-limited prefix-only evaluation uses only four rollouts with a 2,048-token limit. We therefore report the complete results in Tables 4–6 but omit them from Table 1. We agree that the prefix-only AIME results are mixed, so our claim is “most,” not all, benchmark pairs. Separately, the downstream policy improves both AIME24 and AIME25 in Table 2 and AIME24 across all reported
Kin Table 7.