- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces COEVO, a framework that jointly adapts model parameters and system-prompt strategy scaffolds during reinforcement learning. Parameters are updated through on-policy RL, while context revisions are periodically generated and evaluated using the current frozen checkpoint. Policy entropy and section-level attention guide the choice among four editing strategies: Expand, Relax, Clarify, and Consolidate. The main contribution is a policy-aware approach to context–parameter co-evolution, in which evolving guidance shapes subsequent training experience. Validation covers mathematical reasoning and code generation across multiple Qwen model settings, with comparisons against fixed-prompt RL, single-signal ablations, and a reward-driven prompt-evolution baseline. Cross-prompt evaluation further examines whether improvements persist under shared inference contexts.
Critical strengths
-
A compelling formulation of policy-aware context–parameter co-evolution. The paper addresses a meaningful interaction: prompts shape the trajectories used for learning, while parameter updates change which guidance is useful. COEVO operationalizes this interaction through state-conditioned context revision within an RL loop (Secs. 2–3). This is consequential because it treats training context as an adaptive component and offers a promising approach to improving the learning process itself.
-
Encouraging empirical gains with evidence of benefits beyond the evolved prompt. COEVO improves Average@12 across the reported mathematical reasoning and code-generation settings (Table 13). More importantly, shared-prompt evaluation on 4B mathematics shows stronger average and worst-prompt performance (Table 2), supporting the interpretation that context evolution benefits the learned parameters rather than merely producing a better inference-time prompt. This provides substantive preliminary support for the central idea.
Critical weaknesses
-
Limited statistical support for the reported improvements. The mathematics evaluation uses only 30 AIME 2025 problems, and the experiments report a single training seed, without independent training repetitions or confidence intervals. On this evaluation set, a 6.67-percentage-point improvement in Pass@12 corresponds to only two additional solved problems. Consequently, the statistical reliability and training-run stability of the reported gains remain unclear.
-
Insufficient validation of the heuristic intervention rules. The mapping from entropy–attention states to Expand, Relax, Clarify, and Consolidate is intuitively motivated but not directly validated. The single-signal ablations show that the combined method performs better in the reported setting, but do not establish whether, at the same policy checkpoint, the prescribed intervention yields better subsequent learning than the other three directions (Table 5). For example, whether Expand improves subsequent learning more than the alternative strategies when both entropy and attention are low.
-
Additional computational expenditure. COEVO generates eight candidate edits every five RL steps and evaluates the candidates together with the incumbent using fresh rollouts (Appendix A.4; Algorithm 1). This introduces substantial additional sampling and evaluation expenditure relative to fixed-prompt RL. Without computational-cost reporting and matched-budget comparisons, it remains unclear how much of the improvement comes from the proposed policy-aware adaptation rather than the additional search budget.
Questions for authors
-
Were any independent training runs conducted?
-
What evidence supports the specific mapping from entropy–attention states to the four editing strategies?
-
What is COEVO’s total computational overhead relative to fixed-prompt RL, including candidate generation, candidate evaluation, and attention measurement?